2024-09-19T03:58:09.650 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T03:58:09.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T03:58:09.769 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911627' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM4mB0AfXk7lTwCUJTtanH9CWchlcAEWJlw8/hu+f6OIWJ7sJfB+exjgQBoVKOq9hoHOuyt7BIqoJNWObzxKD24= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwtpu1j1sW6wu+SCq4vjRrgDUb+v6XuyyhcrT33zOL4NlwBGgHriGFCCnpM/H4p3bmvnGILNkJWuoWXnFbGcAQ= smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAQR4U2MJsen6vClfAlCmOrQCK4/5BkOH7GCML++9xrty2H4zUF/tjUcTXzY6fuwQvOqfWdCSbu8J10rVyxGAGM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T03:58:09.769 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T03:58:09.770 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T03:58:09.770 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T03:58:09.772 INFO:teuthology.task.internal:Checking packages... 2024-09-19T03:58:09.793 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T03:58:09.793 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T03:58:09.793 INFO:teuthology.packaging:ref: None 2024-09-19T03:58:09.793 INFO:teuthology.packaging:tag: None 2024-09-19T03:58:09.793 INFO:teuthology.packaging:branch: squid 2024-09-19T03:58:09.793 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:58:09.794 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T03:58:10.010 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T03:58:10.011 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T03:58:10.020 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T03:58:10.020 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T03:58:10.043 INFO:teuthology.task.internal:Saving configuration 2024-09-19T03:58:10.064 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T03:58:10.071 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T03:58:10.093 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:52:53.326104', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM4mB0AfXk7lTwCUJTtanH9CWchlcAEWJlw8/hu+f6OIWJ7sJfB+exjgQBoVKOq9hoHOuyt7BIqoJNWObzxKD24='} 2024-09-19T03:58:10.115 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:52:53.327538', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwtpu1j1sW6wu+SCq4vjRrgDUb+v6XuyyhcrT33zOL4NlwBGgHriGFCCnpM/H4p3bmvnGILNkJWuoWXnFbGcAQ='} 2024-09-19T03:58:10.134 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:52:53.324468', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAQR4U2MJsen6vClfAlCmOrQCK4/5BkOH7GCML++9xrty2H4zUF/tjUcTXzY6fuwQvOqfWdCSbu8J10rVyxGAGM='} 2024-09-19T03:58:10.134 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T03:58:10.140 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T03:58:10.140 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T03:58:10.140 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T03:58:10.140 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T03:58:10.226 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5c1d51fbe0>, signals=[15]) 2024-09-19T03:58:10.226 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T03:58:10.232 INFO:teuthology.task.internal:Opening connections... 2024-09-19T03:58:10.233 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-09-19T03:58:10.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:58:10.308 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-09-19T03:58:10.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:58:10.380 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2024-09-19T03:58:10.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:58:10.455 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T03:58:10.462 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-09-19T03:58:10.481 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-09-19T03:58:10.482 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-09-19T03:58:10.539 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-09-19T03:58:10.539 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-09-19T03:58:10.539 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-09-19T03:58:10.539 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-09-19T03:58:10.539 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-09-19T03:58:10.539 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-09-19T03:58:10.539 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T03:58:10.539 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-09-19T03:58:10.540 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-09-19T03:58:10.540 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T03:58:10.540 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-09-19T03:58:10.540 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T03:58:10.540 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T03:58:10.540 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T03:58:10.541 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-09-19T03:58:10.563 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-09-19T03:58:10.581 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-09-19T03:58:10.582 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-09-19T03:58:10.640 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-09-19T03:58:10.640 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T03:58:10.641 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T03:58:10.642 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-09-19T03:58:10.662 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-09-19T03:58:10.681 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-09-19T03:58:10.681 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-09-19T03:58:10.738 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="9" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="9" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el9" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:LOGO="fedora-logo-icon" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2024-09-19T03:58:10.739 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T03:58:10.740 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T03:58:10.740 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T03:58:10.740 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-09-19T03:58:10.762 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T03:58:10.772 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T03:58:10.819 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T03:58:10.819 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:58:10.822 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:58:10.824 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:58:10.842 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T03:58:10.910 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T03:58:10.911 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:58:10.914 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:58:10.916 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:58:11.008 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T03:58:11.082 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T03:58:11.096 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T03:58:11.096 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-19T03:58:11.096 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-19T03:58:11.097 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T03:58:11.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:58:11.114 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-19T03:58:11.271 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T03:58:11.272 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-19T03:58:11.272 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-19T03:58:11.290 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-503.el9.x86_64 2024-09-19T03:58:11.290 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T03:58:11.290 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-19T03:58:11.291 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum upgrade -y linux-firmware 2024-09-19T03:58:13.714 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-19T03:58:19.763 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 20 MB 00:04 2024-09-19T03:58:24.064 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 18 MB/s | 6.5 MB 00:00 2024-09-19T03:58:26.022 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-19T03:58:27.320 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-19T03:58:32.815 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-19T03:58:34.817 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-19T03:58:34.828 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 1 Package 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 388 M 2024-09-19T03:58:34.829 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-19T03:58:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:linux-firmware-20240905-146.el9.noarch.rpm 67 MB/s | 388 MB 00:05 2024-09-19T03:58:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-19T03:58:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:Total 66 MB/s | 388 MB 00:05 2024-09-19T03:58:43.839 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-19T03:58:44.026 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-19T03:58:44.026 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-19T03:58:47.317 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-19T03:58:47.318 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-19T03:58:56.679 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-19T03:58:57.357 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-19T03:58:57.585 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T03:58:58.402 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T03:58:58.402 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-19T03:58:59.136 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T03:58:59.136 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T03:58:59.136 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-09-19T03:58:59.136 INFO:teuthology.orchestra.run.smithi107.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-19T03:58:59.136 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T03:58:59.136 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-19T03:58:59.476 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-19T03:58:59.477 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T03:58:59.628 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-19T03:58:59.629 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-19T03:58:59.631 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi107.front.sepia.ceph.com to complete... 2024-09-19T03:59:05.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19 03:59:05 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-19T03:59:05.934 INFO:teuthology.orchestra.run.smithi107.stderr:FINISHED --2024-09-19 03:59:05-- 2024-09-19T03:59:05.934 INFO:teuthology.orchestra.run.smithi107.stderr:Total wall clock time: 6.3s 2024-09-19T03:59:05.934 INFO:teuthology.orchestra.run.smithi107.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-09-19T03:59:05.937 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-19T03:59:05.937 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-19T03:59:07.992 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying... ######################################## 2024-09-19T03:59:08.139 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing... ######################################## 2024-09-19T03:59:08.199 INFO:teuthology.orchestra.run.smithi107.stdout:Updating / installing... 2024-09-19T03:59:16.184 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-19T03:59:56.280 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-19T03:59:56.280 DEBUG:teuthology.orchestra.run.smithi107:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-19T03:59:56.371 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T03:59:56.371 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T03:59:56.371 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T03:59:56.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-19T03:59:56.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-19T03:59:56.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-19T03:59:56.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-19T03:59:56.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-19T03:59:56.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-19T03:59:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-19T03:59:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-19T03:59:56.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-19T03:59:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-19T03:59:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-19T03:59:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-19T03:59:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-19T03:59:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-19T03:59:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-19T03:59:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-19T03:59:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-19T03:59:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-19T03:59:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-19T03:59:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-19T03:59:56.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-19T03:59:56.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-19T03:59:56.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-19T03:59:56.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-19T03:59:56.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-19T03:59:56.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-19T03:59:56.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-19T03:59:56.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-19T03:59:56.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-19T03:59:56.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-19T03:59:56.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-19T03:59:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-19T03:59:56.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-19T03:59:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-19T03:59:56.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-19T03:59:56.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-19T03:59:56.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-19T03:59:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-19T03:59:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-19T03:59:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-19T03:59:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-19T03:59:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-19T03:59:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-19T03:59:56.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-19T03:59:56.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-19T03:59:56.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-19T03:59:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-19T03:59:56.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-19T03:59:56.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-19T03:59:56.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-19T03:59:56.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-19T03:59:56.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-19T03:59:56.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-19T03:59:56.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-19T03:59:56.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-19T03:59:56.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-19T03:59:56.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-19T03:59:56.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-19T03:59:56.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-19T03:59:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-19T03:59:56.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-19T03:59:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-19T03:59:56.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-19T03:59:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-19T03:59:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-19T03:59:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-19T03:59:56.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-19T03:59:56.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-19T03:59:56.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-19T03:59:56.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-19T03:59:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-19T03:59:56.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-19T03:59:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-19T03:59:56.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-19T03:59:56.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-19T03:59:56.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-19T03:59:56.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-19T03:59:56.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-19T03:59:56.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-19T03:59:56.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-19T03:59:56.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-19T03:59:56.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-19T03:59:56.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-19T03:59:56.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-19T03:59:56.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-19T03:59:56.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-19T03:59:56.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-19T03:59:56.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-19T03:59:56.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-19T03:59:56.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-19T03:59:56.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-19T03:59:56.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-19T03:59:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-19T03:59:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-19T03:59:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-19T03:59:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-19T03:59:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-19T03:59:56.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-19T03:59:56.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-19T03:59:56.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-19T03:59:56.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-19T03:59:56.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-19T03:59:56.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-19T03:59:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-19T03:59:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-19T03:59:56.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-19T03:59:56.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-19T03:59:56.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-19T03:59:56.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-19T03:59:56.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-19T03:59:56.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-19T03:59:56.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-19T03:59:56.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-19T03:59:56.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-19T03:59:56.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-19T03:59:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-19T03:59:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-19T03:59:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-19T03:59:56.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-19T03:59:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-19T03:59:56.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-19T03:59:56.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-19T03:59:56.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-19T03:59:56.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-19T03:59:56.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-19T03:59:56.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-19T03:59:56.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-19T03:59:56.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-19T03:59:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-19T03:59:56.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-19T03:59:56.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-19T03:59:56.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-19T03:59:56.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-19T03:59:56.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-19T03:59:56.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-19T03:59:56.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-19T03:59:56.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-19T03:59:56.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-19T03:59:56.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-19T03:59:56.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-19T03:59:56.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-19T03:59:56.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-19T03:59:56.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-19T03:59:56.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-19T03:59:56.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-19T03:59:56.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-19T03:59:56.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-19T03:59:56.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-19T03:59:56.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-19T03:59:56.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-19T03:59:56.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-19T03:59:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-19T03:59:56.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-19T03:59:56.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-19T03:59:56.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-19T03:59:56.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-19T03:59:56.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-19T03:59:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-19T03:59:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-19T03:59:56.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-19T03:59:56.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-19T03:59:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-19T03:59:56.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-19T03:59:56.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-19T03:59:56.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-19T03:59:56.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-19T03:59:56.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-19T03:59:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-19T03:59:56.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-19T03:59:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-19T03:59:56.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-19T03:59:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-19T03:59:56.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-19T03:59:56.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-19T03:59:56.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-19T03:59:56.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-19T03:59:56.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-19T03:59:56.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-19T03:59:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-19T03:59:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-19T03:59:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-19T03:59:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-19T03:59:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-19T03:59:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-19T03:59:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-19T03:59:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-19T03:59:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-19T03:59:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-19T03:59:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-19T03:59:56.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-19T03:59:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-19T03:59:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-19T03:59:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-19T03:59:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-19T03:59:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-19T03:59:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-19T03:59:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-19T03:59:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-19T03:59:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-19T03:59:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-19T03:59:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-19T03:59:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-19T03:59:56.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-19T03:59:56.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-19T03:59:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-19T03:59:56.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-19T03:59:56.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-19T03:59:56.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-19T03:59:56.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-19T03:59:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-19T03:59:56.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-19T03:59:56.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-19T03:59:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-19T03:59:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-19T03:59:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-19T03:59:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-19T03:59:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-19T03:59:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-19T03:59:56.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-19T03:59:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-19T03:59:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-19T03:59:56.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-19T03:59:56.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-19T03:59:56.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-19T03:59:56.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-19T03:59:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-19T03:59:56.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-19T03:59:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-19T03:59:56.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-19T03:59:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-19T03:59:56.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-19T03:59:56.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-19T03:59:56.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-19T03:59:56.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-19T03:59:56.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-19T03:59:56.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-19T03:59:56.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-19T03:59:56.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-19T03:59:56.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-19T03:59:56.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-19T03:59:56.521 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T03:59:56.521 DEBUG:teuthology.orchestra.run.smithi107:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-19T03:59:56.543 INFO:teuthology.orchestra.run.smithi107.stdout:postinstall scriptlet (using /bin/sh): 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout:for file in vmlinuz System.map config; do 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout: fi 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout:done 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T03:59:56.544 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-09-19T03:59:56.545 INFO:teuthology.orchestra.run.smithi107.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-19T03:59:56.545 INFO:teuthology.orchestra.run.smithi107.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T03:59:56.545 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-09-19T03:59:56.545 INFO:teuthology.orchestra.run.smithi107.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-19T03:59:56.545 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-19T03:59:56.545 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T03:59:56.545 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-09-19T03:59:56.546 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-09-19T03:59:56.617 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-09-19T03:59:56.617 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-09-19T03:59:56.617 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-09-19T03:59:56.617 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 82.el9 2024-09-19T03:59:56.617 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-09-19T03:59:56.617 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T03:59:56.617 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8274670 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T03:59:56.618 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T03:59:56.619 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T03:59:56.619 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-09-19T03:59:56.619 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T03:59:56.619 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T03:59:56.619 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T03:59:56.619 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T03:59:56.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T03:59:57.182 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-09-19T03:59:58.548 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T03:59:58.577 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-09-19T03:59:58.579 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-09-19T03:59:58.594 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.HJHsndAgLK 2024-09-19T03:59:58.595 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HJHsndAgLK 2024-09-19T03:59:58.661 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.HJHsndAgLK 2024-09-19T03:59:58.800 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.HJHsndAgLK is 6KB 2024-09-19T03:59:58.848 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.HJHsndAgLK 2024-09-19T03:59:58.863 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T03:59:58.925 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T03:59:58.925 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T03:59:58.925 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T03:59:58.925 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-19T03:59:58.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T03:59:59.095 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-09-19T04:00:29.098 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T04:00:29.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-19T04:00:29.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:00:47.421 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-19T04:00:56.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-19T04:00:56.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:00:59.489 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-19T04:01:11.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-19T04:01:11.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:02:11.493 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T04:02:26.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-19T04:02:26.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:02:26.844 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-19T04:02:27.342 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-19T04:02:27.343 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-19T04:02:27.343 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-19T04:02:27.376 INFO:teuthology.orchestra.run.smithi107.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:02:27.377 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:02:27.377 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T04:02:27.377 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T04:02:28.378 DEBUG:teuthology.parallel:result is None 2024-09-19T04:02:28.379 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T04:02:28.386 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T04:02:28.386 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:02:28.390 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:02:28.393 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:02:28.424 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T04:02:28.431 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T04:02:28.449 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T04:02:28.449 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:02:28.469 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:02:28.471 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:02:28.598 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T04:02:28.605 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T04:02:28.605 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:02:28.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:02:28.623 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:02:28.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:02:28.641 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:02:28.645 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:02:28.665 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:02:28.730 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:02:28.732 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:02:28.737 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:02:28.754 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:02:28.757 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:02:28.760 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:02:28.765 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T04:02:28.773 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T04:02:28.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:02:28.777 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:02:28.779 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:02:28.829 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T04:02:28.838 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T04:02:28.838 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:02:28.842 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:02:28.848 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:02:28.872 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:02:28.976 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:02:29.055 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:02:29.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:02:29.188 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:02:29.189 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:02:29.243 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:02:29.288 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:02:29.340 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:02:29.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:02:29.467 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:02:29.468 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:02:29.534 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:02:29.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:02:29.633 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:02:29.718 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:02:29.763 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:02:29.763 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:02:29.831 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-09-19T04:02:29.835 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-09-19T04:02:29.837 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2024-09-19T04:02:29.877 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:02:29.905 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:02:29.916 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:02:30.294 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T04:02:30.301 INFO:teuthology.task.internal:Starting timer... 2024-09-19T04:02:30.301 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T04:02:30.312 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T04:02:30.320 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T04:02:30.320 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-09-19T04:02:30.412 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-09-19T04:02:30.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-09-19T04:02:30.481 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-09-19T04:02:30.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2024-09-19T04:02:30.551 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: 2024-09-19T04:02:30.553 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T04:02:30.554 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-09-19T04:02:30.594 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-09-19T04:02:30.595 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-09-19T04:02:30.620 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-09-19T04:02:30.620 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2024-09-19T04:02:30.656 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2024-09-19T04:02:30.656 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi145.front.sepia.ceph.com': 'permissive'} 2024-09-19T04:02:30.656 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:02:30.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:02:30.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:02:30.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:02:30.741 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:02:30.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:02:30.769 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T04:02:30.769 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:02:30.834 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:02:30.863 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:02:30.898 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T04:02:30.908 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T04:02:30.919 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T04:02:30.920 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi115.front.sepia.ceph.com,smithi145.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T04:07:05.648 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi145.front.sepia.ceph.com')] 2024-09-19T04:07:05.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-19T04:07:05.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:07:05.744 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-19T04:07:05.850 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-19T04:07:05.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-19T04:07:05.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:07:05.922 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-19T04:07:06.000 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-19T04:07:06.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-19T04:07:06.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:07:06.077 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-19T04:07:06.148 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-19T04:07:06.149 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T04:07:06.159 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T04:07:06.159 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:07:06.160 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:07:06.164 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:07:06.164 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:07:06.167 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:07:06.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:07:06.194 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:07:06.208 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:07:06.221 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:07:06.221 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:07:06.232 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-09-19T04:07:06.236 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:07:06.243 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-09-19T04:07:06.250 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:07:06.257 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:07:06.261 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2024-09-19T04:07:06.271 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:07:06.272 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2024-09-19T04:07:06.286 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:07:06.300 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:07:06.302 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-09-19T04:07:06.326 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-09-19T04:07:06.328 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:07:06.332 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:07:06.332 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-19T04:07:06.332 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.333 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.333 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.333 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.356 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:07:06.357 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:07:06.361 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:07:06.361 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-09-19T04:07:06.362 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.362 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.362 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.362 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.385 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:07:06.479 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:07:06.486 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:07:06.486 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-19T04:07:06.486 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.486 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.486 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.486 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:07:06.487 INFO:teuthology.run_tasks:Running task install... 2024-09-19T04:07:06.495 DEBUG:teuthology.task.install:project ceph 2024-09-19T04:07:06.496 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T04:07:06.496 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T04:07:06.496 INFO:teuthology.task.install:Using flavor: default 2024-09-19T04:07:06.501 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T04:07:06.501 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T04:07:06.502 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:07:06.502 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:07:06.504 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:07:06.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:07:06.506 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:07:06.506 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:07:06.643 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:07:06.643 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:07:06.652 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:07:06.652 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:07:06.660 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:07:06.660 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:07:06.752 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:07:06.752 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:07:06.752 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:07:06.761 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:07:06.762 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:07:06.762 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:07:06.768 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:07:06.768 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:07:06.768 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:07:06.788 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:07:06.788 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:07:06.795 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:07:06.795 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:07:06.810 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:07:06.810 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:07:06.853 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:07:06.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:07:06.908 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:07:06.929 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:07:06.937 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:07:06.986 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:07:06.995 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2024-09-19T04:07:06.996 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2024-09-19T04:07:07.001 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-09-19T04:07:07.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-09-19T04:07:07.041 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-09-19T04:07:07.044 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-09-19T04:07:07.221 INFO:teuthology.orchestra.run.smithi145.stdout:45 files removed 2024-09-19T04:07:07.226 INFO:teuthology.orchestra.run.smithi115.stdout:45 files removed 2024-09-19T04:07:07.247 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:07:07.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:07:07.306 INFO:teuthology.orchestra.run.smithi107.stdout:45 files removed 2024-09-19T04:07:07.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:07:07.798 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-19T04:07:07.806 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-19T04:07:08.016 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 298 kB/s | 84 kB 00:00 2024-09-19T04:07:08.082 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T04:07:08.090 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T04:07:08.300 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T04:07:08.358 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T04:07:08.358 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-19T04:07:08.568 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-19T04:07:09.058 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-19T04:07:10.162 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.3 MB 00:01 2024-09-19T04:07:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.3 MB 00:02 2024-09-19T04:07:11.184 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-19T04:07:12.300 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-19T04:07:14.426 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 20 MB 00:03 2024-09-19T04:07:16.701 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-19T04:07:16.917 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.5 MB 00:01 2024-09-19T04:07:18.118 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 94 kB/s | 19 kB 00:00 2024-09-19T04:07:18.418 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-19T04:07:19.192 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-19T04:07:21.043 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-19T04:07:22.927 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-19T04:07:24.670 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T04:07:24.869 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 23 MB 00:06 2024-09-19T04:07:26.799 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:07:26.800 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:07:26.813 INFO:teuthology.orchestra.run.smithi115.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:07:26.815 INFO:teuthology.orchestra.run.smithi115.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:07:26.815 INFO:teuthology.orchestra.run.smithi115.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:07:26.817 INFO:teuthology.orchestra.run.smithi115.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:07:26.818 INFO:teuthology.orchestra.run.smithi115.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:07:26.821 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:07:26.822 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:07:26.884 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-19T04:07:26.907 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:07:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:07:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:07:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:07:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:07:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:07:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:07:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:07:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:07:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:07:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:07:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:07:26.919 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:07:26.920 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:07:26.921 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-09-19T04:07:26.921 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:07:26.921 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:07:26.921 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-19T04:07:26.921 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-19T04:07:26.921 INFO:teuthology.orchestra.run.smithi115.stdout:Install 148 Packages 2024-09-19T04:07:26.921 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 8 Packages 2024-09-19T04:07:26.921 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:07:26.922 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 226 M 2024-09-19T04:07:26.922 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-19T04:07:27.713 INFO:teuthology.orchestra.run.smithi115.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 50 kB/s | 6.6 kB 00:00 2024-09-19T04:07:27.717 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 4.8 MB/s | 23 MB 00:04 2024-09-19T04:07:27.922 INFO:teuthology.orchestra.run.smithi115.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.2 MB/s | 896 kB 00:00 2024-09-19T04:07:28.005 INFO:teuthology.orchestra.run.smithi115.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.7 MB/s | 145 kB 00:00 2024-09-19T04:07:28.097 INFO:teuthology.orchestra.run.smithi115.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T04:07:28.214 INFO:teuthology.orchestra.run.smithi115.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 13 MB/s | 1.5 MB 00:00 2024-09-19T04:07:28.264 INFO:teuthology.orchestra.run.smithi115.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 9.2 MB/s | 2.4 MB 00:00 2024-09-19T04:07:28.481 INFO:teuthology.orchestra.run.smithi115.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-19T04:07:28.757 INFO:teuthology.orchestra.run.smithi115.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 18 MB/s | 21 MB 00:01 2024-09-19T04:07:28.798 INFO:teuthology.orchestra.run.smithi115.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 610 kB/s | 25 kB 00:00 2024-09-19T04:07:29.032 INFO:teuthology.orchestra.run.smithi115.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 19 MB/s | 11 MB 00:00 2024-09-19T04:07:29.107 INFO:teuthology.orchestra.run.smithi115.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.0 MB/s | 228 kB 00:00 2024-09-19T04:07:29.191 INFO:teuthology.orchestra.run.smithi115.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 324 kB/s | 27 kB 00:00 2024-09-19T04:07:29.358 INFO:teuthology.orchestra.run.smithi115.stdout:(13/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 15 MB/s | 17 MB 00:01 2024-09-19T04:07:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 171 kB/s | 32 kB 00:00 2024-09-19T04:07:29.419 INFO:teuthology.orchestra.run.smithi115.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 751 kB 00:00 2024-09-19T04:07:29.444 INFO:teuthology.orchestra.run.smithi115.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T04:07:29.478 INFO:teuthology.orchestra.run.smithi115.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T04:07:29.511 INFO:teuthology.orchestra.run.smithi115.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T04:07:29.570 INFO:teuthology.orchestra.run.smithi115.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 773 kB/s | 45 kB 00:00 2024-09-19T04:07:29.679 INFO:teuthology.orchestra.run.smithi115.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.2 MB/s | 136 kB 00:00 2024-09-19T04:07:29.762 INFO:teuthology.orchestra.run.smithi115.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-19T04:07:29.788 INFO:teuthology.orchestra.run.smithi115.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.5 MB/s | 161 kB 00:00 2024-09-19T04:07:29.813 INFO:teuthology.orchestra.run.smithi115.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 6.2 MB/s | 320 kB 00:00 2024-09-19T04:07:29.838 INFO:teuthology.orchestra.run.smithi115.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.9 MB/s | 300 kB 00:00 2024-09-19T04:07:29.863 INFO:teuthology.orchestra.run.smithi115.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T04:07:29.889 INFO:teuthology.orchestra.run.smithi115.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T04:07:29.956 INFO:teuthology.orchestra.run.smithi115.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-19T04:07:30.006 INFO:teuthology.orchestra.run.smithi115.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 22 MB/s | 3.1 MB 00:00 2024-09-19T04:07:30.031 INFO:teuthology.orchestra.run.smithi115.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 425 kB/s | 32 kB 00:00 2024-09-19T04:07:30.056 INFO:teuthology.orchestra.run.smithi115.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T04:07:30.313 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-19T04:07:30.440 INFO:teuthology.orchestra.run.smithi115.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 19 MB/s | 7.4 MB 00:00 2024-09-19T04:07:30.499 INFO:teuthology.orchestra.run.smithi115.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.2 MB/s | 249 kB 00:00 2024-09-19T04:07:30.557 INFO:teuthology.orchestra.run.smithi115.stdout:(33/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 6.8 MB/s | 3.6 MB 00:00 2024-09-19T04:07:30.583 INFO:teuthology.orchestra.run.smithi115.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 590 kB/s | 49 kB 00:00 2024-09-19T04:07:30.608 INFO:teuthology.orchestra.run.smithi115.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 327 kB/s | 16 kB 00:00 2024-09-19T04:07:30.633 INFO:teuthology.orchestra.run.smithi115.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.5 MB/s | 285 kB 00:00 2024-09-19T04:07:30.667 INFO:teuthology.orchestra.run.smithi115.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 13 MB/s | 765 kB 00:00 2024-09-19T04:07:30.684 INFO:teuthology.orchestra.run.smithi115.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 363 kB/s | 18 kB 00:00 2024-09-19T04:07:31.193 INFO:teuthology.orchestra.run.smithi115.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 20 MB/s | 49 MB 00:02 2024-09-19T04:07:31.559 INFO:teuthology.orchestra.run.smithi115.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 89 kB/s | 80 kB 00:00 2024-09-19T04:07:31.743 INFO:teuthology.orchestra.run.smithi115.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 347 kB/s | 191 kB 00:00 2024-09-19T04:07:31.851 INFO:teuthology.orchestra.run.smithi115.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 868 kB/s | 253 kB 00:00 2024-09-19T04:07:31.919 INFO:teuthology.orchestra.run.smithi115.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 649 kB/s | 801 kB 00:01 2024-09-19T04:07:31.969 INFO:teuthology.orchestra.run.smithi115.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 909 kB/s | 106 kB 00:00 2024-09-19T04:07:32.028 INFO:teuthology.orchestra.run.smithi115.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-19T04:07:32.087 INFO:teuthology.orchestra.run.smithi115.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-19T04:07:32.145 INFO:teuthology.orchestra.run.smithi115.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-19T04:07:32.220 INFO:teuthology.orchestra.run.smithi115.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2024-09-19T04:07:32.254 INFO:teuthology.orchestra.run.smithi115.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-19T04:07:32.296 INFO:teuthology.orchestra.run.smithi115.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 523 kB/s | 22 kB 00:00 2024-09-19T04:07:32.329 INFO:teuthology.orchestra.run.smithi115.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-09-19T04:07:32.354 INFO:teuthology.orchestra.run.smithi115.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 797 kB/s | 107 kB 00:00 2024-09-19T04:07:32.380 INFO:teuthology.orchestra.run.smithi115.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-19T04:07:32.405 INFO:teuthology.orchestra.run.smithi115.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-19T04:07:32.430 INFO:teuthology.orchestra.run.smithi115.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-19T04:07:32.456 INFO:teuthology.orchestra.run.smithi115.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T04:07:32.465 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:07:32.466 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:07:32.479 INFO:teuthology.orchestra.run.smithi145.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:07:32.481 INFO:teuthology.orchestra.run.smithi145.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:07:32.482 INFO:teuthology.orchestra.run.smithi145.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:07:32.482 INFO:teuthology.orchestra.run.smithi145.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:07:32.485 INFO:teuthology.orchestra.run.smithi145.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:07:32.487 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:07:32.488 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:07:32.489 INFO:teuthology.orchestra.run.smithi115.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T04:07:32.515 INFO:teuthology.orchestra.run.smithi115.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-19T04:07:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-19T04:07:32.557 INFO:teuthology.orchestra.run.smithi115.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-19T04:07:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================== 2024-09-19T04:07:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-19T04:07:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================== 2024-09-19T04:07:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-19T04:07:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:07:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:07:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:07:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:07:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:07:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:07:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:07:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:07:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:07:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:07:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:07:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:07:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:07:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:07:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:07:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:07:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:07:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:07:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:07:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:07:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:07:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:07:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:07:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:07:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:07:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:07:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:07:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:07:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:07:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:07:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:07:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:07:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:07:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:07:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:07:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:07:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:07:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2024-09-19T04:07:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:07:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:07:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:07:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:07:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:07:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:07:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:07:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:07:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-19T04:07:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:07:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:07:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:07:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:07:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:07:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:07:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:07:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:07:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:07:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:07:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:07:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:07:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:07:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:07:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:07:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:07:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:07:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:07:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:07:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:07:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:07:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:07:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:07:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:07:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:07:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:07:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:07:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:07:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:07:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:07:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:07:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:07:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:07:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:07:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:07:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:07:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:07:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:07:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:07:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:07:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:07:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:07:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:07:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:07:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:07:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:07:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:07:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:07:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:07:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:07:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:07:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:07:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:07:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:07:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:07:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:07:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:07:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:07:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:07:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:07:32.588 INFO:teuthology.orchestra.run.smithi145.stdout:Installing weak dependencies: 2024-09-19T04:07:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:07:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:07:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-19T04:07:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================== 2024-09-19T04:07:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:Install 148 Packages 2024-09-19T04:07:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 8 Packages 2024-09-19T04:07:32.589 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:07:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 226 M 2024-09-19T04:07:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-19T04:07:32.590 INFO:teuthology.orchestra.run.smithi115.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.6 MB/s | 581 kB 00:00 2024-09-19T04:07:32.615 INFO:teuthology.orchestra.run.smithi115.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-19T04:07:32.641 INFO:teuthology.orchestra.run.smithi115.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-19T04:07:32.674 INFO:teuthology.orchestra.run.smithi115.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-19T04:07:32.699 INFO:teuthology.orchestra.run.smithi115.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-19T04:07:32.725 INFO:teuthology.orchestra.run.smithi115.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-19T04:07:32.775 INFO:teuthology.orchestra.run.smithi115.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.1 MB/s | 3.0 MB 00:00 2024-09-19T04:07:32.884 INFO:teuthology.orchestra.run.smithi115.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 27 MB/s | 4.9 MB 00:00 2024-09-19T04:07:32.926 INFO:teuthology.orchestra.run.smithi115.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-19T04:07:32.960 INFO:teuthology.orchestra.run.smithi115.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-19T04:07:32.984 INFO:teuthology.orchestra.run.smithi115.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T04:07:33.043 INFO:teuthology.orchestra.run.smithi115.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-19T04:07:33.085 INFO:teuthology.orchestra.run.smithi115.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-19T04:07:33.119 INFO:teuthology.orchestra.run.smithi115.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-19T04:07:33.153 INFO:teuthology.orchestra.run.smithi115.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-19T04:07:33.164 INFO:teuthology.orchestra.run.smithi145.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 54 kB/s | 6.6 kB 00:00 2024-09-19T04:07:33.228 INFO:teuthology.orchestra.run.smithi115.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-19T04:07:33.253 INFO:teuthology.orchestra.run.smithi115.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-19T04:07:33.278 INFO:teuthology.orchestra.run.smithi115.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-19T04:07:33.296 INFO:teuthology.orchestra.run.smithi115.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 612 kB/s | 26 kB 00:00 2024-09-19T04:07:33.313 INFO:teuthology.orchestra.run.smithi115.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 574 kB/s | 19 kB 00:00 2024-09-19T04:07:33.314 INFO:teuthology.orchestra.run.smithi145.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 5.8 MB/s | 896 kB 00:00 2024-09-19T04:07:33.338 INFO:teuthology.orchestra.run.smithi115.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-19T04:07:33.363 INFO:teuthology.orchestra.run.smithi115.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-19T04:07:33.406 INFO:teuthology.orchestra.run.smithi145.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.5 MB/s | 145 kB 00:00 2024-09-19T04:07:33.447 INFO:teuthology.orchestra.run.smithi115.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-19T04:07:33.472 INFO:teuthology.orchestra.run.smithi115.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-19T04:07:33.497 INFO:teuthology.orchestra.run.smithi115.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-19T04:07:33.522 INFO:teuthology.orchestra.run.smithi115.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T04:07:33.548 INFO:teuthology.orchestra.run.smithi115.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-19T04:07:33.557 INFO:teuthology.orchestra.run.smithi145.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T04:07:33.573 INFO:teuthology.orchestra.run.smithi115.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-19T04:07:33.598 INFO:teuthology.orchestra.run.smithi115.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-19T04:07:33.632 INFO:teuthology.orchestra.run.smithi145.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-19T04:07:33.632 INFO:teuthology.orchestra.run.smithi115.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-19T04:07:33.657 INFO:teuthology.orchestra.run.smithi115.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-09-19T04:07:33.682 INFO:teuthology.orchestra.run.smithi145.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 12 MB/s | 1.5 MB 00:00 2024-09-19T04:07:33.682 INFO:teuthology.orchestra.run.smithi115.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T04:07:33.758 INFO:teuthology.orchestra.run.smithi115.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T04:07:33.774 INFO:teuthology.orchestra.run.smithi115.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-09-19T04:07:33.791 INFO:teuthology.orchestra.run.smithi115.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-19T04:07:33.817 INFO:teuthology.orchestra.run.smithi115.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-09-19T04:07:33.834 INFO:teuthology.orchestra.run.smithi115.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 855 kB/s | 36 kB 00:00 2024-09-19T04:07:33.851 INFO:teuthology.orchestra.run.smithi115.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-19T04:07:33.884 INFO:teuthology.orchestra.run.smithi115.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T04:07:33.901 INFO:teuthology.orchestra.run.smithi115.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T04:07:33.926 INFO:teuthology.orchestra.run.smithi115.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-19T04:07:33.952 INFO:teuthology.orchestra.run.smithi115.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T04:07:33.986 INFO:teuthology.orchestra.run.smithi115.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 763 kB/s | 45 kB 00:00 2024-09-19T04:07:34.033 INFO:teuthology.orchestra.run.smithi145.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-19T04:07:34.094 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-19T04:07:34.241 INFO:teuthology.orchestra.run.smithi145.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 18 MB/s | 21 MB 00:01 2024-09-19T04:07:34.261 INFO:teuthology.orchestra.run.smithi115.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-19T04:07:34.286 INFO:teuthology.orchestra.run.smithi115.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-09-19T04:07:34.299 INFO:teuthology.orchestra.run.smithi145.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 434 kB/s | 25 kB 00:00 2024-09-19T04:07:34.353 INFO:teuthology.orchestra.run.smithi115.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-19T04:07:34.379 INFO:teuthology.orchestra.run.smithi115.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 772 kB/s | 19 kB 00:00 2024-09-19T04:07:34.429 INFO:teuthology.orchestra.run.smithi115.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.1 MB/s | 308 kB 00:00 2024-09-19T04:07:34.454 INFO:teuthology.orchestra.run.smithi115.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T04:07:34.479 INFO:teuthology.orchestra.run.smithi115.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T04:07:34.505 INFO:teuthology.orchestra.run.smithi115.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-19T04:07:34.538 INFO:teuthology.orchestra.run.smithi115.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-19T04:07:34.622 INFO:teuthology.orchestra.run.smithi115.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 839 kB 00:00 2024-09-19T04:07:34.664 INFO:teuthology.orchestra.run.smithi115.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-09-19T04:07:34.700 INFO:teuthology.orchestra.run.smithi145.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 16 MB/s | 11 MB 00:00 2024-09-19T04:07:34.714 INFO:teuthology.orchestra.run.smithi115.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-19T04:07:34.739 INFO:teuthology.orchestra.run.smithi115.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 578 kB/s | 43 kB 00:00 2024-09-19T04:07:34.765 INFO:teuthology.orchestra.run.smithi115.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-19T04:07:34.790 INFO:teuthology.orchestra.run.smithi115.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-19T04:07:34.857 INFO:teuthology.orchestra.run.smithi115.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.2 MB/s | 4.4 MB 00:00 2024-09-19T04:07:34.875 INFO:teuthology.orchestra.run.smithi145.stdout:(11/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 14 MB/s | 17 MB 00:01 2024-09-19T04:07:34.882 INFO:teuthology.orchestra.run.smithi115.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-09-19T04:07:34.900 INFO:teuthology.orchestra.run.smithi145.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.1 MB/s | 228 kB 00:00 2024-09-19T04:07:34.917 INFO:teuthology.orchestra.run.smithi145.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 649 kB/s | 27 kB 00:00 2024-09-19T04:07:34.934 INFO:teuthology.orchestra.run.smithi145.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 950 kB/s | 32 kB 00:00 2024-09-19T04:07:34.976 INFO:teuthology.orchestra.run.smithi145.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 3.8 MB/s | 163 kB 00:00 2024-09-19T04:07:34.983 INFO:teuthology.orchestra.run.smithi115.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-09-19T04:07:35.008 INFO:teuthology.orchestra.run.smithi115.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 1.7 MB/s | 220 kB 00:00 2024-09-19T04:07:35.018 INFO:teuthology.orchestra.run.smithi145.stdout:(16/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 7.3 MB/s | 751 kB 00:00 2024-09-19T04:07:35.033 INFO:teuthology.orchestra.run.smithi115.stdout:(122/156): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-09-19T04:07:35.043 INFO:teuthology.orchestra.run.smithi145.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 1.8 MB/s | 127 kB 00:00 2024-09-19T04:07:35.058 INFO:teuthology.orchestra.run.smithi115.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-19T04:07:35.083 INFO:teuthology.orchestra.run.smithi115.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-19T04:07:35.085 INFO:teuthology.orchestra.run.smithi145.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T04:07:35.109 INFO:teuthology.orchestra.run.smithi115.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-19T04:07:35.134 INFO:teuthology.orchestra.run.smithi115.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-19T04:07:35.152 INFO:teuthology.orchestra.run.smithi145.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 680 kB/s | 45 kB 00:00 2024-09-19T04:07:35.159 INFO:teuthology.orchestra.run.smithi115.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-19T04:07:35.185 INFO:teuthology.orchestra.run.smithi115.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T04:07:35.219 INFO:teuthology.orchestra.run.smithi115.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-19T04:07:35.219 INFO:teuthology.orchestra.run.smithi145.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T04:07:35.252 INFO:teuthology.orchestra.run.smithi115.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-19T04:07:35.286 INFO:teuthology.orchestra.run.smithi145.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.4 MB/s | 161 kB 00:00 2024-09-19T04:07:35.303 INFO:teuthology.orchestra.run.smithi115.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-09-19T04:07:35.328 INFO:teuthology.orchestra.run.smithi115.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-09-19T04:07:35.362 INFO:teuthology.orchestra.run.smithi115.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-09-19T04:07:35.370 INFO:teuthology.orchestra.run.smithi145.stdout:(22/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-19T04:07:35.387 INFO:teuthology.orchestra.run.smithi115.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-19T04:07:35.395 INFO:teuthology.orchestra.run.smithi145.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 2.9 MB/s | 320 kB 00:00 2024-09-19T04:07:35.420 INFO:teuthology.orchestra.run.smithi115.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-19T04:07:35.429 INFO:teuthology.orchestra.run.smithi145.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.0 MB/s | 300 kB 00:00 2024-09-19T04:07:35.446 INFO:teuthology.orchestra.run.smithi115.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-09-19T04:07:35.454 INFO:teuthology.orchestra.run.smithi145.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.7 MB/s | 99 kB 00:00 2024-09-19T04:07:35.479 INFO:teuthology.orchestra.run.smithi115.stdout:(137/156): python3-kubernetes-26.1.0-2.el9.noar 3.2 MB/s | 1.0 MB 00:00 2024-09-19T04:07:35.480 INFO:teuthology.orchestra.run.smithi145.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T04:07:35.505 INFO:teuthology.orchestra.run.smithi115.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 366 kB/s | 31 kB 00:00 2024-09-19T04:07:35.530 INFO:teuthology.orchestra.run.smithi145.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-19T04:07:35.530 INFO:teuthology.orchestra.run.smithi115.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-19T04:07:35.555 INFO:teuthology.orchestra.run.smithi115.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-19T04:07:35.581 INFO:teuthology.orchestra.run.smithi115.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-19T04:07:35.589 INFO:teuthology.orchestra.run.smithi145.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 547 kB/s | 32 kB 00:00 2024-09-19T04:07:35.606 INFO:teuthology.orchestra.run.smithi115.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 985 kB/s | 74 kB 00:00 2024-09-19T04:07:35.631 INFO:teuthology.orchestra.run.smithi115.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-19T04:07:35.647 INFO:teuthology.orchestra.run.smithi145.stdout:(29/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 16 MB/s | 3.1 MB 00:00 2024-09-19T04:07:35.665 INFO:teuthology.orchestra.run.smithi115.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-19T04:07:35.672 INFO:teuthology.orchestra.run.smithi145.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.7 MB/s | 146 kB 00:00 2024-09-19T04:07:35.690 INFO:teuthology.orchestra.run.smithi115.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-19T04:07:35.716 INFO:teuthology.orchestra.run.smithi115.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:07:35.749 INFO:teuthology.orchestra.run.smithi115.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-19T04:07:35.841 INFO:teuthology.orchestra.run.smithi115.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-19T04:07:35.914 INFO:teuthology.orchestra.run.smithi145.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 13 MB/s | 3.6 MB 00:00 2024-09-19T04:07:35.990 INFO:teuthology.orchestra.run.smithi145.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.2 MB/s | 249 kB 00:00 2024-09-19T04:07:36.090 INFO:teuthology.orchestra.run.smithi145.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 495 kB/s | 49 kB 00:00 2024-09-19T04:07:36.190 INFO:teuthology.orchestra.run.smithi145.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 164 kB/s | 16 kB 00:00 2024-09-19T04:07:36.200 INFO:teuthology.orchestra.run.smithi115.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 7.1 MB/s | 3.4 MB 00:00 2024-09-19T04:07:36.250 INFO:teuthology.orchestra.run.smithi115.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 6.3 MB/s | 3.2 MB 00:00 2024-09-19T04:07:36.274 INFO:teuthology.orchestra.run.smithi145.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-19T04:07:36.299 INFO:teuthology.orchestra.run.smithi145.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 2.6 MB/s | 285 kB 00:00 2024-09-19T04:07:36.349 INFO:teuthology.orchestra.run.smithi145.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 10 MB/s | 765 kB 00:00 2024-09-19T04:07:36.374 INFO:teuthology.orchestra.run.smithi145.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 243 kB/s | 18 kB 00:00 2024-09-19T04:07:36.434 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:07:36.435 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:07:36.448 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:07:36.449 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:07:36.450 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:07:36.452 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:07:36.453 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:07:36.456 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:07:36.457 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:07:36.522 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-19T04:07:36.533 INFO:teuthology.orchestra.run.smithi115.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 144 kB/s | 41 kB 00:00 2024-09-19T04:07:36.546 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-19T04:07:36.546 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-19T04:07:36.546 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-19T04:07:36.546 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-19T04:07:36.546 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:07:36.546 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:07:36.546 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:07:36.547 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:07:36.548 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:07:36.549 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:07:36.550 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:07:36.551 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:07:36.552 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:07:36.553 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:07:36.554 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:07:36.555 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:07:36.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:07:36.557 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:07:36.558 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:07:36.559 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:07:36.559 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:07:36.559 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-09-19T04:07:36.559 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:07:36.559 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:07:36.559 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-19T04:07:36.559 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-19T04:07:36.559 INFO:teuthology.orchestra.run.smithi107.stdout:Install 148 Packages 2024-09-19T04:07:36.559 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 8 Packages 2024-09-19T04:07:36.559 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:07:36.560 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 226 M 2024-09-19T04:07:36.560 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-19T04:07:36.759 INFO:teuthology.orchestra.run.smithi115.stdout:(152/156): NetworkManager-1.51.0-1.el9.x86_64.r 2.5 MB/s | 2.3 MB 00:00 2024-09-19T04:07:36.809 INFO:teuthology.orchestra.run.smithi145.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 19 MB/s | 49 MB 00:02 2024-09-19T04:07:36.842 INFO:teuthology.orchestra.run.smithi115.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 807 kB/s | 249 kB 00:00 2024-09-19T04:07:36.892 INFO:teuthology.orchestra.run.smithi145.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 154 kB/s | 80 kB 00:00 2024-09-19T04:07:36.892 INFO:teuthology.orchestra.run.smithi115.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-19T04:07:36.934 INFO:teuthology.orchestra.run.smithi145.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-19T04:07:36.984 INFO:teuthology.orchestra.run.smithi145.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-19T04:07:37.009 INFO:teuthology.orchestra.run.smithi145.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-19T04:07:37.035 INFO:teuthology.orchestra.run.smithi145.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T04:07:37.076 INFO:teuthology.orchestra.run.smithi115.stdout:(155/156): NetworkManager-libnm-1.51.0-1.el9.x8 2.1 MB/s | 1.8 MB 00:00 2024-09-19T04:07:37.077 INFO:teuthology.orchestra.run.smithi145.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-19T04:07:37.102 INFO:teuthology.orchestra.run.smithi145.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-19T04:07:37.136 INFO:teuthology.orchestra.run.smithi145.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-19T04:07:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-19T04:07:37.219 INFO:teuthology.orchestra.run.smithi145.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-09-19T04:07:37.361 INFO:teuthology.orchestra.run.smithi145.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 755 kB/s | 107 kB 00:00 2024-09-19T04:07:37.379 INFO:teuthology.orchestra.run.smithi107.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 5.3 MB/s | 896 kB 00:00 2024-09-19T04:07:37.409 INFO:teuthology.orchestra.run.smithi115.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.2 MB/s | 2.4 MB 00:00 2024-09-19T04:07:37.412 INFO:teuthology.orchestra.run.smithi145.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 435 kB/s | 22 kB 00:00 2024-09-19T04:07:37.416 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:07:37.416 INFO:teuthology.orchestra.run.smithi115.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-19T04:07:37.445 INFO:teuthology.orchestra.run.smithi145.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-09-19T04:07:37.462 INFO:teuthology.orchestra.run.smithi107.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.7 MB/s | 145 kB 00:00 2024-09-19T04:07:37.496 INFO:teuthology.orchestra.run.smithi145.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-19T04:07:37.529 INFO:teuthology.orchestra.run.smithi145.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-19T04:07:37.554 INFO:teuthology.orchestra.run.smithi145.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-09-19T04:07:37.580 INFO:teuthology.orchestra.run.smithi145.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 361 kB/s | 18 kB 00:00 2024-09-19T04:07:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-19T04:07:37.605 INFO:teuthology.orchestra.run.smithi145.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T04:07:37.655 INFO:teuthology.orchestra.run.smithi145.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-09-19T04:07:37.681 INFO:teuthology.orchestra.run.smithi145.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-19T04:07:37.688 INFO:teuthology.orchestra.run.smithi107.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-19T04:07:37.738 INFO:teuthology.orchestra.run.smithi107.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 8.7 MB/s | 2.4 MB 00:00 2024-09-19T04:07:37.847 INFO:teuthology.orchestra.run.smithi145.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 3.4 MB/s | 581 kB 00:00 2024-09-19T04:07:37.881 INFO:teuthology.orchestra.run.smithi145.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-09-19T04:07:37.906 INFO:teuthology.orchestra.run.smithi145.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-19T04:07:37.931 INFO:teuthology.orchestra.run.smithi145.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-19T04:07:37.972 INFO:teuthology.orchestra.run.smithi107.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-19T04:07:38.015 INFO:teuthology.orchestra.run.smithi145.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-19T04:07:38.049 INFO:teuthology.orchestra.run.smithi145.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-19T04:07:38.149 INFO:teuthology.orchestra.run.smithi145.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 288 kB/s | 38 kB 00:00 2024-09-19T04:07:38.272 INFO:teuthology.orchestra.run.smithi107.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 18 MB/s | 21 MB 00:01 2024-09-19T04:07:38.316 INFO:teuthology.orchestra.run.smithi145.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.6 MB/s | 3.0 MB 00:00 2024-09-19T04:07:38.347 INFO:teuthology.orchestra.run.smithi107.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 336 kB/s | 25 kB 00:00 2024-09-19T04:07:38.656 INFO:teuthology.orchestra.run.smithi107.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 16 MB/s | 11 MB 00:00 2024-09-19T04:07:38.657 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-19T04:07:38.658 INFO:teuthology.orchestra.run.smithi145.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-09-19T04:07:38.810 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-19T04:07:38.810 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-19T04:07:38.848 INFO:teuthology.orchestra.run.smithi107.stdout:(11/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 15 MB/s | 17 MB 00:01 2024-09-19T04:07:38.866 INFO:teuthology.orchestra.run.smithi145.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.3 MB/s | 910 kB 00:00 2024-09-19T04:07:38.925 INFO:teuthology.orchestra.run.smithi145.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T04:07:39.023 INFO:teuthology.orchestra.run.smithi107.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 622 kB/s | 228 kB 00:00 2024-09-19T04:07:39.073 INFO:teuthology.orchestra.run.smithi107.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 120 kB/s | 27 kB 00:00 2024-09-19T04:07:39.124 INFO:teuthology.orchestra.run.smithi107.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 317 kB/s | 32 kB 00:00 2024-09-19T04:07:39.134 INFO:teuthology.orchestra.run.smithi145.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.6 MB/s | 4.9 MB 00:01 2024-09-19T04:07:39.165 INFO:teuthology.orchestra.run.smithi107.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 8.0 MB/s | 751 kB 00:00 2024-09-19T04:07:39.191 INFO:teuthology.orchestra.run.smithi107.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T04:07:39.216 INFO:teuthology.orchestra.run.smithi107.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T04:07:39.249 INFO:teuthology.orchestra.run.smithi107.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 8.5 MB/s | 506 kB 00:00 2024-09-19T04:07:39.334 INFO:teuthology.orchestra.run.smithi145.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-09-19T04:07:39.334 INFO:teuthology.orchestra.run.smithi107.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 539 kB/s | 45 kB 00:00 2024-09-19T04:07:39.392 INFO:teuthology.orchestra.run.smithi145.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T04:07:39.459 INFO:teuthology.orchestra.run.smithi107.stdout:(20/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 22 MB/s | 5.3 MB 00:00 2024-09-19T04:07:39.484 INFO:teuthology.orchestra.run.smithi107.stdout:(21/156): python3-ceph-common-19.1.1-361.g2f599 910 kB/s | 136 kB 00:00 2024-09-19T04:07:39.485 INFO:teuthology.orchestra.run.smithi145.stdout:(73/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-09-19T04:07:39.517 INFO:teuthology.orchestra.run.smithi107.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.7 MB/s | 161 kB 00:00 2024-09-19T04:07:39.535 INFO:teuthology.orchestra.run.smithi145.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-19T04:07:39.542 INFO:teuthology.orchestra.run.smithi107.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 5.3 MB/s | 320 kB 00:00 2024-09-19T04:07:39.568 INFO:teuthology.orchestra.run.smithi107.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.8 MB/s | 300 kB 00:00 2024-09-19T04:07:39.593 INFO:teuthology.orchestra.run.smithi107.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T04:07:39.618 INFO:teuthology.orchestra.run.smithi145.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-19T04:07:39.618 INFO:teuthology.orchestra.run.smithi107.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T04:07:39.669 INFO:teuthology.orchestra.run.smithi107.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-19T04:07:39.711 INFO:teuthology.orchestra.run.smithi145.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-19T04:07:39.744 INFO:teuthology.orchestra.run.smithi107.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-19T04:07:39.761 INFO:teuthology.orchestra.run.smithi145.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-19T04:07:39.786 INFO:teuthology.orchestra.run.smithi107.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 273 kB/s | 32 kB 00:00 2024-09-19T04:07:39.802 INFO:teuthology.orchestra.run.smithi145.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 461 kB/s | 19 kB 00:00 2024-09-19T04:07:39.811 INFO:teuthology.orchestra.run.smithi107.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.1 MB/s | 146 kB 00:00 2024-09-19T04:07:39.894 INFO:teuthology.orchestra.run.smithi145.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-19T04:07:39.936 INFO:teuthology.orchestra.run.smithi145.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 835 kB/s | 35 kB 00:00 2024-09-19T04:07:40.020 INFO:teuthology.orchestra.run.smithi145.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 5.3 MB/s | 5.8 MB 00:01 2024-09-19T04:07:40.028 INFO:teuthology.orchestra.run.smithi107.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 15 MB/s | 3.6 MB 00:00 2024-09-19T04:07:40.103 INFO:teuthology.orchestra.run.smithi107.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.3 MB/s | 249 kB 00:00 2024-09-19T04:07:40.104 INFO:teuthology.orchestra.run.smithi145.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-19T04:07:40.171 INFO:teuthology.orchestra.run.smithi145.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-09-19T04:07:40.221 INFO:teuthology.orchestra.run.smithi107.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 18 MB/s | 7.4 MB 00:00 2024-09-19T04:07:40.229 INFO:teuthology.orchestra.run.smithi145.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T04:07:40.246 INFO:teuthology.orchestra.run.smithi107.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 347 kB/s | 49 kB 00:00 2024-09-19T04:07:40.271 INFO:teuthology.orchestra.run.smithi107.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 329 kB/s | 16 kB 00:00 2024-09-19T04:07:40.288 INFO:teuthology.orchestra.run.smithi145.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-09-19T04:07:40.296 INFO:teuthology.orchestra.run.smithi107.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T04:07:40.338 INFO:teuthology.orchestra.run.smithi107.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 11 MB/s | 765 kB 00:00 2024-09-19T04:07:40.363 INFO:teuthology.orchestra.run.smithi107.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 273 kB/s | 18 kB 00:00 2024-09-19T04:07:40.371 INFO:teuthology.orchestra.run.smithi145.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-09-19T04:07:40.472 INFO:teuthology.orchestra.run.smithi145.stdout:(87/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2024-09-19T04:07:40.489 INFO:teuthology.orchestra.run.smithi107.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-19T04:07:40.505 INFO:teuthology.orchestra.run.smithi145.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-19T04:07:40.564 INFO:teuthology.orchestra.run.smithi145.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-09-19T04:07:40.572 INFO:teuthology.orchestra.run.smithi107.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 191 kB 00:00 2024-09-19T04:07:40.589 INFO:teuthology.orchestra.run.smithi145.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-09-19T04:07:40.615 INFO:teuthology.orchestra.run.smithi145.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T04:07:40.615 INFO:teuthology.orchestra.run.smithi107.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.1 MB/s | 801 kB 00:00 2024-09-19T04:07:40.639 INFO:teuthology.orchestra.run.smithi107.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-19T04:07:40.665 INFO:teuthology.orchestra.run.smithi145.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-09-19T04:07:40.671 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-19T04:07:40.672 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-19T04:07:40.715 INFO:teuthology.orchestra.run.smithi145.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 888 kB/s | 44 kB 00:00 2024-09-19T04:07:40.782 INFO:teuthology.orchestra.run.smithi145.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-19T04:07:40.832 INFO:teuthology.orchestra.run.smithi145.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-09-19T04:07:40.891 INFO:teuthology.orchestra.run.smithi145.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-09-19T04:07:40.974 INFO:teuthology.orchestra.run.smithi145.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-19T04:07:41.050 INFO:teuthology.orchestra.run.smithi145.stdout:(98/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-09-19T04:07:41.075 INFO:teuthology.orchestra.run.smithi145.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 576 kB/s | 58 kB 00:00 2024-09-19T04:07:41.098 INFO:teuthology.orchestra.run.smithi107.stdout:(43/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 18 MB/s | 49 MB 00:02 2024-09-19T04:07:41.100 INFO:teuthology.orchestra.run.smithi145.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T04:07:41.123 INFO:teuthology.orchestra.run.smithi107.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 220 kB/s | 106 kB 00:00 2024-09-19T04:07:41.125 INFO:teuthology.orchestra.run.smithi145.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T04:07:41.151 INFO:teuthology.orchestra.run.smithi145.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-19T04:07:41.165 INFO:teuthology.orchestra.run.smithi107.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-09-19T04:07:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-19T04:07:41.216 INFO:teuthology.orchestra.run.smithi107.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-19T04:07:41.266 INFO:teuthology.orchestra.run.smithi107.stdout:(48/156): python3-pycparser-2.20-6.el9.noarch.r 808 kB/s | 135 kB 00:00 2024-09-19T04:07:41.358 INFO:teuthology.orchestra.run.smithi107.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-19T04:07:41.409 INFO:teuthology.orchestra.run.smithi107.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-09-19T04:07:41.409 INFO:teuthology.orchestra.run.smithi145.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-09-19T04:07:41.434 INFO:teuthology.orchestra.run.smithi107.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 289 kB/s | 22 kB 00:00 2024-09-19T04:07:41.459 INFO:teuthology.orchestra.run.smithi107.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 555 kB/s | 107 kB 00:00 2024-09-19T04:07:41.468 INFO:teuthology.orchestra.run.smithi145.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T04:07:41.484 INFO:teuthology.orchestra.run.smithi107.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 658 kB/s | 33 kB 00:00 2024-09-19T04:07:41.502 INFO:teuthology.orchestra.run.smithi145.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T04:07:41.509 INFO:teuthology.orchestra.run.smithi107.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-19T04:07:41.527 INFO:teuthology.orchestra.run.smithi145.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 771 kB/s | 19 kB 00:00 2024-09-19T04:07:41.534 INFO:teuthology.orchestra.run.smithi107.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-19T04:07:41.560 INFO:teuthology.orchestra.run.smithi107.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T04:07:41.585 INFO:teuthology.orchestra.run.smithi107.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T04:07:41.610 INFO:teuthology.orchestra.run.smithi107.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-09-19T04:07:41.694 INFO:teuthology.orchestra.run.smithi107.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-19T04:07:41.719 INFO:teuthology.orchestra.run.smithi107.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-19T04:07:41.744 INFO:teuthology.orchestra.run.smithi107.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 489 kB/s | 24 kB 00:00 2024-09-19T04:07:41.769 INFO:teuthology.orchestra.run.smithi107.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-19T04:07:41.794 INFO:teuthology.orchestra.run.smithi145.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-19T04:07:41.795 INFO:teuthology.orchestra.run.smithi107.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-19T04:07:41.819 INFO:teuthology.orchestra.run.smithi145.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T04:07:41.828 INFO:teuthology.orchestra.run.smithi107.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-19T04:07:41.844 INFO:teuthology.orchestra.run.smithi145.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T04:07:41.854 INFO:teuthology.orchestra.run.smithi107.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-19T04:07:41.870 INFO:teuthology.orchestra.run.smithi145.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T04:07:41.903 INFO:teuthology.orchestra.run.smithi145.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-19T04:07:41.921 INFO:teuthology.orchestra.run.smithi107.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.5 MB/s | 3.0 MB 00:00 2024-09-19T04:07:41.979 INFO:teuthology.orchestra.run.smithi145.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-19T04:07:41.996 INFO:teuthology.orchestra.run.smithi107.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-19T04:07:42.004 INFO:teuthology.orchestra.run.smithi145.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-09-19T04:07:42.046 INFO:teuthology.orchestra.run.smithi145.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-19T04:07:42.071 INFO:teuthology.orchestra.run.smithi145.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-09-19T04:07:42.079 INFO:teuthology.orchestra.run.smithi107.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-19T04:07:42.155 INFO:teuthology.orchestra.run.smithi107.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-19T04:07:42.155 INFO:teuthology.orchestra.run.smithi145.stdout:(116/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 288 kB/s | 43 kB 00:00 2024-09-19T04:07:42.180 INFO:teuthology.orchestra.run.smithi107.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-19T04:07:42.180 INFO:teuthology.orchestra.run.smithi145.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 130 kB/s | 14 kB 00:00 2024-09-19T04:07:42.214 INFO:teuthology.orchestra.run.smithi145.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-19T04:07:42.222 INFO:teuthology.orchestra.run.smithi107.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 3.9 MB/s | 910 kB 00:00 2024-09-19T04:07:42.247 INFO:teuthology.orchestra.run.smithi145.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-19T04:07:42.264 INFO:teuthology.orchestra.run.smithi107.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-19T04:07:42.281 INFO:teuthology.orchestra.run.smithi145.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-19T04:07:42.289 INFO:teuthology.orchestra.run.smithi107.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-19T04:07:42.315 INFO:teuthology.orchestra.run.smithi145.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-19T04:07:42.331 INFO:teuthology.orchestra.run.smithi107.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T04:07:42.348 INFO:teuthology.orchestra.run.smithi145.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T04:07:42.365 INFO:teuthology.orchestra.run.smithi145.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 645 kB/s | 11 kB 00:00 2024-09-19T04:07:42.373 INFO:teuthology.orchestra.run.smithi107.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 5.8 MB/s | 245 kB 00:00 2024-09-19T04:07:42.382 INFO:teuthology.orchestra.run.smithi145.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-19T04:07:42.400 INFO:teuthology.orchestra.run.smithi145.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-19T04:07:42.416 INFO:teuthology.orchestra.run.smithi145.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.2 MB/s | 19 kB 00:00 2024-09-19T04:07:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-19T04:07:42.433 INFO:teuthology.orchestra.run.smithi145.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-19T04:07:42.492 INFO:teuthology.orchestra.run.smithi145.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.7 MB/s | 2.0 MB 00:00 2024-09-19T04:07:42.517 INFO:teuthology.orchestra.run.smithi145.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T04:07:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-09-19T04:07:42.549 INFO:teuthology.orchestra.run.smithi107.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-09-19T04:07:42.551 INFO:teuthology.orchestra.run.smithi145.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-19T04:07:42.576 INFO:teuthology.orchestra.run.smithi107.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 385 kB/s | 19 kB 00:00 2024-09-19T04:07:42.576 INFO:teuthology.orchestra.run.smithi145.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-19T04:07:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-19T04:07:42.632 INFO:teuthology.orchestra.run.smithi107.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 597 kB/s | 35 kB 00:00 2024-09-19T04:07:42.718 INFO:teuthology.orchestra.run.smithi145.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 346 kB/s | 58 kB 00:00 2024-09-19T04:07:42.727 INFO:teuthology.orchestra.run.smithi107.stdout:(82/156): postgresql-server-13.16-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-09-19T04:07:42.743 INFO:teuthology.orchestra.run.smithi145.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-09-19T04:07:42.752 INFO:teuthology.orchestra.run.smithi107.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-19T04:07:42.769 INFO:teuthology.orchestra.run.smithi145.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-09-19T04:07:42.778 INFO:teuthology.orchestra.run.smithi107.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-19T04:07:42.794 INFO:teuthology.orchestra.run.smithi145.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-19T04:07:42.802 INFO:teuthology.orchestra.run.smithi107.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T04:07:42.819 INFO:teuthology.orchestra.run.smithi145.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 566 kB/s | 28 kB 00:00 2024-09-19T04:07:42.828 INFO:teuthology.orchestra.run.smithi107.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-19T04:07:42.853 INFO:teuthology.orchestra.run.smithi107.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-19T04:07:42.879 INFO:teuthology.orchestra.run.smithi107.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-19T04:07:42.904 INFO:teuthology.orchestra.run.smithi107.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-09-19T04:07:42.929 INFO:teuthology.orchestra.run.smithi107.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T04:07:42.978 INFO:teuthology.orchestra.run.smithi145.stdout:(137/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.8 MB/s | 16 MB 00:02 2024-09-19T04:07:43.003 INFO:teuthology.orchestra.run.smithi145.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 167 kB/s | 31 kB 00:00 2024-09-19T04:07:43.011 INFO:teuthology.orchestra.run.smithi107.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 500 kB/s | 54 kB 00:00 2024-09-19T04:07:43.037 INFO:teuthology.orchestra.run.smithi145.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-19T04:07:43.055 INFO:teuthology.orchestra.run.smithi107.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 967 kB/s | 42 kB 00:00 2024-09-19T04:07:43.062 INFO:teuthology.orchestra.run.smithi145.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 778 kB/s | 188 kB 00:00 2024-09-19T04:07:43.087 INFO:teuthology.orchestra.run.smithi145.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2024-09-19T04:07:43.112 INFO:teuthology.orchestra.run.smithi145.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.0 MB/s | 74 kB 00:00 2024-09-19T04:07:43.138 INFO:teuthology.orchestra.run.smithi145.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T04:07:43.138 INFO:teuthology.orchestra.run.smithi107.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.5 MB/s | 5.1 MB 00:00 2024-09-19T04:07:43.163 INFO:teuthology.orchestra.run.smithi145.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T04:07:43.163 INFO:teuthology.orchestra.run.smithi107.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 409 kB/s | 44 kB 00:00 2024-09-19T04:07:43.188 INFO:teuthology.orchestra.run.smithi145.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-19T04:07:43.189 INFO:teuthology.orchestra.run.smithi107.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T04:07:43.214 INFO:teuthology.orchestra.run.smithi107.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-19T04:07:43.222 INFO:teuthology.orchestra.run.smithi145.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-19T04:07:43.239 INFO:teuthology.orchestra.run.smithi107.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-19T04:07:43.247 INFO:teuthology.orchestra.run.smithi145.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-19T04:07:43.265 INFO:teuthology.orchestra.run.smithi107.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T04:07:43.290 INFO:teuthology.orchestra.run.smithi107.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T04:07:43.315 INFO:teuthology.orchestra.run.smithi107.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T04:07:43.322 INFO:teuthology.orchestra.run.smithi145.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-19T04:07:43.341 INFO:teuthology.orchestra.run.smithi107.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T04:07:43.366 INFO:teuthology.orchestra.run.smithi107.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-19T04:07:43.392 INFO:teuthology.orchestra.run.smithi107.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-09-19T04:07:43.442 INFO:teuthology.orchestra.run.smithi107.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-09-19T04:07:43.500 INFO:teuthology.orchestra.run.smithi107.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 4.9 MB/s | 548 kB 00:00 2024-09-19T04:07:43.506 INFO:teuthology.orchestra.run.smithi145.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 3.2 MB 00:00 2024-09-19T04:07:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-19T04:07:43.551 INFO:teuthology.orchestra.run.smithi107.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-19T04:07:43.565 INFO:teuthology.orchestra.run.smithi145.stdout:(150/156): librados2-19.1.1-361.g2f599a46.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-19T04:07:43.576 INFO:teuthology.orchestra.run.smithi107.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T04:07:43.626 INFO:teuthology.orchestra.run.smithi107.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 979 kB/s | 49 kB 00:00 2024-09-19T04:07:43.648 INFO:teuthology.orchestra.run.smithi145.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 491 kB/s | 41 kB 00:00 2024-09-19T04:07:43.660 INFO:teuthology.orchestra.run.smithi107.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-19T04:07:43.685 INFO:teuthology.orchestra.run.smithi107.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-19T04:07:43.732 INFO:teuthology.orchestra.run.smithi145.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-19T04:07:43.777 INFO:teuthology.orchestra.run.smithi107.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.0 MB/s | 839 kB 00:00 2024-09-19T04:07:43.836 INFO:teuthology.orchestra.run.smithi107.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-19T04:07:43.857 INFO:teuthology.orchestra.run.smithi145.stdout:(153/156): NetworkManager-1.51.0-1.el9.x86_64.r 4.3 MB/s | 2.3 MB 00:00 2024-09-19T04:07:43.861 INFO:teuthology.orchestra.run.smithi107.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T04:07:43.887 INFO:teuthology.orchestra.run.smithi107.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T04:07:43.924 INFO:teuthology.orchestra.run.smithi145.stdout:(154/156): NetworkManager-libnm-1.51.0-1.el9.x8 4.3 MB/s | 1.8 MB 00:00 2024-09-19T04:07:43.962 INFO:teuthology.orchestra.run.smithi107.stdout:(116/156): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-19T04:07:43.987 INFO:teuthology.orchestra.run.smithi107.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 141 kB/s | 14 kB 00:00 2024-09-19T04:07:44.030 INFO:teuthology.orchestra.run.smithi107.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-19T04:07:44.041 INFO:teuthology.orchestra.run.smithi145.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.4 MB/s | 1.4 MB 00:00 2024-09-19T04:07:44.063 INFO:teuthology.orchestra.run.smithi107.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-09-19T04:07:44.105 INFO:teuthology.orchestra.run.smithi107.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-19T04:07:44.130 INFO:teuthology.orchestra.run.smithi107.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-19T04:07:44.164 INFO:teuthology.orchestra.run.smithi107.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-19T04:07:44.189 INFO:teuthology.orchestra.run.smithi107.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-19T04:07:44.398 INFO:teuthology.orchestra.run.smithi107.stdout:(124/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-19T04:07:44.423 INFO:teuthology.orchestra.run.smithi107.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 76 kB/s | 18 kB 00:00 2024-09-19T04:07:44.457 INFO:teuthology.orchestra.run.smithi107.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 840 kB/s | 19 kB 00:00 2024-09-19T04:07:44.566 INFO:teuthology.orchestra.run.smithi145.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.4 MB/s | 2.4 MB 00:00 2024-09-19T04:07:44.573 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:07:44.573 INFO:teuthology.orchestra.run.smithi145.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-19T04:07:44.588 INFO:teuthology.orchestra.run.smithi107.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 152 kB/s | 20 kB 00:00 2024-09-19T04:07:44.632 INFO:teuthology.orchestra.run.smithi107.stdout:(128/156): python3-jaraco-collections-3.0.0-8.e 99 kB/s | 23 kB 00:00 2024-09-19T04:07:44.658 INFO:teuthology.orchestra.run.smithi107.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T04:07:44.691 INFO:teuthology.orchestra.run.smithi107.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-19T04:07:44.725 INFO:teuthology.orchestra.run.smithi107.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-19T04:07:44.800 INFO:teuthology.orchestra.run.smithi107.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-09-19T04:07:44.834 INFO:teuthology.orchestra.run.smithi107.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-09-19T04:07:44.859 INFO:teuthology.orchestra.run.smithi107.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-19T04:07:44.884 INFO:teuthology.orchestra.run.smithi107.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-09-19T04:07:44.910 INFO:teuthology.orchestra.run.smithi107.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 568 kB/s | 28 kB 00:00 2024-09-19T04:07:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-19T04:07:44.961 INFO:teuthology.orchestra.run.smithi107.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 713 kB/s | 90 kB 00:00 2024-09-19T04:07:44.986 INFO:teuthology.orchestra.run.smithi107.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T04:07:45.011 INFO:teuthology.orchestra.run.smithi107.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-09-19T04:07:45.036 INFO:teuthology.orchestra.run.smithi107.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-19T04:07:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-19T04:07:45.083 INFO:teuthology.orchestra.run.smithi107.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 4.3 MB/s | 90 kB 00:00 2024-09-19T04:07:45.108 INFO:teuthology.orchestra.run.smithi107.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2024-09-19T04:07:45.134 INFO:teuthology.orchestra.run.smithi107.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-19T04:07:45.159 INFO:teuthology.orchestra.run.smithi107.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:07:45.184 INFO:teuthology.orchestra.run.smithi107.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.4 MB/s | 427 kB 00:00 2024-09-19T04:07:45.268 INFO:teuthology.orchestra.run.smithi107.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-19T04:07:45.393 INFO:teuthology.orchestra.run.smithi107.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 19 MB/s | 2.3 MB 00:00 2024-09-19T04:07:45.468 INFO:teuthology.orchestra.run.smithi107.stdout:(150/156): NetworkManager-libnm-1.51.0-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-09-19T04:07:45.543 INFO:teuthology.orchestra.run.smithi107.stdout:(151/156): librbd1-19.1.1-361.g2f599a46.el9.x86 8.8 MB/s | 3.2 MB 00:00 2024-09-19T04:07:45.569 INFO:teuthology.orchestra.run.smithi107.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 409 kB/s | 41 kB 00:00 2024-09-19T04:07:45.644 INFO:teuthology.orchestra.run.smithi107.stdout:(153/156): librados2-19.1.1-361.g2f599a46.el9.x 7.1 MB/s | 3.4 MB 00:00 2024-09-19T04:07:45.669 INFO:teuthology.orchestra.run.smithi107.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 1.9 MB/s | 249 kB 00:00 2024-09-19T04:07:45.720 INFO:teuthology.orchestra.run.smithi107.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-19T04:07:45.799 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-19T04:07:45.828 INFO:teuthology.orchestra.run.smithi107.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-09-19T04:07:45.834 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:07:45.834 INFO:teuthology.orchestra.run.smithi107.stdout:Total 24 MB/s | 226 MB 00:09 2024-09-19T04:07:45.950 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-19T04:07:45.950 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-19T04:07:47.010 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-19T04:07:47.113 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:07:47.151 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:07:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:07:47.281 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:07:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:07:47.654 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:07:47.761 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-19T04:07:47.761 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-19T04:07:47.773 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:07:47.887 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-19T04:07:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:07:47.924 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:07:47.998 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:07:48.037 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:07:48.064 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-19T04:07:48.065 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-19T04:07:48.192 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:07:48.242 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:07:48.278 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:07:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:07:48.484 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:07:48.506 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:07:48.530 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:07:48.715 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:07:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:07:48.999 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:07:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:07:49.176 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:07:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:07:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:07:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:07:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:07:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:07:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:07:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:07:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:07:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:07:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:07:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:07:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:07:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:07:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:07:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:07:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:07:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:07:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:07:50.442 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-19T04:07:50.442 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-19T04:07:50.479 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:07:50.554 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:07:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:07:50.748 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:07:51.280 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:07:51.362 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:07:51.392 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:07:51.435 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:07:51.473 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:07:51.515 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:07:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:07:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:07:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:07:51.871 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:07:51.902 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:07:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:07:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:07:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:07:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:07:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:07:52.252 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:07:52.300 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:07:52.341 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:07:52.445 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:07:52.483 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:07:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:07:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:07:52.907 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:07:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:07:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:07:53.164 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:07:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:07:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:07:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:07:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:07:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:07:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:07:53.428 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:07:53.470 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:07:53.504 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:07:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:07:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:07:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:07:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:07:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:07:53.991 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-19T04:07:54.111 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:07:54.208 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:07:54.269 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:07:54.312 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:07:54.337 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:07:54.379 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:07:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:07:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:07:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:07:54.702 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:07:54.748 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:07:54.795 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:07:54.813 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:07:54.848 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:07:54.936 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:07:54.937 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:07:54.964 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:07:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:07:55.056 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:07:55.085 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:07:55.168 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:07:55.207 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:07:55.235 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:07:55.285 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:07:55.290 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:07:55.318 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:07:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:07:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:07:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:07:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:07:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:07:55.722 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:07:55.849 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:07:55.934 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:07:55.962 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:07:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:07:56.072 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:07:56.148 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:07:56.368 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:07:56.477 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:07:56.574 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:07:56.603 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:07:56.646 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:07:56.728 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:07:56.789 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:07:56.821 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:07:56.864 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:07:56.897 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:07:56.925 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:07:56.941 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-19T04:07:56.995 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:07:57.048 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:07:57.075 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:07:57.086 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:07:57.130 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:07:57.200 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:07:57.207 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:07:57.241 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:07:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:07:57.285 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:07:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:07:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:07:57.313 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:07:57.342 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:07:57.384 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:07:57.396 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:07:57.401 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:07:57.424 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:07:57.436 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:07:57.457 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:07:57.512 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:07:57.647 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:07:57.697 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:07:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:07:57.780 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:07:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:07:57.893 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:07:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:07:57.958 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:07:57.995 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:07:58.038 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:07:58.077 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:07:58.193 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:07:58.227 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:07:58.240 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:07:58.324 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:07:58.363 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:07:58.406 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:07:58.439 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:07:58.444 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:07:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:07:58.485 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:07:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:07:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:07:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:07:58.578 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:07:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:07:58.650 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:07:58.657 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:07:58.691 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:07:58.722 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:07:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:07:58.776 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:07:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:07:58.836 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:07:58.849 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:07:58.881 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:07:58.917 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:07:58.964 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:07:58.999 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:07:59.051 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:07:59.068 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:07:59.093 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:07:59.239 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:07:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:07:59.286 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:07:59.327 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:07:59.416 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:07:59.430 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:07:59.461 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:07:59.523 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:07:59.572 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:07:59.634 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:07:59.702 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:07:59.727 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:07:59.804 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:07:59.916 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:07:59.929 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:07:59.967 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:07:59.995 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:08:00.029 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:08:00.048 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:08:00.068 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:08:00.100 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:08:00.130 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:08:00.158 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:08:00.173 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:08:00.190 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:08:00.206 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:08:00.234 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:08:00.248 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:08:00.291 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:08:00.316 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:08:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:08:00.377 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:08:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:08:00.421 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:08:00.452 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:08:00.464 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:08:00.502 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:08:00.536 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:08:00.556 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:08:00.612 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:08:00.624 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:08:00.679 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:08:00.716 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:08:00.762 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:08:00.843 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:08:00.906 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:08:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:08:00.955 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:08:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:08:00.983 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:08:01.064 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:08:01.126 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:08:01.219 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:08:01.329 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:08:01.375 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:08:01.376 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:08:01.411 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:08:01.439 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:08:01.509 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:08:01.558 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:08:01.558 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:08:01.603 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:08:01.641 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:08:01.641 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:08:01.641 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:08:01.641 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:01.676 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:08:01.698 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:08:01.738 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:08:01.922 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:08:01.953 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:08:02.030 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:08:02.072 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:08:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:08:02.209 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:08:02.261 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:08:02.299 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:08:02.352 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:08:02.526 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:08:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:08:02.672 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:08:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:08:02.792 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:08:02.805 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:08:02.853 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:08:02.890 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:08:02.948 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:08:02.980 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:08:03.026 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:08:03.076 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:08:03.247 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:08:03.292 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:08:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:08:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:08:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:08:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:08:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:08:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:08:03.720 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:08:03.879 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:08:03.918 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:08:03.955 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:08:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:08:04.014 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:08:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:08:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:08:04.098 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:08:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:08:04.126 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:08:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:08:04.167 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:08:04.183 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:08:04.226 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:08:04.259 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:08:04.359 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:08:04.417 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:08:04.454 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:08:04.486 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:08:04.539 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:08:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:08:04.630 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:08:04.634 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:08:04.660 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:08:04.696 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:08:05.177 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:08:05.242 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:08:05.313 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:08:05.336 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:08:05.521 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:08:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:08:05.612 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:08:05.730 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:08:05.791 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:08:05.844 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:08:05.942 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:08:05.977 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:08:06.199 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:08:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:08:06.325 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:08:07.145 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:08:07.278 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:08:07.662 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:08:07.774 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:08:08.322 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:08:08.360 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:08:08.456 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:08:08.457 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:08:08.457 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:08:08.457 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:08.486 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:08:08.851 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:08:08.975 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:08:09.007 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:08:09.041 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:08:09.371 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:08:09.410 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:08:09.506 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:08:09.541 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:08:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:08:10.109 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:08:10.147 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:08:10.222 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:08:10.249 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:08:13.080 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:08:13.120 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:08:13.733 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:08:13.733 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-09-19T04:08:13.733 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-09-19T04:08:13.733 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-09-19T04:08:13.734 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-09-19T04:08:13.734 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-09-19T04:08:13.734 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-09-19T04:08:13.734 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-09-19T04:08:13.734 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:13.749 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:08:13.790 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:08:13.903 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:08:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:08:13.919 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:13.919 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:08:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:13.925 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:08:13.925 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:08:13.925 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:08:13.925 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:13.975 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:08:15.414 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:08:15.441 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:08:15.441 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:15.441 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:08:15.441 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:08:15.441 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:08:15.441 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:08:15.697 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:08:15.697 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:15.697 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:08:15.697 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:08:15.697 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:08:15.697 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:16.142 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:08:16.165 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:08:16.165 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:16.166 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:08:16.166 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:08:16.166 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:08:16.166 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:16.273 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:08:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:08:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:08:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:08:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:08:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:08:16.802 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:08:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:08:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:08:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:08:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:08:17.473 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:08:17.502 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:08:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:08:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:08:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:08:18.680 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:08:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:08:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:08:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:08:19.830 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:08:19.997 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:08:20.029 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:08:20.052 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:08:20.052 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:20.052 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:08:20.053 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:08:20.053 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:08:20.053 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:20.096 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:08:20.108 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:08:20.207 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:08:20.229 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:08:20.229 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2024-09-19T04:08:20.229 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2024-09-19T04:08:20.229 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2024-09-19T04:08:20.230 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2024-09-19T04:08:20.230 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2024-09-19T04:08:20.230 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /root 2024-09-19T04:08:20.230 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2024-09-19T04:08:20.230 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:20.253 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:08:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:08:20.413 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:08:20.414 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:20.414 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:08:20.414 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:21.179 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:08:21.224 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:08:21.247 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:08:21.248 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:21.248 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:08:21.248 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:08:21.248 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:08:21.248 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:21.300 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:08:21.322 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:08:21.323 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:21.323 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:08:21.323 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:08:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:08:21.399 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:21.399 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:08:21.399 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:08:21.399 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:08:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:08:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:08:21.724 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:21.724 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:08:21.725 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:08:21.725 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:08:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:21.909 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:08:21.935 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:08:21.936 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:21.936 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:08:21.936 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:08:21.936 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:08:21.936 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:22.168 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:08:22.191 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:08:22.191 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:22.191 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:08:22.192 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:08:22.192 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:08:22.192 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:22.626 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:08:22.650 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:08:22.650 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:22.650 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:08:22.650 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:08:22.650 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:08:22.651 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:22.748 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:08:22.813 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:08:22.877 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:08:23.061 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:08:23.110 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:08:23.189 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:08:23.286 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:08:23.423 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:08:23.469 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:08:23.501 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:08:23.529 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:08:23.999 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:08:24.027 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:08:24.059 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:08:24.146 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:08:24.175 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:08:25.222 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:08:25.252 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:08:26.211 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:08:26.235 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:08:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:08:26.312 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:08:26.338 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:08:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:08:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:08:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:08:26.497 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:08:26.538 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:08:26.564 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:08:26.564 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:26.565 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:08:26.565 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:08:26.565 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:08:26.565 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:26.610 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:08:26.613 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:08:26.625 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:08:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:08:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:08:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:08:26.724 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:08:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:08:26.778 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:08:26.911 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:08:26.911 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-09-19T04:08:26.911 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-09-19T04:08:26.911 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-09-19T04:08:26.911 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-09-19T04:08:26.911 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-09-19T04:08:26.911 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-09-19T04:08:26.911 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-09-19T04:08:26.912 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:27.125 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:08:27.169 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:08:27.169 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:27.169 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:08:27.169 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:27.388 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:08:27.430 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:08:27.450 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:08:27.698 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:08:27.733 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:08:27.756 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:08:27.756 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:27.756 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:08:27.757 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:08:27.757 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:08:27.757 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:27.800 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:08:27.823 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:08:27.823 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:27.823 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:08:27.823 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:27.875 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:08:27.899 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:08:27.899 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:27.899 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:08:27.899 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:08:27.899 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:08:27.899 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:28.210 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:08:28.233 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:08:28.233 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:28.233 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:08:28.233 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:08:28.233 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:08:28.233 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:28.275 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:08:28.352 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:08:28.393 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:08:28.477 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:08:28.513 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:08:28.539 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:08:28.565 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:08:28.565 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:08:28.649 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:08:28.649 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:08:28.745 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:08:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:08:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:08:28.780 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:08:28.780 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:08:28.802 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:08:28.802 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:28.803 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:08:28.803 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:08:28.803 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:08:28.803 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:08:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:08:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:08:29.086 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:08:29.096 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:08:29.096 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:29.096 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:08:29.096 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:08:29.096 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:08:29.096 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:08:29.587 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:08:29.638 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:08:29.638 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:29.639 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:08:29.639 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:08:29.639 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:08:29.639 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:29.780 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:08:29.840 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:08:29.904 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:08:30.130 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:08:30.192 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:08:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:08:30.386 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:08:30.559 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:08:30.618 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:08:30.656 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:08:30.682 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:08:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:08:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:08:31.181 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:08:31.249 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:08:31.279 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:08:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:08:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:08:32.788 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:08:32.829 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:08:32.870 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:08:32.902 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:08:32.929 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:08:33.113 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:08:33.159 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:08:33.190 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:08:33.220 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:08:33.266 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:08:33.616 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:08:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:08:33.787 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:08:33.872 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:08:33.913 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:08:33.934 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:08:33.955 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:08:33.992 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:08:34.044 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:08:34.044 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:34.044 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:08:34.044 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:08:34.045 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:08:34.045 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:34.101 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:08:34.127 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:08:34.236 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:08:34.306 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:08:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:08:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:08:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:08:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:08:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:08:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:08:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:08:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:08:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:08:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:08:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:08:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:08:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:08:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:08:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:08:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:08:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:08:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:08:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:08:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:08:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:08:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:08:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:08:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:08:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:08:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:08:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:08:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:08:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:08:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:08:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:08:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:08:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:08:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:08:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:08:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:08:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:08:34.765 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:08:34.851 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:08:34.892 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:08:34.973 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:08:35.012 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:08:35.038 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:08:35.064 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:08:35.064 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:08:35.140 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:08:35.140 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:08:35.224 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:08:35.249 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:08:35.254 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:08:35.254 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:08:35.299 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:08:35.329 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:08:35.379 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:08:35.380 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:35.380 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:08:35.380 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:08:35.380 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:08:35.380 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:35.434 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:08:35.434 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:08:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:08:35.487 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:08:35.487 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:35.487 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:08:35.487 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:35.526 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:08:35.538 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:08:35.590 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:08:35.590 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:35.590 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:08:35.590 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:08:35.591 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:08:35.591 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:35.598 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:08:35.922 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:08:35.973 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:08:35.974 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:08:35.974 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:08:35.974 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:08:35.974 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:08:35.974 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:08:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:37.372 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-09-19T04:08:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:08:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:08:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:08:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:08:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:08:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:08:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:08:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:08:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:08:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:08:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:08:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:08:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:08:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:08:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:08:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:08:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:08:37.386 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-19T04:08:37.783 DEBUG:teuthology.parallel:result is None 2024-09-19T04:08:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:08:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:08:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:08:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:08:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:08:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:08:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:08:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:08:41.430 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:08:41.430 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:08:41.430 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:08:41.430 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:08:41.430 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:08:41.430 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:08:41.430 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:08:41.431 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:08:41.432 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:08:41.444 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:08:41.444 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:08:41.444 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:08:41.444 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:08:41.444 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:08:41.444 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:08:41.444 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:08:41.444 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:08:41.444 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:08:41.444 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:08:41.445 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:08:41.446 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:08:41.447 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:08:41.447 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:08:41.447 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:08:41.447 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:08:41.447 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:08:41.447 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:08:41.447 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:08:41.447 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:08:41.447 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:08:41.447 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:08:41.448 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:08:41.448 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:08:41.448 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:08:41.448 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:08:41.448 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:08:41.448 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:08:41.448 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:08:41.448 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:08:41.448 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:08:41.449 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:08:41.449 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:08:41.449 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:08:41.449 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:08:41.449 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:08:41.449 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:08:41.449 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:08:41.449 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:08:41.449 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:08:41.449 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:08:41.450 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:08:41.451 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:08:41.451 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:08:41.451 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:08:41.451 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:08:41.451 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:08:41.451 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:08:41.451 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:08:41.451 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:08:41.451 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:08:41.451 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:08:41.452 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:08:41.452 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:08:41.452 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:08:41.452 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:08:41.452 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:08:41.452 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:08:41.452 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:08:41.452 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:08:41.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:08:41.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:08:41.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:08:41.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:08:41.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:08:41.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:08:41.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:08:41.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:08:41.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:08:41.454 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:08:41.455 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:08:41.456 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:08:41.456 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:08:41.456 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:08:41.456 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:08:41.456 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:08:41.456 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:08:41.456 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:08:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:08:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:08:42.223 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:08:42.277 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:08:42.318 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:08:43.616 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:08:43.710 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:08:43.766 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:08:43.855 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:08:43.855 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:43.855 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2024-09-19T04:08:43.855 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:43.855 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:43.855 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:43.855 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:43.855 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.855 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.857 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:08:43.858 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:08:43.859 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:08:43.860 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:08:43.860 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:08:43.860 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.860 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.860 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:08:43.860 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.860 INFO:teuthology.orchestra.run.smithi145.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:08:43.860 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:08:43.861 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:08:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:08:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:08:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:08:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:08:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:08:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:08:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:08:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:08:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:08:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:08:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:08:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:08:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:08:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:08:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:08:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:08:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:08:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:08:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:08:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:08:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:08:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:08:43.869 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-19T04:08:43.874 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:08:43.913 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:08:43.938 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:08:43.983 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:08:43.983 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:08:44.115 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:08:44.115 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:08:44.225 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:08:44.248 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:08:44.264 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:08:44.264 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:08:44.291 DEBUG:teuthology.parallel:result is None 2024-09-19T04:08:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:08:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:08:44.611 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:08:44.693 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:08:44.904 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:08:44.905 INFO:teuthology.orchestra.run.smithi107.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-19T04:08:44.905 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:50.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:08:50.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:08:50.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:08:50.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:08:50.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:08:50.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:08:50.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:08:50.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:08:50.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:08:50.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:08:50.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:08:50.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:08:50.394 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:08:50.394 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:08:50.394 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:08:50.405 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:08:50.405 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:08:50.406 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:08:50.407 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:08:50.407 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:08:50.407 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:08:50.407 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:08:50.407 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:08:50.407 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:08:50.407 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:08:50.407 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:08:50.407 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:08:50.408 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:08:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:08:50.410 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:08:50.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:08:50.412 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:08:50.413 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:08:50.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:08:50.415 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:08:50.416 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:08:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:08:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:52.536 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-09-19T04:08:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:08:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:08:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:08:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:08:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:08:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:08:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:08:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:08:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:08:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:08:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:08:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:08:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:08:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:08:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:08:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:08:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:08:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:08:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:08:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:08:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:08:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:08:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:08:52.552 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-19T04:08:53.117 DEBUG:teuthology.parallel:result is None 2024-09-19T04:08:53.118 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:08:53.258 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:08:53.306 INFO:teuthology.orchestra.run.smithi107.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:08:53.306 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:08:53.306 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:08:53.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:08:53.445 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:08:53.472 INFO:teuthology.orchestra.run.smithi115.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:08:53.472 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:08:53.473 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:08:53.474 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:08:53.612 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:08:53.639 INFO:teuthology.orchestra.run.smithi145.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:08:53.639 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:08:53.639 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:08:53.641 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T04:08:53.641 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:08:53.641 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:08:53.700 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:08:53.700 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:08:53.732 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:08:53.732 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:08:53.763 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T04:08:53.764 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:08:53.764 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:08:53.825 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:08:53.922 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:08:53.923 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:08:53.953 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:08:54.022 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:08:54.022 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:08:54.052 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:08:54.121 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T04:08:54.121 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:08:54.122 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:08:54.177 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:08:54.274 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:08:54.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:08:54.300 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:08:54.364 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:08:54.364 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:08:54.394 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:08:54.464 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T04:08:54.464 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:08:54.465 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:08:54.521 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:08:54.617 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:08:54.617 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:08:54.644 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:08:54.709 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:08:54.709 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:08:54.738 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:08:54.806 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T04:08:54.902 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2}} 2024-09-19T04:08:54.902 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:08:54.903 INFO:tasks.cephadm:Cluster fsid is e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:08:54.903 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T04:08:54.903 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.107', 'mon.b': '172.21.15.115', 'mon.c': '172.21.15.145'} 2024-09-19T04:08:54.903 INFO:tasks.cephadm:First mon is mon.a on smithi107 2024-09-19T04:08:54.903 INFO:tasks.cephadm:First mgr is x 2024-09-19T04:08:54.903 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T04:08:54.904 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-09-19T04:08:54.967 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2024-09-19T04:08:54.992 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname $(hostname -s) 2024-09-19T04:08:55.020 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T04:08:55.020 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:08:55.162 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T04:08:55.306 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:08:55.307 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T04:08:55.307 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T04:08:55.308 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:08:55.706 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:08 /home/ubuntu/cephtest/cephadm 2024-09-19T04:08:55.707 DEBUG:teuthology.orchestra.run.smithi115:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:08:56.064 INFO:teuthology.orchestra.run.smithi115.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:08 /home/ubuntu/cephtest/cephadm 2024-09-19T04:08:56.064 DEBUG:teuthology.orchestra.run.smithi145:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:08:56.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:08 /home/ubuntu/cephtest/cephadm 2024-09-19T04:08:56.739 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:08:56.781 DEBUG:teuthology.orchestra.run.smithi115:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:08:56.801 DEBUG:teuthology.orchestra.run.smithi145:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:08:56.835 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T04:08:56.836 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:08:56.840 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:08:56.843 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:08:58.336 INFO:teuthology.orchestra.run.smithi145.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:08:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:08:58.859 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:09:26.331 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-19T04:09:26.331 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:09:26.331 INFO:teuthology.orchestra.run.smithi145.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:09:26.331 INFO:teuthology.orchestra.run.smithi145.stdout: "repo_digests": [ 2024-09-19T04:09:26.331 INFO:teuthology.orchestra.run.smithi145.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:09:26.332 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-19T04:09:26.332 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-19T04:09:46.615 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-19T04:09:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:09:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:09:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2024-09-19T04:09:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:09:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-19T04:09:46.616 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-19T04:09:55.359 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:09:55.359 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:09:55.359 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:09:55.359 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-09-19T04:09:55.360 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:09:55.360 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:09:55.360 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:09:55.390 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-09-19T04:09:55.462 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2024-09-19T04:09:55.496 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph 2024-09-19T04:09:55.531 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-09-19T04:09:55.586 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2024-09-19T04:09:55.614 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /etc/ceph 2024-09-19T04:09:55.642 INFO:tasks.cephadm:Writing seed config... 2024-09-19T04:09:55.644 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T04:09:55.644 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T04:09:55.644 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T04:09:55.644 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T04:09:55.644 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T04:09:55.644 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T04:09:55.644 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T04:09:55.644 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T04:09:55.645 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-19T04:09:55.646 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T04:09:55.647 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-19T04:09:55.648 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:09:55.648 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T04:09:55.683 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e2ec5b0e-763c-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T04:09:55.683 DEBUG:teuthology.orchestra.run.smithi107:mon.a> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T04:09:55.727 DEBUG:teuthology.orchestra.run.smithi107:mgr.x> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T04:09:55.773 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T04:09:55.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:09:56.027 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:09:56.027 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', 'e2ec5b0e-763c-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2024-09-19T04:09:56.028 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T04:09:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2024-09-19T04:09:56.087 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.2.2 2024-09-19T04:09:56.087 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2024-09-19T04:09:56.087 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2024-09-19T04:09:56.101 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T04:09:56.101 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T04:09:56.113 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T04:09:56.113 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-19T04:09:56.126 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-09-19T04:09:56.139 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-09-19T04:09:56.139 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-09-19T04:09:56.139 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2024-09-19T04:09:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.2.2 2024-09-19T04:09:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T04:09:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2024-09-19T04:09:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2024-09-19T04:09:56.205 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T04:09:56.205 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T04:09:56.217 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T04:09:56.217 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-19T04:09:56.230 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-09-19T04:09:56.243 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-09-19T04:09:56.243 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-09-19T04:09:56.243 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2024-09-19T04:09:56.243 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:09:56.243 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 140126275232624 on /run/cephadm/e2ec5b0e-763c-11ef-bceb-c7b262605968.lock 2024-09-19T04:09:56.243 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 140126275232624 acquired on /run/cephadm/e2ec5b0e-763c-11ef-bceb-c7b262605968.lock 2024-09-19T04:09:56.243 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2024-09-19T04:09:56.244 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2024-09-19T04:09:56.244 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-09-19T04:09:56.251 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.107 metric 100 2024-09-19T04:09:56.251 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2024-09-19T04:09:56.257 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T04:09:56.257 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-19T04:09:56.257 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-19T04:09:56.262 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T04:09:56.263 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T04:09:56.263 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T04:09:56.263 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-19T04:09:56.263 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link noprefixroute 2024-09-19T04:09:56.263 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T04:09:56.263 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-09-19T04:09:56.264 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-09-19T04:09:56.264 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T04:09:56.264 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T04:09:56.265 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:09:56.787 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T04:09:56.788 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:09:56.788 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T04:09:56.788 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T04:09:56.788 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T04:09:56.788 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T04:09:56.788 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T04:09:57.797 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T04:09:57.798 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T04:09:57.798 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T04:09:58.829 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2024-09-19T04:09:58.829 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2024-09-19T04:09:59.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQAXpOtmpMmTChAAtaeMaCd4UJAegyU8YvgjSg== 2024-09-19T04:10:00.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQAYpOtm/KNuDhAAQqiWTmuXIW/ue6LeHsBYUA== 2024-09-19T04:10:01.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQAZpOtmE/yaDBAA76QaBRU6+E1IsRpQbarI3w== 2024-09-19T04:10:01.879 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2024-09-19T04:10:02.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T04:10:02.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T04:10:02.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:02.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T04:10:02.860 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for a [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T04:10:02.860 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = quincy 2024-09-19T04:10:02.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:02.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T04:10:02.860 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:10:02.861 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2024-09-19T04:10:03.803 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.a on 2024-09-19T04:10:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T04:10:04.318 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T04:10:04.568 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968.target. 2024-09-19T04:10:04.569 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968.target. 2024-09-19T04:10:04.899 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.a 2024-09-19T04:10:04.899 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T04:10:05.162 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968.target.wants/ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@.service. 2024-09-19T04:10:05.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:05 smithi107 podman[30280]: 2024-09-19 04:10:05.411035306 +0000 UTC m=+0.026264851 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:10:05.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:05 smithi107 podman[30280]: 2024-09-19 04:10:05.751817547 +0000 UTC m=+0.367047093 container init 4459dc934286e0ed837776d8a69627c3e0a7e5bb304c3f6008eb1411781b192b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T04:10:05.879 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:10:05.879 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-19T04:10:05.892 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:10:05.892 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-19T04:10:05.892 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-19T04:10:05.892 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T04:10:05.893 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2024-09-19T04:10:05.893 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2024-09-19T04:10:06.029 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:05 smithi107 podman[30280]: 2024-09-19 04:10:05.758383105 +0000 UTC m=+0.373612630 container start 4459dc934286e0ed837776d8a69627c3e0a7e5bb304c3f6008eb1411781b192b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T04:10:06.029 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:05 smithi107 ceph-mon[30298]: mkfs e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:06.029 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:05 smithi107 ceph-mon[30298]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:10:06.029 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:05 smithi107 bash[30280]: 4459dc934286e0ed837776d8a69627c3e0a7e5bb304c3f6008eb1411781b192b 2024-09-19T04:10:06.029 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:05 smithi107 systemd[1]: Started Ceph mon.a for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:10:06.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:10:06.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: monmap epoch 1 2024-09-19T04:10:06.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:06.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: last_changed 2024-09-19T04:10:02.266251+0000 2024-09-19T04:10:06.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:10:06.951 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: min_mon_release 19 (squid) 2024-09-19T04:10:06.951 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: election_strategy: 1 2024-09-19T04:10:06.951 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:10:06.951 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: fsmap 2024-09-19T04:10:06.951 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T04:10:06.951 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: mgrmap e1: no daemons active 2024-09-19T04:10:06.951 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:06 smithi107 ceph-mon[30298]: from='client.? 172.21.15.107:0/77048585' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.584462s) 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2024-09-19T04:10:06.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T04:10:06.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T04:10:06.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T04:10:06.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.328+0000 7fa947e00640 1 Processor -- start 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.328+0000 7fa947e00640 1 -- start start 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.328+0000 7fa947e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.328+0000 7fa947e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa94807c840 con 0x7fa94807de50 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.329+0000 7fa946a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.329+0000 7fa946a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39670/0 (socket says 172.21.15.107:39670) 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.329+0000 7fa946a00640 1 -- 172.21.15.107:0/2310877803 learned_addr learned my addr 172.21.15.107:0/2310877803 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.329+0000 7fa946a00640 1 -- 172.21.15.107:0/2310877803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94807c9c0 con 0x7fa94807de50 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.332+0000 7fa946a00640 1 --2- 172.21.15.107:0/2310877803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94807c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa938009820 tx=0x7fa93802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f351cac9306d44d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.332+0000 7fa946000640 1 -- 172.21.15.107:0/2310877803 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa938037070 con 0x7fa94807de50 2024-09-19T04:10:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.332+0000 7fa946000640 1 -- 172.21.15.107:0/2310877803 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938032440 con 0x7fa94807de50 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.332+0000 7fa946000640 1 -- 172.21.15.107:0/2310877803 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa938032740 con 0x7fa94807de50 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.333+0000 7fa947e00640 1 -- 172.21.15.107:0/2310877803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 msgr2=0x7fa94807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.333+0000 7fa947e00640 1 --2- 172.21.15.107:0/2310877803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94807c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa938009820 tx=0x7fa93802a300 comp rx=0 tx=0).stop 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.333+0000 7fa947e00640 1 -- 172.21.15.107:0/2310877803 shutdown_connections 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.333+0000 7fa947e00640 1 --2- 172.21.15.107:0/2310877803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94807c300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.333+0000 7fa947e00640 1 -- 172.21.15.107:0/2310877803 >> 172.21.15.107:0/2310877803 conn(0x7fa948102cf0 msgr2=0x7fa948105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.333+0000 7fa947e00640 1 -- 172.21.15.107:0/2310877803 shutdown_connections 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.333+0000 7fa947e00640 1 -- 172.21.15.107:0/2310877803 wait complete. 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.334+0000 7fa947e00640 1 Processor -- start 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.334+0000 7fa947e00640 1 -- start start 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.335+0000 7fa947e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.335+0000 7fa947e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa94807cf60 con 0x7fa94807de50 2024-09-19T04:10:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.335+0000 7fa946a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.335+0000 7fa946a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39678/0 (socket says 172.21.15.107:39678) 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.335+0000 7fa946a00640 1 -- 172.21.15.107:0/77048585 learned_addr learned my addr 172.21.15.107:0/77048585 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.335+0000 7fa946a00640 1 -- 172.21.15.107:0/77048585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94819d420 con 0x7fa94807de50 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.336+0000 7fa946a00640 1 --2- 172.21.15.107:0/77048585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94819cee0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa9380090b0 tx=0x7fa9380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.336+0000 7fa93f400640 1 -- 172.21.15.107:0/77048585 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa938041070 con 0x7fa94807de50 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.336+0000 7fa93f400640 1 -- 172.21.15.107:0/77048585 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93803c7a0 con 0x7fa94807de50 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.336+0000 7fa93f400640 1 -- 172.21.15.107:0/77048585 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa938037040 con 0x7fa94807de50 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.336+0000 7fa947e00640 1 -- 172.21.15.107:0/77048585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94819d6b0 con 0x7fa94807de50 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.336+0000 7fa947e00640 1 -- 172.21.15.107:0/77048585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9481a03a0 con 0x7fa94807de50 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.336+0000 7fa93f400640 1 -- 172.21.15.107:0/77048585 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa938044070 con 0x7fa94807de50 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.336+0000 7fa93f400640 1 -- 172.21.15.107:0/77048585 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa93803d3f0 con 0x7fa94807de50 2024-09-19T04:10:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.337+0000 7fa947e00640 1 -- 172.21.15.107:0/77048585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94807c380 con 0x7fa94807de50 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.340+0000 7fa93f400640 1 -- 172.21.15.107:0/77048585 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa938046e20 con 0x7fa94807de50 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.402+0000 7fa947e00640 1 -- 172.21.15.107:0/77048585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa94802da30 con 0x7fa94807de50 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.402+0000 7fa93f400640 1 -- 172.21.15.107:0/77048585 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fa938046450 con 0x7fa94807de50 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.403+0000 7fa947e00640 1 -- 172.21.15.107:0/77048585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 msgr2=0x7fa94819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.403+0000 7fa947e00640 1 --2- 172.21.15.107:0/77048585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94819cee0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa9380090b0 tx=0x7fa9380046c0 comp rx=0 tx=0).stop 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.404+0000 7fa947e00640 1 -- 172.21.15.107:0/77048585 shutdown_connections 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.404+0000 7fa947e00640 1 --2- 172.21.15.107:0/77048585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa94807de50 0x7fa94819cee0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.404+0000 7fa947e00640 1 -- 172.21.15.107:0/77048585 >> 172.21.15.107:0/77048585 conn(0x7fa948102cf0 msgr2=0x7fa948103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.404+0000 7fa947e00640 1 -- 172.21.15.107:0/77048585 shutdown_connections 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:06.404+0000 7fa947e00640 1 -- 172.21.15.107:0/77048585 wait complete. 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2024-09-19T04:10:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T04:10:08.140 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:07 smithi107 ceph-mon[30298]: from='client.? 172.21.15.107:0/4045848679' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T04:10:08.141 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:07 smithi107 ceph-mon[30298]: from='client.? 172.21.15.107:0/4045848679' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T04:10:08.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-19T04:10:08.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T04:10:08.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:08.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-09-19T04:10:08.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T04:10:08.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T04:10:08.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T04:10:08.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.450+0000 7fac73e00640 1 Processor -- start 2024-09-19T04:10:08.155 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.450+0000 7fac73e00640 1 -- start start 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.450+0000 7fac73e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac741079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.450+0000 7fac73e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac74107f90 con 0x7fac741075c0 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.450+0000 7fac72a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac741079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.450+0000 7fac72a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac741079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39692/0 (socket says 172.21.15.107:39692) 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.450+0000 7fac72a00640 1 -- 172.21.15.107:0/1037228665 learned_addr learned my addr 172.21.15.107:0/1037228665 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.451+0000 7fac72a00640 1 -- 172.21.15.107:0/1037228665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac74108720 con 0x7fac741075c0 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.452+0000 7fac72a00640 1 --2- 172.21.15.107:0/1037228665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac741079c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fac64009820 tx=0x7fac6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1395bb6a48300207 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.452+0000 7fac71600640 1 -- 172.21.15.107:0/1037228665 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac64037070 con 0x7fac741075c0 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.452+0000 7fac71600640 1 -- 172.21.15.107:0/1037228665 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac64032440 con 0x7fac741075c0 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.452+0000 7fac71600640 1 -- 172.21.15.107:0/1037228665 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac64032740 con 0x7fac741075c0 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.452+0000 7fac73e00640 1 -- 172.21.15.107:0/1037228665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 msgr2=0x7fac741079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:08.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.452+0000 7fac73e00640 1 --2- 172.21.15.107:0/1037228665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac741079c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fac64009820 tx=0x7fac6402a270 comp rx=0 tx=0).stop 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.452+0000 7fac73e00640 1 -- 172.21.15.107:0/1037228665 shutdown_connections 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.452+0000 7fac73e00640 1 --2- 172.21.15.107:0/1037228665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac741079c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.452+0000 7fac73e00640 1 -- 172.21.15.107:0/1037228665 >> 172.21.15.107:0/1037228665 conn(0x7fac74102d90 msgr2=0x7fac741051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.453+0000 7fac73e00640 1 -- 172.21.15.107:0/1037228665 shutdown_connections 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.453+0000 7fac73e00640 1 -- 172.21.15.107:0/1037228665 wait complete. 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.454+0000 7fac73e00640 1 Processor -- start 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.454+0000 7fac73e00640 1 -- start start 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.454+0000 7fac73e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac7419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.454+0000 7fac73e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac74108cc0 con 0x7fac741075c0 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.455+0000 7fac72a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac7419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.455+0000 7fac72a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac7419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39702/0 (socket says 172.21.15.107:39702) 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.455+0000 7fac72a00640 1 -- 172.21.15.107:0/4045848679 learned_addr learned my addr 172.21.15.107:0/4045848679 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:08.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.455+0000 7fac72a00640 1 -- 172.21.15.107:0/4045848679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7419d450 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.455+0000 7fac72a00640 1 --2- 172.21.15.107:0/4045848679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac7419cf10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fac640090b0 tx=0x7fac640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.456+0000 7fac78a00640 1 -- 172.21.15.107:0/4045848679 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac64041070 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.456+0000 7fac78a00640 1 -- 172.21.15.107:0/4045848679 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6403c7a0 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.456+0000 7fac73e00640 1 -- 172.21.15.107:0/4045848679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7419d6e0 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.456+0000 7fac78a00640 1 -- 172.21.15.107:0/4045848679 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac64037040 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.456+0000 7fac73e00640 1 -- 172.21.15.107:0/4045848679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac741a03d0 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.456+0000 7fac78a00640 1 -- 172.21.15.107:0/4045848679 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac64044070 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.456+0000 7fac78a00640 1 -- 172.21.15.107:0/4045848679 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac6403d3f0 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.457+0000 7fac73e00640 1 -- 172.21.15.107:0/4045848679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac74109310 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.460+0000 7fac78a00640 1 -- 172.21.15.107:0/4045848679 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fac64046450 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.520+0000 7fac73e00640 1 -- 172.21.15.107:0/4045848679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fac741a1aa0 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.524+0000 7fac78a00640 1 -- 172.21.15.107:0/4045848679 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7fac64046630 con 0x7fac741075c0 2024-09-19T04:10:08.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.525+0000 7fac78a00640 1 -- 172.21.15.107:0/4045848679 <== mon.0 v2:172.21.15.107:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac64056070 con 0x7fac741075c0 2024-09-19T04:10:08.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.526+0000 7fac73e00640 1 -- 172.21.15.107:0/4045848679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 msgr2=0x7fac7419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:08.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.526+0000 7fac73e00640 1 --2- 172.21.15.107:0/4045848679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac7419cf10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fac640090b0 tx=0x7fac640046c0 comp rx=0 tx=0).stop 2024-09-19T04:10:08.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.526+0000 7fac73e00640 1 -- 172.21.15.107:0/4045848679 shutdown_connections 2024-09-19T04:10:08.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.526+0000 7fac73e00640 1 --2- 172.21.15.107:0/4045848679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac741075c0 0x7fac7419cf10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:08.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.526+0000 7fac73e00640 1 -- 172.21.15.107:0/4045848679 >> 172.21.15.107:0/4045848679 conn(0x7fac74102d90 msgr2=0x7fac7407c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:08.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.526+0000 7fac73e00640 1 -- 172.21.15.107:0/4045848679 shutdown_connections 2024-09-19T04:10:08.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:07.526+0000 7fac73e00640 1 -- 172.21.15.107:0/4045848679 wait complete. 2024-09-19T04:10:08.159 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2024-09-19T04:10:08.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:08 smithi107 ceph-mon[30298]: from='client.? 172.21.15.107:0/3257830131' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:10:09.314 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.612+0000 7fb819400640 1 Processor -- start 2024-09-19T04:10:09.314 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.612+0000 7fb819400640 1 -- start start 2024-09-19T04:10:09.314 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.613+0000 7fb819400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb814107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:09.314 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.613+0000 7fb819400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb814108110 con 0x7fb814107740 2024-09-19T04:10:09.314 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.613+0000 7fb812800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb814107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:09.314 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.613+0000 7fb812800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb814107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39708/0 (socket says 172.21.15.107:39708) 2024-09-19T04:10:09.314 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.613+0000 7fb812800640 1 -- 172.21.15.107:0/2469827170 learned_addr learned my addr 172.21.15.107:0/2469827170 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:09.314 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.613+0000 7fb812800640 1 -- 172.21.15.107:0/2469827170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8141088f0 con 0x7fb814107740 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.614+0000 7fb812800640 1 --2- 172.21.15.107:0/2469827170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb814107b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb804009820 tx=0x7fb80402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f74692ae9782bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.615+0000 7fb811e00640 1 -- 172.21.15.107:0/2469827170 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb804037070 con 0x7fb814107740 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.615+0000 7fb811e00640 1 -- 172.21.15.107:0/2469827170 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb804032900 con 0x7fb814107740 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.615+0000 7fb811e00640 1 -- 172.21.15.107:0/2469827170 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb804030850 con 0x7fb814107740 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.615+0000 7fb819400640 1 -- 172.21.15.107:0/2469827170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 msgr2=0x7fb814107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.615+0000 7fb819400640 1 --2- 172.21.15.107:0/2469827170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb814107b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb804009820 tx=0x7fb80402a270 comp rx=0 tx=0).stop 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.615+0000 7fb819400640 1 -- 172.21.15.107:0/2469827170 shutdown_connections 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.615+0000 7fb819400640 1 --2- 172.21.15.107:0/2469827170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb814107b40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.615+0000 7fb819400640 1 -- 172.21.15.107:0/2469827170 >> 172.21.15.107:0/2469827170 conn(0x7fb814102ef0 msgr2=0x7fb814105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.616+0000 7fb819400640 1 -- 172.21.15.107:0/2469827170 shutdown_connections 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.616+0000 7fb819400640 1 -- 172.21.15.107:0/2469827170 wait complete. 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.616+0000 7fb819400640 1 Processor -- start 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.617+0000 7fb819400640 1 -- start start 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.617+0000 7fb819400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb81419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.617+0000 7fb819400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb81410bca0 con 0x7fb814107740 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.617+0000 7fb812800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb81419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.617+0000 7fb812800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb81419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39714/0 (socket says 172.21.15.107:39714) 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.617+0000 7fb812800640 1 -- 172.21.15.107:0/3257830131 learned_addr learned my addr 172.21.15.107:0/3257830131 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.617+0000 7fb812800640 1 -- 172.21.15.107:0/3257830131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81419bca0 con 0x7fb814107740 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.618+0000 7fb812800640 1 --2- 172.21.15.107:0/3257830131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb81419b760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8040097f0 tx=0x7fb804030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.618+0000 7fb80be00640 1 -- 172.21.15.107:0/3257830131 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb804042070 con 0x7fb814107740 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.618+0000 7fb80be00640 1 -- 172.21.15.107:0/3257830131 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb804032ca0 con 0x7fb814107740 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.618+0000 7fb80be00640 1 -- 172.21.15.107:0/3257830131 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb804037040 con 0x7fb814107740 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.618+0000 7fb819400640 1 -- 172.21.15.107:0/3257830131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81419bf30 con 0x7fb814107740 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.618+0000 7fb819400640 1 -- 172.21.15.107:0/3257830131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81419ec20 con 0x7fb814107740 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.618+0000 7fb80be00640 1 -- 172.21.15.107:0/3257830131 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb804045070 con 0x7fb814107740 2024-09-19T04:10:09.316 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.618+0000 7fb80be00640 1 -- 172.21.15.107:0/3257830131 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb804047ad0 con 0x7fb814107740 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.619+0000 7fb819400640 1 -- 172.21.15.107:0/3257830131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb81410be20 con 0x7fb814107740 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.622+0000 7fb80be00640 1 -- 172.21.15.107:0/3257830131 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb80403da60 con 0x7fb814107740 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.682+0000 7fb819400640 1 -- 172.21.15.107:0/3257830131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb81419f100 con 0x7fb814107740 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.683+0000 7fb80be00640 1 -- 172.21.15.107:0/3257830131 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fb80403d430 con 0x7fb814107740 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.684+0000 7fb819400640 1 -- 172.21.15.107:0/3257830131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 msgr2=0x7fb81419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.684+0000 7fb819400640 1 --2- 172.21.15.107:0/3257830131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb81419b760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8040097f0 tx=0x7fb804030ec0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.684+0000 7fb819400640 1 -- 172.21.15.107:0/3257830131 shutdown_connections 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.684+0000 7fb819400640 1 --2- 172.21.15.107:0/3257830131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb814107740 0x7fb81419b760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.684+0000 7fb819400640 1 -- 172.21.15.107:0/3257830131 >> 172.21.15.107:0/3257830131 conn(0x7fb814102ef0 msgr2=0x7fb814103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.684+0000 7fb819400640 1 -- 172.21.15.107:0/3257830131 shutdown_connections 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:08.684+0000 7fb819400640 1 -- 172.21.15.107:0/3257830131 wait complete. 2024-09-19T04:10:09.317 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2024-09-19T04:10:09.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:09 smithi107 systemd[1]: Stopping Ceph mon.a for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:10:09.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:09 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a[30294]: 2024-09-19T04:10:09.684+0000 7f40b8600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:10:09.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:09 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a[30294]: 2024-09-19T04:10:09.684+0000 7f40b8600640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T04:10:09.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:09 smithi107 podman[30538]: 2024-09-19 04:10:09.705530889 +0000 UTC m=+0.154501706 container died 4459dc934286e0ed837776d8a69627c3e0a7e5bb304c3f6008eb1411781b192b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T04:10:10.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:10 smithi107 podman[30538]: 2024-09-19 04:10:10.072433768 +0000 UTC m=+0.521404584 container cleanup 4459dc934286e0ed837776d8a69627c3e0a7e5bb304c3f6008eb1411781b192b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T04:10:10.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:10 smithi107 bash[30538]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a 2024-09-19T04:10:10.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:10 smithi107 podman[30550]: 2024-09-19 04:10:10.231560206 +0000 UTC m=+0.518595272 container remove 4459dc934286e0ed837776d8a69627c3e0a7e5bb304c3f6008eb1411781b192b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T04:10:10.816 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:10 smithi107 systemd[1]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T04:10:10.816 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:10 smithi107 systemd[1]: Stopped Ceph mon.a for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:10:10.817 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:10 smithi107 systemd[1]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.a.service: Consumed 1.208s CPU time. 2024-09-19T04:10:10.817 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:10 smithi107 systemd[1]: Starting Ceph mon.a for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:10:11.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:10 smithi107 podman[30640]: 2024-09-19 04:10:10.815748578 +0000 UTC m=+0.122093606 container create 7e651d195b90d496def2b6acae0421e01f804fabfee60a21c1461bcd30457811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T04:10:11.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:10 smithi107 podman[30640]: 2024-09-19 04:10:10.720388894 +0000 UTC m=+0.026733939 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:10:11.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 podman[30640]: 2024-09-19 04:10:11.001636256 +0000 UTC m=+0.307981283 container init 7e651d195b90d496def2b6acae0421e01f804fabfee60a21c1461bcd30457811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 podman[30640]: 2024-09-19 04:10:11.008324288 +0000 UTC m=+0.314669312 container start 7e651d195b90d496def2b6acae0421e01f804fabfee60a21c1461bcd30457811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: pidfile_write: ignore empty --pid-file 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: load: jerasure load: lrc 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: RocksDB version: 7.9.2 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Git sha 0 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: DB SUMMARY 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: DB Session ID: Q0519FU7MAI7TV6A4JDR 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: CURRENT file: CURRENT 2024-09-19T04:10:11.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T04:10:11.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T04:10:11.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T04:10:11.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105832 ; 2024-09-19T04:10:11.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.error_if_exists: 0 2024-09-19T04:10:11.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.create_if_missing: 0 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.paranoid_checks: 1 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.env: 0x558a368de940 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.info_log: 0x558a761716a0 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.statistics: (nil) 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.use_fsync: 0 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_log_file_size: 0 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T04:10:11.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.allow_fallocate: 1 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.use_direct_reads: 0 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.db_log_dir: 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.wal_dir: 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T04:10:11.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.write_buffer_manager: 0x558a76180dc0 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T04:10:11.099 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.unordered_write: 0 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.row_cache: None 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.wal_filter: None 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.two_write_queues: 0 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T04:10:11.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.wal_compression: 0 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.atomic_flush: 0 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.log_readahead_size: 0 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T04:10:11.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_background_jobs: 2 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_background_compactions: -1 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_subcompactions: 1 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_open_files: -1 2024-09-19T04:10:11.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_background_flushes: -1 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Compression algorithms supported: 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: kZSTD supported: 0 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: kXpressCompression supported: 0 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: kBZip2Compression supported: 0 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: kLZ4Compression supported: 1 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: kZlibCompression supported: 1 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: kSnappyCompression supported: 1 2024-09-19T04:10:11.103 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.merge_operator: 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_filter: None 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558a76171300) 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-09-19T04:10:11.104 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_type: 0 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_index_type: 0 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_shortening: 1 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: checksum: 4 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: no_block_cache: 0 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache: 0x558a761651f0 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_options: 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: capacity : 536870912 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_shard_bits : 4 2024-09-19T04:10:11.105 INFO:journalctl@ceph.mon.a.smithi107.stdout: strict_capacity_limit : 0 2024-09-19T04:10:11.106 INFO:journalctl@ceph.mon.a.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-09-19T04:10:11.106 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_compressed: (nil) 2024-09-19T04:10:11.106 INFO:journalctl@ceph.mon.a.smithi107.stdout: persistent_cache: (nil) 2024-09-19T04:10:11.106 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size: 4096 2024-09-19T04:10:11.106 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size_deviation: 10 2024-09-19T04:10:11.106 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_restart_interval: 16 2024-09-19T04:10:11.106 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_block_restart_interval: 1 2024-09-19T04:10:11.107 INFO:journalctl@ceph.mon.a.smithi107.stdout: metadata_block_size: 4096 2024-09-19T04:10:11.107 INFO:journalctl@ceph.mon.a.smithi107.stdout: partition_filters: 0 2024-09-19T04:10:11.107 INFO:journalctl@ceph.mon.a.smithi107.stdout: use_delta_encoding: 1 2024-09-19T04:10:11.107 INFO:journalctl@ceph.mon.a.smithi107.stdout: filter_policy: bloomfilter 2024-09-19T04:10:11.107 INFO:journalctl@ceph.mon.a.smithi107.stdout: whole_key_filtering: 1 2024-09-19T04:10:11.107 INFO:journalctl@ceph.mon.a.smithi107.stdout: verify_compression: 0 2024-09-19T04:10:11.107 INFO:journalctl@ceph.mon.a.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-09-19T04:10:11.107 INFO:journalctl@ceph.mon.a.smithi107.stdout: format_version: 5 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_index_compression: 1 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_align: 0 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout: max_auto_readahead_size: 262144 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout: prepopulate_block_cache: 0 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout: initial_auto_readahead_size: 8192 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compression: NoCompression 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T04:10:11.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.num_levels: 7 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T04:10:11.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T04:10:11.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T04:10:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T04:10:11.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.inplace_update_support: 0 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.bloom_locality: 0 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.max_successive_merges: 0 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T04:10:11.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.ttl: 2592000 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.enable_blob_files: false 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.min_blob_size: 0 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T04:10:11.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9d9e8aac-e9e3-4fe7-aea3-0727b285b2a6 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726719011049807, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726719011051242, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100843, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98728, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14423, "raw_average_key_size": 49, "raw_value_size": 90934, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726719011, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9d9e8aac-e9e3-4fe7-aea3-0727b285b2a6", "db_session_id": "Q0519FU7MAI7TV6A4JDR", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726719011051515, "job": 1, "event": "recovery_finished"} 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558a76194e00 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: DB pointer 0x558a76290000 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** DB Stats ** 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:10:11.115 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: L0 2/0 100.34 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: Sum 2/0 100.34 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:10:11.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 120.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T04:10:11.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative compaction: 0.00 GB write, 16.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval compaction: 0.00 GB write, 16.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache BinnedLRUCache@0x558a761651f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: starting mon.a rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: mon.a@-1(???) e1 preinit fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: mon.a@-1(???).mds e1 new map 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: mon.a@-1(???).mds e1 print_map 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: e1 2024-09-19T04:10:11.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: btime 2024-09-19T04:10:05:820337+0000 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout: legacy client fscid: -1 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout: No filesystems configured 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T04:10:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 bash[30640]: 7e651d195b90d496def2b6acae0421e01f804fabfee60a21c1461bcd30457811 2024-09-19T04:10:11.120 INFO:teuthology.orchestra.run.smithi107.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T04:10:11.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 systemd[1]: Started Ceph mon.a for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:10:11.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:10:11.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: monmap epoch 1 2024-09-19T04:10:11.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:11.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: last_changed 2024-09-19T04:10:02.266251+0000 2024-09-19T04:10:11.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:10:11.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: min_mon_release 19 (squid) 2024-09-19T04:10:11.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: election_strategy: 1 2024-09-19T04:10:11.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:10:11.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: fsmap 2024-09-19T04:10:11.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T04:10:11.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:11 smithi107 ceph-mon[30659]: mgrmap e1: no daemons active 2024-09-19T04:10:12.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.579+0000 7ffaa0c00640 1 Processor -- start 2024-09-19T04:10:12.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.580+0000 7ffaa0c00640 1 -- start start 2024-09-19T04:10:12.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.580+0000 7ffaa0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:12.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.580+0000 7ffaa0c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c108110 con 0x7ffa9c107740 2024-09-19T04:10:12.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.581+0000 7ffa99e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:12.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.581+0000 7ffa99e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34698/0 (socket says 172.21.15.107:34698) 2024-09-19T04:10:12.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.581+0000 7ffa99e00640 1 -- 172.21.15.107:0/2164721017 learned_addr learned my addr 172.21.15.107:0/2164721017 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:12.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.581+0000 7ffa99e00640 1 -- 172.21.15.107:0/2164721017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9c1088f0 con 0x7ffa9c107740 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.584+0000 7ffa99e00640 1 --2- 172.21.15.107:0/2164721017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c107b40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009820 tx=0x7ffa8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d281afc736734e96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.584+0000 7ffa99400640 1 -- 172.21.15.107:0/2164721017 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa8c037070 con 0x7ffa9c107740 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.584+0000 7ffa99400640 1 -- 172.21.15.107:0/2164721017 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa8c032900 con 0x7ffa9c107740 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.584+0000 7ffa99400640 1 -- 172.21.15.107:0/2164721017 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa8c030850 con 0x7ffa9c107740 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.584+0000 7ffaa0c00640 1 -- 172.21.15.107:0/2164721017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 msgr2=0x7ffa9c107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.584+0000 7ffaa0c00640 1 --2- 172.21.15.107:0/2164721017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c107b40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009820 tx=0x7ffa8c02a270 comp rx=0 tx=0).stop 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.585+0000 7ffaa0c00640 1 -- 172.21.15.107:0/2164721017 shutdown_connections 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.585+0000 7ffaa0c00640 1 --2- 172.21.15.107:0/2164721017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c107b40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.585+0000 7ffaa0c00640 1 -- 172.21.15.107:0/2164721017 >> 172.21.15.107:0/2164721017 conn(0x7ffa9c102ef0 msgr2=0x7ffa9c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.585+0000 7ffaa0c00640 1 -- 172.21.15.107:0/2164721017 shutdown_connections 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.585+0000 7ffaa0c00640 1 -- 172.21.15.107:0/2164721017 wait complete. 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.586+0000 7ffaa0c00640 1 Processor -- start 2024-09-19T04:10:12.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.586+0000 7ffaa0c00640 1 -- start start 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.586+0000 7ffaa0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.586+0000 7ffaa0c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c10bca0 con 0x7ffa9c107740 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.586+0000 7ffa99e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.586+0000 7ffa99e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34706/0 (socket says 172.21.15.107:34706) 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.586+0000 7ffa99e00640 1 -- 172.21.15.107:0/383465781 learned_addr learned my addr 172.21.15.107:0/383465781 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.587+0000 7ffa99e00640 1 -- 172.21.15.107:0/383465781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9c07e0b0 con 0x7ffa9c107740 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.587+0000 7ffa99e00640 1 --2- 172.21.15.107:0/383465781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c07db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009940 tx=0x7ffa8c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.587+0000 7ffa93400640 1 -- 172.21.15.107:0/383465781 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa8c042070 con 0x7ffa9c107740 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.588+0000 7ffa93400640 1 -- 172.21.15.107:0/383465781 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa8c032be0 con 0x7ffa9c107740 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.588+0000 7ffaa0c00640 1 -- 172.21.15.107:0/383465781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9c07c2a0 con 0x7ffa9c107740 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.588+0000 7ffa93400640 1 -- 172.21.15.107:0/383465781 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa8c037040 con 0x7ffa9c107740 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.588+0000 7ffaa0c00640 1 -- 172.21.15.107:0/383465781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9c07c7e0 con 0x7ffa9c107740 2024-09-19T04:10:12.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.588+0000 7ffa93400640 1 -- 172.21.15.107:0/383465781 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffa8c045070 con 0x7ffa9c107740 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.588+0000 7ffa93400640 1 -- 172.21.15.107:0/383465781 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffa8c047a80 con 0x7ffa9c107740 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.589+0000 7ffaa0c00640 1 -- 172.21.15.107:0/383465781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa9c107bc0 con 0x7ffa9c107740 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.592+0000 7ffa93400640 1 -- 172.21.15.107:0/383465781 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ffa8c032d90 con 0x7ffa9c107740 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.651+0000 7ffaa0c00640 1 -- 172.21.15.107:0/383465781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ffa9c07cd10 con 0x7ffa9c107740 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.653+0000 7ffa93400640 1 -- 172.21.15.107:0/383465781 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ffa8c03d430 con 0x7ffa9c107740 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.654+0000 7ffaa0c00640 1 -- 172.21.15.107:0/383465781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 msgr2=0x7ffa9c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.654+0000 7ffaa0c00640 1 --2- 172.21.15.107:0/383465781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c07db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009940 tx=0x7ffa8c0326e0 comp rx=0 tx=0).stop 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.654+0000 7ffaa0c00640 1 -- 172.21.15.107:0/383465781 shutdown_connections 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.654+0000 7ffaa0c00640 1 --2- 172.21.15.107:0/383465781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c107740 0x7ffa9c07db70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.654+0000 7ffaa0c00640 1 -- 172.21.15.107:0/383465781 >> 172.21.15.107:0/383465781 conn(0x7ffa9c102ef0 msgr2=0x7ffa9c103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.655+0000 7ffaa0c00640 1 -- 172.21.15.107:0/383465781 shutdown_connections 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:11.655+0000 7ffaa0c00640 1 -- 172.21.15.107:0/383465781 wait complete. 2024-09-19T04:10:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T04:10:12.282 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:10:12.282 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2024-09-19T04:10:12.282 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T04:10:12.283 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T04:10:12.625 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.x 2024-09-19T04:10:12.625 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T04:10:12.860 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968.target.wants/ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@.service. 2024-09-19T04:10:12.893 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:12 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/383465781' entity='client.admin' 2024-09-19T04:10:13.202 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:12 smithi107 systemd[1]: Starting Ceph mgr.x for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:10:13.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:13 smithi107 podman[30869]: 2024-09-19 04:10:13.200515587 +0000 UTC m=+0.118717658 container create c7ad715785fee54e102083cae12f2cb19eb851520e7a423ef7f9f2bcbddef977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T04:10:13.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:13 smithi107 podman[30869]: 2024-09-19 04:10:13.108312313 +0000 UTC m=+0.026514399 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:10:13.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:13 smithi107 podman[30869]: 2024-09-19 04:10:13.417886684 +0000 UTC m=+0.336088777 container init c7ad715785fee54e102083cae12f2cb19eb851520e7a423ef7f9f2bcbddef977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T04:10:13.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:13 smithi107 podman[30869]: 2024-09-19 04:10:13.423861285 +0000 UTC m=+0.342063358 container start c7ad715785fee54e102083cae12f2cb19eb851520e7a423ef7f9f2bcbddef977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T04:10:14.231 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:13.768+0000 7fe88162f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:10:14.231 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:13.835+0000 7fe88162f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:10:14.231 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:14.023+0000 7fe88162f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:10:14.231 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:14.102+0000 7fe88162f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:10:14.491 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:10:14.491 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-19T04:10:14.505 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:10:14.505 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-19T04:10:14.505 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-19T04:10:14.505 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T04:10:14.522 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:10:14.523 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-19T04:10:14.537 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:10:14.537 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-19T04:10:14.537 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-19T04:10:14.537 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T04:10:14.537 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2024-09-19T04:10:14.537 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2024-09-19T04:10:14.680 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:14.274+0000 7fe88162f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:10:14.680 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:14.440+0000 7fe88162f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:10:14.680 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:14 smithi107 bash[30869]: c7ad715785fee54e102083cae12f2cb19eb851520e7a423ef7f9f2bcbddef977 2024-09-19T04:10:14.680 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:14 smithi107 systemd[1]: Started Ceph mgr.x for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:10:14.980 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:14.825+0000 7fe88162f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:10:14.980 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:14.898+0000 7fe88162f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:10:15.480 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:15 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:15.120+0000 7fe88162f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:10:16.730 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:16 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:16.298+0000 7fe88162f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:10:16.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:16 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:16.379+0000 7fe88162f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:10:17.637 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:17.339+0000 7fe88162f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:10:17.638 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:17.473+0000 7fe88162f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:10:17.638 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:17.636+0000 7fe88162f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:10:17.980 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:17.710+0000 7fe88162f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:10:17.981 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:17.884+0000 7fe88162f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:10:18.480 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:17.966+0000 7fe88162f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:10:18.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:18.036+0000 7fe88162f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:10:18.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:18.225+0000 7fe88162f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:10:18.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:18.307+0000 7fe88162f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:10:18.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:18.389+0000 7fe88162f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:10:18.980 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:18.490+0000 7fe88162f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:10:18.981 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:18.560+0000 7fe88162f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:10:18.981 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:18.632+0000 7fe88162f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:10:19.980 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:19.575+0000 7fe88162f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:10:19.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: Activating manager daemon x 2024-09-19T04:10:19.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: mgrmap e2: x(active, starting, since 0.00427034s) 2024-09-19T04:10:19.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: from='mgr.14100 172.21.15.107:0/991016723' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:10:19.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: from='mgr.14100 172.21.15.107:0/991016723' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:10:19.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: from='mgr.14100 172.21.15.107:0/991016723' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:10:19.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: from='mgr.14100 172.21.15.107:0/991016723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:10:19.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: from='mgr.14100 172.21.15.107:0/991016723' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:10:19.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: Manager daemon x is now available 2024-09-19T04:10:19.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: from='mgr.14100 172.21.15.107:0/991016723' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:10:19.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: from='mgr.14100 172.21.15.107:0/991016723' entity='mgr.x' 2024-09-19T04:10:19.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: from='mgr.14100 172.21.15.107:0/991016723' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:10:19.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: from='mgr.14100 172.21.15.107:0/991016723' entity='mgr.x' 2024-09-19T04:10:19.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:19 smithi107 ceph-mon[30659]: from='mgr.14100 172.21.15.107:0/991016723' entity='mgr.x' 2024-09-19T04:10:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:21 smithi107 ceph-mon[30659]: mgrmap e3: x(active, since 1.00892s) 2024-09-19T04:10:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:22 smithi107 ceph-mon[30659]: mgrmap e4: x(active, since 2s) 2024-09-19T04:10:24.435 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:24 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1799379968' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:10:24.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-19T04:10:24.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:10:24.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:10:24.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:10:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:10:24.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:10:24.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:10:05:820337+0000", 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:10:24.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:10:05.821340+0000", 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.862+0000 7feb8b600640 1 Processor -- start 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.862+0000 7feb8b600640 1 -- start start 2024-09-19T04:10:24.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.862+0000 7feb8b600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb84109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.862+0000 7feb8b600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410a330 con 0x7feb84109960 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.863+0000 7feb88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb84109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.863+0000 7feb88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb84109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50476/0 (socket says 172.21.15.107:50476) 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.863+0000 7feb88c00640 1 -- 172.21.15.107:0/699185797 learned_addr learned my addr 172.21.15.107:0/699185797 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.863+0000 7feb88c00640 1 -- 172.21.15.107:0/699185797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8410aad0 con 0x7feb84109960 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.864+0000 7feb88c00640 1 --2- 172.21.15.107:0/699185797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb84109d60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feb78009820 tx=0x7feb7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da3df83ecbf8836 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.864+0000 7feb83400640 1 -- 172.21.15.107:0/699185797 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb78037070 con 0x7feb84109960 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.865+0000 7feb83400640 1 -- 172.21.15.107:0/699185797 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feb78032900 con 0x7feb84109960 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.865+0000 7feb83400640 1 -- 172.21.15.107:0/699185797 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb78030850 con 0x7feb84109960 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.865+0000 7feb8b600640 1 -- 172.21.15.107:0/699185797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 msgr2=0x7feb84109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.865+0000 7feb8b600640 1 --2- 172.21.15.107:0/699185797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb84109d60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feb78009820 tx=0x7feb7802a270 comp rx=0 tx=0).stop 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.865+0000 7feb8b600640 1 -- 172.21.15.107:0/699185797 shutdown_connections 2024-09-19T04:10:24.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.865+0000 7feb8b600640 1 --2- 172.21.15.107:0/699185797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb84109d60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.865+0000 7feb8b600640 1 -- 172.21.15.107:0/699185797 >> 172.21.15.107:0/699185797 conn(0x7feb8407cf30 msgr2=0x7feb8407d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.865+0000 7feb8b600640 1 -- 172.21.15.107:0/699185797 shutdown_connections 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.866+0000 7feb8b600640 1 -- 172.21.15.107:0/699185797 wait complete. 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.866+0000 7feb8b600640 1 Processor -- start 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.867+0000 7feb8b600640 1 -- start start 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.867+0000 7feb8b600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb8419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.867+0000 7feb8b600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410de80 con 0x7feb84109960 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.867+0000 7feb88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb8419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.867+0000 7feb88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb8419d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50478/0 (socket says 172.21.15.107:50478) 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.867+0000 7feb88c00640 1 -- 172.21.15.107:0/1799379968 learned_addr learned my addr 172.21.15.107:0/1799379968 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.868+0000 7feb88c00640 1 -- 172.21.15.107:0/1799379968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8419de60 con 0x7feb84109960 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.868+0000 7feb88c00640 1 --2- 172.21.15.107:0/1799379968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb8419d920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feb780097f0 tx=0x7feb78030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.868+0000 7feb81600640 1 -- 172.21.15.107:0/1799379968 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb78042070 con 0x7feb84109960 2024-09-19T04:10:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.868+0000 7feb81600640 1 -- 172.21.15.107:0/1799379968 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feb78032ca0 con 0x7feb84109960 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.868+0000 7feb81600640 1 -- 172.21.15.107:0/1799379968 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb78037040 con 0x7feb84109960 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.868+0000 7feb8b600640 1 -- 172.21.15.107:0/1799379968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8419e0f0 con 0x7feb84109960 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.868+0000 7feb8b600640 1 -- 172.21.15.107:0/1799379968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb841a0de0 con 0x7feb84109960 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.869+0000 7feb81600640 1 -- 172.21.15.107:0/1799379968 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7feb78045070 con 0x7feb84109960 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.869+0000 7feb8b600640 1 -- 172.21.15.107:0/1799379968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb8410e000 con 0x7feb84109960 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.870+0000 7feb81600640 1 --2- 172.21.15.107:0/1799379968 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7feb6c0391b0 0x7feb6c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.870+0000 7feb81600640 1 -- 172.21.15.107:0/1799379968 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feb7806c140 con 0x7feb84109960 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.871+0000 7feb83e00640 1 --2- 172.21.15.107:0/1799379968 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7feb6c0391b0 0x7feb6c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.871+0000 7feb83e00640 1 --2- 172.21.15.107:0/1799379968 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7feb6c0391b0 0x7feb6c03b670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb700098c0 tx=0x7feb70006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:23.876+0000 7feb81600640 1 -- 172.21.15.107:0/1799379968 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feb7804b170 con 0x7feb84109960 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.119+0000 7feb8b600640 1 -- 172.21.15.107:0/1799379968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7feb841a1450 con 0x7feb84109960 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.119+0000 7feb81600640 1 -- 172.21.15.107:0/1799379968 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7feb7806bb60 con 0x7feb84109960 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.123+0000 7feb8b600640 1 -- 172.21.15.107:0/1799379968 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7feb6c0391b0 msgr2=0x7feb6c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:24.818 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.123+0000 7feb8b600640 1 --2- 172.21.15.107:0/1799379968 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7feb6c0391b0 0x7feb6c03b670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb700098c0 tx=0x7feb70006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:24.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.123+0000 7feb8b600640 1 -- 172.21.15.107:0/1799379968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 msgr2=0x7feb8419d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:24.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.123+0000 7feb8b600640 1 --2- 172.21.15.107:0/1799379968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb8419d920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feb780097f0 tx=0x7feb78030ec0 comp rx=0 tx=0).stop 2024-09-19T04:10:24.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.123+0000 7feb8b600640 1 -- 172.21.15.107:0/1799379968 shutdown_connections 2024-09-19T04:10:24.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.123+0000 7feb8b600640 1 --2- 172.21.15.107:0/1799379968 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7feb6c0391b0 0x7feb6c03b670 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:24.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.123+0000 7feb8b600640 1 --2- 172.21.15.107:0/1799379968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84109960 0x7feb8419d920 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:24.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.123+0000 7feb8b600640 1 -- 172.21.15.107:0/1799379968 >> 172.21.15.107:0/1799379968 conn(0x7feb8407cf30 msgr2=0x7feb84107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:24.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.123+0000 7feb8b600640 1 -- 172.21.15.107:0/1799379968 shutdown_connections 2024-09-19T04:10:24.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:24.123+0000 7feb8b600640 1 -- 172.21.15.107:0/1799379968 wait complete. 2024-09-19T04:10:24.819 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2024-09-19T04:10:25.772 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:25 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/993433812' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T04:10:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-19T04:10:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T04:10:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T04:10:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.303+0000 7f11b9c00640 1 Processor -- start 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.304+0000 7f11b9c00640 1 -- start start 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.304+0000 7f11b9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.304+0000 7f11b9c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11b407c800 con 0x7f11b407de10 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.304+0000 7f11b3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.304+0000 7f11b3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50488/0 (socket says 172.21.15.107:50488) 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.304+0000 7f11b3e00640 1 -- 172.21.15.107:0/1770453205 learned_addr learned my addr 172.21.15.107:0/1770453205 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.305+0000 7f11b3e00640 1 -- 172.21.15.107:0/1770453205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b407c980 con 0x7f11b407de10 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.306+0000 7f11b3e00640 1 --2- 172.21.15.107:0/1770453205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b407c2c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009820 tx=0x7f11a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673459ca9f164624 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.306+0000 7f11b3400640 1 -- 172.21.15.107:0/1770453205 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11a4037070 con 0x7f11b407de10 2024-09-19T04:10:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.306+0000 7f11b3400640 1 -- 172.21.15.107:0/1770453205 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11a4032900 con 0x7f11b407de10 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.306+0000 7f11b3400640 1 -- 172.21.15.107:0/1770453205 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11a4030850 con 0x7f11b407de10 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.306+0000 7f11b9c00640 1 -- 172.21.15.107:0/1770453205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 msgr2=0x7f11b407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.306+0000 7f11b9c00640 1 --2- 172.21.15.107:0/1770453205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b407c2c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009820 tx=0x7f11a402a270 comp rx=0 tx=0).stop 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.307+0000 7f11b9c00640 1 -- 172.21.15.107:0/1770453205 shutdown_connections 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.307+0000 7f11b9c00640 1 --2- 172.21.15.107:0/1770453205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b407c2c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.307+0000 7f11b9c00640 1 -- 172.21.15.107:0/1770453205 >> 172.21.15.107:0/1770453205 conn(0x7f11b4102cf0 msgr2=0x7f11b4105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.307+0000 7f11b9c00640 1 -- 172.21.15.107:0/1770453205 shutdown_connections 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.307+0000 7f11b9c00640 1 -- 172.21.15.107:0/1770453205 wait complete. 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.308+0000 7f11b9c00640 1 Processor -- start 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.308+0000 7f11b9c00640 1 -- start start 2024-09-19T04:10:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.308+0000 7f11b9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.308+0000 7f11b9c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11b410dbb0 con 0x7f11b407de10 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.309+0000 7f11b3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.309+0000 7f11b3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b419d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50504/0 (socket says 172.21.15.107:50504) 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.309+0000 7f11b3e00640 1 -- 172.21.15.107:0/993433812 learned_addr learned my addr 172.21.15.107:0/993433812 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.309+0000 7f11b3e00640 1 -- 172.21.15.107:0/993433812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b419dc40 con 0x7f11b407de10 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.309+0000 7f11b3e00640 1 --2- 172.21.15.107:0/993433812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b419d700 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f11a40097f0 tx=0x7f11a4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.309+0000 7f11b1600640 1 -- 172.21.15.107:0/993433812 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11a4037040 con 0x7f11b407de10 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.310+0000 7f11b1600640 1 -- 172.21.15.107:0/993433812 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11a4040070 con 0x7f11b407de10 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.310+0000 7f11b9c00640 1 -- 172.21.15.107:0/993433812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b419ded0 con 0x7f11b407de10 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.310+0000 7f11b1600640 1 -- 172.21.15.107:0/993433812 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11a403a390 con 0x7f11b407de10 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.310+0000 7f11b9c00640 1 -- 172.21.15.107:0/993433812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b41a0bc0 con 0x7f11b407de10 2024-09-19T04:10:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.311+0000 7f11b1600640 1 -- 172.21.15.107:0/993433812 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f11a403b430 con 0x7f11b407de10 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.311+0000 7f11b9c00640 1 -- 172.21.15.107:0/993433812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b407c340 con 0x7f11b407de10 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.311+0000 7f11b1600640 1 --2- 172.21.15.107:0/993433812 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f1198039160 0x7f119803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.311+0000 7f11b1600640 1 -- 172.21.15.107:0/993433812 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f11a406c430 con 0x7f11b407de10 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.312+0000 7f11abe00640 1 --2- 172.21.15.107:0/993433812 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f1198039160 0x7f119803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.312+0000 7f11abe00640 1 --2- 172.21.15.107:0/993433812 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f1198039160 0x7f119803b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f119c0098c0 tx=0x7f119c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.316+0000 7f11b1600640 1 -- 172.21.15.107:0/993433812 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f11a40371e0 con 0x7f11b407de10 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.498+0000 7f11b9c00640 1 -- 172.21.15.107:0/993433812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f11b419e350 con 0x7f11b407de10 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.499+0000 7f11b1600640 1 -- 172.21.15.107:0/993433812 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f11a4032dd0 con 0x7f11b407de10 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.502+0000 7f11b9c00640 1 -- 172.21.15.107:0/993433812 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f1198039160 msgr2=0x7f119803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.502+0000 7f11b9c00640 1 --2- 172.21.15.107:0/993433812 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f1198039160 0x7f119803b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f119c0098c0 tx=0x7f119c006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.502+0000 7f11b9c00640 1 -- 172.21.15.107:0/993433812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 msgr2=0x7f11b419d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:26.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.502+0000 7f11b9c00640 1 --2- 172.21.15.107:0/993433812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b419d700 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f11a40097f0 tx=0x7f11a4030d80 comp rx=0 tx=0).stop 2024-09-19T04:10:26.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.502+0000 7f11b9c00640 1 -- 172.21.15.107:0/993433812 shutdown_connections 2024-09-19T04:10:26.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.502+0000 7f11b9c00640 1 --2- 172.21.15.107:0/993433812 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f1198039160 0x7f119803b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:26.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.502+0000 7f11b9c00640 1 --2- 172.21.15.107:0/993433812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11b407de10 0x7f11b419d700 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:26.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.502+0000 7f11b9c00640 1 -- 172.21.15.107:0/993433812 >> 172.21.15.107:0/993433812 conn(0x7f11b4102cf0 msgr2=0x7f11b4103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:26.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.502+0000 7f11b9c00640 1 -- 172.21.15.107:0/993433812 shutdown_connections 2024-09-19T04:10:26.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:25.502+0000 7f11b9c00640 1 -- 172.21.15.107:0/993433812 wait complete. 2024-09-19T04:10:26.183 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2024-09-19T04:10:27.152 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:26 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/823305020' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T04:10:27.152 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:27.153 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:27.153 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:27.150+0000 7f8ec7634040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:10:27.415 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:27.224+0000 7f8ec7634040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:10:27.670 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:27.413+0000 7f8ec7634040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:10:27.670 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:27.496+0000 7f8ec7634040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:10:27.749 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.641+0000 7f68aec00640 1 Processor -- start 2024-09-19T04:10:27.749 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.641+0000 7f68aec00640 1 -- start start 2024-09-19T04:10:27.749 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.641+0000 7f68aec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:27.749 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.641+0000 7f68aec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f68a807c8f0 con 0x7f68a807de70 2024-09-19T04:10:27.749 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.642+0000 7f68a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.642+0000 7f68a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50520/0 (socket says 172.21.15.107:50520) 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.642+0000 7f68a7e00640 1 -- 172.21.15.107:0/1293866875 learned_addr learned my addr 172.21.15.107:0/1293866875 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.642+0000 7f68a7e00640 1 -- 172.21.15.107:0/1293866875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a807ca70 con 0x7f68a807de70 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.643+0000 7f68a7e00640 1 --2- 172.21.15.107:0/1293866875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a807c320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6898009820 tx=0x7f689802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bf73d4ef9feb00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.643+0000 7f68a6a00640 1 -- 172.21.15.107:0/1293866875 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6898037070 con 0x7f68a807de70 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.644+0000 7f68a6a00640 1 -- 172.21.15.107:0/1293866875 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6898032900 con 0x7f68a807de70 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.644+0000 7f68a6a00640 1 -- 172.21.15.107:0/1293866875 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6898030850 con 0x7f68a807de70 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.644+0000 7f68aec00640 1 -- 172.21.15.107:0/1293866875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 msgr2=0x7f68a807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.644+0000 7f68aec00640 1 --2- 172.21.15.107:0/1293866875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a807c320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6898009820 tx=0x7f689802a270 comp rx=0 tx=0).stop 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.644+0000 7f68aec00640 1 -- 172.21.15.107:0/1293866875 shutdown_connections 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.644+0000 7f68aec00640 1 --2- 172.21.15.107:0/1293866875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a807c320 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.644+0000 7f68aec00640 1 -- 172.21.15.107:0/1293866875 >> 172.21.15.107:0/1293866875 conn(0x7f68a8102ef0 msgr2=0x7f68a8105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.644+0000 7f68aec00640 1 -- 172.21.15.107:0/1293866875 shutdown_connections 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.644+0000 7f68aec00640 1 -- 172.21.15.107:0/1293866875 wait complete. 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.645+0000 7f68aec00640 1 Processor -- start 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.645+0000 7f68aec00640 1 -- start start 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.646+0000 7f68aec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.646+0000 7f68aec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f68a810dd30 con 0x7f68a807de70 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.646+0000 7f68a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.646+0000 7f68a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50522/0 (socket says 172.21.15.107:50522) 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.646+0000 7f68a7e00640 1 -- 172.21.15.107:0/823305020 learned_addr learned my addr 172.21.15.107:0/823305020 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.646+0000 7f68a7e00640 1 -- 172.21.15.107:0/823305020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a819de40 con 0x7f68a807de70 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.646+0000 7f68a7e00640 1 --2- 172.21.15.107:0/823305020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a819d900 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f68980097f0 tx=0x7f6898030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.647+0000 7f68a4c00640 1 -- 172.21.15.107:0/823305020 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6898042070 con 0x7f68a807de70 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.647+0000 7f68a4c00640 1 -- 172.21.15.107:0/823305020 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6898032ca0 con 0x7f68a807de70 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.647+0000 7f68aec00640 1 -- 172.21.15.107:0/823305020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a819e0d0 con 0x7f68a807de70 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.647+0000 7f68a4c00640 1 -- 172.21.15.107:0/823305020 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6898037040 con 0x7f68a807de70 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.647+0000 7f68aec00640 1 -- 172.21.15.107:0/823305020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a81a0dc0 con 0x7f68a807de70 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.648+0000 7f68a4c00640 1 -- 172.21.15.107:0/823305020 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f6898045070 con 0x7f68a807de70 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.648+0000 7f68aec00640 1 -- 172.21.15.107:0/823305020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a807c3a0 con 0x7f68a807de70 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.648+0000 7f68a4c00640 1 --2- 172.21.15.107:0/823305020 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f688c0391b0 0x7f688c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.648+0000 7f68a4c00640 1 -- 172.21.15.107:0/823305020 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f689806c1b0 con 0x7f68a807de70 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.649+0000 7f68a7400640 1 --2- 172.21.15.107:0/823305020 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f688c0391b0 0x7f688c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.649+0000 7f68a7400640 1 --2- 172.21.15.107:0/823305020 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f688c0391b0 0x7f688c03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f68940098c0 tx=0x7f6894006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.654+0000 7f68a4c00640 1 -- 172.21.15.107:0/823305020 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f689804b170 con 0x7f68a807de70 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.875+0000 7f68aec00640 1 -- 172.21.15.107:0/823305020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f68a81a1440 con 0x7f68a807de70 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.927+0000 7f68a4c00640 1 -- 172.21.15.107:0/823305020 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f689806bbd0 con 0x7f68a807de70 2024-09-19T04:10:27.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.928+0000 7f68a4c00640 1 -- 172.21.15.107:0/823305020 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f689803d430 con 0x7f68a807de70 2024-09-19T04:10:27.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.930+0000 7f68aec00640 1 -- 172.21.15.107:0/823305020 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f688c0391b0 msgr2=0x7f688c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:27.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.930+0000 7f68aec00640 1 --2- 172.21.15.107:0/823305020 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f688c0391b0 0x7f688c03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f68940098c0 tx=0x7f6894006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:27.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.930+0000 7f68aec00640 1 -- 172.21.15.107:0/823305020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 msgr2=0x7f68a819d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:27.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.930+0000 7f68aec00640 1 --2- 172.21.15.107:0/823305020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a819d900 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f68980097f0 tx=0x7f6898030ec0 comp rx=0 tx=0).stop 2024-09-19T04:10:27.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.930+0000 7f68aec00640 1 -- 172.21.15.107:0/823305020 shutdown_connections 2024-09-19T04:10:27.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.930+0000 7f68aec00640 1 --2- 172.21.15.107:0/823305020 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f688c0391b0 0x7f688c03b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:27.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.930+0000 7f68aec00640 1 --2- 172.21.15.107:0/823305020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a807de70 0x7f68a819d900 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:27.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.930+0000 7f68aec00640 1 -- 172.21.15.107:0/823305020 >> 172.21.15.107:0/823305020 conn(0x7f68a8102ef0 msgr2=0x7f68a8103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:27.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.930+0000 7f68aec00640 1 -- 172.21.15.107:0/823305020 shutdown_connections 2024-09-19T04:10:27.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:26.930+0000 7f68aec00640 1 -- 172.21.15.107:0/823305020 wait complete. 2024-09-19T04:10:27.933 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:27.668+0000 7f8ec7634040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:10:27.933 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:27.839+0000 7f8ec7634040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:10:28.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:27 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/823305020' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T04:10:28.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:27 smithi107 ceph-mon[30659]: mgrmap e5: x(active, since 7s) 2024-09-19T04:10:28.447 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:28 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:28.221+0000 7f8ec7634040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:10:28.448 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:28 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:28.294+0000 7f8ec7634040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:10:28.730 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:28 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:28.512+0000 7f8ec7634040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:10:29.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:28 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3920916651' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T04:10:29.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:10:29.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T04:10:29.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:10:29.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T04:10:29.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.188+0000 7f6524400640 1 Processor -- start 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.189+0000 7f6524400640 1 -- start start 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.189+0000 7f6524400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.189+0000 7f6524400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f651c10a330 con 0x7f651c109960 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.190+0000 7f6521a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.190+0000 7f6521a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50560/0 (socket says 172.21.15.107:50560) 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.190+0000 7f6521a00640 1 -- 172.21.15.107:0/1977250094 learned_addr learned my addr 172.21.15.107:0/1977250094 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.190+0000 7f6521a00640 1 -- 172.21.15.107:0/1977250094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651c10aad0 con 0x7f651c109960 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.191+0000 7f6521a00640 1 --2- 172.21.15.107:0/1977250094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c109d60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6510009820 tx=0x7f651002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b58def8c1cf0ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.191+0000 7f651be00640 1 -- 172.21.15.107:0/1977250094 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6510037070 con 0x7f651c109960 2024-09-19T04:10:29.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.191+0000 7f651be00640 1 -- 172.21.15.107:0/1977250094 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6510032900 con 0x7f651c109960 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.191+0000 7f651be00640 1 -- 172.21.15.107:0/1977250094 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6510030850 con 0x7f651c109960 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.192+0000 7f6524400640 1 -- 172.21.15.107:0/1977250094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 msgr2=0x7f651c109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.192+0000 7f6524400640 1 --2- 172.21.15.107:0/1977250094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c109d60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6510009820 tx=0x7f651002a270 comp rx=0 tx=0).stop 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.192+0000 7f6524400640 1 -- 172.21.15.107:0/1977250094 shutdown_connections 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.192+0000 7f6524400640 1 --2- 172.21.15.107:0/1977250094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c109d60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.192+0000 7f6524400640 1 -- 172.21.15.107:0/1977250094 >> 172.21.15.107:0/1977250094 conn(0x7f651c07cf30 msgr2=0x7f651c07d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.192+0000 7f6524400640 1 -- 172.21.15.107:0/1977250094 shutdown_connections 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.192+0000 7f6524400640 1 -- 172.21.15.107:0/1977250094 wait complete. 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.193+0000 7f6524400640 1 Processor -- start 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.194+0000 7f6524400640 1 -- start start 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.194+0000 7f6524400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.194+0000 7f6524400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f651c10de80 con 0x7f651c109960 2024-09-19T04:10:29.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.194+0000 7f6521a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.194+0000 7f6521a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50564/0 (socket says 172.21.15.107:50564) 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.194+0000 7f6521a00640 1 -- 172.21.15.107:0/3920916651 learned_addr learned my addr 172.21.15.107:0/3920916651 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.194+0000 7f6521a00640 1 -- 172.21.15.107:0/3920916651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651c19de40 con 0x7f651c109960 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.195+0000 7f6521a00640 1 --2- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c19d900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f65100097f0 tx=0x7f6510030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.195+0000 7f651a000640 1 -- 172.21.15.107:0/3920916651 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6510042070 con 0x7f651c109960 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.195+0000 7f651a000640 1 -- 172.21.15.107:0/3920916651 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6510032ca0 con 0x7f651c109960 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.195+0000 7f651a000640 1 -- 172.21.15.107:0/3920916651 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6510037040 con 0x7f651c109960 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.195+0000 7f6524400640 1 -- 172.21.15.107:0/3920916651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651c19e0d0 con 0x7f651c109960 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.195+0000 7f6524400640 1 -- 172.21.15.107:0/3920916651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651c1a0dc0 con 0x7f651c109960 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.196+0000 7f651a000640 1 -- 172.21.15.107:0/3920916651 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f6510045070 con 0x7f651c109960 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.196+0000 7f6524400640 1 -- 172.21.15.107:0/3920916651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651c10e000 con 0x7f651c109960 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.197+0000 7f651a000640 1 --2- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f6504039200 0x7f650403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:29.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.197+0000 7f651a000640 1 -- 172.21.15.107:0/3920916651 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f651006c1a0 con 0x7f651c109960 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.197+0000 7f6521000640 1 -- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f6504039200 msgr2=0x7f650403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2057620332 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.197+0000 7f6521000640 1 --2- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f6504039200 0x7f650403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.206+0000 7f651a000640 1 -- 172.21.15.107:0/3920916651 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6510045360 con 0x7f651c109960 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.398+0000 7f6521000640 1 -- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f6504039200 msgr2=0x7f650403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2057620332 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.398+0000 7f6521000640 1 --2- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f6504039200 0x7f650403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.430+0000 7f6524400640 1 -- 172.21.15.107:0/3920916651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f651c1a1430 con 0x7f651c109960 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.431+0000 7f651a000640 1 -- 172.21.15.107:0/3920916651 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f651006bbc0 con 0x7f651c109960 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.433+0000 7f6524400640 1 -- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f6504039200 msgr2=0x7f650403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.433+0000 7f6524400640 1 --2- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f6504039200 0x7f650403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.433+0000 7f6524400640 1 -- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 msgr2=0x7f651c19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.433+0000 7f6524400640 1 --2- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c19d900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f65100097f0 tx=0x7f6510030ec0 comp rx=0 tx=0).stop 2024-09-19T04:10:29.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.433+0000 7f6524400640 1 -- 172.21.15.107:0/3920916651 shutdown_connections 2024-09-19T04:10:29.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.433+0000 7f6524400640 1 --2- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7f6504039200 0x7f650403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:29.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.433+0000 7f6524400640 1 --2- 172.21.15.107:0/3920916651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c109960 0x7f651c19d900 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:29.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.433+0000 7f6524400640 1 -- 172.21.15.107:0/3920916651 >> 172.21.15.107:0/3920916651 conn(0x7f651c07cf30 msgr2=0x7f651c107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:29.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.433+0000 7f6524400640 1 -- 172.21.15.107:0/3920916651 shutdown_connections 2024-09-19T04:10:29.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:28.433+0000 7f6524400640 1 -- 172.21.15.107:0/3920916651 wait complete. 2024-09-19T04:10:29.369 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-09-19T04:10:29.369 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 5... 2024-09-19T04:10:29.980 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:29 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:29.616+0000 7f8ec7634040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:10:29.981 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:29 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:29.698+0000 7f8ec7634040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:10:30.730 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:30 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:30.608+0000 7f8ec7634040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:10:31.144 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:30 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:30.741+0000 7f8ec7634040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:10:31.144 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:30 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:30.903+0000 7f8ec7634040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:10:31.145 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:30 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:30.974+0000 7f8ec7634040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:10:31.145 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:31.142+0000 7f8ec7634040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:10:31.472 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:31.221+0000 7f8ec7634040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:10:31.472 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:31.292+0000 7f8ec7634040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:10:31.472 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:31.470+0000 7f8ec7634040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:10:31.730 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:31.546+0000 7f8ec7634040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:10:31.730 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:31.623+0000 7f8ec7634040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:10:31.730 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:31.718+0000 7f8ec7634040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:10:32.230 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:31.787+0000 7f8ec7634040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:10:32.230 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:31.855+0000 7f8ec7634040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:10:33.230 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:32.748+0000 7f8ec7634040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: Active manager daemon x restarted 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: Activating manager daemon x 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: mgrmap e6: x(active, starting, since 0.00623544s) 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: Manager daemon x is now available 2024-09-19T04:10:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:33.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:33.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:33.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:32 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:34.084 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:33 smithi107 ceph-mon[30659]: Found migration_current of "None". Setting to last migration. 2024-09-19T04:10:34.084 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:33 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:10:34.085 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:33 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:10:34.085 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:33 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:34.085 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:33 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:34.085 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:33 smithi107 ceph-mon[30659]: mgrmap e7: x(active, since 1.00655s) 2024-09-19T04:10:34.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:10:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T04:10:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T04:10:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:10:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.837+0000 7fc04d200640 1 Processor -- start 2024-09-19T04:10:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.838+0000 7fc04d200640 1 -- start start 2024-09-19T04:10:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.838+0000 7fc04d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc048107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.838+0000 7fc04d200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc048108080 con 0x7fc048107740 2024-09-19T04:10:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.838+0000 7fc045e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc048107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.838+0000 7fc045e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc048107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41868/0 (socket says 172.21.15.107:41868) 2024-09-19T04:10:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.838+0000 7fc045e00640 1 -- 172.21.15.107:0/2454244709 learned_addr learned my addr 172.21.15.107:0/2454244709 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.839+0000 7fc045e00640 1 -- 172.21.15.107:0/2454244709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc048108200 con 0x7fc048107740 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.840+0000 7fc045e00640 1 --2- 172.21.15.107:0/2454244709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc048107b40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc038009820 tx=0x7fc03802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8146d3fc9d27093 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.840+0000 7fc044a00640 1 -- 172.21.15.107:0/2454244709 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc038037070 con 0x7fc048107740 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.840+0000 7fc044a00640 1 -- 172.21.15.107:0/2454244709 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc038032900 con 0x7fc048107740 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.840+0000 7fc044a00640 1 -- 172.21.15.107:0/2454244709 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc038030850 con 0x7fc048107740 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.841+0000 7fc04d200640 1 -- 172.21.15.107:0/2454244709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 msgr2=0x7fc048107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.841+0000 7fc04d200640 1 --2- 172.21.15.107:0/2454244709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc048107b40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc038009820 tx=0x7fc03802a270 comp rx=0 tx=0).stop 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.841+0000 7fc04d200640 1 -- 172.21.15.107:0/2454244709 shutdown_connections 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.841+0000 7fc04d200640 1 --2- 172.21.15.107:0/2454244709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc048107b40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.841+0000 7fc04d200640 1 -- 172.21.15.107:0/2454244709 >> 172.21.15.107:0/2454244709 conn(0x7fc048102f50 msgr2=0x7fc0481053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.841+0000 7fc04d200640 1 -- 172.21.15.107:0/2454244709 shutdown_connections 2024-09-19T04:10:34.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.841+0000 7fc04d200640 1 -- 172.21.15.107:0/2454244709 wait complete. 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.842+0000 7fc04d200640 1 Processor -- start 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.843+0000 7fc04d200640 1 -- start start 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.843+0000 7fc04d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc04807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.843+0000 7fc04d200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc04810bbb0 con 0x7fc048107740 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.843+0000 7fc045e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc04807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.843+0000 7fc045e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc04807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41870/0 (socket says 172.21.15.107:41870) 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.843+0000 7fc045e00640 1 -- 172.21.15.107:0/1145354710 learned_addr learned my addr 172.21.15.107:0/1145354710 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.843+0000 7fc045e00640 1 -- 172.21.15.107:0/1145354710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04807e0b0 con 0x7fc048107740 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.844+0000 7fc045e00640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc04807db70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc038031000 tx=0x7fc0380326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.844+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc038042070 con 0x7fc048107740 2024-09-19T04:10:34.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.844+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc038032be0 con 0x7fc048107740 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.844+0000 7fc04d200640 1 -- 172.21.15.107:0/1145354710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04807c2a0 con 0x7fc048107740 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.844+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc038037040 con 0x7fc048107740 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.844+0000 7fc04d200640 1 -- 172.21.15.107:0/1145354710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04807c7e0 con 0x7fc048107740 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.845+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fc038045070 con 0x7fc048107740 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.846+0000 7fc03ea00640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 0x7fc02c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.846+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 --> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc02c03bd80 con 0x7fc02c0391b0 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.846+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc03806c2c0 con 0x7fc048107740 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.846+0000 7fc045400640 1 -- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 msgr2=0x7fc02c03b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2057620332 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:29.846+0000 7fc045400640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 0x7fc02c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:30.047+0000 7fc045400640 1 -- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 msgr2=0x7fc02c03b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2057620332 2024-09-19T04:10:34.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:30.047+0000 7fc045400640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 0x7fc02c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:30.448+0000 7fc045400640 1 -- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 msgr2=0x7fc02c03b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2057620332 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:30.448+0000 7fc045400640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 0x7fc02c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:31.249+0000 7fc045400640 1 -- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 msgr2=0x7fc02c03b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2057620332 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:31.249+0000 7fc045400640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 0x7fc02c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:32.754+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7fc03806ad80 con 0x7fc048107740 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:32.754+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 msgr2=0x7fc02c03b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:32.754+0000 7fc03ea00640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 0x7fc02c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.753+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7fc03803cb30 con 0x7fc048107740 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.753+0000 7fc03ea00640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fc02c03cae0 0x7fc02c03eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.753+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 --> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc02c03bd80 con 0x7fc02c03cae0 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.754+0000 7fc045400640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fc02c03cae0 0x7fc02c03eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.755+0000 7fc045400640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fc02c03cae0 0x7fc02c03eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc034003980 tx=0x7fc034006e10 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.756+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 <== mgr.14114 v2:172.21.15.107:6800/972711045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fc02c03bd80 con 0x7fc02c03cae0 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.769+0000 7fc04d200640 1 -- 172.21.15.107:0/1145354710 --> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc048107bc0 con 0x7fc02c03cae0 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.769+0000 7fc03ea00640 1 -- 172.21.15.107:0/1145354710 <== mgr.14114 v2:172.21.15.107:6800/972711045 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fc048107bc0 con 0x7fc02c03cae0 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 -- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fc02c03cae0 msgr2=0x7fc02c03eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fc02c03cae0 0x7fc02c03eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc034003980 tx=0x7fc034006e10 comp rx=0 tx=0).stop 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 -- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 msgr2=0x7fc04807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc04807db70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc038031000 tx=0x7fc0380326e0 comp rx=0 tx=0).stop 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 -- 172.21.15.107:0/1145354710 shutdown_connections 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fc02c03cae0 0x7fc02c03eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:6800/2057620332,v1:172.21.15.107:6801/2057620332] conn(0x7fc02c0391b0 0x7fc02c03b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 --2- 172.21.15.107:0/1145354710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048107740 0x7fc04807db70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 -- 172.21.15.107:0/1145354710 >> 172.21.15.107:0/1145354710 conn(0x7fc048102f50 msgr2=0x7fc048103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 -- 172.21.15.107:0/1145354710 shutdown_connections 2024-09-19T04:10:34.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:33.770+0000 7fc04d200640 1 -- 172.21.15.107:0/1145354710 wait complete. 2024-09-19T04:10:34.406 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 5 is available 2024-09-19T04:10:34.406 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2024-09-19T04:10:34.646 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:34.576+0000 7f8e26800640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:10:34] ENGINE Error in HTTPServer.serve 2024-09-19T04:10:34.646 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: Traceback (most recent call last): 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self._connections.run(self.expiration_interval) 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self._run(expiration_interval) 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: s = self.context.wrap_socket( 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:10:34.647 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: return self.sslsocket_class._create( 2024-09-19T04:10:34.648 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:10:34.648 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self.do_handshake() 2024-09-19T04:10:34.648 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:10:34.648 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self._sslobj.do_handshake() 2024-09-19T04:10:34.648 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:10:34.648 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:34.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-mon[30659]: from='client.14118 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T04:10:34.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-mon[30659]: from='client.14118 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T04:10:34.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:34 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:35.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.876+0000 7f59f1c00640 1 Processor -- start 2024-09-19T04:10:35.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.876+0000 7f59f1c00640 1 -- start start 2024-09-19T04:10:35.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.876+0000 7f59f1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:35.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.876+0000 7f59f1c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec07c820 con 0x7f59ec07de30 2024-09-19T04:10:35.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.877+0000 7f59ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:35.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.877+0000 7f59ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41936/0 (socket says 172.21.15.107:41936) 2024-09-19T04:10:35.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.877+0000 7f59ebe00640 1 -- 172.21.15.107:0/537239820 learned_addr learned my addr 172.21.15.107:0/537239820 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:35.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.877+0000 7f59ebe00640 1 -- 172.21.15.107:0/537239820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec07c9a0 con 0x7f59ec07de30 2024-09-19T04:10:35.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.878+0000 7f59ebe00640 1 --2- 172.21.15.107:0/537239820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec07c2e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009820 tx=0x7f59dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c489d543580cb20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:35.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.878+0000 7f59eaa00640 1 -- 172.21.15.107:0/537239820 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59dc037070 con 0x7f59ec07de30 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.878+0000 7f59eaa00640 1 -- 172.21.15.107:0/537239820 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59dc032900 con 0x7f59ec07de30 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.878+0000 7f59eaa00640 1 -- 172.21.15.107:0/537239820 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59dc030850 con 0x7f59ec07de30 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.879+0000 7f59f1c00640 1 -- 172.21.15.107:0/537239820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 msgr2=0x7f59ec07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.879+0000 7f59f1c00640 1 --2- 172.21.15.107:0/537239820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec07c2e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009820 tx=0x7f59dc02a270 comp rx=0 tx=0).stop 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.879+0000 7f59f1c00640 1 -- 172.21.15.107:0/537239820 shutdown_connections 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.879+0000 7f59f1c00640 1 --2- 172.21.15.107:0/537239820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec07c2e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.879+0000 7f59f1c00640 1 -- 172.21.15.107:0/537239820 >> 172.21.15.107:0/537239820 conn(0x7f59ec102cf0 msgr2=0x7f59ec105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.879+0000 7f59f1c00640 1 -- 172.21.15.107:0/537239820 shutdown_connections 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.879+0000 7f59f1c00640 1 -- 172.21.15.107:0/537239820 wait complete. 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.880+0000 7f59f1c00640 1 Processor -- start 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.880+0000 7f59f1c00640 1 -- start start 2024-09-19T04:10:35.693 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.880+0000 7f59f1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.880+0000 7f59f1c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec10db90 con 0x7f59ec07de30 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.881+0000 7f59ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.881+0000 7f59ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41944/0 (socket says 172.21.15.107:41944) 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.881+0000 7f59ebe00640 1 -- 172.21.15.107:0/3813696922 learned_addr learned my addr 172.21.15.107:0/3813696922 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.881+0000 7f59ebe00640 1 -- 172.21.15.107:0/3813696922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec19dc10 con 0x7f59ec07de30 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.881+0000 7f59ebe00640 1 --2- 172.21.15.107:0/3813696922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec19d6d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f59dc032ce0 tx=0x7f59dc032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.881+0000 7f59e8c00640 1 -- 172.21.15.107:0/3813696922 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59dc042070 con 0x7f59ec07de30 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.881+0000 7f59f1c00640 1 -- 172.21.15.107:0/3813696922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ec19dea0 con 0x7f59ec07de30 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.881+0000 7f59e8c00640 1 -- 172.21.15.107:0/3813696922 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59dc0308c0 con 0x7f59ec07de30 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.882+0000 7f59e8c00640 1 -- 172.21.15.107:0/3813696922 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59dc037040 con 0x7f59ec07de30 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.882+0000 7f59f1c00640 1 -- 172.21.15.107:0/3813696922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ec1a0b90 con 0x7f59ec07de30 2024-09-19T04:10:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.883+0000 7f59e8c00640 1 -- 172.21.15.107:0/3813696922 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f59dc045070 con 0x7f59ec07de30 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.883+0000 7f59f1c00640 1 -- 172.21.15.107:0/3813696922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ec07c360 con 0x7f59ec07de30 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.883+0000 7f59e8c00640 1 --2- 172.21.15.107:0/3813696922 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f59d00391b0 0x7f59d003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.883+0000 7f59e8c00640 1 -- 172.21.15.107:0/3813696922 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f59dc06c330 con 0x7f59ec07de30 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.883+0000 7f59eb400640 1 --2- 172.21.15.107:0/3813696922 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f59d00391b0 0x7f59d003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.884+0000 7f59eb400640 1 --2- 172.21.15.107:0/3813696922 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f59d00391b0 0x7f59d003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59d80098c0 tx=0x7f59d8006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:34.888+0000 7f59e8c00640 1 -- 172.21.15.107:0/3813696922 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59dc04b170 con 0x7f59ec07de30 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.079+0000 7f59f1c00640 1 -- 172.21.15.107:0/3813696922 --> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f59ec104000 con 0x7f59d00391b0 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.087+0000 7f59e8c00640 1 -- 172.21.15.107:0/3813696922 <== mgr.14114 v2:172.21.15.107:6800/972711045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59ec104000 con 0x7f59d00391b0 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.089+0000 7f59f1c00640 1 -- 172.21.15.107:0/3813696922 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f59d00391b0 msgr2=0x7f59d003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.089+0000 7f59f1c00640 1 --2- 172.21.15.107:0/3813696922 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f59d00391b0 0x7f59d003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59d80098c0 tx=0x7f59d8006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.089+0000 7f59f1c00640 1 -- 172.21.15.107:0/3813696922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 msgr2=0x7f59ec19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:35.695 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.089+0000 7f59f1c00640 1 --2- 172.21.15.107:0/3813696922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec19d6d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f59dc032ce0 tx=0x7f59dc032d10 comp rx=0 tx=0).stop 2024-09-19T04:10:35.696 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.090+0000 7f59f1c00640 1 -- 172.21.15.107:0/3813696922 shutdown_connections 2024-09-19T04:10:35.696 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.090+0000 7f59f1c00640 1 --2- 172.21.15.107:0/3813696922 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f59d00391b0 0x7f59d003b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:35.696 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.090+0000 7f59f1c00640 1 --2- 172.21.15.107:0/3813696922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec07de30 0x7f59ec19d6d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:35.696 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.090+0000 7f59f1c00640 1 -- 172.21.15.107:0/3813696922 >> 172.21.15.107:0/3813696922 conn(0x7f59ec102cf0 msgr2=0x7f59ec1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:35.696 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.090+0000 7f59f1c00640 1 -- 172.21.15.107:0/3813696922 shutdown_connections 2024-09-19T04:10:35.696 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:35.090+0000 7f59f1c00640 1 -- 172.21.15.107:0/3813696922 wait complete. 2024-09-19T04:10:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:35 smithi107 ceph-mon[30659]: [19/Sep/2024:04:10:34] ENGINE Bus STARTING 2024-09-19T04:10:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:35 smithi107 ceph-mon[30659]: [19/Sep/2024:04:10:34] ENGINE Serving on https://172.21.15.107:7150 2024-09-19T04:10:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:35 smithi107 ceph-mon[30659]: [19/Sep/2024:04:10:34] ENGINE Error in HTTPServer.serve 2024-09-19T04:10:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-09-19T04:10:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:10:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:10:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:10:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-09-19T04:10:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:10:35.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:35 smithi107 ceph-mon[30659]: [19/Sep/2024:04:10:34] ENGINE Serving on http://172.21.15.107:8765 2024-09-19T04:10:35.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:35 smithi107 ceph-mon[30659]: [19/Sep/2024:04:10:34] ENGINE Bus STARTED 2024-09-19T04:10:35.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:35 smithi107 ceph-mon[30659]: mgrmap e8: x(active, since 2s) 2024-09-19T04:10:35.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:35 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:35.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:35 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:37.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:36 smithi107 ceph-mon[30659]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:37.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T04:10:37.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.194+0000 7f7cbd800640 1 Processor -- start 2024-09-19T04:10:37.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.194+0000 7f7cbd800640 1 -- start start 2024-09-19T04:10:37.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.194+0000 7f7cbd800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:37.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.194+0000 7f7cbd800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb807c840 con 0x7f7cb807dd90 2024-09-19T04:10:37.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.195+0000 7f7cb6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:37.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.195+0000 7f7cb6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41946/0 (socket says 172.21.15.107:41946) 2024-09-19T04:10:37.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.195+0000 7f7cb6a00640 1 -- 172.21.15.107:0/3493873207 learned_addr learned my addr 172.21.15.107:0/3493873207 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.195+0000 7f7cb6a00640 1 -- 172.21.15.107:0/3493873207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb807c9c0 con 0x7f7cb807dd90 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.196+0000 7f7cb6a00640 1 --2- 172.21.15.107:0/3493873207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb807c300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009820 tx=0x7f7ca802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8446ae8067bc4439 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.196+0000 7f7cb5600640 1 -- 172.21.15.107:0/3493873207 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ca8037070 con 0x7f7cb807dd90 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.196+0000 7f7cb5600640 1 -- 172.21.15.107:0/3493873207 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7ca8032900 con 0x7f7cb807dd90 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.196+0000 7f7cb5600640 1 -- 172.21.15.107:0/3493873207 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ca8030850 con 0x7f7cb807dd90 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.197+0000 7f7cbd800640 1 -- 172.21.15.107:0/3493873207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 msgr2=0x7f7cb807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.197+0000 7f7cbd800640 1 --2- 172.21.15.107:0/3493873207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb807c300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009820 tx=0x7f7ca802a270 comp rx=0 tx=0).stop 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.197+0000 7f7cbd800640 1 -- 172.21.15.107:0/3493873207 shutdown_connections 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.197+0000 7f7cbd800640 1 --2- 172.21.15.107:0/3493873207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb807c300 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.197+0000 7f7cbd800640 1 -- 172.21.15.107:0/3493873207 >> 172.21.15.107:0/3493873207 conn(0x7f7cb8102f50 msgr2=0x7f7cb81053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.197+0000 7f7cbd800640 1 -- 172.21.15.107:0/3493873207 shutdown_connections 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.197+0000 7f7cbd800640 1 -- 172.21.15.107:0/3493873207 wait complete. 2024-09-19T04:10:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.198+0000 7f7cbd800640 1 Processor -- start 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.198+0000 7f7cbd800640 1 -- start start 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.199+0000 7f7cbd800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.199+0000 7f7cbd800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb810dcc0 con 0x7f7cb807dd90 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.199+0000 7f7cb6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.199+0000 7f7cb6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41960/0 (socket says 172.21.15.107:41960) 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.199+0000 7f7cb6a00640 1 -- 172.21.15.107:0/4227491669 learned_addr learned my addr 172.21.15.107:0/4227491669 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.199+0000 7f7cb6a00640 1 -- 172.21.15.107:0/4227491669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb819de80 con 0x7f7cb807dd90 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.200+0000 7f7cb6a00640 1 --2- 172.21.15.107:0/4227491669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb819d940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80097f0 tx=0x7f7ca8030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.200+0000 7f7caf400640 1 -- 172.21.15.107:0/4227491669 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ca8042070 con 0x7f7cb807dd90 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.200+0000 7f7cbd800640 1 -- 172.21.15.107:0/4227491669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb819e110 con 0x7f7cb807dd90 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.200+0000 7f7caf400640 1 -- 172.21.15.107:0/4227491669 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7ca8032ca0 con 0x7f7cb807dd90 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.200+0000 7f7caf400640 1 -- 172.21.15.107:0/4227491669 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ca8037040 con 0x7f7cb807dd90 2024-09-19T04:10:37.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.200+0000 7f7cbd800640 1 -- 172.21.15.107:0/4227491669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb81a0e00 con 0x7f7cb807dd90 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.201+0000 7f7caf400640 1 -- 172.21.15.107:0/4227491669 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f7ca8045070 con 0x7f7cb807dd90 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.201+0000 7f7cbd800640 1 -- 172.21.15.107:0/4227491669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb807c380 con 0x7f7cb807dd90 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.202+0000 7f7caf400640 1 --2- 172.21.15.107:0/4227491669 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7c9c039200 0x7f7c9c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.202+0000 7f7caf400640 1 -- 172.21.15.107:0/4227491669 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7ca806c2a0 con 0x7f7cb807dd90 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.202+0000 7f7cb6000640 1 --2- 172.21.15.107:0/4227491669 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7c9c039200 0x7f7c9c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.203+0000 7f7cb6000640 1 --2- 172.21.15.107:0/4227491669 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7c9c039200 0x7f7c9c03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40098c0 tx=0x7f7ca4006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.207+0000 7f7caf400640 1 -- 172.21.15.107:0/4227491669 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7ca8045360 con 0x7f7cb807dd90 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.384+0000 7f7cbd800640 1 -- 172.21.15.107:0/4227491669 --> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f7cb8104210 con 0x7f7c9c039200 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.385+0000 7f7caf400640 1 -- 172.21.15.107:0/4227491669 <== mgr.14114 v2:172.21.15.107:6800/972711045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f7cb8104210 con 0x7f7c9c039200 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.387+0000 7f7cbd800640 1 -- 172.21.15.107:0/4227491669 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7c9c039200 msgr2=0x7f7c9c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.387+0000 7f7cbd800640 1 --2- 172.21.15.107:0/4227491669 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7c9c039200 0x7f7c9c03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40098c0 tx=0x7f7ca4006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:37.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.387+0000 7f7cbd800640 1 -- 172.21.15.107:0/4227491669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 msgr2=0x7f7cb819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:37.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.387+0000 7f7cbd800640 1 --2- 172.21.15.107:0/4227491669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb819d940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80097f0 tx=0x7f7ca8030ec0 comp rx=0 tx=0).stop 2024-09-19T04:10:37.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.387+0000 7f7cbd800640 1 -- 172.21.15.107:0/4227491669 shutdown_connections 2024-09-19T04:10:37.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.387+0000 7f7cbd800640 1 --2- 172.21.15.107:0/4227491669 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7c9c039200 0x7f7c9c03b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:37.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.387+0000 7f7cbd800640 1 --2- 172.21.15.107:0/4227491669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb807dd90 0x7f7cb819d940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:37.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.387+0000 7f7cbd800640 1 -- 172.21.15.107:0/4227491669 >> 172.21.15.107:0/4227491669 conn(0x7f7cb8102f50 msgr2=0x7f7cb8103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:37.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.387+0000 7f7cbd800640 1 -- 172.21.15.107:0/4227491669 shutdown_connections 2024-09-19T04:10:37.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:36.387+0000 7f7cbd800640 1 -- 172.21.15.107:0/4227491669 wait complete. 2024-09-19T04:10:37.025 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2024-09-19T04:10:38.160 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:37 smithi107 ceph-mon[30659]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:38.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: Generating public/private rsa key pair. 2024-09-19T04:10:38.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: Your identification has been saved in /tmp/tmpd1vbsozj/key 2024-09-19T04:10:38.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: Your public key has been saved in /tmp/tmpd1vbsozj/key.pub 2024-09-19T04:10:38.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: The key fingerprint is: 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: SHA256:MrBQX0YtV5vH0MsxlVdSvUq3MB/KyeFGBg+fHLLt/is ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: The key's randomart image is: 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: +---[RSA 3072]----+ 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: | . .+. .o..oB| 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: | . . o. o+ *+oo| 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: | . . . o %.=+o| 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: | . o . #o+ | 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: | . o S O X o| 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: | o X o | 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: | o | 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: | E | 2024-09-19T04:10:38.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: | oo.| 2024-09-19T04:10:38.483 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: +----[SHA256]-----+ 2024-09-19T04:10:38.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.535+0000 7fe89bc00640 1 Processor -- start 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.535+0000 7fe89bc00640 1 -- start start 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.535+0000 7fe89bc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe894109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.535+0000 7fe89bc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe89410a330 con 0x7fe894109960 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.536+0000 7fe899200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe894109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.536+0000 7fe899200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe894109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41970/0 (socket says 172.21.15.107:41970) 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.536+0000 7fe899200640 1 -- 172.21.15.107:0/606131276 learned_addr learned my addr 172.21.15.107:0/606131276 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.536+0000 7fe899200640 1 -- 172.21.15.107:0/606131276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89410aad0 con 0x7fe894109960 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.537+0000 7fe899200640 1 --2- 172.21.15.107:0/606131276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe894109d60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe888009820 tx=0x7fe88802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3d000174a42c32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.537+0000 7fe893400640 1 -- 172.21.15.107:0/606131276 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe888037070 con 0x7fe894109960 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.537+0000 7fe893400640 1 -- 172.21.15.107:0/606131276 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe888032900 con 0x7fe894109960 2024-09-19T04:10:38.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.537+0000 7fe893400640 1 -- 172.21.15.107:0/606131276 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe888030850 con 0x7fe894109960 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.538+0000 7fe89bc00640 1 -- 172.21.15.107:0/606131276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 msgr2=0x7fe894109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.538+0000 7fe89bc00640 1 --2- 172.21.15.107:0/606131276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe894109d60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe888009820 tx=0x7fe88802a270 comp rx=0 tx=0).stop 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.538+0000 7fe89bc00640 1 -- 172.21.15.107:0/606131276 shutdown_connections 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.538+0000 7fe89bc00640 1 --2- 172.21.15.107:0/606131276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe894109d60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.538+0000 7fe89bc00640 1 -- 172.21.15.107:0/606131276 >> 172.21.15.107:0/606131276 conn(0x7fe89407cf30 msgr2=0x7fe89407d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.538+0000 7fe89bc00640 1 -- 172.21.15.107:0/606131276 shutdown_connections 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.538+0000 7fe89bc00640 1 -- 172.21.15.107:0/606131276 wait complete. 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.539+0000 7fe89bc00640 1 Processor -- start 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.539+0000 7fe89bc00640 1 -- start start 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.539+0000 7fe89bc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe89419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.539+0000 7fe89bc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe89410de80 con 0x7fe894109960 2024-09-19T04:10:38.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.539+0000 7fe899200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe89419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.540+0000 7fe899200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe89419d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41980/0 (socket says 172.21.15.107:41980) 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.540+0000 7fe899200640 1 -- 172.21.15.107:0/1156233957 learned_addr learned my addr 172.21.15.107:0/1156233957 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.540+0000 7fe899200640 1 -- 172.21.15.107:0/1156233957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89419de40 con 0x7fe894109960 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.540+0000 7fe899200640 1 --2- 172.21.15.107:0/1156233957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe89419d900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe8880097f0 tx=0x7fe888030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.540+0000 7fe891600640 1 -- 172.21.15.107:0/1156233957 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe888042070 con 0x7fe894109960 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.540+0000 7fe891600640 1 -- 172.21.15.107:0/1156233957 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe888032ca0 con 0x7fe894109960 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.540+0000 7fe89bc00640 1 -- 172.21.15.107:0/1156233957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89419e0d0 con 0x7fe894109960 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.540+0000 7fe891600640 1 -- 172.21.15.107:0/1156233957 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe888037040 con 0x7fe894109960 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.541+0000 7fe89bc00640 1 -- 172.21.15.107:0/1156233957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8941a0dc0 con 0x7fe894109960 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.542+0000 7fe891600640 1 -- 172.21.15.107:0/1156233957 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fe888045070 con 0x7fe894109960 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.542+0000 7fe89bc00640 1 -- 172.21.15.107:0/1156233957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89410e000 con 0x7fe894109960 2024-09-19T04:10:38.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.542+0000 7fe891600640 1 --2- 172.21.15.107:0/1156233957 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe87c039200 0x7fe87c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.542+0000 7fe891600640 1 -- 172.21.15.107:0/1156233957 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe88806c210 con 0x7fe894109960 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.542+0000 7fe893e00640 1 --2- 172.21.15.107:0/1156233957 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe87c039200 0x7fe87c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.543+0000 7fe893e00640 1 --2- 172.21.15.107:0/1156233957 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe87c039200 0x7fe87c03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe8800098c0 tx=0x7fe880006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.547+0000 7fe891600640 1 -- 172.21.15.107:0/1156233957 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe88804b170 con 0x7fe894109960 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:37.723+0000 7fe89bc00640 1 -- 172.21.15.107:0/1156233957 --> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe894107c20 con 0x7fe87c039200 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.166+0000 7fe891600640 1 -- 172.21.15.107:0/1156233957 <== mgr.14114 v2:172.21.15.107:6800/972711045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe894107c20 con 0x7fe87c039200 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.168+0000 7fe89bc00640 1 -- 172.21.15.107:0/1156233957 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe87c039200 msgr2=0x7fe87c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.168+0000 7fe89bc00640 1 --2- 172.21.15.107:0/1156233957 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe87c039200 0x7fe87c03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe8800098c0 tx=0x7fe880006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.168+0000 7fe89bc00640 1 -- 172.21.15.107:0/1156233957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 msgr2=0x7fe89419d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.168+0000 7fe89bc00640 1 --2- 172.21.15.107:0/1156233957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe89419d900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe8880097f0 tx=0x7fe888030ec0 comp rx=0 tx=0).stop 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.169+0000 7fe89bc00640 1 -- 172.21.15.107:0/1156233957 shutdown_connections 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.169+0000 7fe89bc00640 1 --2- 172.21.15.107:0/1156233957 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe87c039200 0x7fe87c03b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.169+0000 7fe89bc00640 1 --2- 172.21.15.107:0/1156233957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe894109960 0x7fe89419d900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:38.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.169+0000 7fe89bc00640 1 -- 172.21.15.107:0/1156233957 >> 172.21.15.107:0/1156233957 conn(0x7fe89407cf30 msgr2=0x7fe894107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:38.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.169+0000 7fe89bc00640 1 -- 172.21.15.107:0/1156233957 shutdown_connections 2024-09-19T04:10:38.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:38.169+0000 7fe89bc00640 1 -- 172.21.15.107:0/1156233957 wait complete. 2024-09-19T04:10:39.137 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-mon[30659]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:39.137 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-mon[30659]: Generating ssh key... 2024-09-19T04:10:39.137 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:39.137 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:38 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:40.009 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTR2CZIG5V/+ju5vCbPcP03fnvYPo0IFqkVP+JFR/RFGmW8cUwRLnwDdF1z5EklQCweV6tO31TU5zADXsDxQfUUov/AwkNHdR+zSGSx3COs4cj8NII6BbeRMbXZdSxP2IWs5+PvWIvHtKdweBtvqKNypLxVRz7IM7XI2F8dJEUQWGGfOQMOLDz0KaJSIxIBB+GDZJgqK6AgGjClSJ0BiRSl/njLH0U6kyA1APm7FVndIOzg74lXNBbSEa16m1rz0eAamC6TH2iLkZCEXCzhutS3h42bdnH0WxvFI5DZBEH9jAATiww5QHJpzG/oJHDZbuoS8FvAsCcgX0yZH72yJfX17fyDfSfIzExaWhEQDdg7guHB1D4iS4AWiif7qRCUh4xnHZ35U9ZueESabVJh8O9pLT9FtZGcb8HCaq6E3zLy8saFzRRY6j2E6ScapIQC7fuPnFc0Jl0dLcrG+vGJAiA7s0iPabJ7uQTWC355hRsEdg3oEYwqaSDehFSuZE43zc= ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.240+0000 7fd107200640 1 Processor -- start 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.240+0000 7fd107200640 1 -- start start 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.241+0000 7fd107200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10810af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.241+0000 7fd107200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd10810b510 con 0x7fd10810ab40 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.241+0000 7fd105e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10810af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.241+0000 7fd105e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10810af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58544/0 (socket says 172.21.15.107:58544) 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.241+0000 7fd105e00640 1 -- 172.21.15.107:0/1349649010 learned_addr learned my addr 172.21.15.107:0/1349649010 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.241+0000 7fd105e00640 1 -- 172.21.15.107:0/1349649010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10810b690 con 0x7fd10810ab40 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.242+0000 7fd105e00640 1 --2- 172.21.15.107:0/1349649010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10810af40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009e80 tx=0x7fd0f802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87f99f600ae1d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.243+0000 7fd105400640 1 -- 172.21.15.107:0/1349649010 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0f8036070 con 0x7fd10810ab40 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.243+0000 7fd105400640 1 -- 172.21.15.107:0/1349649010 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd0f80318c0 con 0x7fd10810ab40 2024-09-19T04:10:40.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.243+0000 7fd105400640 1 -- 172.21.15.107:0/1349649010 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0f8007ab0 con 0x7fd10810ab40 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.243+0000 7fd107200640 1 -- 172.21.15.107:0/1349649010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 msgr2=0x7fd10810af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.243+0000 7fd107200640 1 --2- 172.21.15.107:0/1349649010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10810af40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009e80 tx=0x7fd0f802a7a0 comp rx=0 tx=0).stop 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.243+0000 7fd107200640 1 -- 172.21.15.107:0/1349649010 shutdown_connections 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.243+0000 7fd107200640 1 --2- 172.21.15.107:0/1349649010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10810af40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.243+0000 7fd107200640 1 -- 172.21.15.107:0/1349649010 >> 172.21.15.107:0/1349649010 conn(0x7fd10807cd70 msgr2=0x7fd10807d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.243+0000 7fd107200640 1 -- 172.21.15.107:0/1349649010 shutdown_connections 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.243+0000 7fd107200640 1 -- 172.21.15.107:0/1349649010 wait complete. 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.244+0000 7fd107200640 1 Processor -- start 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.245+0000 7fd107200640 1 -- start start 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.245+0000 7fd107200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.245+0000 7fd107200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd10810dba0 con 0x7fd10810ab40 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.245+0000 7fd105e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.245+0000 7fd105e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58548/0 (socket says 172.21.15.107:58548) 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.245+0000 7fd105e00640 1 -- 172.21.15.107:0/3347019273 learned_addr learned my addr 172.21.15.107:0/3347019273 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.245+0000 7fd105e00640 1 -- 172.21.15.107:0/3347019273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10819dc20 con 0x7fd10810ab40 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.246+0000 7fd105e00640 1 --2- 172.21.15.107:0/3347019273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10819d6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd0f802acc0 tx=0x7fd0f8031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.246+0000 7fd0ff400640 1 -- 172.21.15.107:0/3347019273 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0f8036050 con 0x7fd10810ab40 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.246+0000 7fd0ff400640 1 -- 172.21.15.107:0/3347019273 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd0f8037040 con 0x7fd10810ab40 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.246+0000 7fd0ff400640 1 -- 172.21.15.107:0/3347019273 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0f803d850 con 0x7fd10810ab40 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.246+0000 7fd107200640 1 -- 172.21.15.107:0/3347019273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10819deb0 con 0x7fd10810ab40 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.246+0000 7fd107200640 1 -- 172.21.15.107:0/3347019273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1081a0ba0 con 0x7fd10810ab40 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.247+0000 7fd0ff400640 1 -- 172.21.15.107:0/3347019273 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fd0f8044020 con 0x7fd10810ab40 2024-09-19T04:10:40.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.247+0000 7fd107200640 1 -- 172.21.15.107:0/3347019273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd10810dd20 con 0x7fd10810ab40 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.247+0000 7fd0ff400640 1 --2- 172.21.15.107:0/3347019273 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd0ec039200 0x7fd0ec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.248+0000 7fd0ff400640 1 -- 172.21.15.107:0/3347019273 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd0f8042020 con 0x7fd10810ab40 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.248+0000 7fd0fd400640 1 --2- 172.21.15.107:0/3347019273 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd0ec039200 0x7fd0ec03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.248+0000 7fd0fd400640 1 --2- 172.21.15.107:0/3347019273 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd0ec039200 0x7fd0ec03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00098c0 tx=0x7fd0f0006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.253+0000 7fd0ff400640 1 -- 172.21.15.107:0/3347019273 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0f804b020 con 0x7fd10810ab40 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.426+0000 7fd107200640 1 -- 172.21.15.107:0/3347019273 --> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd108107c90 con 0x7fd0ec039200 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.427+0000 7fd0ff400640 1 -- 172.21.15.107:0/3347019273 <== mgr.14114 v2:172.21.15.107:6800/972711045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd108107c90 con 0x7fd0ec039200 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.430+0000 7fd107200640 1 -- 172.21.15.107:0/3347019273 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd0ec039200 msgr2=0x7fd0ec03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.430+0000 7fd107200640 1 --2- 172.21.15.107:0/3347019273 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd0ec039200 0x7fd0ec03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00098c0 tx=0x7fd0f0006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.430+0000 7fd107200640 1 -- 172.21.15.107:0/3347019273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 msgr2=0x7fd10819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:40.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.430+0000 7fd107200640 1 --2- 172.21.15.107:0/3347019273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10819d6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd0f802acc0 tx=0x7fd0f8031ca0 comp rx=0 tx=0).stop 2024-09-19T04:10:40.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.430+0000 7fd107200640 1 -- 172.21.15.107:0/3347019273 shutdown_connections 2024-09-19T04:10:40.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.430+0000 7fd107200640 1 --2- 172.21.15.107:0/3347019273 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd0ec039200 0x7fd0ec03b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:40.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.430+0000 7fd107200640 1 --2- 172.21.15.107:0/3347019273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10810ab40 0x7fd10819d6e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:40.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.430+0000 7fd107200640 1 -- 172.21.15.107:0/3347019273 >> 172.21.15.107:0/3347019273 conn(0x7fd10807cd70 msgr2=0x7fd108107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:40.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.430+0000 7fd107200640 1 -- 172.21.15.107:0/3347019273 shutdown_connections 2024-09-19T04:10:40.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:39.430+0000 7fd107200640 1 -- 172.21.15.107:0/3347019273 wait complete. 2024-09-19T04:10:40.014 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:10:40.014 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T04:10:40.014 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2024-09-19T04:10:40.285 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:40 smithi107 ceph-mon[30659]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:41.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:41 smithi107 ceph-mon[30659]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:42.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:42 smithi107 ceph-mon[30659]: Deploying cephadm binary to smithi107 2024-09-19T04:10:45.423 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.500+0000 7f9e02200640 1 Processor -- start 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.500+0000 7f9e02200640 1 -- start start 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.501+0000 7f9e02200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc1063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.501+0000 7f9e02200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc07be10 con 0x7f9dfc105ff0 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.501+0000 7f9dfb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc1063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.501+0000 7f9dfb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc1063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58564/0 (socket says 172.21.15.107:58564) 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.501+0000 7f9dfb400640 1 -- 172.21.15.107:0/52547643 learned_addr learned my addr 172.21.15.107:0/52547643 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.501+0000 7f9dfb400640 1 -- 172.21.15.107:0/52547643 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dfc106930 con 0x7f9dfc105ff0 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.502+0000 7f9dfb400640 1 --2- 172.21.15.107:0/52547643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc1063f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009820 tx=0x7f9dec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900b0c804a99bdf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.503+0000 7f9dfaa00640 1 -- 172.21.15.107:0/52547643 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dec037070 con 0x7f9dfc105ff0 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.503+0000 7f9dfaa00640 1 -- 172.21.15.107:0/52547643 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9dec032900 con 0x7f9dfc105ff0 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.503+0000 7f9dfaa00640 1 -- 172.21.15.107:0/52547643 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dec030850 con 0x7f9dfc105ff0 2024-09-19T04:10:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.503+0000 7f9e02200640 1 -- 172.21.15.107:0/52547643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 msgr2=0x7f9dfc1063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.503+0000 7f9e02200640 1 --2- 172.21.15.107:0/52547643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc1063f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009820 tx=0x7f9dec02a270 comp rx=0 tx=0).stop 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.503+0000 7f9e02200640 1 -- 172.21.15.107:0/52547643 shutdown_connections 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.503+0000 7f9e02200640 1 --2- 172.21.15.107:0/52547643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc1063f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.503+0000 7f9e02200640 1 -- 172.21.15.107:0/52547643 >> 172.21.15.107:0/52547643 conn(0x7f9dfc101e20 msgr2=0x7f9dfc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.503+0000 7f9e02200640 1 -- 172.21.15.107:0/52547643 shutdown_connections 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.504+0000 7f9e02200640 1 -- 172.21.15.107:0/52547643 wait complete. 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.504+0000 7f9e02200640 1 Processor -- start 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.505+0000 7f9e02200640 1 -- start start 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.505+0000 7f9e02200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.505+0000 7f9e02200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc107a30 con 0x7f9dfc105ff0 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.505+0000 7f9dfb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.505+0000 7f9dfb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58572/0 (socket says 172.21.15.107:58572) 2024-09-19T04:10:45.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.505+0000 7f9dfb400640 1 -- 172.21.15.107:0/2278187403 learned_addr learned my addr 172.21.15.107:0/2278187403 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:45.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.505+0000 7f9dfb400640 1 -- 172.21.15.107:0/2278187403 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dfc19de00 con 0x7f9dfc105ff0 2024-09-19T04:10:45.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.506+0000 7f9dfb400640 1 --2- 172.21.15.107:0/2278187403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc19d8c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0097f0 tx=0x7f9dec030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:45.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.506+0000 7f9df8c00640 1 -- 172.21.15.107:0/2278187403 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dec042070 con 0x7f9dfc105ff0 2024-09-19T04:10:45.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.506+0000 7f9df8c00640 1 -- 172.21.15.107:0/2278187403 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9dec032ca0 con 0x7f9dfc105ff0 2024-09-19T04:10:45.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.506+0000 7f9df8c00640 1 -- 172.21.15.107:0/2278187403 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dec037040 con 0x7f9dfc105ff0 2024-09-19T04:10:45.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.506+0000 7f9e02200640 1 -- 172.21.15.107:0/2278187403 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dfc19e090 con 0x7f9dfc105ff0 2024-09-19T04:10:45.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.506+0000 7f9e02200640 1 -- 172.21.15.107:0/2278187403 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dfc1a0d80 con 0x7f9dfc105ff0 2024-09-19T04:10:45.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.507+0000 7f9df8c00640 1 -- 172.21.15.107:0/2278187403 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9dec045070 con 0x7f9dfc105ff0 2024-09-19T04:10:45.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.507+0000 7f9e02200640 1 -- 172.21.15.107:0/2278187403 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dfc1063f0 con 0x7f9dfc105ff0 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.507+0000 7f9df8c00640 1 --2- 172.21.15.107:0/2278187403 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f9de0039200 0x7f9de003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.508+0000 7f9df8c00640 1 -- 172.21.15.107:0/2278187403 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9dec06c2d0 con 0x7f9dfc105ff0 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.508+0000 7f9df2a00640 1 --2- 172.21.15.107:0/2278187403 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f9de0039200 0x7f9de003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.508+0000 7f9df2a00640 1 --2- 172.21.15.107:0/2278187403 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f9de0039200 0x7f9de003b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9de40098c0 tx=0x7f9de4006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.512+0000 7f9df8c00640 1 -- 172.21.15.107:0/2278187403 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9dec045360 con 0x7f9dfc105ff0 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:40.692+0000 7f9e02200640 1 -- 172.21.15.107:0/2278187403 --> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}) -- 0x7f9dfc02dab0 con 0x7f9de0039200 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.551+0000 7f9df8c00640 1 -- 172.21.15.107:0/2278187403 <== mgr.14114 v2:172.21.15.107:6800/972711045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9dfc02dab0 con 0x7f9de0039200 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.554+0000 7f9e02200640 1 -- 172.21.15.107:0/2278187403 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f9de0039200 msgr2=0x7f9de003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.554+0000 7f9e02200640 1 --2- 172.21.15.107:0/2278187403 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f9de0039200 0x7f9de003b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9de40098c0 tx=0x7f9de4006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.554+0000 7f9e02200640 1 -- 172.21.15.107:0/2278187403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 msgr2=0x7f9dfc19d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.554+0000 7f9e02200640 1 --2- 172.21.15.107:0/2278187403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc19d8c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0097f0 tx=0x7f9dec030ec0 comp rx=0 tx=0).stop 2024-09-19T04:10:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.555+0000 7f9e02200640 1 -- 172.21.15.107:0/2278187403 shutdown_connections 2024-09-19T04:10:45.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.555+0000 7f9e02200640 1 --2- 172.21.15.107:0/2278187403 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f9de0039200 0x7f9de003b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:45.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.555+0000 7f9e02200640 1 --2- 172.21.15.107:0/2278187403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dfc105ff0 0x7f9dfc19d8c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:45.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.555+0000 7f9e02200640 1 -- 172.21.15.107:0/2278187403 >> 172.21.15.107:0/2278187403 conn(0x7f9dfc101e20 msgr2=0x7f9dfc1945a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:45.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.555+0000 7f9e02200640 1 -- 172.21.15.107:0/2278187403 shutdown_connections 2024-09-19T04:10:45.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:44.555+0000 7f9e02200640 1 -- 172.21.15.107:0/2278187403 wait complete. 2024-09-19T04:10:45.428 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mon service... 2024-09-19T04:10:45.677 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:45 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:45.677 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:45 smithi107 ceph-mon[30659]: Added host smithi107 2024-09-19T04:10:45.678 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:45 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:47.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T04:10:47.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.987+0000 7fd257200640 1 Processor -- start 2024-09-19T04:10:47.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.987+0000 7fd257200640 1 -- start start 2024-09-19T04:10:47.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.988+0000 7fd257200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd258109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:47.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.988+0000 7fd257200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd25810a260 con 0x7fd258109870 2024-09-19T04:10:47.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.988+0000 7fd255e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd258109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:47.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.988+0000 7fd255e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd258109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58574/0 (socket says 172.21.15.107:58574) 2024-09-19T04:10:47.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.988+0000 7fd255e00640 1 -- 172.21.15.107:0/3924279158 learned_addr learned my addr 172.21.15.107:0/3924279158 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:47.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.988+0000 7fd255e00640 1 -- 172.21.15.107:0/3924279158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25810a930 con 0x7fd258109870 2024-09-19T04:10:47.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.989+0000 7fd255e00640 1 --2- 172.21.15.107:0/3924279158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd258109c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd248009820 tx=0x7fd24802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccd0bc6204d0e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:47.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.990+0000 7fd254a00640 1 -- 172.21.15.107:0/3924279158 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd248037070 con 0x7fd258109870 2024-09-19T04:10:47.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.990+0000 7fd254a00640 1 -- 172.21.15.107:0/3924279158 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd248032900 con 0x7fd258109870 2024-09-19T04:10:47.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.990+0000 7fd254a00640 1 -- 172.21.15.107:0/3924279158 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd248030850 con 0x7fd258109870 2024-09-19T04:10:47.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.990+0000 7fd257200640 1 -- 172.21.15.107:0/3924279158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 msgr2=0x7fd258109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:47.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.990+0000 7fd257200640 1 --2- 172.21.15.107:0/3924279158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd258109c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd248009820 tx=0x7fd24802a270 comp rx=0 tx=0).stop 2024-09-19T04:10:47.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.990+0000 7fd257200640 1 -- 172.21.15.107:0/3924279158 shutdown_connections 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.990+0000 7fd257200640 1 --2- 172.21.15.107:0/3924279158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd258109c90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.990+0000 7fd257200640 1 -- 172.21.15.107:0/3924279158 >> 172.21.15.107:0/3924279158 conn(0x7fd25807ce50 msgr2=0x7fd25807d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.990+0000 7fd257200640 1 -- 172.21.15.107:0/3924279158 shutdown_connections 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.990+0000 7fd257200640 1 -- 172.21.15.107:0/3924279158 wait complete. 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.991+0000 7fd257200640 1 Processor -- start 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.991+0000 7fd257200640 1 -- start start 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.992+0000 7fd257200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd25819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.992+0000 7fd257200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd25810dce0 con 0x7fd258109870 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.992+0000 7fd255e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd25819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.992+0000 7fd255e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd25819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58584/0 (socket says 172.21.15.107:58584) 2024-09-19T04:10:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.992+0000 7fd255e00640 1 -- 172.21.15.107:0/1162695790 learned_addr learned my addr 172.21.15.107:0/1162695790 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.992+0000 7fd255e00640 1 -- 172.21.15.107:0/1162695790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25819dbe0 con 0x7fd258109870 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.992+0000 7fd255e00640 1 --2- 172.21.15.107:0/1162695790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd25819d6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd2480097f0 tx=0x7fd248030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.993+0000 7fd24ea00640 1 -- 172.21.15.107:0/1162695790 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd248037040 con 0x7fd258109870 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.993+0000 7fd24ea00640 1 -- 172.21.15.107:0/1162695790 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd248040070 con 0x7fd258109870 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.993+0000 7fd257200640 1 -- 172.21.15.107:0/1162695790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25819de70 con 0x7fd258109870 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.993+0000 7fd24ea00640 1 -- 172.21.15.107:0/1162695790 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd24803a390 con 0x7fd258109870 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.993+0000 7fd257200640 1 -- 172.21.15.107:0/1162695790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2581a0b60 con 0x7fd258109870 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.994+0000 7fd24ea00640 1 -- 172.21.15.107:0/1162695790 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fd24803b430 con 0x7fd258109870 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.994+0000 7fd257200640 1 -- 172.21.15.107:0/1162695790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd258109c90 con 0x7fd258109870 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.994+0000 7fd24ea00640 1 --2- 172.21.15.107:0/1162695790 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd23c0391b0 0x7fd23c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:47.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.994+0000 7fd24ea00640 1 -- 172.21.15.107:0/1162695790 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd24806c850 con 0x7fd258109870 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.994+0000 7fd255400640 1 --2- 172.21.15.107:0/1162695790 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd23c0391b0 0x7fd23c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.995+0000 7fd255400640 1 --2- 172.21.15.107:0/1162695790 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd23c0391b0 0x7fd23c03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd2440098c0 tx=0x7fd244006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:45.999+0000 7fd24ea00640 1 -- 172.21.15.107:0/1162695790 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd24806c1a0 con 0x7fd258109870 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.209+0000 7fd257200640 1 -- 172.21.15.107:0/1162695790 --> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd258107b20 con 0x7fd23c0391b0 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.214+0000 7fd24ea00640 1 -- 172.21.15.107:0/1162695790 <== mgr.14114 v2:172.21.15.107:6800/972711045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd258107b20 con 0x7fd23c0391b0 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.216+0000 7fd257200640 1 -- 172.21.15.107:0/1162695790 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd23c0391b0 msgr2=0x7fd23c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.216+0000 7fd257200640 1 --2- 172.21.15.107:0/1162695790 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd23c0391b0 0x7fd23c03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd2440098c0 tx=0x7fd244006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.216+0000 7fd257200640 1 -- 172.21.15.107:0/1162695790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 msgr2=0x7fd25819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.216+0000 7fd257200640 1 --2- 172.21.15.107:0/1162695790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd25819d6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd2480097f0 tx=0x7fd248030d80 comp rx=0 tx=0).stop 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.216+0000 7fd257200640 1 -- 172.21.15.107:0/1162695790 shutdown_connections 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.216+0000 7fd257200640 1 --2- 172.21.15.107:0/1162695790 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fd23c0391b0 0x7fd23c03b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:47.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.216+0000 7fd257200640 1 --2- 172.21.15.107:0/1162695790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd258109870 0x7fd25819d6a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:47.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.216+0000 7fd257200640 1 -- 172.21.15.107:0/1162695790 >> 172.21.15.107:0/1162695790 conn(0x7fd25807ce50 msgr2=0x7fd258107410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:47.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.217+0000 7fd257200640 1 -- 172.21.15.107:0/1162695790 shutdown_connections 2024-09-19T04:10:47.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:46.217+0000 7fd257200640 1 -- 172.21.15.107:0/1162695790 wait complete. 2024-09-19T04:10:47.392 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mgr service... 2024-09-19T04:10:47.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:47 smithi107 ceph-mon[30659]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:47.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:47 smithi107 ceph-mon[30659]: Saving service mon spec with placement count:5 2024-09-19T04:10:47.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:47 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:49.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T04:10:49.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.943+0000 7f0391800640 1 Processor -- start 2024-09-19T04:10:49.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.943+0000 7f0391800640 1 -- start start 2024-09-19T04:10:49.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.943+0000 7f0391800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:49.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.943+0000 7f0391800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f038c07c8f0 con 0x7f038c07de70 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.944+0000 7f038aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.944+0000 7f038aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58600/0 (socket says 172.21.15.107:58600) 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.944+0000 7f038aa00640 1 -- 172.21.15.107:0/2015567470 learned_addr learned my addr 172.21.15.107:0/2015567470 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.944+0000 7f038aa00640 1 -- 172.21.15.107:0/2015567470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038c07ca70 con 0x7f038c07de70 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.945+0000 7f038aa00640 1 --2- 172.21.15.107:0/2015567470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c07c320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f037c009820 tx=0x7f037c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e5bc840de26012 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.945+0000 7f0389600640 1 -- 172.21.15.107:0/2015567470 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f037c037070 con 0x7f038c07de70 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.945+0000 7f0389600640 1 -- 172.21.15.107:0/2015567470 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f037c032900 con 0x7f038c07de70 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.945+0000 7f0389600640 1 -- 172.21.15.107:0/2015567470 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f037c030850 con 0x7f038c07de70 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.946+0000 7f0391800640 1 -- 172.21.15.107:0/2015567470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 msgr2=0x7f038c07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.946+0000 7f0391800640 1 --2- 172.21.15.107:0/2015567470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c07c320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f037c009820 tx=0x7f037c02a270 comp rx=0 tx=0).stop 2024-09-19T04:10:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.946+0000 7f0391800640 1 -- 172.21.15.107:0/2015567470 shutdown_connections 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.946+0000 7f0391800640 1 --2- 172.21.15.107:0/2015567470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c07c320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.946+0000 7f0391800640 1 -- 172.21.15.107:0/2015567470 >> 172.21.15.107:0/2015567470 conn(0x7f038c102ef0 msgr2=0x7f038c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.946+0000 7f0391800640 1 -- 172.21.15.107:0/2015567470 shutdown_connections 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.946+0000 7f0391800640 1 -- 172.21.15.107:0/2015567470 wait complete. 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.947+0000 7f0391800640 1 Processor -- start 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.947+0000 7f0391800640 1 -- start start 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.947+0000 7f0391800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.947+0000 7f0391800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f038c10dd30 con 0x7f038c07de70 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.948+0000 7f038aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.948+0000 7f038aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58612/0 (socket says 172.21.15.107:58612) 2024-09-19T04:10:49.002 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.948+0000 7f038aa00640 1 -- 172.21.15.107:0/565903656 learned_addr learned my addr 172.21.15.107:0/565903656 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.948+0000 7f038aa00640 1 -- 172.21.15.107:0/565903656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038c19de40 con 0x7f038c07de70 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.948+0000 7f038aa00640 1 --2- 172.21.15.107:0/565903656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c19d900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f037c0097f0 tx=0x7f037c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.949+0000 7f0383400640 1 -- 172.21.15.107:0/565903656 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f037c042070 con 0x7f038c07de70 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.949+0000 7f0383400640 1 -- 172.21.15.107:0/565903656 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f037c032ca0 con 0x7f038c07de70 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.949+0000 7f0383400640 1 -- 172.21.15.107:0/565903656 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f037c037040 con 0x7f038c07de70 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.949+0000 7f0391800640 1 -- 172.21.15.107:0/565903656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038c19e0d0 con 0x7f038c07de70 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.949+0000 7f0391800640 1 -- 172.21.15.107:0/565903656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f038c1a0dc0 con 0x7f038c07de70 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.950+0000 7f0383400640 1 -- 172.21.15.107:0/565903656 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f037c045070 con 0x7f038c07de70 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.950+0000 7f0391800640 1 -- 172.21.15.107:0/565903656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f038c07c3a0 con 0x7f038c07de70 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.950+0000 7f0383400640 1 --2- 172.21.15.107:0/565903656 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f0370039200 0x7f037003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.950+0000 7f0383400640 1 -- 172.21.15.107:0/565903656 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f037c06c1f0 con 0x7f038c07de70 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.951+0000 7f038a000640 1 --2- 172.21.15.107:0/565903656 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f0370039200 0x7f037003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.951+0000 7f038a000640 1 --2- 172.21.15.107:0/565903656 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f0370039200 0x7f037003b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f03780098c0 tx=0x7f0378006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:47.956+0000 7f0383400640 1 -- 172.21.15.107:0/565903656 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f037c045360 con 0x7f038c07de70 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.149+0000 7f0391800640 1 -- 172.21.15.107:0/565903656 --> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f038c104190 con 0x7f0370039200 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.154+0000 7f0383400640 1 -- 172.21.15.107:0/565903656 <== mgr.14114 v2:172.21.15.107:6800/972711045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f038c104190 con 0x7f0370039200 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.156+0000 7f0391800640 1 -- 172.21.15.107:0/565903656 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f0370039200 msgr2=0x7f037003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.156+0000 7f0391800640 1 --2- 172.21.15.107:0/565903656 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f0370039200 0x7f037003b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f03780098c0 tx=0x7f0378006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.156+0000 7f0391800640 1 -- 172.21.15.107:0/565903656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 msgr2=0x7f038c19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.156+0000 7f0391800640 1 --2- 172.21.15.107:0/565903656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c19d900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f037c0097f0 tx=0x7f037c030ec0 comp rx=0 tx=0).stop 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.157+0000 7f0391800640 1 -- 172.21.15.107:0/565903656 shutdown_connections 2024-09-19T04:10:49.004 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.157+0000 7f0391800640 1 --2- 172.21.15.107:0/565903656 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f0370039200 0x7f037003b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:49.005 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.157+0000 7f0391800640 1 --2- 172.21.15.107:0/565903656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f038c07de70 0x7f038c19d900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:49.005 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.157+0000 7f0391800640 1 -- 172.21.15.107:0/565903656 >> 172.21.15.107:0/565903656 conn(0x7f038c102ef0 msgr2=0x7f038c103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:49.005 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.157+0000 7f0391800640 1 -- 172.21.15.107:0/565903656 shutdown_connections 2024-09-19T04:10:49.005 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:48.157+0000 7f0391800640 1 -- 172.21.15.107:0/565903656 wait complete. 2024-09-19T04:10:49.279 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:49 smithi107 ceph-mon[30659]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:10:49.279 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:49 smithi107 ceph-mon[30659]: Saving service mgr spec with placement count:2 2024-09-19T04:10:49.279 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:49 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:49.279 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:49 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:49.279 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:49 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:50.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.493+0000 7f539f200640 1 Processor -- start 2024-09-19T04:10:50.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.493+0000 7f539f200640 1 -- start start 2024-09-19T04:10:50.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.493+0000 7f539f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a0107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.493+0000 7f539f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0107f30 con 0x7f53a0107560 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.494+0000 7f539de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a0107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.494+0000 7f539de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a0107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40556/0 (socket says 172.21.15.107:40556) 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.494+0000 7f539de00640 1 -- 172.21.15.107:0/3468790475 learned_addr learned my addr 172.21.15.107:0/3468790475 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.494+0000 7f539de00640 1 -- 172.21.15.107:0/3468790475 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a0108710 con 0x7f53a0107560 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.495+0000 7f539de00640 1 --2- 172.21.15.107:0/3468790475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a0107960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5390009820 tx=0x7f539002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3031089885933a50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.496+0000 7f539ca00640 1 -- 172.21.15.107:0/3468790475 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5390037070 con 0x7f53a0107560 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.496+0000 7f539ca00640 1 -- 172.21.15.107:0/3468790475 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5390032920 con 0x7f53a0107560 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.496+0000 7f539ca00640 1 -- 172.21.15.107:0/3468790475 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5390030810 con 0x7f53a0107560 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.496+0000 7f539f200640 1 -- 172.21.15.107:0/3468790475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 msgr2=0x7f53a0107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.496+0000 7f539f200640 1 --2- 172.21.15.107:0/3468790475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a0107960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5390009820 tx=0x7f539002a270 comp rx=0 tx=0).stop 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.496+0000 7f539f200640 1 -- 172.21.15.107:0/3468790475 shutdown_connections 2024-09-19T04:10:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.496+0000 7f539f200640 1 --2- 172.21.15.107:0/3468790475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a0107960 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.496+0000 7f539f200640 1 -- 172.21.15.107:0/3468790475 >> 172.21.15.107:0/3468790475 conn(0x7f53a0102cf0 msgr2=0x7f53a0105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.496+0000 7f539f200640 1 -- 172.21.15.107:0/3468790475 shutdown_connections 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.497+0000 7f539f200640 1 -- 172.21.15.107:0/3468790475 wait complete. 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.497+0000 7f539f200640 1 Processor -- start 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.498+0000 7f539f200640 1 -- start start 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.498+0000 7f539f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.498+0000 7f539f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53a010bac0 con 0x7f53a0107560 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.498+0000 7f539de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.498+0000 7f539de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40572/0 (socket says 172.21.15.107:40572) 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.498+0000 7f539de00640 1 -- 172.21.15.107:0/4103761461 learned_addr learned my addr 172.21.15.107:0/4103761461 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.498+0000 7f539de00640 1 -- 172.21.15.107:0/4103761461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a007e0b0 con 0x7f53a0107560 2024-09-19T04:10:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.499+0000 7f539de00640 1 --2- 172.21.15.107:0/4103761461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a007db70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5390030560 tx=0x7f5390032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.499+0000 7f5396a00640 1 -- 172.21.15.107:0/4103761461 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5390037040 con 0x7f53a0107560 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.499+0000 7f5396a00640 1 -- 172.21.15.107:0/4103761461 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5390040070 con 0x7f53a0107560 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.499+0000 7f539f200640 1 -- 172.21.15.107:0/4103761461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a007c2a0 con 0x7f53a0107560 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.499+0000 7f5396a00640 1 -- 172.21.15.107:0/4103761461 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f539003a740 con 0x7f53a0107560 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.499+0000 7f539f200640 1 -- 172.21.15.107:0/4103761461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a007c7e0 con 0x7f53a0107560 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.500+0000 7f5396a00640 1 -- 172.21.15.107:0/4103761461 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f539003a9b0 con 0x7f53a0107560 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.501+0000 7f5396a00640 1 --2- 172.21.15.107:0/4103761461 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f53840391b0 0x7f538403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.501+0000 7f5396a00640 1 -- 172.21.15.107:0/4103761461 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f539006c960 con 0x7f53a0107560 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.501+0000 7f539f200640 1 -- 172.21.15.107:0/4103761461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a01079e0 con 0x7f53a0107560 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.501+0000 7f539d400640 1 --2- 172.21.15.107:0/4103761461 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f53840391b0 0x7f538403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.502+0000 7f539d400640 1 --2- 172.21.15.107:0/4103761461 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f53840391b0 0x7f538403b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f538c0098c0 tx=0x7f538c006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.506+0000 7f5396a00640 1 -- 172.21.15.107:0/4103761461 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5390041270 con 0x7f53a0107560 2024-09-19T04:10:50.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.711+0000 7f539f200640 1 -- 172.21.15.107:0/4103761461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f53a007d0c0 con 0x7f53a0107560 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.714+0000 7f5396a00640 1 -- 172.21.15.107:0/4103761461 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f53900328f0 con 0x7f53a0107560 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.717+0000 7f539f200640 1 -- 172.21.15.107:0/4103761461 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f53840391b0 msgr2=0x7f538403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.717+0000 7f539f200640 1 --2- 172.21.15.107:0/4103761461 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f53840391b0 0x7f538403b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f538c0098c0 tx=0x7f538c006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.717+0000 7f539f200640 1 -- 172.21.15.107:0/4103761461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 msgr2=0x7f53a007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.717+0000 7f539f200640 1 --2- 172.21.15.107:0/4103761461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a007db70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5390030560 tx=0x7f5390032670 comp rx=0 tx=0).stop 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.718+0000 7f539f200640 1 -- 172.21.15.107:0/4103761461 shutdown_connections 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.718+0000 7f539f200640 1 --2- 172.21.15.107:0/4103761461 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f53840391b0 0x7f538403b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.718+0000 7f539f200640 1 --2- 172.21.15.107:0/4103761461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a0107560 0x7f53a007db70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.718+0000 7f539f200640 1 -- 172.21.15.107:0/4103761461 >> 172.21.15.107:0/4103761461 conn(0x7f53a0102cf0 msgr2=0x7f53a01038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.718+0000 7f539f200640 1 -- 172.21.15.107:0/4103761461 shutdown_connections 2024-09-19T04:10:50.798 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:49.718+0000 7f539f200640 1 -- 172.21.15.107:0/4103761461 wait complete. 2024-09-19T04:10:50.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:50 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/4103761461' entity='client.admin' 2024-09-19T04:10:52.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:51 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:52.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:51 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/416224619' entity='client.admin' 2024-09-19T04:10:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.352+0000 7f550d600640 1 Processor -- start 2024-09-19T04:10:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.353+0000 7f550d600640 1 -- start start 2024-09-19T04:10:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.353+0000 7f550d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f5508109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.353+0000 7f550d600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f550810a330 con 0x7f5508109960 2024-09-19T04:10:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.353+0000 7f5506a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f5508109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.354+0000 7f5506a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f5508109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40582/0 (socket says 172.21.15.107:40582) 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.354+0000 7f5506a00640 1 -- 172.21.15.107:0/620037223 learned_addr learned my addr 172.21.15.107:0/620037223 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.354+0000 7f5506a00640 1 -- 172.21.15.107:0/620037223 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550810aad0 con 0x7f5508109960 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.355+0000 7f5506a00640 1 --2- 172.21.15.107:0/620037223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f5508109d60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009820 tx=0x7f54f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff697ff6d1010cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.355+0000 7f5506000640 1 -- 172.21.15.107:0/620037223 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54f8037070 con 0x7f5508109960 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.355+0000 7f5506000640 1 -- 172.21.15.107:0/620037223 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f8032920 con 0x7f5508109960 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.355+0000 7f5506000640 1 -- 172.21.15.107:0/620037223 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54f8030810 con 0x7f5508109960 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.356+0000 7f550d600640 1 -- 172.21.15.107:0/620037223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 msgr2=0x7f5508109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.356+0000 7f550d600640 1 --2- 172.21.15.107:0/620037223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f5508109d60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009820 tx=0x7f54f802a270 comp rx=0 tx=0).stop 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.356+0000 7f550d600640 1 -- 172.21.15.107:0/620037223 shutdown_connections 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.356+0000 7f550d600640 1 --2- 172.21.15.107:0/620037223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f5508109d60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.356+0000 7f550d600640 1 -- 172.21.15.107:0/620037223 >> 172.21.15.107:0/620037223 conn(0x7f550807cf30 msgr2=0x7f550807d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.356+0000 7f550d600640 1 -- 172.21.15.107:0/620037223 shutdown_connections 2024-09-19T04:10:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.356+0000 7f550d600640 1 -- 172.21.15.107:0/620037223 wait complete. 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.357+0000 7f550d600640 1 Processor -- start 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.357+0000 7f550d600640 1 -- start start 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.357+0000 7f550d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f550819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.357+0000 7f550d600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f550810de80 con 0x7f5508109960 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.358+0000 7f5506a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f550819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.358+0000 7f5506a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f550819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40596/0 (socket says 172.21.15.107:40596) 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.358+0000 7f5506a00640 1 -- 172.21.15.107:0/416224619 learned_addr learned my addr 172.21.15.107:0/416224619 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.358+0000 7f5506a00640 1 -- 172.21.15.107:0/416224619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550819de80 con 0x7f5508109960 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.358+0000 7f5506a00640 1 --2- 172.21.15.107:0/416224619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f550819d940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f54f8030560 tx=0x7f54f8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.359+0000 7f54ffe00640 1 -- 172.21.15.107:0/416224619 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54f8037040 con 0x7f5508109960 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.359+0000 7f54ffe00640 1 -- 172.21.15.107:0/416224619 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f8040070 con 0x7f5508109960 2024-09-19T04:10:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.359+0000 7f550d600640 1 -- 172.21.15.107:0/416224619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550819e110 con 0x7f5508109960 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.359+0000 7f54ffe00640 1 -- 172.21.15.107:0/416224619 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54f803a6d0 con 0x7f5508109960 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.359+0000 7f550d600640 1 -- 172.21.15.107:0/416224619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55081a0e00 con 0x7f5508109960 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.360+0000 7f54ffe00640 1 -- 172.21.15.107:0/416224619 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f54f803bb20 con 0x7f5508109960 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.360+0000 7f550d600640 1 -- 172.21.15.107:0/416224619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f550810e000 con 0x7f5508109960 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.360+0000 7f54ffe00640 1 --2- 172.21.15.107:0/416224619 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f54ec039200 0x7f54ec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.360+0000 7f54ffe00640 1 -- 172.21.15.107:0/416224619 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f54f806c870 con 0x7f5508109960 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.361+0000 7f54fe000640 1 --2- 172.21.15.107:0/416224619 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f54ec039200 0x7f54ec03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.361+0000 7f54fe000640 1 --2- 172.21.15.107:0/416224619 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f54ec039200 0x7f54ec03b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f54f00098c0 tx=0x7f54f0006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.366+0000 7f54ffe00640 1 -- 172.21.15.107:0/416224619 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f54f80328f0 con 0x7f5508109960 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.546+0000 7f550d600640 1 -- 172.21.15.107:0/416224619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f5508109de0 con 0x7f5508109960 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.548+0000 7f54ffe00640 1 -- 172.21.15.107:0/416224619 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f54f804a200 con 0x7f5508109960 2024-09-19T04:10:52.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.551+0000 7f550d600640 1 -- 172.21.15.107:0/416224619 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f54ec039200 msgr2=0x7f54ec03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:52.124 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.551+0000 7f550d600640 1 --2- 172.21.15.107:0/416224619 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f54ec039200 0x7f54ec03b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f54f00098c0 tx=0x7f54f0006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:52.124 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.551+0000 7f550d600640 1 -- 172.21.15.107:0/416224619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 msgr2=0x7f550819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:52.124 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.551+0000 7f550d600640 1 --2- 172.21.15.107:0/416224619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f550819d940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f54f8030560 tx=0x7f54f8032670 comp rx=0 tx=0).stop 2024-09-19T04:10:52.124 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.551+0000 7f550d600640 1 -- 172.21.15.107:0/416224619 shutdown_connections 2024-09-19T04:10:52.124 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.551+0000 7f550d600640 1 --2- 172.21.15.107:0/416224619 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f54ec039200 0x7f54ec03b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:52.124 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.551+0000 7f550d600640 1 --2- 172.21.15.107:0/416224619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5508109960 0x7f550819d940 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:52.124 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.551+0000 7f550d600640 1 -- 172.21.15.107:0/416224619 >> 172.21.15.107:0/416224619 conn(0x7f550807cf30 msgr2=0x7f5508107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:52.124 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.551+0000 7f550d600640 1 -- 172.21.15.107:0/416224619 shutdown_connections 2024-09-19T04:10:52.124 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:51.551+0000 7f550d600640 1 -- 172.21.15.107:0/416224619 wait complete. 2024-09-19T04:10:52.124 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2024-09-19T04:10:53.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:53 smithi107 ceph-mon[30659]: from='mgr.14114 172.21.15.107:0/68151215' entity='mgr.x' 2024-09-19T04:10:53.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:53 smithi107 ceph-mon[30659]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:10:53.854 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:53 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1364349651' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T04:10:54.112 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:53 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:54.113 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:53 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:54.113 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:54.006+0000 7f59599d3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:10:54.113 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:54.077+0000 7f59599d3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:10:54.411 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:54.276+0000 7f59599d3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:10:54.411 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:54.358+0000 7f59599d3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:10:54.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.587+0000 7faf2c000640 1 Processor -- start 2024-09-19T04:10:54.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.587+0000 7faf2c000640 1 -- start start 2024-09-19T04:10:54.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.587+0000 7faf2c000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:54.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.587+0000 7faf2c000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf2407c8f0 con 0x7faf2407de70 2024-09-19T04:10:54.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.588+0000 7faf29600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:54.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.588+0000 7faf29600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40608/0 (socket says 172.21.15.107:40608) 2024-09-19T04:10:54.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.588+0000 7faf29600640 1 -- 172.21.15.107:0/599811277 learned_addr learned my addr 172.21.15.107:0/599811277 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:54.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.588+0000 7faf29600640 1 -- 172.21.15.107:0/599811277 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2407ca70 con 0x7faf2407de70 2024-09-19T04:10:54.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.589+0000 7faf29600640 1 --2- 172.21.15.107:0/599811277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2407c320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf18009820 tx=0x7faf1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1691e74b332613 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:54.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.589+0000 7faf23e00640 1 -- 172.21.15.107:0/599811277 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf18037070 con 0x7faf2407de70 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.589+0000 7faf23e00640 1 -- 172.21.15.107:0/599811277 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf18032920 con 0x7faf2407de70 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.589+0000 7faf23e00640 1 -- 172.21.15.107:0/599811277 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf18030810 con 0x7faf2407de70 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.590+0000 7faf2c000640 1 -- 172.21.15.107:0/599811277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 msgr2=0x7faf2407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.590+0000 7faf2c000640 1 --2- 172.21.15.107:0/599811277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2407c320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf18009820 tx=0x7faf1802a270 comp rx=0 tx=0).stop 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.590+0000 7faf2c000640 1 -- 172.21.15.107:0/599811277 shutdown_connections 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.590+0000 7faf2c000640 1 --2- 172.21.15.107:0/599811277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2407c320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.590+0000 7faf2c000640 1 -- 172.21.15.107:0/599811277 >> 172.21.15.107:0/599811277 conn(0x7faf24102ef0 msgr2=0x7faf24105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.590+0000 7faf2c000640 1 -- 172.21.15.107:0/599811277 shutdown_connections 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.590+0000 7faf2c000640 1 -- 172.21.15.107:0/599811277 wait complete. 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.591+0000 7faf2c000640 1 Processor -- start 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.591+0000 7faf2c000640 1 -- start start 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.592+0000 7faf2c000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:54.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.592+0000 7faf2c000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf2410dd90 con 0x7faf2407de70 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.592+0000 7faf29600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.592+0000 7faf29600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2419d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40616/0 (socket says 172.21.15.107:40616) 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.592+0000 7faf29600640 1 -- 172.21.15.107:0/1364349651 learned_addr learned my addr 172.21.15.107:0/1364349651 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.592+0000 7faf29600640 1 -- 172.21.15.107:0/1364349651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2419de50 con 0x7faf2407de70 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.593+0000 7faf29600640 1 --2- 172.21.15.107:0/1364349651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2419d910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faf180097f0 tx=0x7faf180327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.593+0000 7faf22000640 1 -- 172.21.15.107:0/1364349651 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf18037040 con 0x7faf2407de70 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.593+0000 7faf22000640 1 -- 172.21.15.107:0/1364349651 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf18030c40 con 0x7faf2407de70 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.593+0000 7faf2c000640 1 -- 172.21.15.107:0/1364349651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2419e0e0 con 0x7faf2407de70 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.593+0000 7faf22000640 1 -- 172.21.15.107:0/1364349651 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf1803a5d0 con 0x7faf2407de70 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.593+0000 7faf2c000640 1 -- 172.21.15.107:0/1364349651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf241a0dd0 con 0x7faf2407de70 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.594+0000 7faf2c000640 1 -- 172.21.15.107:0/1364349651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf2407c3a0 con 0x7faf2407de70 2024-09-19T04:10:54.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.594+0000 7faf22000640 1 -- 172.21.15.107:0/1364349651 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7faf1803a7c0 con 0x7faf2407de70 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.595+0000 7faf22000640 1 --2- 172.21.15.107:0/1364349651 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7faf0c039200 0x7faf0c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.595+0000 7faf22000640 1 -- 172.21.15.107:0/1364349651 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faf1806c940 con 0x7faf2407de70 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.595+0000 7faf28c00640 1 --2- 172.21.15.107:0/1364349651 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7faf0c039200 0x7faf0c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.596+0000 7faf28c00640 1 --2- 172.21.15.107:0/1364349651 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7faf0c039200 0x7faf0c03b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf100098c0 tx=0x7faf10006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.600+0000 7faf22000640 1 -- 172.21.15.107:0/1364349651 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf1803b770 con 0x7faf2407de70 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:52.846+0000 7faf2c000640 1 -- 172.21.15.107:0/1364349651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7faf241a0fa0 con 0x7faf2407de70 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.768+0000 7faf22000640 1 -- 172.21.15.107:0/1364349651 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7faf1806c290 con 0x7faf2407de70 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.769+0000 7faf22000640 1 -- 172.21.15.107:0/1364349651 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7faf18044400 con 0x7faf2407de70 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.771+0000 7faf2c000640 1 -- 172.21.15.107:0/1364349651 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7faf0c039200 msgr2=0x7faf0c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.771+0000 7faf2c000640 1 --2- 172.21.15.107:0/1364349651 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7faf0c039200 0x7faf0c03b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf100098c0 tx=0x7faf10006db0 comp rx=0 tx=0).stop 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.771+0000 7faf2c000640 1 -- 172.21.15.107:0/1364349651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 msgr2=0x7faf2419d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.771+0000 7faf2c000640 1 --2- 172.21.15.107:0/1364349651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2419d910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faf180097f0 tx=0x7faf180327b0 comp rx=0 tx=0).stop 2024-09-19T04:10:54.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.771+0000 7faf2c000640 1 -- 172.21.15.107:0/1364349651 shutdown_connections 2024-09-19T04:10:54.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.771+0000 7faf2c000640 1 --2- 172.21.15.107:0/1364349651 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7faf0c039200 0x7faf0c03b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:54.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.771+0000 7faf2c000640 1 --2- 172.21.15.107:0/1364349651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf2407de70 0x7faf2419d910 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:54.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.771+0000 7faf2c000640 1 -- 172.21.15.107:0/1364349651 >> 172.21.15.107:0/1364349651 conn(0x7faf24102ef0 msgr2=0x7faf24103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:54.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.771+0000 7faf2c000640 1 -- 172.21.15.107:0/1364349651 shutdown_connections 2024-09-19T04:10:54.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:53.771+0000 7faf2c000640 1 -- 172.21.15.107:0/1364349651 wait complete. 2024-09-19T04:10:54.699 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:54.540+0000 7f59599d3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:10:54.980 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:54.706+0000 7f59599d3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:10:54.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:54 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1364349651' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T04:10:54.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:54 smithi107 ceph-mon[30659]: mgrmap e9: x(active, since 21s) 2024-09-19T04:10:55.248 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:55 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:55.093+0000 7f59599d3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:10:55.249 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:55 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:55.165+0000 7f59599d3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:10:55.606 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:55 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:55.395+0000 7f59599d3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:10:55.916 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:55 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1715511470' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T04:10:55.950 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:10:55.950 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T04:10:55.950 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.005+0000 7f7b61c00640 1 Processor -- start 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.005+0000 7f7b61c00640 1 -- start start 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.006+0000 7f7b61c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c10a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.006+0000 7f7b61c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c07bdd0 con 0x7f7b5c108040 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.006+0000 7f7b5be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.006+0000 7f7b5be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c10a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40644/0 (socket says 172.21.15.107:40644) 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.006+0000 7f7b5be00640 1 -- 172.21.15.107:0/3733495211 learned_addr learned my addr 172.21.15.107:0/3733495211 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.007+0000 7f7b5be00640 1 -- 172.21.15.107:0/3733495211 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c10a970 con 0x7f7b5c108040 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.008+0000 7f7b5be00640 1 --2- 172.21.15.107:0/3733495211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c10a430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009820 tx=0x7f7b4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4334d8cd7bb6d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.008+0000 7f7b5b400640 1 -- 172.21.15.107:0/3733495211 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b4c037070 con 0x7f7b5c108040 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.008+0000 7f7b5b400640 1 -- 172.21.15.107:0/3733495211 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4c032920 con 0x7f7b5c108040 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.008+0000 7f7b5b400640 1 -- 172.21.15.107:0/3733495211 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b4c030810 con 0x7f7b5c108040 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.009+0000 7f7b61c00640 1 -- 172.21.15.107:0/3733495211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 msgr2=0x7f7b5c10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.009+0000 7f7b61c00640 1 --2- 172.21.15.107:0/3733495211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c10a430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009820 tx=0x7f7b4c02a270 comp rx=0 tx=0).stop 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.009+0000 7f7b61c00640 1 -- 172.21.15.107:0/3733495211 shutdown_connections 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.009+0000 7f7b61c00640 1 --2- 172.21.15.107:0/3733495211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c10a430 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.009+0000 7f7b61c00640 1 -- 172.21.15.107:0/3733495211 >> 172.21.15.107:0/3733495211 conn(0x7f7b5c101c40 msgr2=0x7f7b5c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.009+0000 7f7b61c00640 1 -- 172.21.15.107:0/3733495211 shutdown_connections 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.009+0000 7f7b61c00640 1 -- 172.21.15.107:0/3733495211 wait complete. 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.010+0000 7f7b61c00640 1 Processor -- start 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.010+0000 7f7b61c00640 1 -- start start 2024-09-19T04:10:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.010+0000 7f7b61c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c1078c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.010+0000 7f7b61c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c10b930 con 0x7f7b5c108040 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.011+0000 7f7b5be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c1078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.011+0000 7f7b5be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c1078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40646/0 (socket says 172.21.15.107:40646) 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.011+0000 7f7b5be00640 1 -- 172.21.15.107:0/1715511470 learned_addr learned my addr 172.21.15.107:0/1715511470 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.011+0000 7f7b5be00640 1 -- 172.21.15.107:0/1715511470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c107e00 con 0x7f7b5c108040 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.011+0000 7f7b5be00640 1 --2- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c1078c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0097f0 tx=0x7f7b4c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.011+0000 7f7b59600640 1 -- 172.21.15.107:0/1715511470 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b4c037040 con 0x7f7b5c108040 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.011+0000 7f7b59600640 1 -- 172.21.15.107:0/1715511470 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4c030cc0 con 0x7f7b5c108040 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.011+0000 7f7b61c00640 1 -- 172.21.15.107:0/1715511470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5c105ff0 con 0x7f7b5c108040 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.012+0000 7f7b59600640 1 -- 172.21.15.107:0/1715511470 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b4c03a6b0 con 0x7f7b5c108040 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.012+0000 7f7b61c00640 1 -- 172.21.15.107:0/1715511470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5c106530 con 0x7f7b5c108040 2024-09-19T04:10:55.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.013+0000 7f7b59600640 1 -- 172.21.15.107:0/1715511470 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f7b4c03a850 con 0x7f7b5c108040 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.013+0000 7f7b61c00640 1 -- 172.21.15.107:0/1715511470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5c1084a0 con 0x7f7b5c108040 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.013+0000 7f7b59600640 1 --2- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7b40039260 0x7f7b4003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.013+0000 7f7b59600640 1 -- 172.21.15.107:0/1715511470 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7b4c06c9f0 con 0x7f7b5c108040 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.014+0000 7f7b53400640 1 -- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7b40039260 msgr2=0x7f7b4003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/972711045 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.014+0000 7f7b53400640 1 --2- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7b40039260 0x7f7b4003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.018+0000 7f7b59600640 1 -- 172.21.15.107:0/1715511470 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b4c06c340 con 0x7f7b5c108040 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.215+0000 7f7b53400640 1 -- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7b40039260 msgr2=0x7f7b4003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/972711045 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.215+0000 7f7b53400640 1 --2- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7b40039260 0x7f7b4003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.246+0000 7f7b61c00640 1 -- 172.21.15.107:0/1715511470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7b5c1067b0 con 0x7f7b5c108040 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.247+0000 7f7b59600640 1 -- 172.21.15.107:0/1715511470 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f7b4c044920 con 0x7f7b5c108040 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.249+0000 7f7b61c00640 1 -- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7b40039260 msgr2=0x7f7b4003b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:10:55.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.249+0000 7f7b61c00640 1 --2- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7b40039260 0x7f7b4003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:55.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.249+0000 7f7b61c00640 1 -- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 msgr2=0x7f7b5c1078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:55.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.249+0000 7f7b61c00640 1 --2- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c1078c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0097f0 tx=0x7f7b4c0327b0 comp rx=0 tx=0).stop 2024-09-19T04:10:55.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.250+0000 7f7b61c00640 1 -- 172.21.15.107:0/1715511470 shutdown_connections 2024-09-19T04:10:55.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.250+0000 7f7b61c00640 1 --2- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7f7b40039260 0x7f7b4003b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:55.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.250+0000 7f7b61c00640 1 --2- 172.21.15.107:0/1715511470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b5c108040 0x7f7b5c1078c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:55.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.250+0000 7f7b61c00640 1 -- 172.21.15.107:0/1715511470 >> 172.21.15.107:0/1715511470 conn(0x7f7b5c101c40 msgr2=0x7f7b5c1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:55.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.250+0000 7f7b61c00640 1 -- 172.21.15.107:0/1715511470 shutdown_connections 2024-09-19T04:10:55.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:55.250+0000 7f7b61c00640 1 -- 172.21.15.107:0/1715511470 wait complete. 2024-09-19T04:10:55.955 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-09-19T04:10:55.955 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 9... 2024-09-19T04:10:56.825 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:56 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:56.570+0000 7f59599d3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:10:56.825 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:56 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:56.652+0000 7f59599d3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:10:57.846 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:57 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:57.568+0000 7f59599d3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:10:57.846 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:57 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:57.703+0000 7f59599d3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:10:58.106 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:57 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:57.864+0000 7f59599d3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:10:58.106 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:57 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:57.935+0000 7f59599d3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:10:58.106 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:58.104+0000 7f59599d3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:10:58.433 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:58.185+0000 7f59599d3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:10:58.433 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:58.255+0000 7f59599d3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:10:58.433 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:58.431+0000 7f59599d3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:10:58.730 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:58.508+0000 7f59599d3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:10:58.730 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:58.585+0000 7f59599d3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:10:58.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:58.679+0000 7f59599d3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:10:59.230 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:58.748+0000 7f59599d3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:10:59.230 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:58.816+0000 7f59599d3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:10:59.980 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:10:59.718+0000 7f59599d3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:10:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: Active manager daemon x restarted 2024-09-19T04:10:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: Activating manager daemon x 2024-09-19T04:10:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T04:10:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: mgrmap e10: x(active, starting, since 0.00466394s) 2024-09-19T04:10:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:10:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:10:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:10:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:10:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:10:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: Manager daemon x is now available 2024-09-19T04:10:59.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:10:59.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:10:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:11:00.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:11:00.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:00 smithi107 ceph-mon[30659]: mgrmap e11: x(active, since 1.00773s) 2024-09-19T04:11:01.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:11:01.369+0000 7f58b6400640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:11:01] ENGINE Error in HTTPServer.serve 2024-09-19T04:11:01.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: Traceback (most recent call last): 2024-09-19T04:11:01.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:11:01.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self._connections.run(self.expiration_interval) 2024-09-19T04:11:01.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:11:01.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self._run(expiration_interval) 2024-09-19T04:11:01.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:11:01.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:11:01.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: s = self.context.wrap_socket( 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: return self.sslsocket_class._create( 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self.do_handshake() 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self._sslobj.do_handshake() 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:11:01.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:11:01.921 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-mon[30659]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T04:11:01.921 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-mon[30659]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T04:11:01.922 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:01 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:01.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:11:01.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T04:11:01.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T04:11:01.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:11:01.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.432+0000 7fe192400640 1 Processor -- start 2024-09-19T04:11:01.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.432+0000 7fe192400640 1 -- start start 2024-09-19T04:11:01.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.432+0000 7fe192400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c1079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.432+0000 7fe192400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c107f90 con 0x7fe18c1075c0 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.433+0000 7fe191000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c1079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.433+0000 7fe191000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c1079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40656/0 (socket says 172.21.15.107:40656) 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.433+0000 7fe191000640 1 -- 172.21.15.107:0/3856636166 learned_addr learned my addr 172.21.15.107:0/3856636166 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.433+0000 7fe191000640 1 -- 172.21.15.107:0/3856636166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c108720 con 0x7fe18c1075c0 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.434+0000 7fe191000640 1 --2- 172.21.15.107:0/3856636166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c1079c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe180009820 tx=0x7fe18002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a95f4a24b9c6d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.434+0000 7fe18b400640 1 -- 172.21.15.107:0/3856636166 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe180037070 con 0x7fe18c1075c0 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.434+0000 7fe18b400640 1 -- 172.21.15.107:0/3856636166 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe180032920 con 0x7fe18c1075c0 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.434+0000 7fe18b400640 1 -- 172.21.15.107:0/3856636166 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe180030810 con 0x7fe18c1075c0 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.435+0000 7fe192400640 1 -- 172.21.15.107:0/3856636166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 msgr2=0x7fe18c1079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:01.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.435+0000 7fe192400640 1 --2- 172.21.15.107:0/3856636166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c1079c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe180009820 tx=0x7fe18002a270 comp rx=0 tx=0).stop 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.435+0000 7fe192400640 1 -- 172.21.15.107:0/3856636166 shutdown_connections 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.435+0000 7fe192400640 1 --2- 172.21.15.107:0/3856636166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c1079c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.435+0000 7fe192400640 1 -- 172.21.15.107:0/3856636166 >> 172.21.15.107:0/3856636166 conn(0x7fe18c102d90 msgr2=0x7fe18c1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.435+0000 7fe192400640 1 -- 172.21.15.107:0/3856636166 shutdown_connections 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.435+0000 7fe192400640 1 -- 172.21.15.107:0/3856636166 wait complete. 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.436+0000 7fe192400640 1 Processor -- start 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.437+0000 7fe192400640 1 -- start start 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.437+0000 7fe192400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c194550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.437+0000 7fe192400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10bad0 con 0x7fe18c1075c0 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.437+0000 7fe191000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c194550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:01.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.437+0000 7fe191000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c194550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40668/0 (socket says 172.21.15.107:40668) 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.437+0000 7fe191000640 1 -- 172.21.15.107:0/3719836364 learned_addr learned my addr 172.21.15.107:0/3719836364 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.438+0000 7fe191000640 1 -- 172.21.15.107:0/3719836364 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c194a90 con 0x7fe18c1075c0 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.438+0000 7fe191000640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c194550 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe1800097f0 tx=0x7fe1800327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.438+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe180037040 con 0x7fe18c1075c0 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.438+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe180030cc0 con 0x7fe18c1075c0 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.438+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe18003a640 con 0x7fe18c1075c0 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.438+0000 7fe192400640 1 -- 172.21.15.107:0/3719836364 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c194d20 con 0x7fe18c1075c0 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.438+0000 7fe192400640 1 -- 172.21.15.107:0/3719836364 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c1912e0 con 0x7fe18c1075c0 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.440+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fe18003b3f0 con 0x7fe18c1075c0 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.441+0000 7fe189600640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 0x7fe17403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.441+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 --> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe17403be30 con 0x7fe174039260 2024-09-19T04:11:01.937 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.441+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe18006c960 con 0x7fe18c1075c0 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.441+0000 7fe18be00640 1 -- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 msgr2=0x7fe17403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/972711045 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.441+0000 7fe18be00640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 0x7fe17403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.642+0000 7fe18be00640 1 -- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 msgr2=0x7fe17403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/972711045 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:56.642+0000 7fe18be00640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 0x7fe17403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:57.043+0000 7fe18be00640 1 -- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 msgr2=0x7fe17403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/972711045 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:57.043+0000 7fe18be00640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 0x7fe17403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:57.844+0000 7fe18be00640 1 -- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 msgr2=0x7fe17403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/972711045 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:57.844+0000 7fe18be00640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 0x7fe17403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:59.445+0000 7fe18be00640 1 -- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 msgr2=0x7fe17403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/972711045 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:59.446+0000 7fe18be00640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 0x7fe17403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:59.723+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7fe18003a7e0 con 0x7fe18c1075c0 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:59.723+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 msgr2=0x7fe17403b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:11:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:10:59.723+0000 7fe189600640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 0x7fe17403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.725+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fe18003fd10 con 0x7fe18c1075c0 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.725+0000 7fe189600640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe17403cd20 0x7fe17403f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.725+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe1800659c0 con 0x7fe17403cd20 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.726+0000 7fe18be00640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe17403cd20 0x7fe17403f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.727+0000 7fe18be00640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe17403cd20 0x7fe17403f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe178003980 tx=0x7fe1780092b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.728+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fe1800659c0 con 0x7fe17403cd20 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.735+0000 7fe192400640 1 -- 172.21.15.107:0/3719836364 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe18c194ef0 con 0x7fe17403cd20 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe189600640 1 -- 172.21.15.107:0/3719836364 <== mgr.14146 v2:172.21.15.107:6800/3991722475 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fe18c194ef0 con 0x7fe17403cd20 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe192400640 1 -- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe17403cd20 msgr2=0x7fe17403f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe192400640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe17403cd20 0x7fe17403f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe178003980 tx=0x7fe1780092b0 comp rx=0 tx=0).stop 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe192400640 1 -- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 msgr2=0x7fe18c194550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe192400640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c194550 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe1800097f0 tx=0x7fe1800327b0 comp rx=0 tx=0).stop 2024-09-19T04:11:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe192400640 1 -- 172.21.15.107:0/3719836364 shutdown_connections 2024-09-19T04:11:01.940 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe192400640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe17403cd20 0x7fe17403f110 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:01.940 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe192400640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:6800/972711045,v1:172.21.15.107:6801/972711045] conn(0x7fe174039260 0x7fe17403b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:01.940 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe192400640 1 --2- 172.21.15.107:0/3719836364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe18c1075c0 0x7fe18c194550 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:01.940 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe192400640 1 -- 172.21.15.107:0/3719836364 >> 172.21.15.107:0/3719836364 conn(0x7fe18c102d90 msgr2=0x7fe18c1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:01.940 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.736+0000 7fe192400640 1 -- 172.21.15.107:0/3719836364 shutdown_connections 2024-09-19T04:11:01.940 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:00.737+0000 7fe192400640 1 -- 172.21.15.107:0/3719836364 wait complete. 2024-09-19T04:11:01.940 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 9 is available 2024-09-19T04:11:01.940 INFO:teuthology.orchestra.run.smithi107.stdout:Generating a dashboard self-signed certificate... 2024-09-19T04:11:03.092 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:02 smithi107 ceph-mon[30659]: [19/Sep/2024:04:11:01] ENGINE Bus STARTING 2024-09-19T04:11:03.092 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:02 smithi107 ceph-mon[30659]: [19/Sep/2024:04:11:01] ENGINE Serving on https://172.21.15.107:7150 2024-09-19T04:11:03.092 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:02 smithi107 ceph-mon[30659]: [19/Sep/2024:04:11:01] ENGINE Error in HTTPServer.serve 2024-09-19T04:11:03.092 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-09-19T04:11:03.092 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:11:03.092 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:11:03.092 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-09-19T04:11:03.093 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:11:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-09-19T04:11:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:11:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:02 smithi107 ceph-mon[30659]: [19/Sep/2024:04:11:01] ENGINE Serving on http://172.21.15.107:8765 2024-09-19T04:11:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:02 smithi107 ceph-mon[30659]: [19/Sep/2024:04:11:01] ENGINE Bus STARTED 2024-09-19T04:11:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:02 smithi107 ceph-mon[30659]: mgrmap e12: x(active, since 2s) 2024-09-19T04:11:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:03.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T04:11:03.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.482+0000 7fb773c00640 1 Processor -- start 2024-09-19T04:11:03.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.483+0000 7fb773c00640 1 -- start start 2024-09-19T04:11:03.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.483+0000 7fb773c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:03.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.483+0000 7fb773c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1080c0 con 0x7fb76c1076f0 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.483+0000 7fb771200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.483+0000 7fb771200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46662/0 (socket says 172.21.15.107:46662) 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.483+0000 7fb771200640 1 -- 172.21.15.107:0/2256089931 learned_addr learned my addr 172.21.15.107:0/2256089931 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.484+0000 7fb771200640 1 -- 172.21.15.107:0/2256089931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c1088f0 con 0x7fb76c1076f0 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.485+0000 7fb771200640 1 --2- 172.21.15.107:0/2256089931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c107af0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009820 tx=0x7fb75c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00f25bf84b4993a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.485+0000 7fb76be00640 1 -- 172.21.15.107:0/2256089931 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c037070 con 0x7fb76c1076f0 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.485+0000 7fb76be00640 1 -- 172.21.15.107:0/2256089931 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb75c032920 con 0x7fb76c1076f0 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.485+0000 7fb76be00640 1 -- 172.21.15.107:0/2256089931 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c030810 con 0x7fb76c1076f0 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.486+0000 7fb773c00640 1 -- 172.21.15.107:0/2256089931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 msgr2=0x7fb76c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.486+0000 7fb773c00640 1 --2- 172.21.15.107:0/2256089931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c107af0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009820 tx=0x7fb75c02a270 comp rx=0 tx=0).stop 2024-09-19T04:11:03.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.486+0000 7fb773c00640 1 -- 172.21.15.107:0/2256089931 shutdown_connections 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.486+0000 7fb773c00640 1 --2- 172.21.15.107:0/2256089931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c107af0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.486+0000 7fb773c00640 1 -- 172.21.15.107:0/2256089931 >> 172.21.15.107:0/2256089931 conn(0x7fb76c102ea0 msgr2=0x7fb76c1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.486+0000 7fb773c00640 1 -- 172.21.15.107:0/2256089931 shutdown_connections 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.486+0000 7fb773c00640 1 -- 172.21.15.107:0/2256089931 wait complete. 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.487+0000 7fb773c00640 1 Processor -- start 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.487+0000 7fb773c00640 1 -- start start 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.487+0000 7fb773c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.487+0000 7fb773c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c10bca0 con 0x7fb76c1076f0 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.488+0000 7fb771200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.488+0000 7fb771200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46670/0 (socket says 172.21.15.107:46670) 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.488+0000 7fb771200640 1 -- 172.21.15.107:0/1509712440 learned_addr learned my addr 172.21.15.107:0/1509712440 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.488+0000 7fb771200640 1 -- 172.21.15.107:0/1509712440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c19bc20 con 0x7fb76c1076f0 2024-09-19T04:11:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.488+0000 7fb771200640 1 --2- 172.21.15.107:0/1509712440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c19b6e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0097f0 tx=0x7fb75c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.489+0000 7fb76a000640 1 -- 172.21.15.107:0/1509712440 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c037040 con 0x7fb76c1076f0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.489+0000 7fb76a000640 1 -- 172.21.15.107:0/1509712440 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb75c030cc0 con 0x7fb76c1076f0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.489+0000 7fb773c00640 1 -- 172.21.15.107:0/1509712440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c19beb0 con 0x7fb76c1076f0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.489+0000 7fb76a000640 1 -- 172.21.15.107:0/1509712440 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c03a640 con 0x7fb76c1076f0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.489+0000 7fb773c00640 1 -- 172.21.15.107:0/1509712440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c19eba0 con 0x7fb76c1076f0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.490+0000 7fb76a000640 1 -- 172.21.15.107:0/1509712440 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fb75c03a940 con 0x7fb76c1076f0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.490+0000 7fb773c00640 1 -- 172.21.15.107:0/1509712440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76c10be20 con 0x7fb76c1076f0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.490+0000 7fb76a000640 1 --2- 172.21.15.107:0/1509712440 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7500390f0 0x7fb75003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.490+0000 7fb76a000640 1 -- 172.21.15.107:0/1509712440 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb75c06c7a0 con 0x7fb76c1076f0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.491+0000 7fb763e00640 1 --2- 172.21.15.107:0/1509712440 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7500390f0 0x7fb75003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.491+0000 7fb763e00640 1 --2- 172.21.15.107:0/1509712440 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7500390f0 0x7fb75003b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7540098c0 tx=0x7fb754006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:03.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.500+0000 7fb76a000640 1 -- 172.21.15.107:0/1509712440 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb75c03ba40 con 0x7fb76c1076f0 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.503+0000 7fb76a000640 1 -- 172.21.15.107:0/1509712440 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fb75c044920 con 0x7fb76c1076f0 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.688+0000 7fb773c00640 1 -- 172.21.15.107:0/1509712440 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb76c104100 con 0x7fb7500390f0 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.766+0000 7fb76a000640 1 -- 172.21.15.107:0/1509712440 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb76c104100 con 0x7fb7500390f0 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.768+0000 7fb773c00640 1 -- 172.21.15.107:0/1509712440 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7500390f0 msgr2=0x7fb75003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.768+0000 7fb773c00640 1 --2- 172.21.15.107:0/1509712440 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7500390f0 0x7fb75003b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7540098c0 tx=0x7fb754006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.768+0000 7fb773c00640 1 -- 172.21.15.107:0/1509712440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 msgr2=0x7fb76c19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.768+0000 7fb773c00640 1 --2- 172.21.15.107:0/1509712440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c19b6e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0097f0 tx=0x7fb75c0327b0 comp rx=0 tx=0).stop 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.768+0000 7fb773c00640 1 -- 172.21.15.107:0/1509712440 shutdown_connections 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.768+0000 7fb773c00640 1 --2- 172.21.15.107:0/1509712440 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7500390f0 0x7fb75003b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.768+0000 7fb773c00640 1 --2- 172.21.15.107:0/1509712440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb76c1076f0 0x7fb76c19b6e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.768+0000 7fb773c00640 1 -- 172.21.15.107:0/1509712440 >> 172.21.15.107:0/1509712440 conn(0x7fb76c102ea0 msgr2=0x7fb76c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.768+0000 7fb773c00640 1 -- 172.21.15.107:0/1509712440 shutdown_connections 2024-09-19T04:11:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:02.768+0000 7fb773c00640 1 -- 172.21.15.107:0/1509712440 wait complete. 2024-09-19T04:11:03.377 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2024-09-19T04:11:03.928 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:03 smithi107 ceph-mon[30659]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:05.498 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:05 smithi107 ceph-mon[30659]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:05.499 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:05.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8WzSMQcYbYjAqg6d6t.oDOfUcSGvrG4BoHoZ3Hta93kGt0PKC4fgK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726719064, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T04:11:05.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.796+0000 7fec42200640 1 Processor -- start 2024-09-19T04:11:05.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.796+0000 7fec42200640 1 -- start start 2024-09-19T04:11:05.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.796+0000 7fec42200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:05.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.796+0000 7fec42200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c10a260 con 0x7fec3c109870 2024-09-19T04:11:05.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.797+0000 7fec40e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:05.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.797+0000 7fec40e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46680/0 (socket says 172.21.15.107:46680) 2024-09-19T04:11:05.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.797+0000 7fec40e00640 1 -- 172.21.15.107:0/3394469277 learned_addr learned my addr 172.21.15.107:0/3394469277 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:05.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.797+0000 7fec40e00640 1 -- 172.21.15.107:0/3394469277 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c10a930 con 0x7fec3c109870 2024-09-19T04:11:05.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.798+0000 7fec40e00640 1 --2- 172.21.15.107:0/3394469277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c109c90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec30009820 tx=0x7fec3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a0eed070bd766c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.798+0000 7fec3b400640 1 -- 172.21.15.107:0/3394469277 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec30037070 con 0x7fec3c109870 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.798+0000 7fec3b400640 1 -- 172.21.15.107:0/3394469277 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30032920 con 0x7fec3c109870 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.798+0000 7fec3b400640 1 -- 172.21.15.107:0/3394469277 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec30030810 con 0x7fec3c109870 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.799+0000 7fec42200640 1 -- 172.21.15.107:0/3394469277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 msgr2=0x7fec3c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.799+0000 7fec42200640 1 --2- 172.21.15.107:0/3394469277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c109c90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec30009820 tx=0x7fec3002a270 comp rx=0 tx=0).stop 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.799+0000 7fec42200640 1 -- 172.21.15.107:0/3394469277 shutdown_connections 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.799+0000 7fec42200640 1 --2- 172.21.15.107:0/3394469277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c109c90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.799+0000 7fec42200640 1 -- 172.21.15.107:0/3394469277 >> 172.21.15.107:0/3394469277 conn(0x7fec3c07ce50 msgr2=0x7fec3c07d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.799+0000 7fec42200640 1 -- 172.21.15.107:0/3394469277 shutdown_connections 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.799+0000 7fec42200640 1 -- 172.21.15.107:0/3394469277 wait complete. 2024-09-19T04:11:05.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.800+0000 7fec42200640 1 Processor -- start 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.801+0000 7fec42200640 1 -- start start 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.801+0000 7fec42200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.801+0000 7fec42200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c10dce0 con 0x7fec3c109870 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.801+0000 7fec40e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.801+0000 7fec40e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46692/0 (socket says 172.21.15.107:46692) 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.801+0000 7fec40e00640 1 -- 172.21.15.107:0/895646611 learned_addr learned my addr 172.21.15.107:0/895646611 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.802+0000 7fec40e00640 1 -- 172.21.15.107:0/895646611 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c19dc20 con 0x7fec3c109870 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.802+0000 7fec40e00640 1 --2- 172.21.15.107:0/895646611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c19d6e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec300097f0 tx=0x7fec300327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.802+0000 7fec39600640 1 -- 172.21.15.107:0/895646611 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec30037040 con 0x7fec3c109870 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.802+0000 7fec39600640 1 -- 172.21.15.107:0/895646611 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30030cc0 con 0x7fec3c109870 2024-09-19T04:11:05.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.802+0000 7fec42200640 1 -- 172.21.15.107:0/895646611 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c19deb0 con 0x7fec3c109870 2024-09-19T04:11:05.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.802+0000 7fec39600640 1 -- 172.21.15.107:0/895646611 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec3003a6b0 con 0x7fec3c109870 2024-09-19T04:11:05.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.802+0000 7fec42200640 1 -- 172.21.15.107:0/895646611 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c1a0ba0 con 0x7fec3c109870 2024-09-19T04:11:05.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.804+0000 7fec39600640 1 -- 172.21.15.107:0/895646611 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fec3003b3f0 con 0x7fec3c109870 2024-09-19T04:11:05.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.804+0000 7fec42200640 1 -- 172.21.15.107:0/895646611 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c109c90 con 0x7fec3c109870 2024-09-19T04:11:05.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.804+0000 7fec39600640 1 --2- 172.21.15.107:0/895646611 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec24039260 0x7fec2403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:05.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.804+0000 7fec39600640 1 -- 172.21.15.107:0/895646611 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fec3006c970 con 0x7fec3c109870 2024-09-19T04:11:05.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.804+0000 7fec3be00640 1 --2- 172.21.15.107:0/895646611 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec24039260 0x7fec2403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:05.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.805+0000 7fec3be00640 1 --2- 172.21.15.107:0/895646611 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec24039260 0x7fec2403b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec280098c0 tx=0x7fec28006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:05.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:03.809+0000 7fec39600640 1 -- 172.21.15.107:0/895646611 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fec3003a850 con 0x7fec3c109870 2024-09-19T04:11:05.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.007+0000 7fec42200640 1 -- 172.21.15.107:0/895646611 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fec3c1a0d70 con 0x7fec24039260 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.235+0000 7fec39600640 1 -- 172.21.15.107:0/895646611 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fec3c1a0d70 con 0x7fec24039260 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.238+0000 7fec42200640 1 -- 172.21.15.107:0/895646611 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec24039260 msgr2=0x7fec2403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.238+0000 7fec42200640 1 --2- 172.21.15.107:0/895646611 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec24039260 0x7fec2403b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec280098c0 tx=0x7fec28006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.238+0000 7fec42200640 1 -- 172.21.15.107:0/895646611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 msgr2=0x7fec3c19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.238+0000 7fec42200640 1 --2- 172.21.15.107:0/895646611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c19d6e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec300097f0 tx=0x7fec300327b0 comp rx=0 tx=0).stop 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.238+0000 7fec42200640 1 -- 172.21.15.107:0/895646611 shutdown_connections 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.238+0000 7fec42200640 1 --2- 172.21.15.107:0/895646611 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec24039260 0x7fec2403b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.238+0000 7fec42200640 1 --2- 172.21.15.107:0/895646611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3c109870 0x7fec3c19d6e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.238+0000 7fec42200640 1 -- 172.21.15.107:0/895646611 >> 172.21.15.107:0/895646611 conn(0x7fec3c07ce50 msgr2=0x7fec3c107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.238+0000 7fec42200640 1 -- 172.21.15.107:0/895646611 shutdown_connections 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:04.238+0000 7fec42200640 1 -- 172.21.15.107:0/895646611 wait complete. 2024-09-19T04:11:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2024-09-19T04:11:06.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:06 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2477144135' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T04:11:06.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T04:11:06.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.973+0000 7f81a3e00640 1 Processor -- start 2024-09-19T04:11:06.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.973+0000 7f81a3e00640 1 -- start start 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.973+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c10a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.973+0000 7f81a3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f819c07bdf0 con 0x7f819c108220 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.974+0000 7f81a1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c10a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.974+0000 7f81a1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c10a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46702/0 (socket says 172.21.15.107:46702) 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.974+0000 7f81a1400640 1 -- 172.21.15.107:0/1200126104 learned_addr learned my addr 172.21.15.107:0/1200126104 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.974+0000 7f81a1400640 1 -- 172.21.15.107:0/1200126104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c10ab50 con 0x7f819c108220 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.975+0000 7f81a1400640 1 --2- 172.21.15.107:0/1200126104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c10a610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8190009820 tx=0x7f819002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38684ae52c79d400 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.975+0000 7f819be00640 1 -- 172.21.15.107:0/1200126104 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8190037070 con 0x7f819c108220 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.975+0000 7f819be00640 1 -- 172.21.15.107:0/1200126104 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8190032920 con 0x7f819c108220 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.975+0000 7f819be00640 1 -- 172.21.15.107:0/1200126104 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8190030810 con 0x7f819c108220 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.976+0000 7f81a3e00640 1 -- 172.21.15.107:0/1200126104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 msgr2=0x7f819c10a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.976+0000 7f81a3e00640 1 --2- 172.21.15.107:0/1200126104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c10a610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8190009820 tx=0x7f819002a270 comp rx=0 tx=0).stop 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.976+0000 7f81a3e00640 1 -- 172.21.15.107:0/1200126104 shutdown_connections 2024-09-19T04:11:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.976+0000 7f81a3e00640 1 --2- 172.21.15.107:0/1200126104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c10a610 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.976+0000 7f81a3e00640 1 -- 172.21.15.107:0/1200126104 >> 172.21.15.107:0/1200126104 conn(0x7f819c101e20 msgr2=0x7f819c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.976+0000 7f81a3e00640 1 -- 172.21.15.107:0/1200126104 shutdown_connections 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.976+0000 7f81a3e00640 1 -- 172.21.15.107:0/1200126104 wait complete. 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.977+0000 7f81a3e00640 1 Processor -- start 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.977+0000 7f81a3e00640 1 -- start start 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.977+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.977+0000 7f81a3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f819c10bb70 con 0x7f819c108220 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.978+0000 7f81a1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.978+0000 7f81a1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46706/0 (socket says 172.21.15.107:46706) 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.978+0000 7f81a1400640 1 -- 172.21.15.107:0/2477144135 learned_addr learned my addr 172.21.15.107:0/2477144135 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.978+0000 7f81a1400640 1 -- 172.21.15.107:0/2477144135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c19ded0 con 0x7f819c108220 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.978+0000 7f81a1400640 1 --2- 172.21.15.107:0/2477144135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c19d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8190030420 tx=0x7f8190032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.978+0000 7f819a000640 1 -- 172.21.15.107:0/2477144135 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8190037040 con 0x7f819c108220 2024-09-19T04:11:06.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.978+0000 7f819a000640 1 -- 172.21.15.107:0/2477144135 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8190040070 con 0x7f819c108220 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.978+0000 7f81a3e00640 1 -- 172.21.15.107:0/2477144135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c19e160 con 0x7f819c108220 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.978+0000 7f819a000640 1 -- 172.21.15.107:0/2477144135 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f819003a6e0 con 0x7f819c108220 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.979+0000 7f81a3e00640 1 -- 172.21.15.107:0/2477144135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c1a0e50 con 0x7f819c108220 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.980+0000 7f819a000640 1 -- 172.21.15.107:0/2477144135 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f819003bb20 con 0x7f819c108220 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.980+0000 7f81a3e00640 1 -- 172.21.15.107:0/2477144135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819c108680 con 0x7f819c108220 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.980+0000 7f819a000640 1 --2- 172.21.15.107:0/2477144135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8184039200 0x7f818403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.980+0000 7f819a000640 1 -- 172.21.15.107:0/2477144135 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f819006caf0 con 0x7f819c108220 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.980+0000 7f81a0a00640 1 --2- 172.21.15.107:0/2477144135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8184039200 0x7f818403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.981+0000 7f81a0a00640 1 --2- 172.21.15.107:0/2477144135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8184039200 0x7f818403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f81880098c0 tx=0x7f8188006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:05.985+0000 7f819a000640 1 -- 172.21.15.107:0/2477144135 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8190038280 con 0x7f819c108220 2024-09-19T04:11:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.162+0000 7f81a3e00640 1 -- 172.21.15.107:0/2477144135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f819c1a1020 con 0x7f819c108220 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.163+0000 7f819a000640 1 -- 172.21.15.107:0/2477144135 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f819003b7a0 con 0x7f819c108220 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.165+0000 7f81a3e00640 1 -- 172.21.15.107:0/2477144135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8184039200 msgr2=0x7f818403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.165+0000 7f81a3e00640 1 --2- 172.21.15.107:0/2477144135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8184039200 0x7f818403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f81880098c0 tx=0x7f8188006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.165+0000 7f81a3e00640 1 -- 172.21.15.107:0/2477144135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 msgr2=0x7f819c19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.165+0000 7f81a3e00640 1 --2- 172.21.15.107:0/2477144135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c19d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8190030420 tx=0x7f8190032670 comp rx=0 tx=0).stop 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.165+0000 7f81a3e00640 1 -- 172.21.15.107:0/2477144135 shutdown_connections 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.165+0000 7f81a3e00640 1 --2- 172.21.15.107:0/2477144135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8184039200 0x7f818403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.165+0000 7f81a3e00640 1 --2- 172.21.15.107:0/2477144135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c108220 0x7f819c19d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.165+0000 7f81a3e00640 1 -- 172.21.15.107:0/2477144135 >> 172.21.15.107:0/2477144135 conn(0x7f819c101e20 msgr2=0x7f819c1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.166+0000 7f81a3e00640 1 -- 172.21.15.107:0/2477144135 shutdown_connections 2024-09-19T04:11:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:06.166+0000 7f81a3e00640 1 -- 172.21.15.107:0/2477144135 wait complete. 2024-09-19T04:11:06.748 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:11:06.748 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-19T04:11:06.762 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:11:06.762 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-19T04:11:06.762 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-19T04:11:06.762 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T04:11:06.764 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2024-09-19T04:11:06.764 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:06.764 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107.front.sepia.ceph.com:8443/ 2024-09-19T04:11:06.764 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2024-09-19T04:11:06.765 INFO:teuthology.orchestra.run.smithi107.stdout: Password: y6sqopqkq0 2024-09-19T04:11:06.765 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:06.765 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config directory 2024-09-19T04:11:07.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:07 smithi107 ceph-mon[30659]: mgrmap e13: x(active, since 6s) 2024-09-19T04:11:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.271+0000 7fc5a4800640 1 Processor -- start 2024-09-19T04:11:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.271+0000 7fc5a4800640 1 -- start start 2024-09-19T04:11:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.272+0000 7fc5a4800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.272+0000 7fc5a4800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1080c0 con 0x7fc59c1076f0 2024-09-19T04:11:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.272+0000 7fc5a1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.272+0000 7fc5a1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46722/0 (socket says 172.21.15.107:46722) 2024-09-19T04:11:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.272+0000 7fc5a1e00640 1 -- 172.21.15.107:0/2419365193 learned_addr learned my addr 172.21.15.107:0/2419365193 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.272+0000 7fc5a1e00640 1 -- 172.21.15.107:0/2419365193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c1088f0 con 0x7fc59c1076f0 2024-09-19T04:11:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.273+0000 7fc5a1e00640 1 --2- 172.21.15.107:0/2419365193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c107af0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc590009820 tx=0x7fc59002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ba8d9e746a834 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.273+0000 7fc5a0a00640 1 -- 172.21.15.107:0/2419365193 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc590037070 con 0x7fc59c1076f0 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.273+0000 7fc5a0a00640 1 -- 172.21.15.107:0/2419365193 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc590032920 con 0x7fc59c1076f0 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.274+0000 7fc5a0a00640 1 -- 172.21.15.107:0/2419365193 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc590030810 con 0x7fc59c1076f0 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.274+0000 7fc5a4800640 1 -- 172.21.15.107:0/2419365193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 msgr2=0x7fc59c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.274+0000 7fc5a4800640 1 --2- 172.21.15.107:0/2419365193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c107af0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc590009820 tx=0x7fc59002a270 comp rx=0 tx=0).stop 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.274+0000 7fc5a4800640 1 -- 172.21.15.107:0/2419365193 shutdown_connections 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.274+0000 7fc5a4800640 1 --2- 172.21.15.107:0/2419365193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c107af0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.274+0000 7fc5a4800640 1 -- 172.21.15.107:0/2419365193 >> 172.21.15.107:0/2419365193 conn(0x7fc59c102ec0 msgr2=0x7fc59c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.274+0000 7fc5a4800640 1 -- 172.21.15.107:0/2419365193 shutdown_connections 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.274+0000 7fc5a4800640 1 -- 172.21.15.107:0/2419365193 wait complete. 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.275+0000 7fc5a4800640 1 Processor -- start 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.275+0000 7fc5a4800640 1 -- start start 2024-09-19T04:11:08.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.275+0000 7fc5a4800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.275+0000 7fc5a4800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c10bca0 con 0x7fc59c1076f0 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.276+0000 7fc5a1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.276+0000 7fc5a1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46728/0 (socket says 172.21.15.107:46728) 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.276+0000 7fc5a1e00640 1 -- 172.21.15.107:0/3549797142 learned_addr learned my addr 172.21.15.107:0/3549797142 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.276+0000 7fc5a1e00640 1 -- 172.21.15.107:0/3549797142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c07e0b0 con 0x7fc59c1076f0 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.276+0000 7fc5a1e00640 1 --2- 172.21.15.107:0/3549797142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c07db70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc590030560 tx=0x7fc590032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.277+0000 7fc596a00640 1 -- 172.21.15.107:0/3549797142 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc590037040 con 0x7fc59c1076f0 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.277+0000 7fc596a00640 1 -- 172.21.15.107:0/3549797142 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc590040070 con 0x7fc59c1076f0 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.277+0000 7fc5a4800640 1 -- 172.21.15.107:0/3549797142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59c07c2a0 con 0x7fc59c1076f0 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.277+0000 7fc596a00640 1 -- 172.21.15.107:0/3549797142 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59003a740 con 0x7fc59c1076f0 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.277+0000 7fc5a4800640 1 -- 172.21.15.107:0/3549797142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59c07c7e0 con 0x7fc59c1076f0 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.278+0000 7fc596a00640 1 -- 172.21.15.107:0/3549797142 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fc59003bb90 con 0x7fc59c1076f0 2024-09-19T04:11:08.149 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.278+0000 7fc5a4800640 1 -- 172.21.15.107:0/3549797142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59c107b70 con 0x7fc59c1076f0 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.278+0000 7fc596a00640 1 --2- 172.21.15.107:0/3549797142 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5840392a0 0x7fc58403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.279+0000 7fc596a00640 1 -- 172.21.15.107:0/3549797142 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc59006cb30 con 0x7fc59c1076f0 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.279+0000 7fc5a1400640 1 --2- 172.21.15.107:0/3549797142 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5840392a0 0x7fc58403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.279+0000 7fc5a1400640 1 --2- 172.21.15.107:0/3549797142 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5840392a0 0x7fc58403b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0098c0 tx=0x7fc58c006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.284+0000 7fc596a00640 1 -- 172.21.15.107:0/3549797142 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc590042960 con 0x7fc59c1076f0 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.545+0000 7fc5a4800640 1 -- 172.21.15.107:0/3549797142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc59c07ca60 con 0x7fc59c1076f0 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.547+0000 7fc596a00640 1 -- 172.21.15.107:0/3549797142 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fc59004a200 con 0x7fc59c1076f0 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.549+0000 7fc5a4800640 1 -- 172.21.15.107:0/3549797142 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5840392a0 msgr2=0x7fc58403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.549+0000 7fc5a4800640 1 --2- 172.21.15.107:0/3549797142 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5840392a0 0x7fc58403b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0098c0 tx=0x7fc58c006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.549+0000 7fc5a4800640 1 -- 172.21.15.107:0/3549797142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 msgr2=0x7fc59c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:08.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.549+0000 7fc5a4800640 1 --2- 172.21.15.107:0/3549797142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c07db70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc590030560 tx=0x7fc590032670 comp rx=0 tx=0).stop 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.550+0000 7fc5a4800640 1 -- 172.21.15.107:0/3549797142 shutdown_connections 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.550+0000 7fc5a4800640 1 --2- 172.21.15.107:0/3549797142 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5840392a0 0x7fc58403b760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.550+0000 7fc5a4800640 1 --2- 172.21.15.107:0/3549797142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc59c1076f0 0x7fc59c07db70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.550+0000 7fc5a4800640 1 -- 172.21.15.107:0/3549797142 >> 172.21.15.107:0/3549797142 conn(0x7fc59c102ec0 msgr2=0x7fc59c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.550+0000 7fc5a4800640 1 -- 172.21.15.107:0/3549797142 shutdown_connections 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-19T04:11:07.550+0000 7fc5a4800640 1 -- 172.21.15.107:0/3549797142 wait complete. 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T04:11:08.151 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:08.152 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T04:11:08.152 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:08.152 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2024-09-19T04:11:08.152 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:08.152 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2024-09-19T04:11:08.152 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:08.152 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T04:11:08.152 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:08.152 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2024-09-19T04:11:08.205 INFO:tasks.cephadm:Fetching config... 2024-09-19T04:11:08.205 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:11:08.205 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T04:11:08.241 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T04:11:08.241 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:11:08.241 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T04:11:08.318 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T04:11:08.318 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:11:08.318 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T04:11:08.415 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T04:11:08.415 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:11:08.415 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T04:11:08.490 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T04:11:08.491 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTR2CZIG5V/+ju5vCbPcP03fnvYPo0IFqkVP+JFR/RFGmW8cUwRLnwDdF1z5EklQCweV6tO31TU5zADXsDxQfUUov/AwkNHdR+zSGSx3COs4cj8NII6BbeRMbXZdSxP2IWs5+PvWIvHtKdweBtvqKNypLxVRz7IM7XI2F8dJEUQWGGfOQMOLDz0KaJSIxIBB+GDZJgqK6AgGjClSJ0BiRSl/njLH0U6kyA1APm7FVndIOzg74lXNBbSEa16m1rz0eAamC6TH2iLkZCEXCzhutS3h42bdnH0WxvFI5DZBEH9jAATiww5QHJpzG/oJHDZbuoS8FvAsCcgX0yZH72yJfX17fyDfSfIzExaWhEQDdg7guHB1D4iS4AWiif7qRCUh4xnHZ35U9ZueESabVJh8O9pLT9FtZGcb8HCaq6E3zLy8saFzRRY6j2E6ScapIQC7fuPnFc0Jl0dLcrG+vGJAiA7s0iPabJ7uQTWC355hRsEdg3oEYwqaSDehFSuZE43zc= ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:11:08.614 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTR2CZIG5V/+ju5vCbPcP03fnvYPo0IFqkVP+JFR/RFGmW8cUwRLnwDdF1z5EklQCweV6tO31TU5zADXsDxQfUUov/AwkNHdR+zSGSx3COs4cj8NII6BbeRMbXZdSxP2IWs5+PvWIvHtKdweBtvqKNypLxVRz7IM7XI2F8dJEUQWGGfOQMOLDz0KaJSIxIBB+GDZJgqK6AgGjClSJ0BiRSl/njLH0U6kyA1APm7FVndIOzg74lXNBbSEa16m1rz0eAamC6TH2iLkZCEXCzhutS3h42bdnH0WxvFI5DZBEH9jAATiww5QHJpzG/oJHDZbuoS8FvAsCcgX0yZH72yJfX17fyDfSfIzExaWhEQDdg7guHB1D4iS4AWiif7qRCUh4xnHZ35U9ZueESabVJh8O9pLT9FtZGcb8HCaq6E3zLy8saFzRRY6j2E6ScapIQC7fuPnFc0Jl0dLcrG+vGJAiA7s0iPabJ7uQTWC355hRsEdg3oEYwqaSDehFSuZE43zc= ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:08.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:08 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3549797142' entity='client.admin' 2024-09-19T04:11:08.639 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTR2CZIG5V/+ju5vCbPcP03fnvYPo0IFqkVP+JFR/RFGmW8cUwRLnwDdF1z5EklQCweV6tO31TU5zADXsDxQfUUov/AwkNHdR+zSGSx3COs4cj8NII6BbeRMbXZdSxP2IWs5+PvWIvHtKdweBtvqKNypLxVRz7IM7XI2F8dJEUQWGGfOQMOLDz0KaJSIxIBB+GDZJgqK6AgGjClSJ0BiRSl/njLH0U6kyA1APm7FVndIOzg74lXNBbSEa16m1rz0eAamC6TH2iLkZCEXCzhutS3h42bdnH0WxvFI5DZBEH9jAATiww5QHJpzG/oJHDZbuoS8FvAsCcgX0yZH72yJfX17fyDfSfIzExaWhEQDdg7guHB1D4iS4AWiif7qRCUh4xnHZ35U9ZueESabVJh8O9pLT9FtZGcb8HCaq6E3zLy8saFzRRY6j2E6ScapIQC7fuPnFc0Jl0dLcrG+vGJAiA7s0iPabJ7uQTWC355hRsEdg3oEYwqaSDehFSuZE43zc= ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:11:08.683 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTR2CZIG5V/+ju5vCbPcP03fnvYPo0IFqkVP+JFR/RFGmW8cUwRLnwDdF1z5EklQCweV6tO31TU5zADXsDxQfUUov/AwkNHdR+zSGSx3COs4cj8NII6BbeRMbXZdSxP2IWs5+PvWIvHtKdweBtvqKNypLxVRz7IM7XI2F8dJEUQWGGfOQMOLDz0KaJSIxIBB+GDZJgqK6AgGjClSJ0BiRSl/njLH0U6kyA1APm7FVndIOzg74lXNBbSEa16m1rz0eAamC6TH2iLkZCEXCzhutS3h42bdnH0WxvFI5DZBEH9jAATiww5QHJpzG/oJHDZbuoS8FvAsCcgX0yZH72yJfX17fyDfSfIzExaWhEQDdg7guHB1D4iS4AWiif7qRCUh4xnHZ35U9ZueESabVJh8O9pLT9FtZGcb8HCaq6E3zLy8saFzRRY6j2E6ScapIQC7fuPnFc0Jl0dLcrG+vGJAiA7s0iPabJ7uQTWC355hRsEdg3oEYwqaSDehFSuZE43zc= ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:08.695 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTR2CZIG5V/+ju5vCbPcP03fnvYPo0IFqkVP+JFR/RFGmW8cUwRLnwDdF1z5EklQCweV6tO31TU5zADXsDxQfUUov/AwkNHdR+zSGSx3COs4cj8NII6BbeRMbXZdSxP2IWs5+PvWIvHtKdweBtvqKNypLxVRz7IM7XI2F8dJEUQWGGfOQMOLDz0KaJSIxIBB+GDZJgqK6AgGjClSJ0BiRSl/njLH0U6kyA1APm7FVndIOzg74lXNBbSEa16m1rz0eAamC6TH2iLkZCEXCzhutS3h42bdnH0WxvFI5DZBEH9jAATiww5QHJpzG/oJHDZbuoS8FvAsCcgX0yZH72yJfX17fyDfSfIzExaWhEQDdg7guHB1D4iS4AWiif7qRCUh4xnHZ35U9ZueESabVJh8O9pLT9FtZGcb8HCaq6E3zLy8saFzRRY6j2E6ScapIQC7fuPnFc0Jl0dLcrG+vGJAiA7s0iPabJ7uQTWC355hRsEdg3oEYwqaSDehFSuZE43zc= ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:11:08.738 INFO:teuthology.orchestra.run.smithi145.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTR2CZIG5V/+ju5vCbPcP03fnvYPo0IFqkVP+JFR/RFGmW8cUwRLnwDdF1z5EklQCweV6tO31TU5zADXsDxQfUUov/AwkNHdR+zSGSx3COs4cj8NII6BbeRMbXZdSxP2IWs5+PvWIvHtKdweBtvqKNypLxVRz7IM7XI2F8dJEUQWGGfOQMOLDz0KaJSIxIBB+GDZJgqK6AgGjClSJ0BiRSl/njLH0U6kyA1APm7FVndIOzg74lXNBbSEa16m1rz0eAamC6TH2iLkZCEXCzhutS3h42bdnH0WxvFI5DZBEH9jAATiww5QHJpzG/oJHDZbuoS8FvAsCcgX0yZH72yJfX17fyDfSfIzExaWhEQDdg7guHB1D4iS4AWiif7qRCUh4xnHZ35U9ZueESabVJh8O9pLT9FtZGcb8HCaq6E3zLy8saFzRRY6j2E6ScapIQC7fuPnFc0Jl0dLcrG+vGJAiA7s0iPabJ7uQTWC355hRsEdg3oEYwqaSDehFSuZE43zc= ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:08.750 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T04:11:09.115 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.660+0000 7f6512000640 1 -- 172.21.15.107:0/709316575 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6504031760 con 0x7f6514103820 2024-09-19T04:11:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.661+0000 7f651a600640 1 -- 172.21.15.107:0/709316575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6514103820 msgr2=0x7f6514103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.661+0000 7f651a600640 1 --2- 172.21.15.107:0/709316575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6514103820 0x7f6514103c20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6504009880 tx=0x7f650402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:11:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.661+0000 7f651a600640 1 -- 172.21.15.107:0/709316575 shutdown_connections 2024-09-19T04:11:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.661+0000 7f651a600640 1 --2- 172.21.15.107:0/709316575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6514103820 0x7f6514103c20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.661+0000 7f651a600640 1 -- 172.21.15.107:0/709316575 >> 172.21.15.107:0/709316575 conn(0x7f65140feff0 msgr2=0x7f6514101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.661+0000 7f651a600640 1 -- 172.21.15.107:0/709316575 shutdown_connections 2024-09-19T04:11:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.661+0000 7f651a600640 1 -- 172.21.15.107:0/709316575 wait complete. 2024-09-19T04:11:09.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.662+0000 7f651a600640 1 Processor -- start 2024-09-19T04:11:09.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.662+0000 7f651a600640 1 -- start start 2024-09-19T04:11:09.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.662+0000 7f651a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6514103820 0x7f6514198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:09.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.662+0000 7f651a600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6514109930 con 0x7f6514103820 2024-09-19T04:11:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.663+0000 7f6513400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6514103820 0x7f6514198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.663+0000 7f6513400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6514103820 0x7f6514198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34066/0 (socket says 172.21.15.107:34066) 2024-09-19T04:11:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.663+0000 7f6513400640 1 -- 172.21.15.107:0/1687362437 learned_addr learned my addr 172.21.15.107:0/1687362437 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.663+0000 7f6513400640 1 -- 172.21.15.107:0/1687362437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65141994c0 con 0x7f6514103820 2024-09-19T04:11:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.663+0000 7f6513400640 1 --2- 172.21.15.107:0/1687362437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6514103820 0x7f6514198f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6504004f60 tx=0x7f65040048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:09.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.664+0000 7f650be00640 1 -- 172.21.15.107:0/1687362437 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f650403f070 con 0x7f6514103820 2024-09-19T04:11:09.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.664+0000 7f650be00640 1 -- 172.21.15.107:0/1687362437 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6504004d40 con 0x7f6514103820 2024-09-19T04:11:09.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.664+0000 7f650be00640 1 -- 172.21.15.107:0/1687362437 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6504035030 con 0x7f6514103820 2024-09-19T04:11:09.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.664+0000 7f651a600640 1 -- 172.21.15.107:0/1687362437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6514199750 con 0x7f6514103820 2024-09-19T04:11:09.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.664+0000 7f651a600640 1 -- 172.21.15.107:0/1687362437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651419c440 con 0x7f6514103820 2024-09-19T04:11:09.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.665+0000 7f651a600640 1 -- 172.21.15.107:0/1687362437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651410a0b0 con 0x7f6514103820 2024-09-19T04:11:09.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.666+0000 7f650be00640 1 -- 172.21.15.107:0/1687362437 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f6504042070 con 0x7f6514103820 2024-09-19T04:11:09.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.666+0000 7f650be00640 1 --2- 172.21.15.107:0/1687362437 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f64f80392a0 0x7f64f803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:09.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.666+0000 7f650be00640 1 -- 172.21.15.107:0/1687362437 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f650406d0c0 con 0x7f6514103820 2024-09-19T04:11:09.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.666+0000 7f6512a00640 1 --2- 172.21.15.107:0/1687362437 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f64f80392a0 0x7f64f803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:09.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.667+0000 7f6512a00640 1 --2- 172.21.15.107:0/1687362437 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f64f80392a0 0x7f64f803b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f65000098c0 tx=0x7f6500006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:09.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.671+0000 7f650be00640 1 -- 172.21.15.107:0/1687362437 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f650404a260 con 0x7f6514103820 2024-09-19T04:11:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.869+0000 7f651a600640 1 -- 172.21.15.107:0/1687362437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f651402d830 con 0x7f6514103820 2024-09-19T04:11:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.873+0000 7f650be00640 1 -- 172.21.15.107:0/1687362437 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f65040035f0 con 0x7f6514103820 2024-09-19T04:11:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.875+0000 7f651a600640 1 -- 172.21.15.107:0/1687362437 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f64f80392a0 msgr2=0x7f64f803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.875+0000 7f651a600640 1 --2- 172.21.15.107:0/1687362437 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f64f80392a0 0x7f64f803b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f65000098c0 tx=0x7f6500006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.875+0000 7f651a600640 1 -- 172.21.15.107:0/1687362437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6514103820 msgr2=0x7f6514198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.875+0000 7f651a600640 1 --2- 172.21.15.107:0/1687362437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6514103820 0x7f6514198f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6504004f60 tx=0x7f65040048f0 comp rx=0 tx=0).stop 2024-09-19T04:11:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.875+0000 7f651a600640 1 -- 172.21.15.107:0/1687362437 shutdown_connections 2024-09-19T04:11:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.875+0000 7f651a600640 1 --2- 172.21.15.107:0/1687362437 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f64f80392a0 0x7f64f803b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.876+0000 7f651a600640 1 --2- 172.21.15.107:0/1687362437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6514103820 0x7f6514198f80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.876+0000 7f651a600640 1 -- 172.21.15.107:0/1687362437 >> 172.21.15.107:0/1687362437 conn(0x7f65140feff0 msgr2=0x7f65140ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.876+0000 7f651a600640 1 -- 172.21.15.107:0/1687362437 shutdown_connections 2024-09-19T04:11:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:09.876+0000 7f651a600640 1 -- 172.21.15.107:0/1687362437 wait complete. 2024-09-19T04:11:10.524 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T04:11:10.524 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T04:11:11.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:10 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1687362437' entity='client.admin' 2024-09-19T04:11:11.676 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:12.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.195+0000 7f6a15800640 1 -- 172.21.15.107:0/1825648566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10101620 msgr2=0x7f6a10101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:12.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.195+0000 7f6a15800640 1 --2- 172.21.15.107:0/1825648566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10101620 0x7f6a10101a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009880 tx=0x7f6a0002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.195+0000 7f6a15800640 1 -- 172.21.15.107:0/1825648566 shutdown_connections 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.195+0000 7f6a15800640 1 --2- 172.21.15.107:0/1825648566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10101620 0x7f6a10101a20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.195+0000 7f6a15800640 1 -- 172.21.15.107:0/1825648566 >> 172.21.15.107:0/1825648566 conn(0x7f6a100fcdf0 msgr2=0x7f6a100ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.195+0000 7f6a15800640 1 -- 172.21.15.107:0/1825648566 shutdown_connections 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.195+0000 7f6a15800640 1 -- 172.21.15.107:0/1825648566 wait complete. 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.196+0000 7f6a15800640 1 Processor -- start 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.196+0000 7f6a15800640 1 -- start start 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.197+0000 7f6a15800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10101620 0x7f6a10190660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.197+0000 7f6a15800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10107730 con 0x7f6a10101620 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.197+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10101620 0x7f6a10190660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.197+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10101620 0x7f6a10190660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34096/0 (socket says 172.21.15.107:34096) 2024-09-19T04:11:12.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.197+0000 7f6a0ea00640 1 -- 172.21.15.107:0/1273995863 learned_addr learned my addr 172.21.15.107:0/1273995863 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:12.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.197+0000 7f6a0ea00640 1 -- 172.21.15.107:0/1273995863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a10190ba0 con 0x7f6a10101620 2024-09-19T04:11:12.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.198+0000 7f6a0ea00640 1 --2- 172.21.15.107:0/1273995863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10101620 0x7f6a10190660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a00004f60 tx=0x7f6a000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:12.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.198+0000 7f6a07400640 1 -- 172.21.15.107:0/1273995863 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a0003f070 con 0x7f6a10101620 2024-09-19T04:11:12.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.198+0000 7f6a07400640 1 -- 172.21.15.107:0/1273995863 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a00004cc0 con 0x7f6a10101620 2024-09-19T04:11:12.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.198+0000 7f6a07400640 1 -- 172.21.15.107:0/1273995863 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a00035030 con 0x7f6a10101620 2024-09-19T04:11:12.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.198+0000 7f6a15800640 1 -- 172.21.15.107:0/1273995863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a10190e30 con 0x7f6a10101620 2024-09-19T04:11:12.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.198+0000 7f6a15800640 1 -- 172.21.15.107:0/1273995863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a1018d3f0 con 0x7f6a10101620 2024-09-19T04:11:12.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.199+0000 7f6a07400640 1 -- 172.21.15.107:0/1273995863 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f6a00042070 con 0x7f6a10101620 2024-09-19T04:11:12.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.199+0000 7f6a15800640 1 -- 172.21.15.107:0/1273995863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a1018de30 con 0x7f6a10101620 2024-09-19T04:11:12.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.199+0000 7f6a07400640 1 --2- 172.21.15.107:0/1273995863 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69f4039300 0x7f69f403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:12.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.200+0000 7f6a07400640 1 -- 172.21.15.107:0/1273995863 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6a0006d110 con 0x7f6a10101620 2024-09-19T04:11:12.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.200+0000 7f6a0e000640 1 --2- 172.21.15.107:0/1273995863 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69f4039300 0x7f69f403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:12.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.200+0000 7f6a0e000640 1 --2- 172.21.15.107:0/1273995863 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69f4039300 0x7f69f403b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0098c0 tx=0x7f69fc006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:12.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.207+0000 7f6a07400640 1 -- 172.21.15.107:0/1273995863 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6a000303a0 con 0x7f6a10101620 2024-09-19T04:11:12.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.395+0000 7f6a15800640 1 -- 172.21.15.107:0/1273995863 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6a10192480 con 0x7f69f4039300 2024-09-19T04:11:12.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:12.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:12.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:12.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:12.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:12.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.399+0000 7f6a07400640 1 -- 172.21.15.107:0/1273995863 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a10192480 con 0x7f69f4039300 2024-09-19T04:11:12.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.402+0000 7f6a15800640 1 -- 172.21.15.107:0/1273995863 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69f4039300 msgr2=0x7f69f403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:12.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.402+0000 7f6a15800640 1 --2- 172.21.15.107:0/1273995863 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69f4039300 0x7f69f403b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0098c0 tx=0x7f69fc006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:12.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.402+0000 7f6a15800640 1 -- 172.21.15.107:0/1273995863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10101620 msgr2=0x7f6a10190660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:12.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.402+0000 7f6a15800640 1 --2- 172.21.15.107:0/1273995863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10101620 0x7f6a10190660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a00004f60 tx=0x7f6a000048f0 comp rx=0 tx=0).stop 2024-09-19T04:11:12.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.402+0000 7f6a15800640 1 -- 172.21.15.107:0/1273995863 shutdown_connections 2024-09-19T04:11:12.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.402+0000 7f6a15800640 1 --2- 172.21.15.107:0/1273995863 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69f4039300 0x7f69f403b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:12.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.402+0000 7f6a15800640 1 --2- 172.21.15.107:0/1273995863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10101620 0x7f6a10190660 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:12.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.402+0000 7f6a15800640 1 -- 172.21.15.107:0/1273995863 >> 172.21.15.107:0/1273995863 conn(0x7f6a100fcdf0 msgr2=0x7f6a100fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:12.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.402+0000 7f6a15800640 1 -- 172.21.15.107:0/1273995863 shutdown_connections 2024-09-19T04:11:12.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:12.402+0000 7f6a15800640 1 -- 172.21.15.107:0/1273995863 wait complete. 2024-09-19T04:11:13.486 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi115 2024-09-19T04:11:13.487 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:11:13.487 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.conf 2024-09-19T04:11:13.505 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:11:13.505 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:13.563 INFO:tasks.cephadm:Adding host smithi115 to orchestrator... 2024-09-19T04:11:13.563 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch host add smithi115 2024-09-19T04:11:13.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:13 smithi107 ceph-mon[30659]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:13.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:13 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:13.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:13 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:13.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:13 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:13.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:13 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:13.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:13 smithi107 ceph-mon[30659]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-19T04:11:14.064 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.630+0000 7fe635000640 1 -- 172.21.15.107:0/3802690166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630100360 msgr2=0x7fe630100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.630+0000 7fe635000640 1 --2- 172.21.15.107:0/3802690166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630100360 0x7fe630100740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe620009880 tx=0x7fe62002a670 comp rx=0 tx=0).stop 2024-09-19T04:11:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.630+0000 7fe635000640 1 -- 172.21.15.107:0/3802690166 shutdown_connections 2024-09-19T04:11:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.630+0000 7fe635000640 1 --2- 172.21.15.107:0/3802690166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630100360 0x7fe630100740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.630+0000 7fe635000640 1 -- 172.21.15.107:0/3802690166 >> 172.21.15.107:0/3802690166 conn(0x7fe6300fbfb0 msgr2=0x7fe6300fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.630+0000 7fe635000640 1 -- 172.21.15.107:0/3802690166 shutdown_connections 2024-09-19T04:11:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.630+0000 7fe635000640 1 -- 172.21.15.107:0/3802690166 wait complete. 2024-09-19T04:11:14.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.631+0000 7fe635000640 1 Processor -- start 2024-09-19T04:11:14.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.631+0000 7fe635000640 1 -- start start 2024-09-19T04:11:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.632+0000 7fe635000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630100360 0x7fe63019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.632+0000 7fe635000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe6301120f0 con 0x7fe630100360 2024-09-19T04:11:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.632+0000 7fe62f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630100360 0x7fe63019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.632+0000 7fe62f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630100360 0x7fe63019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34114/0 (socket says 172.21.15.107:34114) 2024-09-19T04:11:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.632+0000 7fe62f400640 1 -- 172.21.15.107:0/3815339372 learned_addr learned my addr 172.21.15.107:0/3815339372 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.632+0000 7fe62f400640 1 -- 172.21.15.107:0/3815339372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63019db20 con 0x7fe630100360 2024-09-19T04:11:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.633+0000 7fe62f400640 1 --2- 172.21.15.107:0/3815339372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630100360 0x7fe63019d5e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe620009230 tx=0x7fe620003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.633+0000 7fe62cc00640 1 -- 172.21.15.107:0/3815339372 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe62003f070 con 0x7fe630100360 2024-09-19T04:11:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.633+0000 7fe62cc00640 1 -- 172.21.15.107:0/3815339372 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe62003bd40 con 0x7fe630100360 2024-09-19T04:11:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.633+0000 7fe635000640 1 -- 172.21.15.107:0/3815339372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63019ddb0 con 0x7fe630100360 2024-09-19T04:11:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.633+0000 7fe62cc00640 1 -- 172.21.15.107:0/3815339372 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe620035030 con 0x7fe630100360 2024-09-19T04:11:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.633+0000 7fe635000640 1 -- 172.21.15.107:0/3815339372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63019e210 con 0x7fe630100360 2024-09-19T04:11:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.634+0000 7fe635000640 1 -- 172.21.15.107:0/3815339372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63006a7c0 con 0x7fe630100360 2024-09-19T04:11:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.634+0000 7fe62cc00640 1 -- 172.21.15.107:0/3815339372 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fe620042070 con 0x7fe630100360 2024-09-19T04:11:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.635+0000 7fe62cc00640 1 --2- 172.21.15.107:0/3815339372 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe614039300 0x7fe61403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.635+0000 7fe62cc00640 1 -- 172.21.15.107:0/3815339372 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe62006cf80 con 0x7fe630100360 2024-09-19T04:11:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.635+0000 7fe626a00640 1 --2- 172.21.15.107:0/3815339372 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe614039300 0x7fe61403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.636+0000 7fe626a00640 1 --2- 172.21.15.107:0/3815339372 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe614039300 0x7fe61403b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe6180098c0 tx=0x7fe618006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.640+0000 7fe62cc00640 1 -- 172.21.15.107:0/3815339372 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe6200412c0 con 0x7fe630100360 2024-09-19T04:11:14.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:14 smithi107 ceph-mon[30659]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:14.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:14 smithi107 ceph-mon[30659]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:14.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:14 smithi107 ceph-mon[30659]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:11:14.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:14 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:14.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:14 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:14.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:14 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:14.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:14.819+0000 7fe635000640 1 -- 172.21.15.107:0/3815339372 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}) -- 0x7fe63019e9a0 con 0x7fe614039300 2024-09-19T04:11:15.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:15 smithi107 ceph-mon[30659]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:16.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:16 smithi107 ceph-mon[30659]: Deploying cephadm binary to smithi115 2024-09-19T04:11:17.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.939+0000 7fe62cc00640 1 -- 172.21.15.107:0/3815339372 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fe63019e9a0 con 0x7fe614039300 2024-09-19T04:11:17.941 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi115' with addr '172.21.15.115' 2024-09-19T04:11:17.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.941+0000 7fe635000640 1 -- 172.21.15.107:0/3815339372 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe614039300 msgr2=0x7fe61403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:17.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.941+0000 7fe635000640 1 --2- 172.21.15.107:0/3815339372 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe614039300 0x7fe61403b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe6180098c0 tx=0x7fe618006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:17.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.941+0000 7fe635000640 1 -- 172.21.15.107:0/3815339372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630100360 msgr2=0x7fe63019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:17.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.941+0000 7fe635000640 1 --2- 172.21.15.107:0/3815339372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630100360 0x7fe63019d5e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe620009230 tx=0x7fe620003d80 comp rx=0 tx=0).stop 2024-09-19T04:11:17.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.941+0000 7fe635000640 1 -- 172.21.15.107:0/3815339372 shutdown_connections 2024-09-19T04:11:17.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.941+0000 7fe635000640 1 --2- 172.21.15.107:0/3815339372 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe614039300 0x7fe61403b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:17.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.941+0000 7fe635000640 1 --2- 172.21.15.107:0/3815339372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630100360 0x7fe63019d5e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:17.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.941+0000 7fe635000640 1 -- 172.21.15.107:0/3815339372 >> 172.21.15.107:0/3815339372 conn(0x7fe6300fbfb0 msgr2=0x7fe6300fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:17.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.941+0000 7fe635000640 1 -- 172.21.15.107:0/3815339372 shutdown_connections 2024-09-19T04:11:17.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:17.942+0000 7fe635000640 1 -- 172.21.15.107:0/3815339372 wait complete. 2024-09-19T04:11:18.539 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T04:11:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:19.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:18 smithi107 ceph-mon[30659]: Added host smithi115 2024-09-19T04:11:19.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:19.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:19.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:19.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.375+0000 7ff519a00640 1 -- 172.21.15.107:0/2773993342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514108280 msgr2=0x7ff514108660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:19.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.375+0000 7ff519a00640 1 --2- 172.21.15.107:0/2773993342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514108280 0x7ff514108660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff504009880 tx=0x7ff50402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:11:19.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.375+0000 7ff519a00640 1 -- 172.21.15.107:0/2773993342 shutdown_connections 2024-09-19T04:11:19.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.375+0000 7ff519a00640 1 --2- 172.21.15.107:0/2773993342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514108280 0x7ff514108660 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:19.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.375+0000 7ff519a00640 1 -- 172.21.15.107:0/2773993342 >> 172.21.15.107:0/2773993342 conn(0x7ff5140775f0 msgr2=0x7ff514079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:19.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.375+0000 7ff519a00640 1 -- 172.21.15.107:0/2773993342 shutdown_connections 2024-09-19T04:11:19.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.375+0000 7ff519a00640 1 -- 172.21.15.107:0/2773993342 wait complete. 2024-09-19T04:11:19.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.376+0000 7ff519a00640 1 Processor -- start 2024-09-19T04:11:19.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.376+0000 7ff519a00640 1 -- start start 2024-09-19T04:11:19.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.377+0000 7ff519a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514108280 0x7ff51419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:19.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.377+0000 7ff519a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff514112290 con 0x7ff514108280 2024-09-19T04:11:19.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.377+0000 7ff512a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514108280 0x7ff51419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:19.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.377+0000 7ff512a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514108280 0x7ff51419d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35470/0 (socket says 172.21.15.107:35470) 2024-09-19T04:11:19.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.377+0000 7ff512a00640 1 -- 172.21.15.107:0/843389375 learned_addr learned my addr 172.21.15.107:0/843389375 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:19.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.377+0000 7ff512a00640 1 -- 172.21.15.107:0/843389375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51419dc80 con 0x7ff514108280 2024-09-19T04:11:19.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.377+0000 7ff512a00640 1 --2- 172.21.15.107:0/843389375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514108280 0x7ff51419d740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff504009230 tx=0x7ff504003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:19.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.378+0000 7ff50be00640 1 -- 172.21.15.107:0/843389375 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff50403f070 con 0x7ff514108280 2024-09-19T04:11:19.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.378+0000 7ff50be00640 1 -- 172.21.15.107:0/843389375 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50403bd40 con 0x7ff514108280 2024-09-19T04:11:19.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.378+0000 7ff50be00640 1 -- 172.21.15.107:0/843389375 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff504035030 con 0x7ff514108280 2024-09-19T04:11:19.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.378+0000 7ff519a00640 1 -- 172.21.15.107:0/843389375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51419df70 con 0x7ff514108280 2024-09-19T04:11:19.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.378+0000 7ff519a00640 1 -- 172.21.15.107:0/843389375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a2010 con 0x7ff514108280 2024-09-19T04:11:19.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.379+0000 7ff50be00640 1 -- 172.21.15.107:0/843389375 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ff504042070 con 0x7ff514108280 2024-09-19T04:11:19.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.379+0000 7ff519a00640 1 -- 172.21.15.107:0/843389375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff514109330 con 0x7ff514108280 2024-09-19T04:11:19.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.379+0000 7ff50be00640 1 --2- 172.21.15.107:0/843389375 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff4f8039300 0x7ff4f803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:19.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.379+0000 7ff50be00640 1 -- 172.21.15.107:0/843389375 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff50406cf60 con 0x7ff514108280 2024-09-19T04:11:19.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.380+0000 7ff50a000640 1 --2- 172.21.15.107:0/843389375 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff4f8039300 0x7ff4f803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:19.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.380+0000 7ff50a000640 1 --2- 172.21.15.107:0/843389375 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff4f8039300 0x7ff4f803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0098c0 tx=0x7ff4fc006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:19.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.385+0000 7ff50be00640 1 -- 172.21.15.107:0/843389375 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff504030d60 con 0x7ff514108280 2024-09-19T04:11:19.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.566+0000 7ff519a00640 1 -- 172.21.15.107:0/843389375 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff51419eaa0 con 0x7ff4f8039300 2024-09-19T04:11:19.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.567+0000 7ff50be00640 1 -- 172.21.15.107:0/843389375 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7ff51419eaa0 con 0x7ff4f8039300 2024-09-19T04:11:19.569 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:19.569 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}] 2024-09-19T04:11:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.571+0000 7ff519a00640 1 -- 172.21.15.107:0/843389375 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff4f8039300 msgr2=0x7ff4f803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.571+0000 7ff519a00640 1 --2- 172.21.15.107:0/843389375 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff4f8039300 0x7ff4f803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0098c0 tx=0x7ff4fc006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:19.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.571+0000 7ff519a00640 1 -- 172.21.15.107:0/843389375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514108280 msgr2=0x7ff51419d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:19.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.571+0000 7ff519a00640 1 --2- 172.21.15.107:0/843389375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514108280 0x7ff51419d740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff504009230 tx=0x7ff504003d80 comp rx=0 tx=0).stop 2024-09-19T04:11:19.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.571+0000 7ff519a00640 1 -- 172.21.15.107:0/843389375 shutdown_connections 2024-09-19T04:11:19.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.571+0000 7ff519a00640 1 --2- 172.21.15.107:0/843389375 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff4f8039300 0x7ff4f803b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:19.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.571+0000 7ff519a00640 1 --2- 172.21.15.107:0/843389375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514108280 0x7ff51419d740 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:19.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.571+0000 7ff519a00640 1 -- 172.21.15.107:0/843389375 >> 172.21.15.107:0/843389375 conn(0x7ff5140775f0 msgr2=0x7ff51410f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:19.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.571+0000 7ff519a00640 1 -- 172.21.15.107:0/843389375 shutdown_connections 2024-09-19T04:11:19.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:19.571+0000 7ff519a00640 1 -- 172.21.15.107:0/843389375 wait complete. 2024-09-19T04:11:20.240 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi145 2024-09-19T04:11:20.240 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:11:20.240 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.conf 2024-09-19T04:11:20.259 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:11:20.259 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:20.318 INFO:tasks.cephadm:Adding host smithi145 to orchestrator... 2024-09-19T04:11:20.318 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch host add smithi145 2024-09-19T04:11:20.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:20 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:20.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:20 smithi107 ceph-mon[30659]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:11:20.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:20 smithi107 ceph-mon[30659]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:20.672 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.166+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1634025745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe0107460 msgr2=0x7f8fe0107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.166+0000 7f8fe4c00640 1 --2- 172.21.15.107:0/1634025745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe0107460 0x7f8fe0107840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009880 tx=0x7f8fd002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.166+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1634025745 shutdown_connections 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.166+0000 7f8fe4c00640 1 --2- 172.21.15.107:0/1634025745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe0107460 0x7f8fe0107840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.166+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1634025745 >> 172.21.15.107:0/1634025745 conn(0x7f8fe00fd0b0 msgr2=0x7f8fe00ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.166+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1634025745 shutdown_connections 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.166+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1634025745 wait complete. 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.167+0000 7f8fe4c00640 1 Processor -- start 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.167+0000 7f8fe4c00640 1 -- start start 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.167+0000 7f8fe4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe0107460 0x7f8fe01991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.167+0000 7f8fe4c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe010ba90 con 0x7f8fe0107460 2024-09-19T04:11:21.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.167+0000 7f8fdf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe0107460 0x7f8fe01991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.167+0000 7f8fdf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe0107460 0x7f8fe01991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35482/0 (socket says 172.21.15.107:35482) 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.167+0000 7f8fdf400640 1 -- 172.21.15.107:0/1213908859 learned_addr learned my addr 172.21.15.107:0/1213908859 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.168+0000 7f8fdf400640 1 -- 172.21.15.107:0/1213908859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe01996e0 con 0x7f8fe0107460 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.168+0000 7f8fdf400640 1 --2- 172.21.15.107:0/1213908859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe0107460 0x7f8fe01991a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009230 tx=0x7f8fd0003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.168+0000 7f8fdcc00640 1 -- 172.21.15.107:0/1213908859 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fd003f070 con 0x7f8fe0107460 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.168+0000 7f8fdcc00640 1 -- 172.21.15.107:0/1213908859 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd003bd40 con 0x7f8fe0107460 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.168+0000 7f8fdcc00640 1 -- 172.21.15.107:0/1213908859 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fd0035030 con 0x7f8fe0107460 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.168+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1213908859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe01999d0 con 0x7f8fe0107460 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.168+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1213908859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe019da70 con 0x7f8fe0107460 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.169+0000 7f8fdcc00640 1 -- 172.21.15.107:0/1213908859 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f8fd0042070 con 0x7f8fe0107460 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.170+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1213908859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe0102b10 con 0x7f8fe0107460 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.170+0000 7f8fdcc00640 1 --2- 172.21.15.107:0/1213908859 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8fc40392a0 0x7f8fc403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.170+0000 7f8fdcc00640 1 -- 172.21.15.107:0/1213908859 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8fd006cf60 con 0x7f8fe0107460 2024-09-19T04:11:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.170+0000 7f8fd7e00640 1 --2- 172.21.15.107:0/1213908859 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8fc40392a0 0x7f8fc403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.171+0000 7f8fd7e00640 1 --2- 172.21.15.107:0/1213908859 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8fc40392a0 0x7f8fc403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80098c0 tx=0x7f8fc8006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.175+0000 7f8fdcc00640 1 -- 172.21.15.107:0/1213908859 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8fd0036df0 con 0x7f8fe0107460 2024-09-19T04:11:21.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:21.356+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1213908859 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}) -- 0x7f8fe019a500 con 0x7f8fc40392a0 2024-09-19T04:11:23.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:22 smithi107 ceph-mon[30659]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:23.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:22 smithi107 ceph-mon[30659]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:23.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:23.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:23.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:23.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:23.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:23.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:23.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:24.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:23 smithi107 ceph-mon[30659]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-19T04:11:24.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:23 smithi107 ceph-mon[30659]: Deploying cephadm binary to smithi145 2024-09-19T04:11:24.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:23 smithi107 ceph-mon[30659]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:24.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:23 smithi107 ceph-mon[30659]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:24.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:23 smithi107 ceph-mon[30659]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:11:24.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:24.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:24.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:24.569 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi145' with addr '172.21.15.145' 2024-09-19T04:11:24.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.567+0000 7f8fdcc00640 1 -- 172.21.15.107:0/1213908859 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f8fe019a500 con 0x7f8fc40392a0 2024-09-19T04:11:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.570+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1213908859 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8fc40392a0 msgr2=0x7f8fc403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.570+0000 7f8fe4c00640 1 --2- 172.21.15.107:0/1213908859 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8fc40392a0 0x7f8fc403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80098c0 tx=0x7f8fc8006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.570+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1213908859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe0107460 msgr2=0x7f8fe01991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.570+0000 7f8fe4c00640 1 --2- 172.21.15.107:0/1213908859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe0107460 0x7f8fe01991a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009230 tx=0x7f8fd0003d80 comp rx=0 tx=0).stop 2024-09-19T04:11:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.570+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1213908859 shutdown_connections 2024-09-19T04:11:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.570+0000 7f8fe4c00640 1 --2- 172.21.15.107:0/1213908859 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8fc40392a0 0x7f8fc403b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.570+0000 7f8fe4c00640 1 --2- 172.21.15.107:0/1213908859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe0107460 0x7f8fe01991a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.570+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1213908859 >> 172.21.15.107:0/1213908859 conn(0x7f8fe00fd0b0 msgr2=0x7f8fe0074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.570+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1213908859 shutdown_connections 2024-09-19T04:11:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:24.570+0000 7f8fe4c00640 1 -- 172.21.15.107:0/1213908859 wait complete. 2024-09-19T04:11:25.202 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:24 smithi107 ceph-mon[30659]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:25.202 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:24 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:25.202 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:24 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:25.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T04:11:25.620 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:25.923 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:25 smithi107 ceph-mon[30659]: Added host smithi145 2024-09-19T04:11:25.923 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:25 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:25.923 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:25 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:26.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.163+0000 7f2dd8800640 1 -- 172.21.15.107:0/3998088702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dd0100310 msgr2=0x7f2dd01006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:26.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.163+0000 7f2dd8800640 1 --2- 172.21.15.107:0/3998088702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dd0100310 0x7f2dd01006f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009880 tx=0x7f2dc402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:11:26.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.163+0000 7f2dd8800640 1 -- 172.21.15.107:0/3998088702 shutdown_connections 2024-09-19T04:11:26.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.163+0000 7f2dd8800640 1 --2- 172.21.15.107:0/3998088702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dd0100310 0x7f2dd01006f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:26.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.163+0000 7f2dd8800640 1 -- 172.21.15.107:0/3998088702 >> 172.21.15.107:0/3998088702 conn(0x7f2dd00fc1a0 msgr2=0x7f2dd00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:26.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.163+0000 7f2dd8800640 1 -- 172.21.15.107:0/3998088702 shutdown_connections 2024-09-19T04:11:26.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.163+0000 7f2dd8800640 1 -- 172.21.15.107:0/3998088702 wait complete. 2024-09-19T04:11:26.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.165+0000 7f2dd8800640 1 Processor -- start 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.165+0000 7f2dd8800640 1 -- start start 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.165+0000 7f2dd8800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dd0100310 0x7f2dd0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.165+0000 7f2dd8800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd010ded0 con 0x7f2dd0100310 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.165+0000 7f2dd5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dd0100310 0x7f2dd0199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.166+0000 7f2dd5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dd0100310 0x7f2dd0199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35502/0 (socket says 172.21.15.107:35502) 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.166+0000 7f2dd5e00640 1 -- 172.21.15.107:0/2707027001 learned_addr learned my addr 172.21.15.107:0/2707027001 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.166+0000 7f2dd5e00640 1 -- 172.21.15.107:0/2707027001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd01998b0 con 0x7f2dd0100310 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.166+0000 7f2dd5e00640 1 --2- 172.21.15.107:0/2707027001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dd0100310 0x7f2dd0199370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009230 tx=0x7f2dc4003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.166+0000 7f2dcaa00640 1 -- 172.21.15.107:0/2707027001 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc403f070 con 0x7f2dd0100310 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.166+0000 7f2dcaa00640 1 -- 172.21.15.107:0/2707027001 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc403bd40 con 0x7f2dd0100310 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.166+0000 7f2dcaa00640 1 -- 172.21.15.107:0/2707027001 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc4035030 con 0x7f2dd0100310 2024-09-19T04:11:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.166+0000 7f2dd8800640 1 -- 172.21.15.107:0/2707027001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd0199b40 con 0x7f2dd0100310 2024-09-19T04:11:26.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.167+0000 7f2dd8800640 1 -- 172.21.15.107:0/2707027001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd0199fa0 con 0x7f2dd0100310 2024-09-19T04:11:26.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.168+0000 7f2dcaa00640 1 -- 172.21.15.107:0/2707027001 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f2dc4042070 con 0x7f2dd0100310 2024-09-19T04:11:26.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.168+0000 7f2dd8800640 1 -- 172.21.15.107:0/2707027001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd006a7c0 con 0x7f2dd0100310 2024-09-19T04:11:26.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.168+0000 7f2dcaa00640 1 --2- 172.21.15.107:0/2707027001 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2db8039300 0x7f2db803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:26.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.168+0000 7f2dcaa00640 1 -- 172.21.15.107:0/2707027001 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2dc406cfc0 con 0x7f2dd0100310 2024-09-19T04:11:26.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.168+0000 7f2dd5400640 1 --2- 172.21.15.107:0/2707027001 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2db8039300 0x7f2db803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:26.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.169+0000 7f2dd5400640 1 --2- 172.21.15.107:0/2707027001 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2db8039300 0x7f2db803b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00098c0 tx=0x7f2dc0006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:26.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.173+0000 7f2dcaa00640 1 -- 172.21.15.107:0/2707027001 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2dc40412c0 con 0x7f2dd0100310 2024-09-19T04:11:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.351+0000 7f2dd8800640 1 -- 172.21.15.107:0/2707027001 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2dd019a730 con 0x7f2db8039300 2024-09-19T04:11:26.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.352+0000 7f2dcaa00640 1 -- 172.21.15.107:0/2707027001 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f2dd019a730 con 0x7f2db8039300 2024-09-19T04:11:26.354 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:26.354 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}, {"addr": "172.21.15.145", "hostname": "smithi145", "labels": [], "status": ""}] 2024-09-19T04:11:26.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.354+0000 7f2dd8800640 1 -- 172.21.15.107:0/2707027001 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2db8039300 msgr2=0x7f2db803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:26.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.354+0000 7f2dd8800640 1 --2- 172.21.15.107:0/2707027001 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2db8039300 0x7f2db803b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00098c0 tx=0x7f2dc0006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:26.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.354+0000 7f2dd8800640 1 -- 172.21.15.107:0/2707027001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dd0100310 msgr2=0x7f2dd0199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.354+0000 7f2dd8800640 1 --2- 172.21.15.107:0/2707027001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dd0100310 0x7f2dd0199370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009230 tx=0x7f2dc4003d80 comp rx=0 tx=0).stop 2024-09-19T04:11:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.355+0000 7f2dd8800640 1 -- 172.21.15.107:0/2707027001 shutdown_connections 2024-09-19T04:11:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.355+0000 7f2dd8800640 1 --2- 172.21.15.107:0/2707027001 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2db8039300 0x7f2db803b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.355+0000 7f2dd8800640 1 --2- 172.21.15.107:0/2707027001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dd0100310 0x7f2dd0199370 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.355+0000 7f2dd8800640 1 -- 172.21.15.107:0/2707027001 >> 172.21.15.107:0/2707027001 conn(0x7f2dd00fc1a0 msgr2=0x7f2dd00fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.355+0000 7f2dd8800640 1 -- 172.21.15.107:0/2707027001 shutdown_connections 2024-09-19T04:11:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:26.355+0000 7f2dd8800640 1 -- 172.21.15.107:0/2707027001 wait complete. 2024-09-19T04:11:26.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:26 smithi107 ceph-mon[30659]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:26.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:27.065 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T04:11:27.065 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T04:11:27.421 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.930+0000 7fa07fa00640 1 -- 172.21.15.107:0/2991007212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa078074620 msgr2=0x7fa078074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.930+0000 7fa07fa00640 1 --2- 172.21.15.107:0/2991007212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa078074620 0x7fa078074a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009880 tx=0x7fa06c02a670 comp rx=0 tx=0).stop 2024-09-19T04:11:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.930+0000 7fa07fa00640 1 -- 172.21.15.107:0/2991007212 shutdown_connections 2024-09-19T04:11:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.930+0000 7fa07fa00640 1 --2- 172.21.15.107:0/2991007212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa078074620 0x7fa078074a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.930+0000 7fa07fa00640 1 -- 172.21.15.107:0/2991007212 >> 172.21.15.107:0/2991007212 conn(0x7fa0780fd2b0 msgr2=0x7fa0780ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.930+0000 7fa07fa00640 1 -- 172.21.15.107:0/2991007212 shutdown_connections 2024-09-19T04:11:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.930+0000 7fa07fa00640 1 -- 172.21.15.107:0/2991007212 wait complete. 2024-09-19T04:11:27.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.931+0000 7fa07fa00640 1 Processor -- start 2024-09-19T04:11:27.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.931+0000 7fa07fa00640 1 -- start start 2024-09-19T04:11:27.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.932+0000 7fa07fa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa078074620 0x7fa078199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:27.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.932+0000 7fa07fa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa07810df00 con 0x7fa078074620 2024-09-19T04:11:27.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.932+0000 7fa07d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa078074620 0x7fa078199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:27.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.932+0000 7fa07d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa078074620 0x7fa078199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35520/0 (socket says 172.21.15.107:35520) 2024-09-19T04:11:27.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.932+0000 7fa07d000640 1 -- 172.21.15.107:0/1143435285 learned_addr learned my addr 172.21.15.107:0/1143435285 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:27.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.932+0000 7fa07d000640 1 -- 172.21.15.107:0/1143435285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa078199970 con 0x7fa078074620 2024-09-19T04:11:27.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.933+0000 7fa07d000640 1 --2- 172.21.15.107:0/1143435285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa078074620 0x7fa078199430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009230 tx=0x7fa06c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:27.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.933+0000 7fa075600640 1 -- 172.21.15.107:0/1143435285 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa06c03f070 con 0x7fa078074620 2024-09-19T04:11:27.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.933+0000 7fa075600640 1 -- 172.21.15.107:0/1143435285 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa06c03bc80 con 0x7fa078074620 2024-09-19T04:11:27.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.933+0000 7fa07fa00640 1 -- 172.21.15.107:0/1143435285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa078199c00 con 0x7fa078074620 2024-09-19T04:11:27.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.933+0000 7fa075600640 1 -- 172.21.15.107:0/1143435285 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa06c035030 con 0x7fa078074620 2024-09-19T04:11:27.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.933+0000 7fa07fa00640 1 -- 172.21.15.107:0/1143435285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07819a060 con 0x7fa078074620 2024-09-19T04:11:27.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.934+0000 7fa075600640 1 -- 172.21.15.107:0/1143435285 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fa06c042070 con 0x7fa078074620 2024-09-19T04:11:27.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.934+0000 7fa07fa00640 1 -- 172.21.15.107:0/1143435285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa078075cd0 con 0x7fa078074620 2024-09-19T04:11:27.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.934+0000 7fa075600640 1 --2- 172.21.15.107:0/1143435285 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa060039300 0x7fa06003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:27.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.935+0000 7fa075600640 1 -- 172.21.15.107:0/1143435285 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa06c06d070 con 0x7fa078074620 2024-09-19T04:11:27.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.935+0000 7fa077e00640 1 --2- 172.21.15.107:0/1143435285 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa060039300 0x7fa06003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:27.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.935+0000 7fa077e00640 1 --2- 172.21.15.107:0/1143435285 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa060039300 0x7fa06003b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0640098c0 tx=0x7fa064006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:27.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:27.939+0000 7fa075600640 1 -- 172.21.15.107:0/1143435285 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa06c0372d0 con 0x7fa078074620 2024-09-19T04:11:28.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.119+0000 7fa07fa00640 1 -- 172.21.15.107:0/1143435285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa07819a7f0 con 0x7fa078074620 2024-09-19T04:11:28.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:27 smithi107 ceph-mon[30659]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:11:28.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.852+0000 7fa075600640 1 -- 172.21.15.107:0/1143435285 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fa06c031dd0 con 0x7fa078074620 2024-09-19T04:11:28.854 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-09-19T04:11:28.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.856+0000 7fa07fa00640 1 -- 172.21.15.107:0/1143435285 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa060039300 msgr2=0x7fa06003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:28.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.856+0000 7fa07fa00640 1 --2- 172.21.15.107:0/1143435285 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa060039300 0x7fa06003b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0640098c0 tx=0x7fa064006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:28.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.856+0000 7fa07fa00640 1 -- 172.21.15.107:0/1143435285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa078074620 msgr2=0x7fa078199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:28.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.856+0000 7fa07fa00640 1 --2- 172.21.15.107:0/1143435285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa078074620 0x7fa078199430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009230 tx=0x7fa06c003d40 comp rx=0 tx=0).stop 2024-09-19T04:11:28.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.856+0000 7fa07fa00640 1 -- 172.21.15.107:0/1143435285 shutdown_connections 2024-09-19T04:11:28.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.856+0000 7fa07fa00640 1 --2- 172.21.15.107:0/1143435285 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa060039300 0x7fa06003b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:28.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.856+0000 7fa07fa00640 1 --2- 172.21.15.107:0/1143435285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa078074620 0x7fa078199430 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:28.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.856+0000 7fa07fa00640 1 -- 172.21.15.107:0/1143435285 >> 172.21.15.107:0/1143435285 conn(0x7fa0780fd2b0 msgr2=0x7fa078105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:28.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.856+0000 7fa07fa00640 1 -- 172.21.15.107:0/1143435285 shutdown_connections 2024-09-19T04:11:28.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:28.856+0000 7fa07fa00640 1 -- 172.21.15.107:0/1143435285 wait complete. 2024-09-19T04:11:29.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:28 smithi107 ceph-mon[30659]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:29.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:28 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1143435285' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:11:29.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:29.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:29.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:29.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:29.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:29.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:29.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:29.597 INFO:tasks.cephadm:Adding mon.a on smithi107 2024-09-19T04:11:29.597 INFO:tasks.cephadm:Adding mon.b on smithi115 2024-09-19T04:11:29.597 INFO:tasks.cephadm:Adding mon.c on smithi145 2024-09-19T04:11:29.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi107:172.21.15.107=a;smithi115:172.21.15.115=b;smithi145:172.21.15.145=c' 2024-09-19T04:11:29.834 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:30.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:29 smithi107 ceph-mon[30659]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-19T04:11:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:29 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1143435285' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:11:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:29 smithi107 ceph-mon[30659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:29 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:29 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:29 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:30.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.231+0000 7fa871077640 1 -- 172.21.15.145:0/2818297479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa86c100990 msgr2=0x7fa86c100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:30.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.231+0000 7fa871077640 1 --2- 172.21.15.145:0/2818297479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa86c100990 0x7fa86c100d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009880 tx=0x7fa85c02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:11:30.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.231+0000 7fa871077640 1 -- 172.21.15.145:0/2818297479 shutdown_connections 2024-09-19T04:11:30.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.231+0000 7fa871077640 1 --2- 172.21.15.145:0/2818297479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa86c100990 0x7fa86c100d70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:30.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.231+0000 7fa871077640 1 -- 172.21.15.145:0/2818297479 >> 172.21.15.145:0/2818297479 conn(0x7fa86c076250 msgr2=0x7fa86c076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:30.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.231+0000 7fa871077640 1 -- 172.21.15.145:0/2818297479 shutdown_connections 2024-09-19T04:11:30.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.231+0000 7fa871077640 1 -- 172.21.15.145:0/2818297479 wait complete. 2024-09-19T04:11:30.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.232+0000 7fa871077640 1 Processor -- start 2024-09-19T04:11:30.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.232+0000 7fa871077640 1 -- start start 2024-09-19T04:11:30.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.233+0000 7fa871077640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa86c100990 0x7fa86c197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:30.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.233+0000 7fa871077640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c10ab90 con 0x7fa86c100990 2024-09-19T04:11:30.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.233+0000 7fa86ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa86c100990 0x7fa86c197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:30.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.233+0000 7fa86ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa86c100990 0x7fa86c197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.145:55344/0 (socket says 172.21.15.145:55344) 2024-09-19T04:11:30.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.233+0000 7fa86ad9d640 1 -- 172.21.15.145:0/750730976 learned_addr learned my addr 172.21.15.145:0/750730976 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:11:30.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.233+0000 7fa86ad9d640 1 -- 172.21.15.145:0/750730976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c197670 con 0x7fa86c100990 2024-09-19T04:11:30.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.233+0000 7fa86ad9d640 1 --2- 172.21.15.145:0/750730976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa86c100990 0x7fa86c197130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009230 tx=0x7fa85c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:30.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.234+0000 7fa863fff640 1 -- 172.21.15.145:0/750730976 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa85c03f070 con 0x7fa86c100990 2024-09-19T04:11:30.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.234+0000 7fa871077640 1 -- 172.21.15.145:0/750730976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86c197960 con 0x7fa86c100990 2024-09-19T04:11:30.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.234+0000 7fa863fff640 1 -- 172.21.15.145:0/750730976 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa85c03bc80 con 0x7fa86c100990 2024-09-19T04:11:30.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.234+0000 7fa863fff640 1 -- 172.21.15.145:0/750730976 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa85c035030 con 0x7fa86c100990 2024-09-19T04:11:30.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.234+0000 7fa871077640 1 -- 172.21.15.145:0/750730976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c19ba00 con 0x7fa86c100990 2024-09-19T04:11:30.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.235+0000 7fa871077640 1 -- 172.21.15.145:0/750730976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa86c101c40 con 0x7fa86c100990 2024-09-19T04:11:30.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.235+0000 7fa863fff640 1 -- 172.21.15.145:0/750730976 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fa85c042070 con 0x7fa86c100990 2024-09-19T04:11:30.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.235+0000 7fa863fff640 1 --2- 172.21.15.145:0/750730976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa858039300 0x7fa85803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:30.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.235+0000 7fa863fff640 1 -- 172.21.15.145:0/750730976 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa85c06cf70 con 0x7fa86c100990 2024-09-19T04:11:30.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.236+0000 7fa86a59c640 1 --2- 172.21.15.145:0/750730976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa858039300 0x7fa85803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:30.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.236+0000 7fa86a59c640 1 --2- 172.21.15.145:0/750730976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa858039300 0x7fa85803b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098c0 tx=0x7fa854006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:30.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.241+0000 7fa863fff640 1 -- 172.21.15.145:0/750730976 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa85c0372d0 con 0x7fa86c100990 2024-09-19T04:11:30.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.420+0000 7fa871077640 1 -- 172.21.15.145:0/750730976 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi115:172.21.15.115=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}) -- 0x7fa86c198490 con 0x7fa858039300 2024-09-19T04:11:30.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.426+0000 7fa863fff640 1 -- 172.21.15.145:0/750730976 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa86c198490 con 0x7fa858039300 2024-09-19T04:11:30.428 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mon update... 2024-09-19T04:11:30.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.428+0000 7fa871077640 1 -- 172.21.15.145:0/750730976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa858039300 msgr2=0x7fa85803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:30.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.428+0000 7fa871077640 1 --2- 172.21.15.145:0/750730976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa858039300 0x7fa85803b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098c0 tx=0x7fa854006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:30.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.428+0000 7fa871077640 1 -- 172.21.15.145:0/750730976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa86c100990 msgr2=0x7fa86c197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:30.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.428+0000 7fa871077640 1 --2- 172.21.15.145:0/750730976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa86c100990 0x7fa86c197130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009230 tx=0x7fa85c003d40 comp rx=0 tx=0).stop 2024-09-19T04:11:30.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.428+0000 7fa871077640 1 -- 172.21.15.145:0/750730976 shutdown_connections 2024-09-19T04:11:30.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.428+0000 7fa871077640 1 --2- 172.21.15.145:0/750730976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa858039300 0x7fa85803b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:30.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.428+0000 7fa871077640 1 --2- 172.21.15.145:0/750730976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa86c100990 0x7fa86c197130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:30.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.428+0000 7fa871077640 1 -- 172.21.15.145:0/750730976 >> 172.21.15.145:0/750730976 conn(0x7fa86c076250 msgr2=0x7fa86c1042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:30.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.428+0000 7fa871077640 1 -- 172.21.15.145:0/750730976 shutdown_connections 2024-09-19T04:11:30.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:30.428+0000 7fa871077640 1 -- 172.21.15.145:0/750730976 wait complete. 2024-09-19T04:11:31.042 DEBUG:teuthology.orchestra.run.smithi115:mon.b> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T04:11:31.045 DEBUG:teuthology.orchestra.run.smithi145:mon.c> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T04:11:31.047 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:11:31.047 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:11:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:11:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:31.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:30 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:32.013 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:32.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:31 smithi107 ceph-mon[30659]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi115:172.21.15.115=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:32.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:31 smithi107 ceph-mon[30659]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi115:172.21.15.115=b;smithi145:172.21.15.145=c;count:3 2024-09-19T04:11:32.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:31 smithi107 ceph-mon[30659]: Deploying daemon mon.c on smithi145 2024-09-19T04:11:33.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:32 smithi107 ceph-mon[30659]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:33.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.426+0000 7fb225c62640 1 -- 172.21.15.145:0/800833410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb22010c470 msgr2=0x7fb22010c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:33.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.426+0000 7fb225c62640 1 --2- 172.21.15.145:0/800833410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb22010c470 0x7fb22010c850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb210009880 tx=0x7fb21002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:11:33.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.426+0000 7fb225c62640 1 -- 172.21.15.145:0/800833410 shutdown_connections 2024-09-19T04:11:33.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.426+0000 7fb225c62640 1 --2- 172.21.15.145:0/800833410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb22010c470 0x7fb22010c850 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:33.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.426+0000 7fb225c62640 1 -- 172.21.15.145:0/800833410 >> 172.21.15.145:0/800833410 conn(0x7fb22006ad60 msgr2=0x7fb22006b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:33.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.426+0000 7fb225c62640 1 -- 172.21.15.145:0/800833410 shutdown_connections 2024-09-19T04:11:33.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.426+0000 7fb225c62640 1 -- 172.21.15.145:0/800833410 wait complete. 2024-09-19T04:11:33.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.427+0000 7fb225c62640 1 Processor -- start 2024-09-19T04:11:33.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.427+0000 7fb225c62640 1 -- start start 2024-09-19T04:11:33.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.427+0000 7fb225c62640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb22010c470 0x7fb2201a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:33.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.427+0000 7fb225c62640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb220116680 con 0x7fb22010c470 2024-09-19T04:11:33.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.428+0000 7fb21f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb22010c470 0x7fb2201a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:33.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.428+0000 7fb21f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb22010c470 0x7fb2201a1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.145:48470/0 (socket says 172.21.15.145:48470) 2024-09-19T04:11:33.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.428+0000 7fb21f7fe640 1 -- 172.21.15.145:0/3463091060 learned_addr learned my addr 172.21.15.145:0/3463091060 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:11:33.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.428+0000 7fb21f7fe640 1 -- 172.21.15.145:0/3463091060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2201a2160 con 0x7fb22010c470 2024-09-19T04:11:33.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.428+0000 7fb21f7fe640 1 --2- 172.21.15.145:0/3463091060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb22010c470 0x7fb2201a1c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb210009230 tx=0x7fb210003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:33.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.428+0000 7fb21cff9640 1 -- 172.21.15.145:0/3463091060 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb21003f070 con 0x7fb22010c470 2024-09-19T04:11:33.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.428+0000 7fb225c62640 1 -- 172.21.15.145:0/3463091060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2201a2450 con 0x7fb22010c470 2024-09-19T04:11:33.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.428+0000 7fb21cff9640 1 -- 172.21.15.145:0/3463091060 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb21003bd40 con 0x7fb22010c470 2024-09-19T04:11:33.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.428+0000 7fb21cff9640 1 -- 172.21.15.145:0/3463091060 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb210035030 con 0x7fb22010c470 2024-09-19T04:11:33.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.429+0000 7fb225c62640 1 -- 172.21.15.145:0/3463091060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2201a6520 con 0x7fb22010c470 2024-09-19T04:11:33.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.430+0000 7fb21cff9640 1 -- 172.21.15.145:0/3463091060 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fb210042070 con 0x7fb22010c470 2024-09-19T04:11:33.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.430+0000 7fb21cff9640 1 --2- 172.21.15.145:0/3463091060 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb2080392a0 0x7fb20803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:33.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.430+0000 7fb21cff9640 1 -- 172.21.15.145:0/3463091060 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb21006cf00 con 0x7fb22010c470 2024-09-19T04:11:33.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.430+0000 7fb225c62640 1 -- 172.21.15.145:0/3463091060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22010d720 con 0x7fb22010c470 2024-09-19T04:11:33.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.430+0000 7fb21effd640 1 --2- 172.21.15.145:0/3463091060 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb2080392a0 0x7fb20803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:33.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.431+0000 7fb21effd640 1 --2- 172.21.15.145:0/3463091060 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb2080392a0 0x7fb20803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2140098c0 tx=0x7fb214006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:33.438 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.436+0000 7fb21cff9640 1 -- 172.21.15.145:0/3463091060 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb21003ad80 con 0x7fb22010c470 2024-09-19T04:11:33.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.694+0000 7fb225c62640 1 -- 172.21.15.145:0/3463091060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb22010c8d0 con 0x7fb22010c470 2024-09-19T04:11:33.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.694+0000 7fb21cff9640 1 -- 172.21.15.145:0/3463091060 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fb21003b860 con 0x7fb22010c470 2024-09-19T04:11:33.697 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:11:33.697 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":1,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","modified":"2024-09-19T04:10:02.266251Z","created":"2024-09-19T04:10:02.266251Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T04:11:33.697 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 1 2024-09-19T04:11:33.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.697+0000 7fb225c62640 1 -- 172.21.15.145:0/3463091060 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb2080392a0 msgr2=0x7fb20803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:33.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.697+0000 7fb225c62640 1 --2- 172.21.15.145:0/3463091060 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb2080392a0 0x7fb20803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2140098c0 tx=0x7fb214006db0 comp rx=0 tx=0).stop 2024-09-19T04:11:33.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.697+0000 7fb225c62640 1 -- 172.21.15.145:0/3463091060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb22010c470 msgr2=0x7fb2201a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:33.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.697+0000 7fb225c62640 1 --2- 172.21.15.145:0/3463091060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb22010c470 0x7fb2201a1c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb210009230 tx=0x7fb210003d80 comp rx=0 tx=0).stop 2024-09-19T04:11:33.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.697+0000 7fb225c62640 1 -- 172.21.15.145:0/3463091060 shutdown_connections 2024-09-19T04:11:33.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.697+0000 7fb225c62640 1 --2- 172.21.15.145:0/3463091060 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb2080392a0 0x7fb20803b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:33.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.697+0000 7fb225c62640 1 --2- 172.21.15.145:0/3463091060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb22010c470 0x7fb2201a1c20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:33.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.697+0000 7fb225c62640 1 -- 172.21.15.145:0/3463091060 >> 172.21.15.145:0/3463091060 conn(0x7fb22006ad60 msgr2=0x7fb22010dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:33.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.697+0000 7fb225c62640 1 -- 172.21.15.145:0/3463091060 shutdown_connections 2024-09-19T04:11:33.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:33.697+0000 7fb225c62640 1 -- 172.21.15.145:0/3463091060 wait complete. 2024-09-19T04:11:34.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:33 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/3463091060' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:34.419 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 systemd[1]: Starting Ceph mon.c for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:11:34.691 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 podman[35077]: 2024-09-19 04:11:34.593053182 +0000 UTC m=+0.095123293 container create 63c8ed9438d42ad5c054dd3a42c26cfbf567e53f0712c9e39685fdc2da0be182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T04:11:34.692 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 podman[35077]: 2024-09-19 04:11:34.513692346 +0000 UTC m=+0.015762422 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:11:35.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 podman[35077]: 2024-09-19 04:11:34.786961145 +0000 UTC m=+0.289031216 container init 63c8ed9438d42ad5c054dd3a42c26cfbf567e53f0712c9e39685fdc2da0be182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T04:11:35.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 podman[35077]: 2024-09-19 04:11:34.789997451 +0000 UTC m=+0.292067517 container start 63c8ed9438d42ad5c054dd3a42c26cfbf567e53f0712c9e39685fdc2da0be182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T04:11:35.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T04:11:35.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T04:11:35.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: pidfile_write: ignore empty --pid-file 2024-09-19T04:11:35.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: load: jerasure load: lrc 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: RocksDB version: 7.9.2 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Git sha 0 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: DB SUMMARY 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: DB Session ID: 4Z2FLH0K5ZM71FK34IHP 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: CURRENT file: CURRENT 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.error_if_exists: 0 2024-09-19T04:11:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.create_if_missing: 0 2024-09-19T04:11:35.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.paranoid_checks: 1 2024-09-19T04:11:35.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T04:11:35.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T04:11:35.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T04:11:35.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.env: 0x55bc1f179940 2024-09-19T04:11:35.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T04:11:35.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.info_log: 0x55bc211af6e0 2024-09-19T04:11:35.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T04:11:35.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.statistics: (nil) 2024-09-19T04:11:35.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.use_fsync: 0 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_log_file_size: 0 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.allow_fallocate: 1 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.use_direct_reads: 0 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.db_log_dir: 2024-09-19T04:11:35.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.wal_dir: 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.write_buffer_manager: 0x55bc211bedc0 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T04:11:35.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.unordered_write: 0 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.row_cache: None 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.wal_filter: None 2024-09-19T04:11:35.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.two_write_queues: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.wal_compression: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.atomic_flush: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.log_readahead_size: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T04:11:35.232 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_background_jobs: 2 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_background_compactions: -1 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_subcompactions: 1 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T04:11:35.233 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_open_files: -1 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_background_flushes: -1 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Compression algorithms supported: 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: kZSTD supported: 0 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: kXpressCompression supported: 0 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: kBZip2Compression supported: 0 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: kLZ4Compression supported: 1 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: kZlibCompression supported: 1 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T04:11:35.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: kSnappyCompression supported: 1 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.merge_operator: 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_filter: None 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bc211af320) 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout: cache_index_and_filter_blocks: 1 2024-09-19T04:11:35.235 INFO:journalctl@ceph.mon.c.smithi145.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: pin_top_level_index_and_filter: 1 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: index_type: 0 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: data_block_index_type: 0 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: index_shortening: 1 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: checksum: 4 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: no_block_cache: 0 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache: 0x55bc211a31f0 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache_name: BinnedLRUCache 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache_options: 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: capacity : 536870912 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: num_shard_bits : 4 2024-09-19T04:11:35.236 INFO:journalctl@ceph.mon.c.smithi145.stdout: strict_capacity_limit : 0 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: high_pri_pool_ratio: 0.000 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache_compressed: (nil) 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: persistent_cache: (nil) 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_size: 4096 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_size_deviation: 10 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_restart_interval: 16 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: index_block_restart_interval: 1 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: metadata_block_size: 4096 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: partition_filters: 0 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: use_delta_encoding: 1 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: filter_policy: bloomfilter 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: whole_key_filtering: 1 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: verify_compression: 0 2024-09-19T04:11:35.237 INFO:journalctl@ceph.mon.c.smithi145.stdout: read_amp_bytes_per_bit: 0 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout: format_version: 5 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout: enable_index_compression: 1 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_align: 0 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout: max_auto_readahead_size: 262144 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout: prepopulate_block_cache: 0 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout: initial_auto_readahead_size: 8192 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compression: NoCompression 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T04:11:35.238 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.num_levels: 7 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T04:11:35.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T04:11:35.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T04:11:35.241 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T04:11:35.243 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T04:11:35.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.inplace_update_support: 0 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.bloom_locality: 0 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.max_successive_merges: 0 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T04:11:35.245 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.ttl: 2592000 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.enable_blob_files: false 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.min_blob_size: 0 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T04:11:35.246 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7361e04c-5ae4-4a67-9c2b-0191a189247a 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726719094816500, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726719094816909, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726719094, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7361e04c-5ae4-4a67-9c2b-0191a189247a", "db_session_id": "4Z2FLH0K5ZM71FK34IHP", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726719094817002, "job": 1, "event": "recovery_finished"} 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bc211d0e00 2024-09-19T04:11:35.247 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: DB pointer 0x55bc212cc000 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** DB Stats ** 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** Compaction Stats [default] ** 2024-09-19T04:11:35.248 INFO:journalctl@ceph.mon.c.smithi145.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** Compaction Stats [default] ** 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T04:11:35.249 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-19T04:11:35.250 INFO:journalctl@ceph.mon.c.smithi145.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:11:35.250 INFO:journalctl@ceph.mon.c.smithi145.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T04:11:35.250 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T04:11:35.250 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T04:11:35.250 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T04:11:35.250 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T04:11:35.250 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:11:35.250 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:11:35.250 INFO:journalctl@ceph.mon.c.smithi145.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout: Block cache BinnedLRUCache@0x55bc211a31f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: using public_addr v2:172.21.15.145:0/0 -> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: starting mon.c rank -1 at public addrs [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] at bind addrs [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(???) e0 preinit fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).mds e1 new map 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout: e1 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout: btime 2024-09-19T04:10:05:820337+0000 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:11:35.251 INFO:journalctl@ceph.mon.c.smithi145.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout: legacy client fscid: -1 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout: No filesystems configured 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:11:35.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:11:35.253 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-19T04:11:35.253 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Deploying cephadm binary to smithi145 2024-09-19T04:11:35.253 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:35.253 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:35.253 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:11:35.253 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Added host smithi145 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:35.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1143435285' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1143435285' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.255 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi115:172.21.15.115=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi115:172.21.15.115=b;smithi145:172.21.15.145=c;count:3 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: Deploying daemon mon.c on smithi145 2024-09-19T04:11:35.256 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:35.257 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/3463091060' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:35.257 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 ceph-mon[35096]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T04:11:35.257 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 bash[35077]: 63c8ed9438d42ad5c054dd3a42c26cfbf567e53f0712c9e39685fdc2da0be182 2024-09-19T04:11:35.257 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:34 smithi145 systemd[1]: Started Ceph mon.c for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:11:35.452 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:11:35.452 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:11:35.703 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:11:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:11:40.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: mon.a calling monitor election 2024-09-19T04:11:40.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:40.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: mon.c calling monitor election 2024-09-19T04:11:40.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:40.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:40.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:11:40.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: monmap epoch 2 2024-09-19T04:11:40.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:40.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: last_changed 2024-09-19T04:11:34.844546+0000 2024-09-19T04:11:40.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:11:40.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: min_mon_release 19 (squid) 2024-09-19T04:11:40.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: election_strategy: 1 2024-09-19T04:11:40.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: fsmap 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: mgrmap e13: x(active, since 40s) 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: overall HEALTH_OK 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:40.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:39 smithi145 ceph-mon[35096]: Deploying daemon mon.b on smithi115 2024-09-19T04:11:40.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:11:40.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: mon.a calling monitor election 2024-09-19T04:11:40.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:40.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: mon.c calling monitor election 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: monmap epoch 2 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: last_changed 2024-09-19T04:11:34.844546+0000 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: min_mon_release 19 (squid) 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: election_strategy: 1 2024-09-19T04:11:40.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: fsmap 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: mgrmap e13: x(active, since 40s) 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: overall HEALTH_OK 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:39 smithi107 ceph-mon[30659]: Deploying daemon mon.b on smithi115 2024-09-19T04:11:40.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.542+0000 7f732b518640 1 -- 172.21.15.145:0/474626559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7310003530 msgr2=0x7f73100059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:40.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.542+0000 7f732b518640 1 --2- 172.21.15.145:0/474626559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7310003530 0x7f73100059c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f732406a240 tx=0x7f732002c490 comp rx=0 tx=0).stop 2024-09-19T04:11:40.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.542+0000 7f732b518640 1 -- 172.21.15.145:0/474626559 shutdown_connections 2024-09-19T04:11:40.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.542+0000 7f732b518640 1 --2- 172.21.15.145:0/474626559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7310003530 0x7f73100059c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:40.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.542+0000 7f732b518640 1 -- 172.21.15.145:0/474626559 >> 172.21.15.145:0/474626559 conn(0x7f73240fc280 msgr2=0x7f73240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:40.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.542+0000 7f732b518640 1 -- 172.21.15.145:0/474626559 shutdown_connections 2024-09-19T04:11:40.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.543+0000 7f732b518640 1 -- 172.21.15.145:0/474626559 wait complete. 2024-09-19T04:11:40.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.543+0000 7f732b518640 1 Processor -- start 2024-09-19T04:11:40.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f732b518640 1 -- start start 2024-09-19T04:11:40.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f732b518640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7310003530 0x7f73241004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:40.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f732b518640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73241009e0 0x7f7324100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:40.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f732b518640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f73241097a0 con 0x7f7310003530 2024-09-19T04:11:40.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f732b518640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7324109620 con 0x7f73241009e0 2024-09-19T04:11:40.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f7328ab3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73241009e0 0x7f7324100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f7328ab3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73241009e0 0x7f7324100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56326/0 (socket says 172.21.15.145:56326) 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f7328ab3640 1 -- 172.21.15.145:0/2604136952 learned_addr learned my addr 172.21.15.145:0/2604136952 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f7328ab3640 1 -- 172.21.15.145:0/2604136952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73241009e0 msgr2=0x7f7324100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f7328ab3640 1 -- 172.21.15.145:0/2604136952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73241009e0 msgr2=0x7f7324100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f7328ab3640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73241009e0 0x7f7324100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.544+0000 7f7328ab3640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73241009e0 0x7f7324100e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.545+0000 7f73292b4640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7310003530 0x7f73241004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.545+0000 7f73292b4640 1 -- 172.21.15.145:0/2604136952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73241009e0 msgr2=0x7f7324100e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.545+0000 7f73292b4640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73241009e0 0x7f7324100e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.545+0000 7f73292b4640 1 -- 172.21.15.145:0/2604136952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7324102340 con 0x7f7310003530 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.545+0000 7f73292b4640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7310003530 0x7f73241004a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7320036040 tx=0x7f7320033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.546+0000 7f731a7fc640 1 -- 172.21.15.145:0/2604136952 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7320032920 con 0x7f7310003530 2024-09-19T04:11:40.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.546+0000 7f731a7fc640 1 -- 172.21.15.145:0/2604136952 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7320041070 con 0x7f7310003530 2024-09-19T04:11:40.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.546+0000 7f731a7fc640 1 -- 172.21.15.145:0/2604136952 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7320004970 con 0x7f7310003530 2024-09-19T04:11:40.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.546+0000 7f732b518640 1 -- 172.21.15.145:0/2604136952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73241a35b0 con 0x7f7310003530 2024-09-19T04:11:40.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.546+0000 7f732b518640 1 -- 172.21.15.145:0/2604136952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241a3b40 con 0x7f7310003530 2024-09-19T04:11:40.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.547+0000 7f731a7fc640 1 -- 172.21.15.145:0/2604136952 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f7320004c20 con 0x7f7310003530 2024-09-19T04:11:40.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.547+0000 7f732b518640 1 -- 172.21.15.145:0/2604136952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732406a4f0 con 0x7f7310003530 2024-09-19T04:11:40.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.547+0000 7f731a7fc640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7308039090 0x7f730803b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:40.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.547+0000 7f731a7fc640 1 -- 172.21.15.145:0/2604136952 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f732006e6a0 con 0x7f7310003530 2024-09-19T04:11:40.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.548+0000 7f7328ab3640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7308039090 0x7f730803b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:40.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.548+0000 7f7328ab3640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7308039090 0x7f730803b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7314009100 tx=0x7f7314008040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:40.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.553+0000 7f731a7fc640 1 -- 172.21.15.145:0/2604136952 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f73200381e0 con 0x7f7310003530 2024-09-19T04:11:40.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.785+0000 7f732b518640 1 -- 172.21.15.145:0/2604136952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7324190f80 con 0x7f7310003530 2024-09-19T04:11:40.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.786+0000 7f731a7fc640 1 -- 172.21.15.145:0/2604136952 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f7320042d60 con 0x7f7310003530 2024-09-19T04:11:40.788 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:11:40.788 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":2,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","modified":"2024-09-19T04:11:34.844546Z","created":"2024-09-19T04:10:02.266251Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:11:40.788 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 2 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 -- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7308039090 msgr2=0x7f730803b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7308039090 0x7f730803b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7314009100 tx=0x7f7314008040 comp rx=0 tx=0).stop 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 -- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7310003530 msgr2=0x7f73241004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7310003530 0x7f73241004a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7320036040 tx=0x7f7320033680 comp rx=0 tx=0).stop 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 -- 172.21.15.145:0/2604136952 shutdown_connections 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7308039090 0x7f730803b550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73241009e0 0x7f7324100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 --2- 172.21.15.145:0/2604136952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7310003530 0x7f73241004a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 -- 172.21.15.145:0/2604136952 >> 172.21.15.145:0/2604136952 conn(0x7f73240fc280 msgr2=0x7f73240fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 -- 172.21.15.145:0/2604136952 shutdown_connections 2024-09-19T04:11:40.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:40.788+0000 7f732b518640 1 -- 172.21.15.145:0/2604136952 wait complete. 2024-09-19T04:11:41.135 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:40 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/2604136952' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:41.135 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:40 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:41.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:40 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/2604136952' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:41.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:40 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:42.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:41 smithi145 ceph-mon[35096]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:42.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:41 smithi107 ceph-mon[30659]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:42.407 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:11:42.407 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:11:42.658 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:11:43.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.100+0000 7ff3ff662640 1 -- 172.21.15.145:0/1772924270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f810c470 msgr2=0x7ff3f810c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:43.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.100+0000 7ff3ff662640 1 --2- 172.21.15.145:0/1772924270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f810c470 0x7ff3f810c850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009880 tx=0x7ff3f402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:11:43.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.100+0000 7ff3ff662640 1 -- 172.21.15.145:0/1772924270 shutdown_connections 2024-09-19T04:11:43.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.100+0000 7ff3ff662640 1 --2- 172.21.15.145:0/1772924270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f810c470 0x7ff3f810c850 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:43.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.100+0000 7ff3ff662640 1 -- 172.21.15.145:0/1772924270 >> 172.21.15.145:0/1772924270 conn(0x7ff3f806b3f0 msgr2=0x7ff3f806b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:43.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.101+0000 7ff3ff662640 1 -- 172.21.15.145:0/1772924270 shutdown_connections 2024-09-19T04:11:43.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.101+0000 7ff3ff662640 1 -- 172.21.15.145:0/1772924270 wait complete. 2024-09-19T04:11:43.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.101+0000 7ff3ff662640 1 Processor -- start 2024-09-19T04:11:43.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.101+0000 7ff3ff662640 1 -- start start 2024-09-19T04:11:43.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3ff662640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f810c470 0x7ff3f81a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:43.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3ff662640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f81a22a0 0x7ff3f81a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:43.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3ff662640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8116720 con 0x7ff3f81a22a0 2024-09-19T04:11:43.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3ff662640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f81165a0 con 0x7ff3f810c470 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fd3fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f810c470 0x7ff3f81a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fd3fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f810c470 0x7ff3f81a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40094/0 (socket says 172.21.15.145:40094) 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fd3fe640 1 -- 172.21.15.145:0/3612646397 learned_addr learned my addr 172.21.15.145:0/3612646397 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fd3fe640 1 -- 172.21.15.145:0/3612646397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f810c470 msgr2=0x7ff3f81a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fd3fe640 1 -- 172.21.15.145:0/3612646397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f810c470 msgr2=0x7ff3f81a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fd3fe640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f810c470 0x7ff3f81a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fcbfd640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f81a22a0 0x7ff3f81a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fd3fe640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f810c470 0x7ff3f81a1d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fcbfd640 1 -- 172.21.15.145:0/3612646397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f810c470 msgr2=0x7ff3f81a1d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fcbfd640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f810c470 0x7ff3f81a1d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:43.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fcbfd640 1 -- 172.21.15.145:0/3612646397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f81a28c0 con 0x7ff3f81a22a0 2024-09-19T04:11:43.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.102+0000 7ff3fcbfd640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f81a22a0 0x7ff3f81a6630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800b3d0 tx=0x7ff3e800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:43.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.103+0000 7ff3ee7fc640 1 -- 172.21.15.145:0/3612646397 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff3e8011630 con 0x7ff3f81a22a0 2024-09-19T04:11:43.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.103+0000 7ff3ff662640 1 -- 172.21.15.145:0/3612646397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f81a6b70 con 0x7ff3f81a22a0 2024-09-19T04:11:43.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.103+0000 7ff3ee7fc640 1 -- 172.21.15.145:0/3612646397 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e8011cb0 con 0x7ff3f81a22a0 2024-09-19T04:11:43.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.103+0000 7ff3ee7fc640 1 -- 172.21.15.145:0/3612646397 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff3e8010df0 con 0x7ff3f81a22a0 2024-09-19T04:11:43.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.103+0000 7ff3ff662640 1 -- 172.21.15.145:0/3612646397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f81a70b0 con 0x7ff3f81a22a0 2024-09-19T04:11:43.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.104+0000 7ff3ee7fc640 1 -- 172.21.15.145:0/3612646397 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ff3e8018020 con 0x7ff3f81a22a0 2024-09-19T04:11:43.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.104+0000 7ff3ff662640 1 -- 172.21.15.145:0/3612646397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f810d7c0 con 0x7ff3f81a22a0 2024-09-19T04:11:43.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.104+0000 7ff3ee7fc640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3e4039040 0x7ff3e403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:43.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.104+0000 7ff3ee7fc640 1 -- 172.21.15.145:0/3612646397 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff3e804bed0 con 0x7ff3f81a22a0 2024-09-19T04:11:43.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.104+0000 7ff3fd3fe640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3e4039040 0x7ff3e403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:43.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.105+0000 7ff3fd3fe640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3e4039040 0x7ff3e403b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40096c0 tx=0x7ff3f4005a40 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:43.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.110+0000 7ff3ee7fc640 1 -- 172.21.15.145:0/3612646397 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff3e801f020 con 0x7ff3f81a22a0 2024-09-19T04:11:43.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.338+0000 7ff3ff662640 1 -- 172.21.15.145:0/3612646397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff3f81a73c0 con 0x7ff3f81a22a0 2024-09-19T04:11:43.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.338+0000 7ff3ee7fc640 1 -- 172.21.15.145:0/3612646397 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7ff3e8017500 con 0x7ff3f81a22a0 2024-09-19T04:11:43.340 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:11:43.340 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":2,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","modified":"2024-09-19T04:11:34.844546Z","created":"2024-09-19T04:10:02.266251Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:11:43.340 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 2 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 -- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3e4039040 msgr2=0x7ff3e403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3e4039040 0x7ff3e403b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40096c0 tx=0x7ff3f4005a40 comp rx=0 tx=0).stop 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 -- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f81a22a0 msgr2=0x7ff3f81a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f81a22a0 0x7ff3f81a6630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800b3d0 tx=0x7ff3e800b890 comp rx=0 tx=0).stop 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 -- 172.21.15.145:0/3612646397 shutdown_connections 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3e4039040 0x7ff3e403b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f81a22a0 0x7ff3f81a6630 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 --2- 172.21.15.145:0/3612646397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f810c470 0x7ff3f81a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 -- 172.21.15.145:0/3612646397 >> 172.21.15.145:0/3612646397 conn(0x7ff3f806b3f0 msgr2=0x7ff3f810dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 -- 172.21.15.145:0/3612646397 shutdown_connections 2024-09-19T04:11:43.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:43.340+0000 7ff3ff662640 1 -- 172.21.15.145:0/3612646397 wait complete. 2024-09-19T04:11:43.503 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-19T04:11:43.503 INFO:journalctl@ceph.mon.b.smithi115.stdout: e1 2024-09-19T04:11:43.503 INFO:journalctl@ceph.mon.b.smithi115.stdout: btime 2024-09-19T04:10:05:820337+0000 2024-09-19T04:11:43.504 INFO:journalctl@ceph.mon.b.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:11:43.504 INFO:journalctl@ceph.mon.b.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:11:43.504 INFO:journalctl@ceph.mon.b.smithi115.stdout: legacy client fscid: -1 2024-09-19T04:11:43.504 INFO:journalctl@ceph.mon.b.smithi115.stdout: 2024-09-19T04:11:43.504 INFO:journalctl@ceph.mon.b.smithi115.stdout: No filesystems configured 2024-09-19T04:11:43.504 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T04:11:43.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:43 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/3612646397' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:43.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:43 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/3612646397' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:43.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T04:11:43.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T04:11:43.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T04:11:43.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T04:11:43.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T04:11:43.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:11:43.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:11:43.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Deploying cephadm binary to smithi145 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:43.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Added host smithi145 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1143435285' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:11:43.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:43.950 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:43.950 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-19T04:11:43.950 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1143435285' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:11:43.950 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:43.950 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.950 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.950 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.950 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi115:172.21.15.115=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi115:172.21.15.115=b;smithi145:172.21.15.145=c;count:3 2024-09-19T04:11:43.951 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Deploying daemon mon.c on smithi145 2024-09-19T04:11:43.952 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:43.952 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/3463091060' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:43.952 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:11:43.952 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:43.952 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.a calling monitor election 2024-09-19T04:11:43.952 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:43.952 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:43.952 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:43.952 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.c calling monitor election 2024-09-19T04:11:43.952 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: monmap epoch 2 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: last_changed 2024-09-19T04:11:34.844546+0000 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: min_mon_release 19 (squid) 2024-09-19T04:11:43.953 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: election_strategy: 1 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: fsmap 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mgrmap e13: x(active, since 40s) 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: overall HEALTH_OK 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:43.954 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: Deploying daemon mon.b on smithi115 2024-09-19T04:11:43.955 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/2604136952' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:43.955 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:43.955 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:43.955 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/3612646397' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:43.955 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 ceph-mon[35477]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T04:11:43.955 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 bash[35458]: 96e16a26aaa9b97091cd71e657dc916ae9ba18f10c99991a503dc3337523f7f7 2024-09-19T04:11:43.955 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:43 smithi115 systemd[1]: Started Ceph mon.b for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:11:44.919 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:11:44.919 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:11:45.163 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:11:48.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.632+0000 7fbb6ff54640 1 -- 172.21.15.145:0/2776062826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb68106650 msgr2=0x7fbb54005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:48.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.632+0000 7fbb6ff54640 1 --2- 172.21.15.145:0/2776062826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb68106650 0x7fbb54005580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb64011c50 tx=0x7fbb6402be50 comp rx=0 tx=0).stop 2024-09-19T04:11:48.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.632+0000 7fbb6ff54640 1 -- 172.21.15.145:0/2776062826 shutdown_connections 2024-09-19T04:11:48.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.632+0000 7fbb6ff54640 1 --2- 172.21.15.145:0/2776062826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb68106650 0x7fbb54005580 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:48.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.632+0000 7fbb6ff54640 1 -- 172.21.15.145:0/2776062826 >> 172.21.15.145:0/2776062826 conn(0x7fbb680fc280 msgr2=0x7fbb680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:48.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.632+0000 7fbb6ff54640 1 -- 172.21.15.145:0/2776062826 shutdown_connections 2024-09-19T04:11:48.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.632+0000 7fbb6ff54640 1 -- 172.21.15.145:0/2776062826 wait complete. 2024-09-19T04:11:48.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.633+0000 7fbb6ff54640 1 Processor -- start 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.633+0000 7fbb6ff54640 1 -- start start 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.633+0000 7fbb6ff54640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb68106650 0x7fbb68196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.633+0000 7fbb6ff54640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb68196540 0x7fbb681969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6ff54640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb681a3670 0x7fbb681a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6ff54640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68108680 con 0x7fbb68106650 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6ff54640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68108500 con 0x7fbb68196540 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6ff54640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68108800 con 0x7fbb681a3670 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6e4f1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb681a3670 0x7fbb681a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6e4f1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb681a3670 0x7fbb681a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40098/0 (socket says 172.21.15.145:40098) 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6e4f1640 1 -- 172.21.15.145:0/3783277619 learned_addr learned my addr 172.21.15.145:0/3783277619 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6d4ef640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb68196540 0x7fbb681969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:48.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6dcf0640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb68106650 0x7fbb68196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6e4f1640 1 -- 172.21.15.145:0/3783277619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb681a3670 msgr2=0x7fbb681a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6e4f1640 1 -- 172.21.15.145:0/3783277619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb681a3670 msgr2=0x7fbb681a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6e4f1640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb681a3670 0x7fbb681a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6e4f1640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb681a3670 0x7fbb681a5a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6d4ef640 1 -- 172.21.15.145:0/3783277619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb68196540 msgr2=0x7fbb681969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6d4ef640 1 -- 172.21.15.145:0/3783277619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb68196540 msgr2=0x7fbb681969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6d4ef640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb68196540 0x7fbb681969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6d4ef640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb68196540 0x7fbb681969a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6dcf0640 1 -- 172.21.15.145:0/3783277619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb681a3670 msgr2=0x7fbb681a5a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6dcf0640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb681a3670 0x7fbb681a5a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6dcf0640 1 -- 172.21.15.145:0/3783277619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb68196540 msgr2=0x7fbb681969a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:11:48.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6dcf0640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb68196540 0x7fbb681969a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:48.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.634+0000 7fbb6dcf0640 1 -- 172.21.15.145:0/3783277619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb681a5fd0 con 0x7fbb68106650 2024-09-19T04:11:48.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.635+0000 7fbb6dcf0640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb68106650 0x7fbb68196000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb64011c50 tx=0x7fbb6402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:48.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.635+0000 7fbb5effd640 1 -- 172.21.15.145:0/3783277619 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64032040 con 0x7fbb68106650 2024-09-19T04:11:48.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.635+0000 7fbb5effd640 1 -- 172.21.15.145:0/3783277619 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb64043070 con 0x7fbb68106650 2024-09-19T04:11:48.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.635+0000 7fbb6ff54640 1 -- 172.21.15.145:0/3783277619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb681a6260 con 0x7fbb68106650 2024-09-19T04:11:48.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.635+0000 7fbb5effd640 1 -- 172.21.15.145:0/3783277619 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64004ae0 con 0x7fbb68106650 2024-09-19T04:11:48.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.635+0000 7fbb6ff54640 1 -- 172.21.15.145:0/3783277619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb681a6870 con 0x7fbb68106650 2024-09-19T04:11:48.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.636+0000 7fbb5effd640 1 -- 172.21.15.145:0/3783277619 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fbb64004c80 con 0x7fbb68106650 2024-09-19T04:11:48.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.636+0000 7fbb5effd640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbb4c039730 0x7fbb4c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:48.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.636+0000 7fbb6ff54640 1 -- 172.21.15.145:0/3783277619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb681097f0 con 0x7fbb68106650 2024-09-19T04:11:48.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.636+0000 7fbb5effd640 1 -- 172.21.15.145:0/3783277619 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbb6406e030 con 0x7fbb68106650 2024-09-19T04:11:48.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.637+0000 7fbb6d4ef640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbb4c039730 0x7fbb4c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:48.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.637+0000 7fbb6d4ef640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbb4c039730 0x7fbb4c03bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb58006fd0 tx=0x7fbb58008040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:48.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.642+0000 7fbb5effd640 1 -- 172.21.15.145:0/3783277619 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbb6402cd60 con 0x7fbb68106650 2024-09-19T04:11:48.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.884+0000 7fbb6ff54640 1 -- 172.21.15.145:0/3783277619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbb6806a200 con 0x7fbb68106650 2024-09-19T04:11:48.886 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: mon.c calling monitor election 2024-09-19T04:11:48.886 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:11:48.886 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: mon.a calling monitor election 2024-09-19T04:11:48.886 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.886 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:48.886 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:48.886 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.886 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.886 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:48.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:48.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:11:48.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: monmap epoch 3 2024-09-19T04:11:48.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:48.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: last_changed 2024-09-19T04:11:43.508260+0000 2024-09-19T04:11:48.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: min_mon_release 19 (squid) 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: election_strategy: 1 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: fsmap 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: mgrmap e13: x(active, since 48s) 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: overall HEALTH_OK 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:48.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:48.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.884+0000 7fbb5effd640 1 -- 172.21.15.145:0/3783277619 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fbb6402de20 con 0x7fbb68106650 2024-09-19T04:11:48.889 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:11:48.889 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":3,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","modified":"2024-09-19T04:11:43.508260Z","created":"2024-09-19T04:10:02.266251Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:11:48.890 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 3 2024-09-19T04:11:48.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 -- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbb4c039730 msgr2=0x7fbb4c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:48.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbb4c039730 0x7fbb4c03bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb58006fd0 tx=0x7fbb58008040 comp rx=0 tx=0).stop 2024-09-19T04:11:48.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 -- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb68106650 msgr2=0x7fbb68196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:48.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb68106650 0x7fbb68196000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb64011c50 tx=0x7fbb6402cc70 comp rx=0 tx=0).stop 2024-09-19T04:11:48.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 -- 172.21.15.145:0/3783277619 shutdown_connections 2024-09-19T04:11:48.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbb4c039730 0x7fbb4c03bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:48.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb681a3670 0x7fbb681a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:48.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb68196540 0x7fbb681969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:48.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 --2- 172.21.15.145:0/3783277619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb68106650 0x7fbb68196000 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:48.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 -- 172.21.15.145:0/3783277619 >> 172.21.15.145:0/3783277619 conn(0x7fbb680fc280 msgr2=0x7fbb680fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:48.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 -- 172.21.15.145:0/3783277619 shutdown_connections 2024-09-19T04:11:48.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:48.887+0000 7fbb6ff54640 1 -- 172.21.15.145:0/3783277619 wait complete. 2024-09-19T04:11:48.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: mon.c calling monitor election 2024-09-19T04:11:48.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:11:48.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: mon.a calling monitor election 2024-09-19T04:11:48.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:48.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:48.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: monmap epoch 3 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: last_changed 2024-09-19T04:11:43.508260+0000 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:11:48.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: min_mon_release 19 (squid) 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: election_strategy: 1 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: fsmap 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: mgrmap e13: x(active, since 48s) 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: overall HEALTH_OK 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:49.434 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T04:11:49.434 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T04:11:49.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:49 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:49.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:49 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/3783277619' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:49.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:49 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:49.803 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:49.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:49 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:49.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:49 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/3783277619' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:49.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:49 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:50.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.322+0000 7f77a3a00640 1 -- 172.21.15.107:0/1070930672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f779c076720 msgr2=0x7f779c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:50.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.322+0000 7f77a3a00640 1 --2- 172.21.15.107:0/1070930672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f779c076720 0x7f779c076b00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7790009880 tx=0x7f779002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:11:50.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.323+0000 7f77a3a00640 1 -- 172.21.15.107:0/1070930672 shutdown_connections 2024-09-19T04:11:50.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.323+0000 7f77a3a00640 1 --2- 172.21.15.107:0/1070930672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f779c076720 0x7f779c076b00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:50.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.323+0000 7f77a3a00640 1 -- 172.21.15.107:0/1070930672 >> 172.21.15.107:0/1070930672 conn(0x7f779c0ff4e0 msgr2=0x7f779c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:50.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.323+0000 7f77a3a00640 1 -- 172.21.15.107:0/1070930672 shutdown_connections 2024-09-19T04:11:50.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.323+0000 7f77a3a00640 1 -- 172.21.15.107:0/1070930672 wait complete. 2024-09-19T04:11:50.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.324+0000 7f77a3a00640 1 Processor -- start 2024-09-19T04:11:50.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.324+0000 7f77a3a00640 1 -- start start 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a3a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f779c076720 0x7f779c1a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a3a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f779c1a3be0 0x7f779c19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a3a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c19dd60 0x7f779c19e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a3a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1122d0 con 0x7f779c1a3be0 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a3a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f779c112150 con 0x7f779c19dd60 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a3a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f779c112450 con 0x7f779c076720 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f779be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f779c1a3be0 0x7f779c19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f779be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f779c1a3be0 0x7f779c19d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44638/0 (socket says 172.21.15.107:44638) 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f779be00640 1 -- 172.21.15.107:0/2428854734 learned_addr learned my addr 172.21.15.107:0/2428854734 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a1000640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f779c076720 0x7f779c1a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a1a00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c19dd60 0x7f779c19e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:50.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a1a00640 1 -- 172.21.15.107:0/2428854734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c19dd60 msgr2=0x7f779c19e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a1a00640 1 -- 172.21.15.107:0/2428854734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c19dd60 msgr2=0x7f779c19e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a1a00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c19dd60 0x7f779c19e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f779be00640 1 -- 172.21.15.107:0/2428854734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f779c076720 msgr2=0x7f779c1a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f779be00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f779c076720 0x7f779c1a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f77a1a00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c19dd60 0x7f779c19e1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f779be00640 1 -- 172.21.15.107:0/2428854734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c19dd60 msgr2=0x7f779c19e1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f779be00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c19dd60 0x7f779c19e1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.325+0000 7f779be00640 1 -- 172.21.15.107:0/2428854734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c1aa500 con 0x7f779c1a3be0 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.326+0000 7f779be00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f779c1a3be0 0x7f779c19d820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f778800b520 tx=0x7f778800b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.326+0000 7f7799600640 1 -- 172.21.15.107:0/2428854734 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788004230 con 0x7f779c1a3be0 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.326+0000 7f7799600640 1 -- 172.21.15.107:0/2428854734 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77880043d0 con 0x7f779c1a3be0 2024-09-19T04:11:50.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.326+0000 7f7799600640 1 -- 172.21.15.107:0/2428854734 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788003d60 con 0x7f779c1a3be0 2024-09-19T04:11:50.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.326+0000 7f77a3a00640 1 -- 172.21.15.107:0/2428854734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779c1aa730 con 0x7f779c1a3be0 2024-09-19T04:11:50.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.326+0000 7f77a3a00640 1 -- 172.21.15.107:0/2428854734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779c1aacc0 con 0x7f779c1a3be0 2024-09-19T04:11:50.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.327+0000 7f77a3a00640 1 -- 172.21.15.107:0/2428854734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f779c077e60 con 0x7f779c1a3be0 2024-09-19T04:11:50.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.328+0000 7f7799600640 1 -- 172.21.15.107:0/2428854734 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f778800e070 con 0x7f779c1a3be0 2024-09-19T04:11:50.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.329+0000 7f7799600640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7784039730 0x7f778403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:50.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.329+0000 7f7799600640 1 -- 172.21.15.107:0/2428854734 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f778804bd60 con 0x7f779c1a3be0 2024-09-19T04:11:50.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.329+0000 7f77a1000640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7784039730 0x7f778403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:50.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.330+0000 7f77a1000640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7784039730 0x7f778403bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7790000c00 tx=0x7f7790005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:50.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.334+0000 7f7799600640 1 -- 172.21.15.107:0/2428854734 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7788051020 con 0x7f779c1a3be0 2024-09-19T04:11:50.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.522+0000 7f77a3a00640 1 -- 172.21.15.107:0/2428854734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f779c1aafe0 con 0x7f779c1a3be0 2024-09-19T04:11:50.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.525+0000 7f7799600640 1 -- 172.21.15.107:0/2428854734 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f7788014020 con 0x7f779c1a3be0 2024-09-19T04:11:50.527 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:50.527 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-09-19T04:11:50.527 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:50.527 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] 2024-09-19T04:11:50.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.528+0000 7f77a3a00640 1 -- 172.21.15.107:0/2428854734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7784039730 msgr2=0x7f778403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:50.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.528+0000 7f77a3a00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7784039730 0x7f778403bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7790000c00 tx=0x7f7790005a20 comp rx=0 tx=0).stop 2024-09-19T04:11:50.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.528+0000 7f77a3a00640 1 -- 172.21.15.107:0/2428854734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f779c1a3be0 msgr2=0x7f779c19d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:50.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.528+0000 7f77a3a00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f779c1a3be0 0x7f779c19d820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f778800b520 tx=0x7f778800b9e0 comp rx=0 tx=0).stop 2024-09-19T04:11:50.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.528+0000 7f77a3a00640 1 -- 172.21.15.107:0/2428854734 shutdown_connections 2024-09-19T04:11:50.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.528+0000 7f77a3a00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7784039730 0x7f778403bbf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:50.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.528+0000 7f77a3a00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c19dd60 0x7f779c19e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:50.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.528+0000 7f77a3a00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f779c1a3be0 0x7f779c19d820 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:50.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.528+0000 7f77a3a00640 1 --2- 172.21.15.107:0/2428854734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f779c076720 0x7f779c1a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:50.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.528+0000 7f77a3a00640 1 -- 172.21.15.107:0/2428854734 >> 172.21.15.107:0/2428854734 conn(0x7f779c0ff4e0 msgr2=0x7f779c0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:50.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.529+0000 7f77a3a00640 1 -- 172.21.15.107:0/2428854734 shutdown_connections 2024-09-19T04:11:50.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:50.529+0000 7f77a3a00640 1 -- 172.21.15.107:0/2428854734 wait complete. 2024-09-19T04:11:50.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: mon.c calling monitor election 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: mon.a calling monitor election 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: monmap epoch 3 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: last_changed 2024-09-19T04:11:43.508260+0000 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: min_mon_release 19 (squid) 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: election_strategy: 1 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: fsmap 2024-09-19T04:11:50.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:50.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: mgrmap e13: x(active, since 48s) 2024-09-19T04:11:50.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: overall HEALTH_OK 2024-09-19T04:11:50.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:50.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:50.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:50.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:50.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:50.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/3783277619' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:11:50.949 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:51.292 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T04:11:51.292 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:11:51.292 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:11:51.350 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:11:51.350 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:51.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: mon.b calling monitor election 2024-09-19T04:11:51.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-19T04:11:51.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-19T04:11:51.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-19T04:11:51.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: mon.b calling monitor election 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: mon.c calling monitor election 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: mon.a calling monitor election 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: monmap epoch 3 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: last_changed 2024-09-19T04:11:43.508260+0000 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: min_mon_release 19 (squid) 2024-09-19T04:11:51.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: election_strategy: 1 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: fsmap 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: mgrmap e13: x(active, since 50s) 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: overall HEALTH_OK 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2428854734' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:51.447 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.447 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: mon.b calling monitor election 2024-09-19T04:11:51.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-19T04:11:51.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-19T04:11:51.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-19T04:11:51.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: mon.b calling monitor election 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: mon.c calling monitor election 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: mon.a calling monitor election 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: monmap epoch 3 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: last_changed 2024-09-19T04:11:43.508260+0000 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: min_mon_release 19 (squid) 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: election_strategy: 1 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:11:51.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: fsmap 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: mgrmap e13: x(active, since 50s) 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: overall HEALTH_OK 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2428854734' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.450 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:11:51.450 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:11:51.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: mon.b calling monitor election 2024-09-19T04:11:51.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-19T04:11:51.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-19T04:11:51.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-19T04:11:51.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: mon.b calling monitor election 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: mon.c calling monitor election 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: mon.a calling monitor election 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: monmap epoch 3 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: last_changed 2024-09-19T04:11:43.508260+0000 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: created 2024-09-19T04:10:02.266251+0000 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: min_mon_release 19 (squid) 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: election_strategy: 1 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-19T04:11:51.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: fsmap 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: mgrmap e13: x(active, since 50s) 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: overall HEALTH_OK 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2428854734' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:51.478 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:11:51.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:51.544 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:11:51.544 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:11:51.575 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:11:51.575 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:11:51.643 INFO:tasks.cephadm:Adding mgr.x on smithi107 2024-09-19T04:11:51.643 INFO:tasks.cephadm:Adding mgr.y on smithi115 2024-09-19T04:11:51.643 INFO:tasks.cephadm:Adding mgr.z on smithi145 2024-09-19T04:11:51.643 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi107=x;smithi115=y;smithi145=z' 2024-09-19T04:11:51.923 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:11:52.364 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.364 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.364 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.364 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.364 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.364 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:11:52.364 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:52.365 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:11:52.365 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:52.365 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: Reconfiguring daemon mon.a on smithi107 2024-09-19T04:11:52.365 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:52.365 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:52 smithi145 ceph-mon[35096]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.363+0000 7fdfee8bf640 1 -- 172.21.15.145:0/3382320065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfe81005e0 msgr2=0x7fdfe81009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.363+0000 7fdfee8bf640 1 --2- 172.21.15.145:0/3382320065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfe81005e0 0x7fdfe81009e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009880 tx=0x7fdfd802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:11:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.363+0000 7fdfee8bf640 1 -- 172.21.15.145:0/3382320065 shutdown_connections 2024-09-19T04:11:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.363+0000 7fdfee8bf640 1 --2- 172.21.15.145:0/3382320065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfe81005e0 0x7fdfe81009e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.363+0000 7fdfee8bf640 1 -- 172.21.15.145:0/3382320065 >> 172.21.15.145:0/3382320065 conn(0x7fdfe80fbdb0 msgr2=0x7fdfe80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.363+0000 7fdfee8bf640 1 -- 172.21.15.145:0/3382320065 shutdown_connections 2024-09-19T04:11:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.363+0000 7fdfee8bf640 1 -- 172.21.15.145:0/3382320065 wait complete. 2024-09-19T04:11:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfee8bf640 1 Processor -- start 2024-09-19T04:11:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfee8bf640 1 -- start start 2024-09-19T04:11:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfee8bf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfe81005e0 0x7fdfe8195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfee8bf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfe8196410 0x7fdfe819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfee8bf640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfe8196890 0x7fdfe819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfee8bf640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81066d0 con 0x7fdfe81005e0 2024-09-19T04:11:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfee8bf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8106550 con 0x7fdfe8196410 2024-09-19T04:11:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfee8bf640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8106850 con 0x7fdfe8196890 2024-09-19T04:11:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfece5c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfe8196890 0x7fdfe819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfece5c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfe8196890 0x7fdfe819da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37988/0 (socket says 172.21.15.145:37988) 2024-09-19T04:11:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfece5c640 1 -- 172.21.15.145:0/460065652 learned_addr learned my addr 172.21.15.145:0/460065652 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:11:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfe7fff640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfe81005e0 0x7fdfe8195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.364+0000 7fdfdffff640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfe8196410 0x7fdfe819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfece5c640 1 -- 172.21.15.145:0/460065652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfe8196410 msgr2=0x7fdfe819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfece5c640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfe8196410 0x7fdfe819b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfece5c640 1 -- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfe81005e0 msgr2=0x7fdfe8195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfece5c640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfe81005e0 0x7fdfe8195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfece5c640 1 -- 172.21.15.145:0/460065652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe819e2d0 con 0x7fdfe8196890 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfe7fff640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfe81005e0 0x7fdfe8195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfece5c640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfe8196890 0x7fdfe819da10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000df40 tx=0x7fdfe000df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfe5ffb640 1 -- 172.21.15.145:0/460065652 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000ea60 con 0x7fdfe8196890 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfe5ffb640 1 -- 172.21.15.145:0/460065652 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe000ec00 con 0x7fdfe8196890 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfee8bf640 1 -- 172.21.15.145:0/460065652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe8069c10 con 0x7fdfe8196890 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfe5ffb640 1 -- 172.21.15.145:0/460065652 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0016780 con 0x7fdfe8196890 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.365+0000 7fdfee8bf640 1 -- 172.21.15.145:0/460065652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe806a1d0 con 0x7fdfe8196890 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.366+0000 7fdfe5ffb640 1 -- 172.21.15.145:0/460065652 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fdfe0016a00 con 0x7fdfe8196890 2024-09-19T04:11:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.366+0000 7fdfee8bf640 1 -- 172.21.15.145:0/460065652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe8100a60 con 0x7fdfe8196890 2024-09-19T04:11:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.366+0000 7fdfe5ffb640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfd4039730 0x7fdfd403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.367+0000 7fdfe5ffb640 1 -- 172.21.15.145:0/460065652 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdfe004e1c0 con 0x7fdfe8196890 2024-09-19T04:11:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.367+0000 7fdfe7fff640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfd4039730 0x7fdfd403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.367+0000 7fdfe7fff640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfd4039730 0x7fdfd403bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009230 tx=0x7fdfd80315d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:52.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.371+0000 7fdfe5ffb640 1 -- 172.21.15.145:0/460065652 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdfe000ee20 con 0x7fdfe8196890 2024-09-19T04:11:52.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:11:52.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:52.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:11:52.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:52.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: Reconfiguring daemon mon.a on smithi107 2024-09-19T04:11:52.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:52.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:52 smithi115 ceph-mon[35477]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:52.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:52.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:11:52.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:52.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:11:52.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:52.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: Reconfiguring daemon mon.a on smithi107 2024-09-19T04:11:52.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:11:52.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:52 smithi107 ceph-mon[30659]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:52.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.549+0000 7fdfee8bf640 1 -- 172.21.15.145:0/460065652 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi115=y;smithi145=z", "target": ["mon-mgr", ""]}) -- 0x7fdfe806a4f0 con 0x7fdfd4039730 2024-09-19T04:11:52.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.555+0000 7fdfe5ffb640 1 -- 172.21.15.145:0/460065652 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdfe806a4f0 con 0x7fdfd4039730 2024-09-19T04:11:52.556 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mgr update... 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 -- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfd4039730 msgr2=0x7fdfd403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfd4039730 0x7fdfd403bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009230 tx=0x7fdfd80315d0 comp rx=0 tx=0).stop 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 -- 172.21.15.145:0/460065652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfe8196890 msgr2=0x7fdfe819da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfe8196890 0x7fdfe819da10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000df40 tx=0x7fdfe000df70 comp rx=0 tx=0).stop 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 -- 172.21.15.145:0/460065652 shutdown_connections 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfd4039730 0x7fdfd403bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfe8196890 0x7fdfe819da10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfe8196410 0x7fdfe819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 --2- 172.21.15.145:0/460065652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfe81005e0 0x7fdfe8195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 -- 172.21.15.145:0/460065652 >> 172.21.15.145:0/460065652 conn(0x7fdfe80fbdb0 msgr2=0x7fdfe80fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 -- 172.21.15.145:0/460065652 shutdown_connections 2024-09-19T04:11:52.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:11:52.557+0000 7fdfee8bf640 1 -- 172.21.15.145:0/460065652 wait complete. 2024-09-19T04:11:53.129 DEBUG:teuthology.orchestra.run.smithi115:mgr.y> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T04:11:53.131 DEBUG:teuthology.orchestra.run.smithi145:mgr.z> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T04:11:53.134 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T04:11:53.134 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:11:53.134 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:11:53.170 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:11:53.170 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-09-19T04:11:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:11:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:53.246 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 642 Links: 1 2024-09-19T04:11:53.246 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:53.246 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:53.246 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-19 04:11:12.278176842 +0000 2024-09-19T04:11:53.246 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-19 04:06:40.931074389 +0000 2024-09-19T04:11:53.246 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-19 04:06:40.931074389 +0000 2024-09-19T04:11:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-19 04:06:40.931074389 +0000 2024-09-19T04:11:53.247 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:11:53.343 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-19T04:11:53.343 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-19T04:11:53.343 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.00029253 s, 1.8 MB/s 2024-09-19T04:11:53.345 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:11:53.385 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-09-19T04:11:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:11:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:53.461 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 659 Links: 1 2024-09-19T04:11:53.461 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:53.461 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:53.461 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-19 04:11:12.316176230 +0000 2024-09-19T04:11:53.461 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-19 04:06:41.415066597 +0000 2024-09-19T04:11:53.461 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-19 04:06:41.415066597 +0000 2024-09-19T04:11:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-19 04:06:41.415066597 +0000 2024-09-19T04:11:53.462 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:11:53.554 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-19T04:11:53.555 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-19T04:11:53.555 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000260372 s, 2.0 MB/s 2024-09-19T04:11:53.558 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:11:53.594 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-09-19T04:11:53.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:53 smithi107 ceph-mon[30659]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi115=y;smithi145=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:53.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:53 smithi107 ceph-mon[30659]: Saving service mgr spec with placement smithi107=x;smithi115=y;smithi145=z;count:3 2024-09-19T04:11:53.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:53.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:53.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:53.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:11:53.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:11:53.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:11:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:53.662 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 679 Links: 1 2024-09-19T04:11:53.662 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:53.663 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:53.663 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-19 04:11:12.354175618 +0000 2024-09-19T04:11:53.663 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-19 04:06:41.865059354 +0000 2024-09-19T04:11:53.663 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-19 04:06:41.865059354 +0000 2024-09-19T04:11:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-19 04:06:41.865059354 +0000 2024-09-19T04:11:53.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:11:53.756 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-19T04:11:53.757 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-19T04:11:53.757 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000284156 s, 1.8 MB/s 2024-09-19T04:11:53.759 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:11:53.798 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-09-19T04:11:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:11:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:53.874 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 696 Links: 1 2024-09-19T04:11:53.874 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:53.874 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:53.874 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-19 04:11:12.391175023 +0000 2024-09-19T04:11:53.874 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-19 04:06:42.330051868 +0000 2024-09-19T04:11:53.874 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-19 04:06:42.330051868 +0000 2024-09-19T04:11:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-19 04:06:42.330051868 +0000 2024-09-19T04:11:53.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:11:53.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:53 smithi115 ceph-mon[35477]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi115=y;smithi145=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:53.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:53 smithi115 ceph-mon[35477]: Saving service mgr spec with placement smithi107=x;smithi115=y;smithi145=z;count:3 2024-09-19T04:11:53.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:53.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:53.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:53.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:11:53.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:11:53.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:53.968 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-19T04:11:53.968 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-19T04:11:53.968 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000456083 s, 1.1 MB/s 2024-09-19T04:11:53.970 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:11:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:53 smithi145 ceph-mon[35096]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi115=y;smithi145=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:11:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:53 smithi145 ceph-mon[35096]: Saving service mgr spec with placement smithi107=x;smithi115=y;smithi145=z;count:3 2024-09-19T04:11:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:53.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:53.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:11:53.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:11:53.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:54.006 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-19T04:11:54.006 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:11:54.024 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:11:54.025 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-09-19T04:11:54.082 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:11:54.082 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:54.082 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T04:11:54.083 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:54.083 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:54.083 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-19 04:11:42.964744004 +0000 2024-09-19T04:11:54.083 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-19 04:06:12.023186325 +0000 2024-09-19T04:11:54.083 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-19 04:06:12.023186325 +0000 2024-09-19T04:11:54.083 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-19 04:06:12.023186325 +0000 2024-09-19T04:11:54.083 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:11:54.148 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-19T04:11:54.148 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-19T04:11:54.148 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000222421 s, 2.3 MB/s 2024-09-19T04:11:54.149 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:11:54.206 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-09-19T04:11:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:11:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:54.262 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T04:11:54.263 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:54.263 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:54.263 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-19 04:11:42.965743985 +0000 2024-09-19T04:11:54.263 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-19 04:06:12.345180047 +0000 2024-09-19T04:11:54.263 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-19 04:06:12.345180047 +0000 2024-09-19T04:11:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-19 04:06:12.345180047 +0000 2024-09-19T04:11:54.263 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:11:54.326 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-19T04:11:54.326 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-19T04:11:54.327 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000195897 s, 2.6 MB/s 2024-09-19T04:11:54.328 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:11:54.385 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-09-19T04:11:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:11:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:54.442 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-19T04:11:54.442 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:54.442 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:54.442 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-19 04:11:42.965743985 +0000 2024-09-19T04:11:54.442 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-19 04:06:12.668173749 +0000 2024-09-19T04:11:54.442 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-19 04:06:12.668173749 +0000 2024-09-19T04:11:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-19 04:06:12.668173749 +0000 2024-09-19T04:11:54.443 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:11:54.505 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-19T04:11:54.506 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-19T04:11:54.506 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000228214 s, 2.2 MB/s 2024-09-19T04:11:54.507 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:11:54.564 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-09-19T04:11:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:11:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:54.620 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T04:11:54.620 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:54.620 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:54.620 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-19 04:11:42.966743965 +0000 2024-09-19T04:11:54.620 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-19 04:06:12.989167491 +0000 2024-09-19T04:11:54.620 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-19 04:06:12.989167491 +0000 2024-09-19T04:11:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-19 04:06:12.989167491 +0000 2024-09-19T04:11:54.620 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:11:54.685 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-19T04:11:54.685 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-19T04:11:54.685 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000192946 s, 2.7 MB/s 2024-09-19T04:11:54.686 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:11:54.742 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-19T04:11:54.743 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:11:54.762 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:11:54.762 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2024-09-19T04:11:54.822 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:11:54.822 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:54.822 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T04:11:54.822 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:54.822 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:54.822 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-19 04:11:34.225917923 +0000 2024-09-19T04:11:54.822 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-19 04:06:08.659925905 +0000 2024-09-19T04:11:54.822 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-19 04:06:08.659925905 +0000 2024-09-19T04:11:54.822 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-19 04:06:08.659925905 +0000 2024-09-19T04:11:54.823 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:11:54.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:54 smithi145 ceph-mon[35096]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:11:54.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:54 smithi145 ceph-mon[35096]: Reconfiguring daemon mgr.x on smithi107 2024-09-19T04:11:54.887 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:54 smithi145 ceph-mon[35096]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:54.890 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-19T04:11:54.890 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-19T04:11:54.890 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000225635 s, 2.3 MB/s 2024-09-19T04:11:54.891 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:11:54.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:54 smithi115 ceph-mon[35477]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:11:54.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:54 smithi115 ceph-mon[35477]: Reconfiguring daemon mgr.x on smithi107 2024-09-19T04:11:54.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:54 smithi115 ceph-mon[35477]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:54.949 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2024-09-19T04:11:54.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:54 smithi107 ceph-mon[30659]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:11:54.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:54 smithi107 ceph-mon[30659]: Reconfiguring daemon mgr.x on smithi107 2024-09-19T04:11:54.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:54 smithi107 ceph-mon[30659]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:55.007 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:11:55.007 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:55.007 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T04:11:55.007 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:55.007 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:55.007 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-19 04:11:34.225917923 +0000 2024-09-19T04:11:55.007 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-19 04:06:08.970918793 +0000 2024-09-19T04:11:55.007 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-19 04:06:08.970918793 +0000 2024-09-19T04:11:55.008 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-19 04:06:08.970918793 +0000 2024-09-19T04:11:55.008 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:11:55.072 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-19T04:11:55.072 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-19T04:11:55.072 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000234981 s, 2.2 MB/s 2024-09-19T04:11:55.073 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:11:55.131 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2024-09-19T04:11:55.187 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:11:55.187 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:55.187 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-19T04:11:55.187 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:55.187 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:55.187 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-19 04:11:34.226917901 +0000 2024-09-19T04:11:55.187 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-19 04:06:09.293911406 +0000 2024-09-19T04:11:55.188 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-19 04:06:09.293911406 +0000 2024-09-19T04:11:55.188 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-19 04:06:09.293911406 +0000 2024-09-19T04:11:55.188 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:11:55.251 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-19T04:11:55.251 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-19T04:11:55.252 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000265124 s, 1.9 MB/s 2024-09-19T04:11:55.253 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:11:55.310 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2024-09-19T04:11:55.366 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:11:55.366 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:11:55.366 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T04:11:55.366 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:11:55.366 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:11:55.367 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-19 04:11:34.226917901 +0000 2024-09-19T04:11:55.367 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-19 04:06:09.608904202 +0000 2024-09-19T04:11:55.367 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-19 04:06:09.608904202 +0000 2024-09-19T04:11:55.367 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-19 04:06:09.608904202 +0000 2024-09-19T04:11:55.367 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:11:55.430 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-19T04:11:55.430 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-19T04:11:55.430 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000233843 s, 2.2 MB/s 2024-09-19T04:11:55.432 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:11:55.488 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2024-09-19T04:11:55.488 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:11:55.836 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:55 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:55.836 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:55 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:55.836 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:55 smithi107 ceph-mon[30659]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:11:55.836 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:55 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:55.836 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:55 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:11:55.836 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:55 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:55.836 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:55 smithi107 ceph-mon[30659]: Reconfiguring daemon mon.b on smithi115 2024-09-19T04:11:55.870 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:55.909 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:55 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:55.910 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:55 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:55.910 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:55 smithi115 ceph-mon[35477]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:11:55.910 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:55 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:55.910 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:55 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:11:55.910 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:55 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:55.910 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:55 smithi115 ceph-mon[35477]: Reconfiguring daemon mon.b on smithi115 2024-09-19T04:11:55.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:55 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:55.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:55 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:55.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:55 smithi145 ceph-mon[35096]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:11:55.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:55 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:55.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:55 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:11:55.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:55 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:55.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:55 smithi145 ceph-mon[35096]: Reconfiguring daemon mon.b on smithi115 2024-09-19T04:11:56.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:56 smithi115 ceph-mon[35477]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:56.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:56.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:56.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:56.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:11:56.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:56.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:56 smithi145 ceph-mon[35096]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:56.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:56.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:56.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:56.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:11:56.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:56.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:56 smithi107 ceph-mon[30659]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:56.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:56.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:56.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:11:56.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:11:56.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:57.892 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:11:57.892 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: Reconfiguring daemon mon.c on smithi145 2024-09-19T04:11:57.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:57.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:57.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:57.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:57.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:57.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:57.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:11:57.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:11:57.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:11:57.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:57.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:11:57.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: Reconfiguring daemon mon.c on smithi145 2024-09-19T04:11:57.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:57.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:57.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:57.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:57.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:57.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:57.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:11:57.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:11:57.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:11:57.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:11:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: Reconfiguring daemon mon.c on smithi145 2024-09-19T04:11:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:11:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:11:57.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:11:57.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:11:57.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:11:57.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:11:57.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:11:58.805 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:58 smithi107 ceph-mon[30659]: Deploying daemon mgr.z on smithi145 2024-09-19T04:11:58.805 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:11:58 smithi107 ceph-mon[30659]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:11:58.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-09-19T04:11:58.929 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:58 smithi145 ceph-mon[35096]: Deploying daemon mgr.z on smithi145 2024-09-19T04:11:58.929 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:11:58 smithi145 ceph-mon[35096]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:58.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:58 smithi115 ceph-mon[35477]: Deploying daemon mgr.z on smithi145 2024-09-19T04:11:58.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:11:58 smithi115 ceph-mon[35477]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:11:59.200 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:11:59.224 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 podman[36898]: 2024-09-19 04:11:59.04245116 +0000 UTC m=+0.015920381 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:11:59.518 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 podman[36898]: 2024-09-19 04:11:59.333287993 +0000 UTC m=+0.306757215 container init 42f6183e2e30dcd4d9dd99ebcd790a88edd1701eba63fef57b9bdd24cf940500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T04:11:59.518 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 podman[36898]: 2024-09-19 04:11:59.335950454 +0000 UTC m=+0.309419677 container start 42f6183e2e30dcd4d9dd99ebcd790a88edd1701eba63fef57b9bdd24cf940500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T04:11:59.518 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 ceph-mgr[36917]: -- 172.21.15.145:0/4279992281 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c77e270e0 con 0x561c77e05000 2024-09-19T04:11:59.519 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 bash[36898]: 42f6183e2e30dcd4d9dd99ebcd790a88edd1701eba63fef57b9bdd24cf940500 2024-09-19T04:11:59.519 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 systemd[1]: Started Ceph mgr.z for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:11:59.519 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:11:59.457+0000 7f239d29e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:11:59.519 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:11:59.516+0000 7f239d29e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:11:59.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.677+0000 7fecb4400640 1 -- 172.21.15.107:0/876228690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecac103e00 msgr2=0x7fecac104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:59.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.677+0000 7fecb4400640 1 --2- 172.21.15.107:0/876228690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecac103e00 0x7fecac104200 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feca0009900 tx=0x7feca002a410 comp rx=0 tx=0).stop 2024-09-19T04:11:59.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.678+0000 7fecb4400640 1 -- 172.21.15.107:0/876228690 shutdown_connections 2024-09-19T04:11:59.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.678+0000 7fecb4400640 1 --2- 172.21.15.107:0/876228690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac105920 0x7fecac10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:59.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.678+0000 7fecb4400640 1 --2- 172.21.15.107:0/876228690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecac104f60 0x7fecac1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:59.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.678+0000 7fecb4400640 1 --2- 172.21.15.107:0/876228690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecac103e00 0x7fecac104200 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:59.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.678+0000 7fecb4400640 1 -- 172.21.15.107:0/876228690 >> 172.21.15.107:0/876228690 conn(0x7fecac0ff5b0 msgr2=0x7fecac1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:11:59.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.678+0000 7fecb4400640 1 -- 172.21.15.107:0/876228690 shutdown_connections 2024-09-19T04:11:59.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.678+0000 7fecb4400640 1 -- 172.21.15.107:0/876228690 wait complete. 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.679+0000 7fecb4400640 1 Processor -- start 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.679+0000 7fecb4400640 1 -- start start 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.679+0000 7fecb4400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac103e00 0x7fecac19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.679+0000 7fecb4400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecac104f60 0x7fecac19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.679+0000 7fecb4400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecac105920 0x7fecac1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.679+0000 7fecb4400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fecac10fae0 con 0x7fecac104f60 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.679+0000 7fecb4400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fecac10f960 con 0x7fecac103e00 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.679+0000 7fecb4400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fecac10fc60 con 0x7fecac105920 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb2400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecac105920 0x7fecac1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb1a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac103e00 0x7fecac19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:59.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb2400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecac105920 0x7fecac1a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46758/0 (socket says 172.21.15.107:46758) 2024-09-19T04:11:59.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb1a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac103e00 0x7fecac19a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:46588/0 (socket says 172.21.15.107:46588) 2024-09-19T04:11:59.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb2400640 1 -- 172.21.15.107:0/718858630 learned_addr learned my addr 172.21.15.107:0/718858630 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:11:59.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb1000640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecac104f60 0x7fecac19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:59.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb2400640 1 -- 172.21.15.107:0/718858630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac103e00 msgr2=0x7fecac19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:59.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb2400640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac103e00 0x7fecac19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:59.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb2400640 1 -- 172.21.15.107:0/718858630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecac104f60 msgr2=0x7fecac19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:11:59.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb2400640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecac104f60 0x7fecac19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:11:59.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb2400640 1 -- 172.21.15.107:0/718858630 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac1a1cd0 con 0x7fecac105920 2024-09-19T04:11:59.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.680+0000 7fecb2400640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecac105920 0x7fecac1a1660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feca400eb10 tx=0x7feca400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:59.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.681+0000 7fecaa000640 1 -- 172.21.15.107:0/718858630 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4004b20 con 0x7fecac105920 2024-09-19T04:11:59.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.681+0000 7fecb4400640 1 -- 172.21.15.107:0/718858630 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecac1a1fc0 con 0x7fecac105920 2024-09-19T04:11:59.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.681+0000 7fecaa000640 1 -- 172.21.15.107:0/718858630 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca4004cc0 con 0x7fecac105920 2024-09-19T04:11:59.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.681+0000 7fecaa000640 1 -- 172.21.15.107:0/718858630 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4010760 con 0x7fecac105920 2024-09-19T04:11:59.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.681+0000 7fecb4400640 1 -- 172.21.15.107:0/718858630 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecac1a2470 con 0x7fecac105920 2024-09-19T04:11:59.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.682+0000 7fecaa000640 1 -- 172.21.15.107:0/718858630 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7feca4005020 con 0x7fecac105920 2024-09-19T04:11:59.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.682+0000 7fecb4400640 1 -- 172.21.15.107:0/718858630 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecac10b1a0 con 0x7fecac105920 2024-09-19T04:11:59.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.682+0000 7fecaa000640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec94039500 0x7fec9403b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:11:59.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.682+0000 7fecaa000640 1 -- 172.21.15.107:0/718858630 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feca404ccd0 con 0x7fecac105920 2024-09-19T04:11:59.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.683+0000 7fecb1a00640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec94039500 0x7fec9403b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:11:59.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.683+0000 7fecb1a00640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec94039500 0x7fec9403b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feca0031440 tx=0x7feca00315b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:11:59.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.691+0000 7fecaa000640 1 -- 172.21.15.107:0/718858630 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feca400a250 con 0x7fecac105920 2024-09-19T04:11:59.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:11:59.877+0000 7fecb4400640 1 -- 172.21.15.107:0/718858630 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fecac062630 con 0x7fec94039500 2024-09-19T04:11:59.880 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:11:59.663+0000 7f239d29e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:11:59.880 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:11:59.732+0000 7f239d29e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:11:59.880 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:11:59 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:11:59.879+0000 7f239d29e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:12:00.224 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:00.016+0000 7f239d29e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:12:00.724 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:00.326+0000 7f239d29e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:12:00.724 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:00.386+0000 7f239d29e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:12:00.724 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:00.572+0000 7f239d29e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: Deploying daemon mgr.y on smithi115 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:00.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:00.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:00.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:00.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: Deploying daemon mgr.y on smithi115 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:00.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:00.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:00.814 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: Deploying daemon mgr.y on smithi115 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:00.815 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:00.816 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:01.446 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:01 smithi115 systemd[1]: Starting Ceph mgr.y for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:12:01.724 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:01 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:01.450+0000 7f239d29e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:12:01.724 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:01 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:01.516+0000 7f239d29e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:12:01.760 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:01 smithi115 podman[36823]: 2024-09-19 04:12:01.370154518 +0000 UTC m=+0.015585358 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:01.760 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:01 smithi115 podman[36823]: 2024-09-19 04:12:01.473946593 +0000 UTC m=+0.119377432 container create e28287cf2a5ba9254500b884708586e98d2d6d457e3312f907d34c8eb697a6a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T04:12:01.761 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:01 smithi115 podman[36823]: 2024-09-19 04:12:01.684753106 +0000 UTC m=+0.330183943 container init e28287cf2a5ba9254500b884708586e98d2d6d457e3312f907d34c8eb697a6a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T04:12:01.761 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:01 smithi115 podman[36823]: 2024-09-19 04:12:01.687345737 +0000 UTC m=+0.332776573 container start e28287cf2a5ba9254500b884708586e98d2d6d457e3312f907d34c8eb697a6a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T04:12:01.761 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:01 smithi115 ceph-mgr[36843]: -- 172.21.15.115:0/3493608841 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba2b3910e0 con 0x55ba2b36f400 2024-09-19T04:12:01.761 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:01 smithi115 bash[36823]: e28287cf2a5ba9254500b884708586e98d2d6d457e3312f907d34c8eb697a6a7 2024-09-19T04:12:02.022 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:01 smithi115 systemd[1]: Started Ceph mgr.y for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:12:02.022 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:01 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:01.808+0000 7fb5f9ad5040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:12:02.022 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:01 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:01.867+0000 7fb5f9ad5040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:12:02.022 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:02.020+0000 7fb5f9ad5040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:12:02.410 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:02.090+0000 7fb5f9ad5040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:12:02.410 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:02.247+0000 7fb5f9ad5040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:12:02.410 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:02.408+0000 7fb5f9ad5040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:12:02.535 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:02.261+0000 7f239d29e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:12:02.536 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:02.391+0000 7f239d29e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:12:02.536 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:02.534+0000 7f239d29e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:12:02.830 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:02.599+0000 7f239d29e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:12:02.830 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:02.758+0000 7f239d29e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:12:02.990 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:02.733+0000 7fb5f9ad5040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:12:02.991 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:02.794+0000 7fb5f9ad5040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:12:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/642161227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9218bb1e-b075-4d2f-9533-da54ea586a54"}]: dispatch 2024-09-19T04:12:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9218bb1e-b075-4d2f-9533-da54ea586a54"}]: dispatch 2024-09-19T04:12:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9218bb1e-b075-4d2f-9533-da54ea586a54"}]': finished 2024-09-19T04:12:02.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:12:02.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:03.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:03.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:03.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:03.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:03.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:03.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:03.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/642161227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9218bb1e-b075-4d2f-9533-da54ea586a54"}]: dispatch 2024-09-19T04:12:03.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9218bb1e-b075-4d2f-9533-da54ea586a54"}]: dispatch 2024-09-19T04:12:03.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9218bb1e-b075-4d2f-9533-da54ea586a54"}]': finished 2024-09-19T04:12:03.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:12:03.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:03.133 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:02.831+0000 7f239d29e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:12:03.134 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:02.892+0000 7f239d29e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:12:03.134 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:03.066+0000 7f239d29e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:12:03.134 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:03.130+0000 7f239d29e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:12:03.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:03.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:03.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:03.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:03.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:03.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:03.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/642161227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9218bb1e-b075-4d2f-9533-da54ea586a54"}]: dispatch 2024-09-19T04:12:03.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9218bb1e-b075-4d2f-9533-da54ea586a54"}]: dispatch 2024-09-19T04:12:03.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9218bb1e-b075-4d2f-9533-da54ea586a54"}]': finished 2024-09-19T04:12:03.180 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:12:03.180 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:03.405 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:03.203+0000 7f239d29e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:12:03.405 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:03.286+0000 7f239d29e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:12:03.405 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:03.345+0000 7f239d29e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:12:03.405 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:03.403+0000 7f239d29e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:12:03.446 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:02.989+0000 7fb5f9ad5040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:12:04.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:03.876+0000 7fb5f9ad5040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:12:04.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:03.944+0000 7fb5f9ad5040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:12:04.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2314215070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:04.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.224 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:12:04 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:12:04.196+0000 7f239d29e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:12:04.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:04.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:04.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:04.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2314215070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:04.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:04.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:04.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:04.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2314215070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:04.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:04.924 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:04 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:04.665+0000 7fb5f9ad5040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:12:04.924 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:04 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:04.780+0000 7fb5f9ad5040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:12:04.924 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:04 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:04.922+0000 7fb5f9ad5040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:12:05.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:05.058+0000 7fecaa000640 1 -- 172.21.15.107:0/718858630 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7feca4021960 con 0x7fecac105920 2024-09-19T04:12:05.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:04 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:04.982+0000 7fb5f9ad5040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:12:05.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:05.131+0000 7fb5f9ad5040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:12:05.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-mon[35477]: Standby manager daemon z started 2024-09-19T04:12:05.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:12:05.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:12:05.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:12:05.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:12:05.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:05.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:05 smithi145 ceph-mon[35096]: Standby manager daemon z started 2024-09-19T04:12:05.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:05 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:12:05.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:05 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:12:05.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:05 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:12:05.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:05 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:12:05.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:05 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:05 smithi107 ceph-mon[30659]: Standby manager daemon z started 2024-09-19T04:12:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:05 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:12:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:05 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:12:05.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:05 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:12:05.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:05 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.145:0/2599409179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:12:05.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:05.481 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:05.199+0000 7fb5f9ad5040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:12:05.481 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:05.258+0000 7fb5f9ad5040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:12:05.481 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:05.414+0000 7fb5f9ad5040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:12:05.482 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:05.480+0000 7fb5f9ad5040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:12:05.748 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:05.546+0000 7fb5f9ad5040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:12:05.748 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:05.628+0000 7fb5f9ad5040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:12:05.748 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:05.687+0000 7fb5f9ad5040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:12:05.748 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:05.746+0000 7fb5f9ad5040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:12:06.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:06 smithi115 ceph-mon[35477]: mgrmap e14: x(active, since 65s), standbys: z 2024-09-19T04:12:06.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:06 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:12:06.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:06 smithi115 ceph-mon[35477]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:06.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:06 smithi145 ceph-mon[35096]: mgrmap e14: x(active, since 65s), standbys: z 2024-09-19T04:12:06.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:06 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:12:06.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:06 smithi145 ceph-mon[35096]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:06.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:06 smithi107 ceph-mon[30659]: mgrmap e14: x(active, since 65s), standbys: z 2024-09-19T04:12:06.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:06 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:12:06.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:06 smithi107 ceph-mon[30659]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:06.946 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:12:06 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:12:06.528+0000 7fb5f9ad5040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:12:07.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:07.071+0000 7fecaa000640 1 -- 172.21.15.107:0/718858630 <== mon.1 v2:172.21.15.145:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7feca401f340 con 0x7fecac105920 2024-09-19T04:12:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:07 smithi115 ceph-mon[35477]: Standby manager daemon y started 2024-09-19T04:12:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:07 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:12:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:07 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:12:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:07 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:12:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:07 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:12:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:07 smithi145 ceph-mon[35096]: Standby manager daemon y started 2024-09-19T04:12:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:07 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:12:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:07 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:12:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:07 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:12:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:07 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:12:07.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:07 smithi107 ceph-mon[30659]: Standby manager daemon y started 2024-09-19T04:12:07.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:07 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:12:07.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:07 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:12:07.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:07 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:12:07.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:07 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.115:0/3946079496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:12:08.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:08 smithi107 ceph-mon[30659]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-19T04:12:08.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:08 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:12:08.098 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:08 smithi107 ceph-mon[30659]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:08.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:08 smithi115 ceph-mon[35477]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-19T04:12:08.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:08 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:12:08.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:08 smithi115 ceph-mon[35477]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:08.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:08 smithi145 ceph-mon[35096]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-19T04:12:08.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:08 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:12:08.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:08 smithi145 ceph-mon[35096]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:10.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:10 smithi107 ceph-mon[30659]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:11.149 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:10 smithi115 ceph-mon[35477]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:11.177 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:10 smithi145 ceph-mon[35096]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:11.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:12:11.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:12.161 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:12:12.162 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:12.190 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:12:12.190 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:12.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:12 smithi107 ceph-mon[30659]: Deploying daemon osd.0 on smithi107 2024-09-19T04:12:12.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:12 smithi107 ceph-mon[30659]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:12 smithi115 ceph-mon[35477]: Deploying daemon osd.0 on smithi107 2024-09-19T04:12:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:12 smithi115 ceph-mon[35477]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:12 smithi145 ceph-mon[35096]: Deploying daemon osd.0 on smithi107 2024-09-19T04:12:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:12 smithi145 ceph-mon[35096]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:14.833 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:14 smithi107 ceph-mon[30659]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:14 smithi115 ceph-mon[35477]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:14 smithi145 ceph-mon[35096]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:16 smithi115 ceph-mon[35477]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:16 smithi145 ceph-mon[35096]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:17.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:16 smithi107 ceph-mon[30659]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:18.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:18 smithi107 ceph-mon[30659]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:18 smithi115 ceph-mon[35477]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:18 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:18 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:18 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:18 smithi145 ceph-mon[35096]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:18 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:18 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:18 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:21.091 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:20 smithi107 ceph-mon[30659]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:20 smithi115 ceph-mon[35477]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:20 smithi145 ceph-mon[35096]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:22.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:22.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:22.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:22.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:22.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:22.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:22 smithi145 ceph-mon[35096]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:22.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:22.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:22.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:22.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:22.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:22.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:22 smithi107 ceph-mon[30659]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:22.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:22.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:22.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:22.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:22.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:22.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:22 smithi115 ceph-mon[35477]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:23.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.828+0000 7fecaa000640 1 -- 172.21.15.107:0/718858630 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fecac062630 con 0x7fec94039500 2024-09-19T04:12:23.829 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2024-09-19T04:12:23.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 -- 172.21.15.107:0/718858630 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec94039500 msgr2=0x7fec9403b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:23.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec94039500 0x7fec9403b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feca0031440 tx=0x7feca00315b0 comp rx=0 tx=0).stop 2024-09-19T04:12:23.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 -- 172.21.15.107:0/718858630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecac105920 msgr2=0x7fecac1a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:23.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecac105920 0x7fecac1a1660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feca400eb10 tx=0x7feca400c2d0 comp rx=0 tx=0).stop 2024-09-19T04:12:23.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 -- 172.21.15.107:0/718858630 shutdown_connections 2024-09-19T04:12:23.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec94039500 0x7fec9403b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:23.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecac105920 0x7fecac1a1660 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:23.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecac104f60 0x7fecac19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:23.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 --2- 172.21.15.107:0/718858630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac103e00 0x7fecac19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:23.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 -- 172.21.15.107:0/718858630 >> 172.21.15.107:0/718858630 conn(0x7fecac0ff5b0 msgr2=0x7fecac108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:23.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 -- 172.21.15.107:0/718858630 shutdown_connections 2024-09-19T04:12:23.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:12:23.831+0000 7fecb4400640 1 -- 172.21.15.107:0/718858630 wait complete. 2024-09-19T04:12:24.490 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T04:12:24.494 INFO:tasks.cephadm:Deploying osd.1 on smithi115 with /dev/vg_nvme/lv_4... 2024-09-19T04:12:24.494 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:12:24.732 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:12:24.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:24 smithi107 ceph-mon[30659]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:24.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:24 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:24.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:24 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:24.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:24 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:24.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:24 smithi107 ceph-mon[30659]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:12:25.071 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:24 smithi115 ceph-mon[35477]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:25.072 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:24 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:25.072 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:24 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:25.072 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:24 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:25.072 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:24 smithi115 ceph-mon[35477]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:12:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:24 smithi145 ceph-mon[35096]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:24 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:24 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:25.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:24 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:25.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:24 smithi145 ceph-mon[35096]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:12:25.925 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:25 smithi107 ceph-mon[30659]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:12:25.925 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:25 smithi107 ceph-mon[30659]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:12:25.925 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:25 smithi107 ceph-mon[30659]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:12:25.925 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:25 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:26.043 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:25 smithi115 ceph-mon[35477]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:12:26.043 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:25 smithi115 ceph-mon[35477]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:12:26.044 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:25 smithi115 ceph-mon[35477]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:12:26.044 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:25 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:26.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:25 smithi145 ceph-mon[35096]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:12:26.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:25 smithi145 ceph-mon[35096]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:12:26.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:25 smithi145 ceph-mon[35096]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:12:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:25 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:26.481 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:12:26 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0[43968]: 2024-09-19T04:12:26.328+0000 7f136e200640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T04:12:26.482 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:12:26 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0[43968]: 2024-09-19T04:12:26.346+0000 7f1364200640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:12:26.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:26 smithi107 ceph-mon[30659]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:26 smithi107 ceph-mon[30659]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:12:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:26 smithi107 ceph-mon[30659]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:12:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:26 smithi107 ceph-mon[30659]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' 2024-09-19T04:12:27.163 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:26 smithi115 ceph-mon[35477]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:27.163 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:26 smithi115 ceph-mon[35477]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:12:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:26 smithi115 ceph-mon[35477]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:12:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:26 smithi115 ceph-mon[35477]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' 2024-09-19T04:12:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:12:27.192 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2024-09-19T04:12:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:26 smithi145 ceph-mon[35096]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:26 smithi145 ceph-mon[35096]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:12:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:26 smithi145 ceph-mon[35096]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:12:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:27.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:27.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:26 smithi145 ceph-mon[35096]: from='osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384]' entity='osd.0' 2024-09-19T04:12:27.440 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:12:27.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.867+0000 7f2e10470640 1 -- 172.21.15.115:0/4088057205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e081078c0 msgr2=0x7f2e08109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:27.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.867+0000 7f2e10470640 1 --2- 172.21.15.115:0/4088057205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e081078c0 0x7f2e08109d10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009ca0 tx=0x7f2e0002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:12:27.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.867+0000 7f2e10470640 1 -- 172.21.15.115:0/4088057205 shutdown_connections 2024-09-19T04:12:27.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.867+0000 7f2e10470640 1 --2- 172.21.15.115:0/4088057205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e081078c0 0x7f2e08109d10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:27.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.867+0000 7f2e10470640 1 --2- 172.21.15.115:0/4088057205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e08104f90 0x7f2e08107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:27.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.867+0000 7f2e10470640 1 --2- 172.21.15.115:0/4088057205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e08102660 0x7f2e08104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:27.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.867+0000 7f2e10470640 1 -- 172.21.15.115:0/4088057205 >> 172.21.15.115:0/4088057205 conn(0x7f2e080fc280 msgr2=0x7f2e080fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:27.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.867+0000 7f2e10470640 1 -- 172.21.15.115:0/4088057205 shutdown_connections 2024-09-19T04:12:27.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.867+0000 7f2e10470640 1 -- 172.21.15.115:0/4088057205 wait complete. 2024-09-19T04:12:27.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.867+0000 7f2e10470640 1 Processor -- start 2024-09-19T04:12:27.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e10470640 1 -- start start 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e10470640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e08102660 0x7f2e08101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e10470640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e08104f90 0x7f2e08100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e10470640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e081078c0 0x7f2e08100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e10470640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810d8d0 con 0x7f2e08102660 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e10470640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810d750 con 0x7f2e081078c0 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e10470640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810da50 con 0x7f2e08104f90 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0ea0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e081078c0 0x7f2e08100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0ea0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e081078c0 0x7f2e08100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49430/0 (socket says 172.21.15.115:49430) 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0ea0d640 1 -- 172.21.15.115:0/2676870032 learned_addr learned my addr 172.21.15.115:0/2676870032 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0e20c640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e08102660 0x7f2e08101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0da0b640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e08104f90 0x7f2e08100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:27.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0ea0d640 1 -- 172.21.15.115:0/2676870032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e08104f90 msgr2=0x7f2e08100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0ea0d640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e08104f90 0x7f2e08100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0ea0d640 1 -- 172.21.15.115:0/2676870032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e08102660 msgr2=0x7f2e08101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0ea0d640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e08102660 0x7f2e08101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0ea0d640 1 -- 172.21.15.115:0/2676870032 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e08101410 con 0x7f2e081078c0 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.868+0000 7f2e0ea0d640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e081078c0 0x7f2e08100b50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e000319f0 tx=0x7f2e0002f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.869+0000 7f2dff7fe640 1 -- 172.21.15.115:0/2676870032 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00003c70 con 0x7f2e081078c0 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.869+0000 7f2dff7fe640 1 -- 172.21.15.115:0/2676870032 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e00003e10 con 0x7f2e081078c0 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.869+0000 7f2e10470640 1 -- 172.21.15.115:0/2676870032 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e081a2050 con 0x7f2e081078c0 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.869+0000 7f2dff7fe640 1 -- 172.21.15.115:0/2676870032 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0002fd30 con 0x7f2e081078c0 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.869+0000 7f2e10470640 1 -- 172.21.15.115:0/2676870032 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e081a2560 con 0x7f2e081078c0 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.870+0000 7f2e10470640 1 -- 172.21.15.115:0/2676870032 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e080fdd70 con 0x7f2e081078c0 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.871+0000 7f2dff7fe640 1 -- 172.21.15.115:0/2676870032 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f2e00004040 con 0x7f2e081078c0 2024-09-19T04:12:27.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.872+0000 7f2dff7fe640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2df40a31b0 0x7f2df40a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:27.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.872+0000 7f2dff7fe640 1 -- 172.21.15.115:0/2676870032 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f2e000ef410 con 0x7f2e081078c0 2024-09-19T04:12:27.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.872+0000 7f2e0e20c640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2df40a31b0 0x7f2df40a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:27.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.872+0000 7f2e0e20c640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2df40a31b0 0x7f2df40a5670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400bbe0 tx=0x7f2e0400b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:27.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:27.876+0000 7f2dff7fe640 1 -- 172.21.15.115:0/2676870032 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2e000bdba0 con 0x7f2e081078c0 2024-09-19T04:12:28.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:28.064+0000 7f2e10470640 1 -- 172.21.15.115:0/2676870032 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2e08062630 con 0x7f2df40a31b0 2024-09-19T04:12:28.066 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:27 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:12:28.066 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:27 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:12:28.066 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:27 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:28.066 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:27 smithi115 ceph-mon[35477]: osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] boot 2024-09-19T04:12:28.066 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:27 smithi115 ceph-mon[35477]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:12:28.066 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:27 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:27 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:12:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:27 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:12:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:27 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:27 smithi145 ceph-mon[35096]: osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] boot 2024-09-19T04:12:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:27 smithi145 ceph-mon[35096]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:12:28.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:27 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:27 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:12:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:27 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:12:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:27 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:27 smithi107 ceph-mon[30659]: osd.0 [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] boot 2024-09-19T04:12:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:27 smithi107 ceph-mon[30659]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:12:28.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:27 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:12:29.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:29.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.126 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.126 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:29.126 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.126 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:29.126 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:29.126 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.126 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:29.126 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:29.126 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:29.126 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:28 smithi107 ceph-mon[30659]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:12:29.189 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:29.189 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.189 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.190 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:29.190 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.190 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:29.190 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:29.190 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.190 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:29.190 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:29.190 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:29.190 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:28 smithi115 ceph-mon[35477]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:12:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:12:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:29.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:29.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:29.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:29.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:29.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:29.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:29.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:28 smithi145 ceph-mon[35096]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:12:30.067 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:29 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi107 2024-09-19T04:12:30.067 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:29 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi107 to 17089M 2024-09-19T04:12:30.067 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:29 smithi115 ceph-mon[35477]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:30.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:29 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi107 2024-09-19T04:12:30.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:29 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi107 to 17089M 2024-09-19T04:12:30.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:29 smithi145 ceph-mon[35096]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:30.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:29 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi107 2024-09-19T04:12:30.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:29 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi107 to 17089M 2024-09-19T04:12:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:29 smithi107 ceph-mon[30659]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:30.939 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:30 smithi115 ceph-mon[35477]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:30.939 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:30 smithi115 ceph-mon[35477]: from='client.? 172.21.15.115:0/3613799959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bed3c76-ab42-4406-9064-a558f56ee785"}]: dispatch 2024-09-19T04:12:30.939 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:30 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bed3c76-ab42-4406-9064-a558f56ee785"}]: dispatch 2024-09-19T04:12:30.940 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:30 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4bed3c76-ab42-4406-9064-a558f56ee785"}]': finished 2024-09-19T04:12:30.940 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:30 smithi115 ceph-mon[35477]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:12:30.940 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:30 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:30 smithi145 ceph-mon[35096]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:30 smithi145 ceph-mon[35096]: from='client.? 172.21.15.115:0/3613799959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bed3c76-ab42-4406-9064-a558f56ee785"}]: dispatch 2024-09-19T04:12:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:30 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bed3c76-ab42-4406-9064-a558f56ee785"}]: dispatch 2024-09-19T04:12:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:30 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4bed3c76-ab42-4406-9064-a558f56ee785"}]': finished 2024-09-19T04:12:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:30 smithi145 ceph-mon[35096]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:12:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:30 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:31.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:30 smithi107 ceph-mon[30659]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:31.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:30 smithi107 ceph-mon[30659]: from='client.? 172.21.15.115:0/3613799959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bed3c76-ab42-4406-9064-a558f56ee785"}]: dispatch 2024-09-19T04:12:31.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:30 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bed3c76-ab42-4406-9064-a558f56ee785"}]: dispatch 2024-09-19T04:12:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:30 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4bed3c76-ab42-4406-9064-a558f56ee785"}]': finished 2024-09-19T04:12:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:30 smithi107 ceph-mon[30659]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:12:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:30 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:32.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:31 smithi115 ceph-mon[35477]: from='client.? 172.21.15.115:0/2966834459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:32.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:31 smithi145 ceph-mon[35096]: from='client.? 172.21.15.115:0/2966834459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:32.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:31 smithi107 ceph-mon[30659]: from='client.? 172.21.15.115:0/2966834459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:32 smithi115 ceph-mon[35477]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:32 smithi145 ceph-mon[35096]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:33.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:32 smithi107 ceph-mon[30659]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:34 smithi115 ceph-mon[35477]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:34 smithi145 ceph-mon[35096]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:35.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:34 smithi107 ceph-mon[30659]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:36 smithi115 ceph-mon[35477]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:36 smithi145 ceph-mon[35096]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:36 smithi107 ceph-mon[30659]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:39.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:38 smithi107 ceph-mon[30659]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:39.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:12:39.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:38 smithi115 ceph-mon[35477]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:12:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:38 smithi145 ceph-mon[35096]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:12:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:40.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:39 smithi115 ceph-mon[35477]: Deploying daemon osd.1 on smithi115 2024-09-19T04:12:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:39 smithi145 ceph-mon[35096]: Deploying daemon osd.1 on smithi115 2024-09-19T04:12:40.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:39 smithi107 ceph-mon[30659]: Deploying daemon osd.1 on smithi115 2024-09-19T04:12:41.059 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:40 smithi115 ceph-mon[35477]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:41.125 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:40 smithi145 ceph-mon[35096]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:41.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:40 smithi107 ceph-mon[30659]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:42 smithi115 ceph-mon[35477]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:42 smithi145 ceph-mon[35096]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:43.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:42 smithi107 ceph-mon[30659]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:44 smithi115 ceph-mon[35477]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:45.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:44 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:45.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:44 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:45.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:44 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:44 smithi145 ceph-mon[35096]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:44 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:44 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:44 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:45.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:44 smithi107 ceph-mon[30659]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:45.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:44 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:45.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:44 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:45.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:44 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:46 smithi115 ceph-mon[35477]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:47.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:47.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:46 smithi145 ceph-mon[35096]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:46 smithi107 ceph-mon[30659]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.974+0000 7f2dff7fe640 1 -- 172.21.15.115:0/2676870032 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2e08062630 con 0x7f2df40a31b0 2024-09-19T04:12:47.976 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 1 on host 'smithi115' 2024-09-19T04:12:47.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 -- 172.21.15.115:0/2676870032 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2df40a31b0 msgr2=0x7f2df40a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:47.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2df40a31b0 0x7f2df40a5670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400bbe0 tx=0x7f2e0400b390 comp rx=0 tx=0).stop 2024-09-19T04:12:47.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 -- 172.21.15.115:0/2676870032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e081078c0 msgr2=0x7f2e08100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:47.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e081078c0 0x7f2e08100b50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e000319f0 tx=0x7f2e0002f450 comp rx=0 tx=0).stop 2024-09-19T04:12:47.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 -- 172.21.15.115:0/2676870032 shutdown_connections 2024-09-19T04:12:47.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2df40a31b0 0x7f2df40a5670 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:47.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e081078c0 0x7f2e08100b50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:47.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e08104f90 0x7f2e08100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:47.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 --2- 172.21.15.115:0/2676870032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e08102660 0x7f2e08101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:47.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 -- 172.21.15.115:0/2676870032 >> 172.21.15.115:0/2676870032 conn(0x7f2e080fc280 msgr2=0x7f2e08102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:47.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 -- 172.21.15.115:0/2676870032 shutdown_connections 2024-09-19T04:12:47.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:12:47.977+0000 7f2e10470640 1 -- 172.21.15.115:0/2676870032 wait complete. 2024-09-19T04:12:48.656 DEBUG:teuthology.orchestra.run.smithi115:osd.1> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T04:12:48.658 INFO:tasks.cephadm:Deploying osd.2 on smithi145 with /dev/vg_nvme/lv_4... 2024-09-19T04:12:48.659 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:12:48.912 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:12:48.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:48 smithi145 ceph-mon[35096]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:48.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:48.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:48.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:48 smithi115 ceph-mon[35477]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:48 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:48 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:48 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:48 smithi107 ceph-mon[30659]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:49.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:12:49.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:49.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:50.755 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:12:50 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1[39953]: 2024-09-19T04:12:50.489+0000 7fd62e098640 -1 osd.1 0 log_to_monitors true 2024-09-19T04:12:51.085 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:50 smithi115 ceph-mon[35477]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:51.086 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:50 smithi115 ceph-mon[35477]: from='osd.1 [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:12:51.086 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:50 smithi115 ceph-mon[35477]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:12:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:50 smithi145 ceph-mon[35096]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:50 smithi145 ceph-mon[35096]: from='osd.1 [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:12:51.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:50 smithi145 ceph-mon[35096]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:12:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:50 smithi107 ceph-mon[30659]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:50 smithi107 ceph-mon[30659]: from='osd.1 [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:12:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:50 smithi107 ceph-mon[30659]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:12:51.257 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:12:51.283 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_4 2024-09-19T04:12:51.536 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:12:51.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 -- 172.21.15.145:0/1919130073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1314076220 msgr2=0x7f1314074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:51.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 --2- 172.21.15.145:0/1919130073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1314076220 0x7f1314074680 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1304009900 tx=0x7f130402a410 comp rx=0 tx=0).stop 2024-09-19T04:12:51.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 -- 172.21.15.145:0/1919130073 shutdown_connections 2024-09-19T04:12:51.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 --2- 172.21.15.145:0/1919130073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1314109a90 0x7f131410bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:51.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 --2- 172.21.15.145:0/1919130073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1314074c50 0x7f13140750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:51.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 --2- 172.21.15.145:0/1919130073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1314076220 0x7f1314074680 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:51.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 -- 172.21.15.145:0/1919130073 >> 172.21.15.145:0/1919130073 conn(0x7f13140fd3c0 msgr2=0x7f13140ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:12:51.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 -- 172.21.15.145:0/1919130073 shutdown_connections 2024-09-19T04:12:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 -- 172.21.15.145:0/1919130073 wait complete. 2024-09-19T04:12:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 Processor -- start 2024-09-19T04:12:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.975+0000 7f131a65e640 1 -- start start 2024-09-19T04:12:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f131a65e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1314074c50 0x7f131419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f131a65e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1314076220 0x7f131419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f131a65e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1314109a90 0x7f13141a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f131a65e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f131410fb40 con 0x7f1314109a90 2024-09-19T04:12:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f131a65e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f131410f9c0 con 0x7f1314076220 2024-09-19T04:12:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f131a65e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f131410fcc0 con 0x7f1314074c50 2024-09-19T04:12:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1313fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1314074c50 0x7f131419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1313fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1314074c50 0x7f131419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50980/0 (socket says 172.21.15.145:50980) 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1313fff640 1 -- 172.21.15.145:0/786805879 learned_addr learned my addr 172.21.15.145:0/786805879 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f13137fe640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1314076220 0x7f131419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1318bfb640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1314109a90 0x7f13141a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1313fff640 1 -- 172.21.15.145:0/786805879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1314076220 msgr2=0x7f131419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1313fff640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1314076220 0x7f131419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1313fff640 1 -- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1314109a90 msgr2=0x7f13141a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1313fff640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1314109a90 0x7f13141a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1313fff640 1 -- 172.21.15.145:0/786805879 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13141a1d90 con 0x7f1314074c50 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1318bfb640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1314109a90 0x7f13141a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f1313fff640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1314074c50 0x7f131419a160 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1304031440 tx=0x7f1304032e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.976+0000 7f13117fa640 1 -- 172.21.15.145:0/786805879 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130403f030 con 0x7f1314074c50 2024-09-19T04:12:52.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.977+0000 7f13117fa640 1 -- 172.21.15.145:0/786805879 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1304003ce0 con 0x7f1314074c50 2024-09-19T04:12:52.002 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.977+0000 7f131a65e640 1 -- 172.21.15.145:0/786805879 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13141a2020 con 0x7f1314074c50 2024-09-19T04:12:52.002 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.977+0000 7f13117fa640 1 -- 172.21.15.145:0/786805879 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304035070 con 0x7f1314074c50 2024-09-19T04:12:52.002 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.977+0000 7f131a65e640 1 -- 172.21.15.145:0/786805879 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13141a2580 con 0x7f1314074c50 2024-09-19T04:12:52.002 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.978+0000 7f131a65e640 1 -- 172.21.15.145:0/786805879 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1314108810 con 0x7f1314074c50 2024-09-19T04:12:52.002 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.979+0000 7f13117fa640 1 -- 172.21.15.145:0/786805879 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f1304042030 con 0x7f1314074c50 2024-09-19T04:12:52.002 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.979+0000 7f13117fa640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f13000a31e0 0x7f13000a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:12:52.002 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.980+0000 7f13117fa640 1 -- 172.21.15.145:0/786805879 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f13040ee550 con 0x7f1314074c50 2024-09-19T04:12:52.002 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.980+0000 7f13137fe640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f13000a31e0 0x7f13000a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:12:52.002 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.980+0000 7f13137fe640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f13000a31e0 0x7f13000a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f131419b680 tx=0x7f12fc00b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:12:52.002 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:51.984+0000 7f13117fa640 1 -- 172.21.15.145:0/786805879 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f13040bcbe0 con 0x7f1314074c50 2024-09-19T04:12:52.164 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:12:52.162+0000 7f131a65e640 1 -- 172.21.15.145:0/786805879 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1314062630 con 0x7f13000a31e0 2024-09-19T04:12:52.164 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:12:52.164 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='osd.1 [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:12:52.164 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:12:52.164 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:52.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:12:52.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:52.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:52.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:52.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:12:52.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='osd.1 [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:12:52.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:12:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:12:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:12:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='osd.1 [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:12:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:12:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:12:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:12:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:52.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:12:52.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:12:52.696 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1[39953]: 2024-09-19T04:12:52.268+0000 7fd62a01b640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T04:12:52.696 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1[39953]: 2024-09-19T04:12:52.271+0000 7fd621e3b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:12:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi115 2024-09-19T04:12:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi115 to 17088M 2024-09-19T04:12:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:12:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:12:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:53.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:53.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:53.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:53.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:53.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:52 smithi115 ceph-mon[35477]: from='osd.1 ' entity='osd.1' 2024-09-19T04:12:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi115 2024-09-19T04:12:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi115 to 17088M 2024-09-19T04:12:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:12:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:12:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:52 smithi145 ceph-mon[35096]: from='osd.1 ' entity='osd.1' 2024-09-19T04:12:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi115 2024-09-19T04:12:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi115 to 17088M 2024-09-19T04:12:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:12:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:12:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:12:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:12:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:12:53.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:52 smithi107 ceph-mon[30659]: from='osd.1 ' entity='osd.1' 2024-09-19T04:12:54.080 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:53 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:12:54.080 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:53 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:12:54.080 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:53 smithi145 ceph-mon[35096]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:54.080 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:54.080 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:53 smithi145 ceph-mon[35096]: osd.1 [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] boot 2024-09-19T04:12:54.080 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:53 smithi145 ceph-mon[35096]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:12:54.080 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:53 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:53 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:53 smithi115 ceph-mon[35477]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:53 smithi115 ceph-mon[35477]: osd.1 [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] boot 2024-09-19T04:12:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:53 smithi115 ceph-mon[35477]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:12:54.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:54.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:53 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:12:54.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:53 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:12:54.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:53 smithi107 ceph-mon[30659]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:12:54.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:54.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:53 smithi107 ceph-mon[30659]: osd.1 [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] boot 2024-09-19T04:12:54.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:53 smithi107 ceph-mon[30659]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:12:54.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:12:54.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:54 smithi115 ceph-mon[35477]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:54.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:54 smithi115 ceph-mon[35477]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T04:12:54.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:54 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/2995989385' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bfdcafa-f7f4-4f78-af4b-af1a16e24be7"}]: dispatch 2024-09-19T04:12:54.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:54 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bfdcafa-f7f4-4f78-af4b-af1a16e24be7"}]: dispatch 2024-09-19T04:12:54.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:54 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bfdcafa-f7f4-4f78-af4b-af1a16e24be7"}]': finished 2024-09-19T04:12:54.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:54 smithi115 ceph-mon[35477]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:12:54.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:54 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:12:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:54 smithi145 ceph-mon[35096]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:54 smithi145 ceph-mon[35096]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T04:12:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:54 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/2995989385' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bfdcafa-f7f4-4f78-af4b-af1a16e24be7"}]: dispatch 2024-09-19T04:12:55.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:54 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bfdcafa-f7f4-4f78-af4b-af1a16e24be7"}]: dispatch 2024-09-19T04:12:55.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:54 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bfdcafa-f7f4-4f78-af4b-af1a16e24be7"}]': finished 2024-09-19T04:12:55.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:54 smithi145 ceph-mon[35096]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:12:55.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:54 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:12:55.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:54 smithi107 ceph-mon[30659]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:12:55.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:54 smithi107 ceph-mon[30659]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T04:12:55.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:54 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/2995989385' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bfdcafa-f7f4-4f78-af4b-af1a16e24be7"}]: dispatch 2024-09-19T04:12:55.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:54 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bfdcafa-f7f4-4f78-af4b-af1a16e24be7"}]: dispatch 2024-09-19T04:12:55.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:54 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bfdcafa-f7f4-4f78-af4b-af1a16e24be7"}]': finished 2024-09-19T04:12:55.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:54 smithi107 ceph-mon[30659]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:12:55.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:54 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:12:56.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:55 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/2665494508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:56.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:55 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/2665494508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:56.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:55 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/2665494508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:12:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:56 smithi115 ceph-mon[35477]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:12:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:56 smithi145 ceph-mon[35096]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:12:57.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:56 smithi107 ceph-mon[30659]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:12:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:12:57 smithi115 ceph-mon[35477]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:12:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:12:57 smithi145 ceph-mon[35096]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:12:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:12:57 smithi107 ceph-mon[30659]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:01.071 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:00 smithi145 ceph-mon[35096]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:00 smithi115 ceph-mon[35477]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:00 smithi107 ceph-mon[30659]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:03.064 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:02 smithi145 ceph-mon[35096]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:03.064 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:13:03.064 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:02 smithi115 ceph-mon[35477]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:13:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:02 smithi107 ceph-mon[30659]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:13:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:04.051 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:03 smithi145 ceph-mon[35096]: Deploying daemon osd.2 on smithi145 2024-09-19T04:13:04.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:03 smithi115 ceph-mon[35477]: Deploying daemon osd.2 on smithi145 2024-09-19T04:13:04.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:03 smithi107 ceph-mon[30659]: Deploying daemon osd.2 on smithi145 2024-09-19T04:13:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:04 smithi115 ceph-mon[35477]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:04 smithi145 ceph-mon[35096]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:05.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:04 smithi107 ceph-mon[30659]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:07.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:06 smithi115 ceph-mon[35477]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:06 smithi145 ceph-mon[35096]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:06 smithi107 ceph-mon[30659]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:09.097 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:08 smithi145 ceph-mon[35096]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:09.097 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:08 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:09.097 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:08 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:09.097 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:08 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:08 smithi115 ceph-mon[35477]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:08 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:08 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:08 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:09.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:08 smithi107 ceph-mon[30659]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:09.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:08 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:09.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:08 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:09.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:08 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:10.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:10 smithi145 ceph-mon[35096]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:10 smithi115 ceph-mon[35477]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:11.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:10 smithi107 ceph-mon[30659]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:12.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.065+0000 7f13117fa640 1 -- 172.21.15.145:0/786805879 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1314062630 con 0x7f13000a31e0 2024-09-19T04:13:12.067 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 2 on host 'smithi145' 2024-09-19T04:13:12.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 -- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f13000a31e0 msgr2=0x7f13000a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:12.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f13000a31e0 0x7f13000a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f131419b680 tx=0x7f12fc00b390 comp rx=0 tx=0).stop 2024-09-19T04:13:12.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 -- 172.21.15.145:0/786805879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1314074c50 msgr2=0x7f131419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:12.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1314074c50 0x7f131419a160 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1304031440 tx=0x7f1304032e20 comp rx=0 tx=0).stop 2024-09-19T04:13:12.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 -- 172.21.15.145:0/786805879 shutdown_connections 2024-09-19T04:13:12.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f13000a31e0 0x7f13000a56a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:12.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1314109a90 0x7f13141a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:12.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1314076220 0x7f131419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:12.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 --2- 172.21.15.145:0/786805879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1314074c50 0x7f131419a160 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:12.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 -- 172.21.15.145:0/786805879 >> 172.21.15.145:0/786805879 conn(0x7f13140fd3c0 msgr2=0x7f13141059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:12.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 -- 172.21.15.145:0/786805879 shutdown_connections 2024-09-19T04:13:12.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:13:12.068+0000 7f131a65e640 1 -- 172.21.15.145:0/786805879 wait complete. 2024-09-19T04:13:12.361 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.361 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.361 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:12.361 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:12.362 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.362 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:12 smithi145 ceph-mon[35096]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:12.362 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:12.362 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.362 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:12.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:12.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:12 smithi107 ceph-mon[30659]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:12.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:12.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.666 DEBUG:teuthology.orchestra.run.smithi145:osd.2> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T04:13:12.668 INFO:tasks.cephadm:Deploying osd.3 on smithi107 with /dev/vg_nvme/lv_3... 2024-09-19T04:13:12.669 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:13:12.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:12.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:12.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:12 smithi115 ceph-mon[35477]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:13.037 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:14.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:14 smithi107 ceph-mon[30659]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:14.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:14 smithi107 ceph-mon[30659]: from='osd.2 [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:13:14.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:14 smithi107 ceph-mon[30659]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:13:15.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:14 smithi145 ceph-mon[35096]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:15.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:14 smithi145 ceph-mon[35096]: from='osd.2 [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:13:15.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:14 smithi145 ceph-mon[35096]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:13:15.092 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2[40074]: 2024-09-19T04:13:14.741+0000 7ff69b977640 -1 osd.2 0 log_to_monitors true 2024-09-19T04:13:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:14 smithi115 ceph-mon[35477]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:14 smithi115 ceph-mon[35477]: from='osd.2 [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:13:15.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:14 smithi115 ceph-mon[35477]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:13:15.936 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:13:15.968 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-09-19T04:13:16.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='osd.2 [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:13:16.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='osd.2 [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:13:16.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:13:16.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:16.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:13:16.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:13:16.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:16.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:16.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:13:16.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='osd.2 [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:13:16.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:13:16.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:16.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:13:16.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:13:16.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:16.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:16.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:16.345 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:13:16.724 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2[40074]: 2024-09-19T04:13:16.240+0000 7ff6978fa640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T04:13:16.724 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2[40074]: 2024-09-19T04:13:16.244+0000 7ff68f71a640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:13:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.877+0000 7f75df200640 1 -- 172.21.15.107:0/1601320601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e0103c50 msgr2=0x7f75e0104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.877+0000 7f75df200640 1 --2- 172.21.15.107:0/1601320601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e0103c50 0x7f75e0104050 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009860 tx=0x7f75d002a340 comp rx=0 tx=0).stop 2024-09-19T04:13:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.877+0000 7f75df200640 1 -- 172.21.15.107:0/1601320601 shutdown_connections 2024-09-19T04:13:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.877+0000 7f75df200640 1 --2- 172.21.15.107:0/1601320601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0105810 0x7f75e010c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.877+0000 7f75df200640 1 --2- 172.21.15.107:0/1601320601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75e0104e50 0x7f75e01052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.877+0000 7f75df200640 1 --2- 172.21.15.107:0/1601320601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e0103c50 0x7f75e0104050 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.877+0000 7f75df200640 1 -- 172.21.15.107:0/1601320601 >> 172.21.15.107:0/1601320601 conn(0x7f75e00ff3e0 msgr2=0x7f75e0101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.877+0000 7f75df200640 1 -- 172.21.15.107:0/1601320601 shutdown_connections 2024-09-19T04:13:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.877+0000 7f75df200640 1 -- 172.21.15.107:0/1601320601 wait complete. 2024-09-19T04:13:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.878+0000 7f75df200640 1 Processor -- start 2024-09-19T04:13:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.878+0000 7f75df200640 1 -- start start 2024-09-19T04:13:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75df200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75e0103c50 0x7f75e0199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75df200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0104e50 0x7f75e019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75df200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e0105810 0x7f75e01a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75df200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75e010f8e0 con 0x7f75e0103c50 2024-09-19T04:13:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75df200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75e010f760 con 0x7f75e0104e50 2024-09-19T04:13:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75df200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75e010fa60 con 0x7f75e0105810 2024-09-19T04:13:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75e0103c50 0x7f75e0199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75e0103c50 0x7f75e0199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52354/0 (socket says 172.21.15.107:52354) 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75dde00640 1 -- 172.21.15.107:0/2935421151 learned_addr learned my addr 172.21.15.107:0/2935421151 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75de800640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e0105810 0x7f75e01a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75dd400640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0104e50 0x7f75e019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75dde00640 1 -- 172.21.15.107:0/2935421151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e0105810 msgr2=0x7f75e01a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75dde00640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e0105810 0x7f75e01a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75dde00640 1 -- 172.21.15.107:0/2935421151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0104e50 msgr2=0x7f75e019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75dde00640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0104e50 0x7f75e019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.879+0000 7f75dde00640 1 -- 172.21.15.107:0/2935421151 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e01a1b60 con 0x7f75e0103c50 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.880+0000 7f75dde00640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75e0103c50 0x7f75e0199f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f75d0035040 tx=0x7f75d0033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.880+0000 7f75d6a00640 1 -- 172.21.15.107:0/2935421151 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0037040 con 0x7f75e0103c50 2024-09-19T04:13:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.880+0000 7f75d6a00640 1 -- 172.21.15.107:0/2935421151 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75d0030850 con 0x7f75e0103c50 2024-09-19T04:13:16.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.880+0000 7f75df200640 1 -- 172.21.15.107:0/2935421151 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e01a1df0 con 0x7f75e0103c50 2024-09-19T04:13:16.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.880+0000 7f75d6a00640 1 -- 172.21.15.107:0/2935421151 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0004920 con 0x7f75e0103c50 2024-09-19T04:13:16.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.880+0000 7f75df200640 1 -- 172.21.15.107:0/2935421151 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e01a2240 con 0x7f75e0103c50 2024-09-19T04:13:16.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.881+0000 7f75df200640 1 -- 172.21.15.107:0/2935421151 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e010b000 con 0x7f75e0103c50 2024-09-19T04:13:16.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.883+0000 7f75d6a00640 1 -- 172.21.15.107:0/2935421151 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f75d0004050 con 0x7f75e0103c50 2024-09-19T04:13:16.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.885+0000 7f75d6a00640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f75c40a3260 0x7f75c40a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:16.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.885+0000 7f75d6a00640 1 -- 172.21.15.107:0/2935421151 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f75d00ee050 con 0x7f75e0103c50 2024-09-19T04:13:16.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.885+0000 7f75dd400640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f75c40a3260 0x7f75c40a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:16.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.886+0000 7f75dd400640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f75c40a3260 0x7f75c40a5720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f75e019b450 tx=0x7f75cc009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:16.890+0000 7f75d6a00640 1 -- 172.21.15.107:0/2935421151 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f75d00bc420 con 0x7f75e0103c50 2024-09-19T04:13:17.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:17.073+0000 7f75df200640 1 -- 172.21.15.107:0/2935421151 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f75e0062630 con 0x7f75c40a3260 2024-09-19T04:13:17.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:16 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi145 2024-09-19T04:13:17.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:16 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi145 to 17088M 2024-09-19T04:13:17.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:16 smithi107 ceph-mon[30659]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:17.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:16 smithi107 ceph-mon[30659]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:13:17.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:16 smithi107 ceph-mon[30659]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T04:13:17.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:16 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:17.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:16 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:17.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:16 smithi107 ceph-mon[30659]: from='osd.2 ' entity='osd.2' 2024-09-19T04:13:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:16 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi145 2024-09-19T04:13:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:16 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi145 to 17088M 2024-09-19T04:13:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:16 smithi115 ceph-mon[35477]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:16 smithi115 ceph-mon[35477]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:13:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:16 smithi115 ceph-mon[35477]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T04:13:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:16 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:16 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:16 smithi115 ceph-mon[35477]: from='osd.2 ' entity='osd.2' 2024-09-19T04:13:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:16 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi145 2024-09-19T04:13:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:16 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi145 to 17088M 2024-09-19T04:13:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:16 smithi145 ceph-mon[35096]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:17.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:16 smithi145 ceph-mon[35096]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:13:17.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:16 smithi145 ceph-mon[35096]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T04:13:17.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:16 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:17.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:16 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:17.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:16 smithi145 ceph-mon[35096]: from='osd.2 ' entity='osd.2' 2024-09-19T04:13:18.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:18 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:13:18.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:18 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:13:18.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:18.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:13:18.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:13:18.677 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:18.677 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:18 smithi107 ceph-mon[30659]: osd.2 [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] boot 2024-09-19T04:13:18.677 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:18 smithi107 ceph-mon[30659]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:13:18.677 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:18.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:18 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:13:18.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:18 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:13:18.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:18 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:18.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:18 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:13:18.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:18 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:13:18.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:18 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:18.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:18 smithi115 ceph-mon[35477]: osd.2 [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] boot 2024-09-19T04:13:18.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:18 smithi115 ceph-mon[35477]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:13:18.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:18 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:18.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:18 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:13:18.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:18 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:13:18.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:18 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:18.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:18 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:13:18.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:18 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:13:18.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:18 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:18.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:18 smithi145 ceph-mon[35096]: osd.2 [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] boot 2024-09-19T04:13:18.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:18 smithi145 ceph-mon[35096]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:13:18.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:18 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:13:19.679 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:19 smithi107 ceph-mon[30659]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:19.679 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:19 smithi107 ceph-mon[30659]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:19.679 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:19 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:13:19.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:19 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:13:19.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:19 smithi107 ceph-mon[30659]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T04:13:19.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:19 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:13:19.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:19 smithi115 ceph-mon[35477]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:19.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:19 smithi115 ceph-mon[35477]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:19.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:19 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:13:19.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:19 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:13:19.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:19 smithi115 ceph-mon[35477]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T04:13:19.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:19 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:13:19.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:19 smithi145 ceph-mon[35096]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:19.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:19 smithi145 ceph-mon[35096]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:13:19.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:19 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:13:19.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:19 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:13:19.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:19 smithi145 ceph-mon[35096]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T04:13:19.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:19 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:13:20.230 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:13:19 smithi107 sudo[48688]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:13:20.230 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:13:19 smithi107 sudo[48688]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:20.231 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:13:19 smithi107 sudo[48688]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:20.231 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:13:19 smithi107 sudo[48688]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:20.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:20.334+0000 7f75d6a00640 1 -- 172.21.15.107:0/2935421151 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f75d00b8390 con 0x7f75e0103c50 2024-09-19T04:13:20.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:20 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:13:20.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:20 smithi115 ceph-mon[35477]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:13:20.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:20 smithi115 ceph-mon[35477]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:20.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:20 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1711368694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12"}]: dispatch 2024-09-19T04:13:20.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:20 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1711368694' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12"}]': finished 2024-09-19T04:13:20.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:20 smithi115 ceph-mon[35477]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:13:20.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:20 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:20 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:13:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:20 smithi145 ceph-mon[35096]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:13:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:20 smithi145 ceph-mon[35096]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:20 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1711368694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12"}]: dispatch 2024-09-19T04:13:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:20 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1711368694' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12"}]': finished 2024-09-19T04:13:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:20 smithi145 ceph-mon[35096]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:13:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:20 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:20.730 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:13:20 smithi107 sudo[48722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:13:20.730 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:13:20 smithi107 sudo[48722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:20.730 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:13:20 smithi107 sudo[48722]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:20.731 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:13:20 smithi107 sudo[48722]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:20.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:20 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:13:20.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:20 smithi107 ceph-mon[30659]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:13:20.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:20 smithi107 ceph-mon[30659]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:20.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:20 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1711368694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12"}]: dispatch 2024-09-19T04:13:20.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:20 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1711368694' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12"}]': finished 2024-09-19T04:13:20.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:20 smithi107 ceph-mon[30659]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:13:20.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:20 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:21.068 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:13:20 smithi115 sudo[44077]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:13:21.068 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:13:20 smithi115 sudo[44077]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:21.068 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:13:20 smithi115 sudo[44077]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:21.068 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:13:20 smithi115 sudo[44077]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:21.328 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:13:21 smithi115 sudo[44080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:13:21.328 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:13:21 smithi115 sudo[44080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:21.328 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:13:21 smithi115 sudo[44080]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:21.328 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:13:21 smithi115 sudo[44080]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:21.595 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:21 smithi145 ceph-mon[35096]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:13:21.595 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:21 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:21.595 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:21 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1344407997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:13:21.595 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:21 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:21.596 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:21 smithi145 sudo[43708]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:13:21.596 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:21 smithi145 sudo[43708]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:21.596 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:21 smithi145 sudo[43708]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:21.596 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:21 smithi145 sudo[43708]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:21.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:21 smithi115 ceph-mon[35477]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:13:21.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:21 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:21.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:21 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1344407997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:13:21.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:21 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:21.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:21 smithi107 ceph-mon[30659]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:13:21.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:21 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:21.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:21 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1344407997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:13:21.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:21 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:21.974 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:21 smithi145 sudo[43711]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:13:21.974 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:21 smithi145 sudo[43711]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:21.974 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:21 smithi145 sudo[43711]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:21.974 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:13:21 smithi145 sudo[43711]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:22.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:22 smithi107 sudo[48963]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:13:22.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:22 smithi107 sudo[48963]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:22.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:22 smithi107 sudo[48963]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:22.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:22 smithi107 sudo[48963]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:22 smithi145 ceph-mon[35096]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:13:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:22 smithi145 sudo[43714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:13:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:22 smithi145 sudo[43714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:22 smithi145 sudo[43714]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:22 smithi145 sudo[43714]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:22 smithi107 sudo[49156]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:13:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:22 smithi107 sudo[49156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:22 smithi107 sudo[49156]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:22 smithi107 sudo[49156]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:22.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:22.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:22 smithi107 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:13:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:22 smithi115 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:13:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 sudo[43717]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:13:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 sudo[43717]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 sudo[43717]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 sudo[43717]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:23.711 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 sudo[44083]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:13:23.711 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 sudo[44083]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:23.711 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 sudo[44083]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:23.711 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 sudo[44083]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:23.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:23.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:13:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:13:23.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:23.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:13:23.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:13:23.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:13:23.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:13:23.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:13:23.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:13:23.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:13:23.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:13:23.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:13:23.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:13:23.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:13:23.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:13:24.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 sudo[44086]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:13:24.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 sudo[44086]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:13:24.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 sudo[44086]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:13:24.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 sudo[44086]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:13:24.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:24.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:13:24.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:13:24.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:13:24.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:13:24.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:13:24.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:13:24.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:13:24.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:13:24.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:13:24.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:13:24.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:13:24.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:13:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:24 smithi145 ceph-mon[35096]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:13:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:24 smithi145 ceph-mon[35096]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:13:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:24 smithi145 ceph-mon[35096]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:24.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:24 smithi107 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:13:24.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:24 smithi107 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:13:24.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:24 smithi107 ceph-mon[30659]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:24.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:24 smithi115 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:13:24.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:24 smithi115 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:13:24.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:24 smithi115 ceph-mon[35477]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:26.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:26 smithi107 ceph-mon[30659]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:26 smithi115 ceph-mon[35477]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:26 smithi145 ceph-mon[35096]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:29.116 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:28 smithi107 ceph-mon[30659]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:28 smithi115 ceph-mon[35477]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:28 smithi145 ceph-mon[35096]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:29.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:29 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:13:29.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:29 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:30.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:29 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:13:30.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:29 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:30.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:29 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:13:30.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:29 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:30 smithi115 ceph-mon[35477]: Deploying daemon osd.3 on smithi107 2024-09-19T04:13:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:30 smithi115 ceph-mon[35477]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:31.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:30 smithi107 ceph-mon[30659]: Deploying daemon osd.3 on smithi107 2024-09-19T04:13:31.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:30 smithi107 ceph-mon[30659]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:30 smithi145 ceph-mon[35096]: Deploying daemon osd.3 on smithi107 2024-09-19T04:13:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:30 smithi145 ceph-mon[35096]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:32.905 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:32 smithi107 ceph-mon[30659]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:32 smithi115 ceph-mon[35477]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:32 smithi145 ceph-mon[35096]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:35.083 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:34 smithi107 ceph-mon[30659]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:34 smithi115 ceph-mon[35477]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:34 smithi145 ceph-mon[35096]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:36 smithi115 ceph-mon[35477]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:36 smithi145 ceph-mon[35096]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:36 smithi107 ceph-mon[30659]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:37 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:37 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:37 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:37 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:37 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:37 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:38.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:37 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:38.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:37 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:38.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:37 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:38.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:38 smithi107 ceph-mon[30659]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:38 smithi115 ceph-mon[35477]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:38 smithi145 ceph-mon[35096]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:40.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:40 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:40.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:40 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:40.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:40 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:40.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:40 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:40.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:40 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:40.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:40 smithi115 ceph-mon[35477]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:40.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:40 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:40.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:40 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:40.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:40 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:40.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:40 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:40.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:40 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:40.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:40 smithi145 ceph-mon[35096]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:40.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:40 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:40.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:40 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:40.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:40 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:40.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:40 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:40.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:40 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:40.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:40 smithi107 ceph-mon[30659]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:41.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.626+0000 7f75d6a00640 1 -- 172.21.15.107:0/2935421151 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f75e0062630 con 0x7f75c40a3260 2024-09-19T04:13:41.628 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 3 on host 'smithi107' 2024-09-19T04:13:41.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 -- 172.21.15.107:0/2935421151 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f75c40a3260 msgr2=0x7f75c40a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f75c40a3260 0x7f75c40a5720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f75e019b450 tx=0x7f75cc009260 comp rx=0 tx=0).stop 2024-09-19T04:13:41.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 -- 172.21.15.107:0/2935421151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75e0103c50 msgr2=0x7f75e0199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75e0103c50 0x7f75e0199f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f75d0035040 tx=0x7f75d0033d90 comp rx=0 tx=0).stop 2024-09-19T04:13:41.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 -- 172.21.15.107:0/2935421151 shutdown_connections 2024-09-19T04:13:41.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f75c40a3260 0x7f75c40a5720 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e0105810 0x7f75e01a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0104e50 0x7f75e019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 --2- 172.21.15.107:0/2935421151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75e0103c50 0x7f75e0199f30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 -- 172.21.15.107:0/2935421151 >> 172.21.15.107:0/2935421151 conn(0x7f75e00ff3e0 msgr2=0x7f75e01081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 -- 172.21.15.107:0/2935421151 shutdown_connections 2024-09-19T04:13:41.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:13:41.629+0000 7f75df200640 1 -- 172.21.15.107:0/2935421151 wait complete. 2024-09-19T04:13:41.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:41 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:41.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:41 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:41.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:41 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:41.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:41 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:41.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:41 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:41.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:41 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:41.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:41 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:13:41.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:41 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:41.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:41 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:42.343 DEBUG:teuthology.orchestra.run.smithi107:osd.3> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T04:13:42.347 INFO:tasks.cephadm:Deploying osd.4 on smithi115 with /dev/vg_nvme/lv_3... 2024-09-19T04:13:42.347 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:13:42.598 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:13:42.944 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:42 smithi115 ceph-mon[35477]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:42.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:42 smithi145 ceph-mon[35096]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:42.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:42 smithi107 ceph-mon[30659]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:42.981 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:13:42 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3[50730]: 2024-09-19T04:13:42.855+0000 7f1b70e2c640 -1 osd.3 0 log_to_monitors true 2024-09-19T04:13:43.828 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:43 smithi107 ceph-mon[30659]: from='osd.3 [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:13:43.828 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:43 smithi107 ceph-mon[30659]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:13:43.931 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:43 smithi115 ceph-mon[35477]: from='osd.3 [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:13:43.931 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:43 smithi115 ceph-mon[35477]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:13:43.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:43 smithi145 ceph-mon[35096]: from='osd.3 [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:13:43.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:43 smithi145 ceph-mon[35096]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:13:44.888 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:44 smithi115 ceph-mon[35477]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:13:44.888 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:44 smithi115 ceph-mon[35477]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:13:44.888 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:44 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:44.889 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:44 smithi115 ceph-mon[35477]: from='osd.3 [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:13:44.889 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:44 smithi115 ceph-mon[35477]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:13:44.889 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:44 smithi115 ceph-mon[35477]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:44.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:44 smithi145 ceph-mon[35096]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:13:44.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:44 smithi145 ceph-mon[35096]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:13:44.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:44 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:44.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:44 smithi145 ceph-mon[35096]: from='osd.3 [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:13:44.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:44 smithi145 ceph-mon[35096]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:13:44.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:44 smithi145 ceph-mon[35096]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:44.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:44 smithi107 ceph-mon[30659]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:13:44.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:44 smithi107 ceph-mon[30659]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:13:44.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:44 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:44.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:44 smithi107 ceph-mon[30659]: from='osd.3 [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:13:44.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:44 smithi107 ceph-mon[30659]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:13:44.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:44 smithi107 ceph-mon[30659]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:13:45.212 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2024-09-19T04:13:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:13:45.584 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:13:45 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3[50730]: 2024-09-19T04:13:45.248+0000 7f1b6ba00640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T04:13:45.585 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:13:45 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3[50730]: 2024-09-19T04:13:45.268+0000 7f1b61a00640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:13:45.828 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:45 smithi115 ceph-mon[35477]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:13:45.829 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:45 smithi115 ceph-mon[35477]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:13:45.829 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:45 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:45.829 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:45 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:45.829 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:45 smithi115 ceph-mon[35477]: from='osd.3 ' entity='osd.3' 2024-09-19T04:13:45.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:45 smithi107 ceph-mon[30659]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:13:45.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:45 smithi107 ceph-mon[30659]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:13:45.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:45 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:45.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:45 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:45.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:45 smithi107 ceph-mon[30659]: from='osd.3 ' entity='osd.3' 2024-09-19T04:13:45.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.910+0000 7f43c8fcd640 1 -- 172.21.15.115:0/1491554495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c4101d10 msgr2=0x7f43c4102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.910+0000 7f43c8fcd640 1 --2- 172.21.15.115:0/1491554495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c4101d10 0x7f43c4102190 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009900 tx=0x7f43b802a410 comp rx=0 tx=0).stop 2024-09-19T04:13:45.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.910+0000 7f43c8fcd640 1 -- 172.21.15.115:0/1491554495 shutdown_connections 2024-09-19T04:13:45.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.910+0000 7f43c8fcd640 1 --2- 172.21.15.115:0/1491554495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43c41026d0 0x7f43c4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.910+0000 7f43c8fcd640 1 --2- 172.21.15.115:0/1491554495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c4101d10 0x7f43c4102190 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.910+0000 7f43c8fcd640 1 --2- 172.21.15.115:0/1491554495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43c4100b10 0x7f43c4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.910+0000 7f43c8fcd640 1 -- 172.21.15.115:0/1491554495 >> 172.21.15.115:0/1491554495 conn(0x7f43c40fc2a0 msgr2=0x7f43c40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.910+0000 7f43c8fcd640 1 -- 172.21.15.115:0/1491554495 shutdown_connections 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.910+0000 7f43c8fcd640 1 -- 172.21.15.115:0/1491554495 wait complete. 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c8fcd640 1 Processor -- start 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c8fcd640 1 -- start start 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c8fcd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43c4100b10 0x7f43c4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c8fcd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43c4101d10 0x7f43c4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c8fcd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c41026d0 0x7f43c419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c8fcd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f43c410d8c0 con 0x7f43c4100b10 2024-09-19T04:13:45.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c8fcd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f43c410d740 con 0x7f43c41026d0 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c8fcd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f43c410da40 con 0x7f43c4101d10 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c2d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c41026d0 0x7f43c419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c2d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c41026d0 0x7f43c419f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55186/0 (socket says 172.21.15.115:55186) 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c2d9d640 1 -- 172.21.15.115:0/450696381 learned_addr learned my addr 172.21.15.115:0/450696381 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c1d9b640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43c4101d10 0x7f43c4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c2d9d640 1 -- 172.21.15.115:0/450696381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43c4101d10 msgr2=0x7f43c4198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c2d9d640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43c4101d10 0x7f43c4198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c2d9d640 1 -- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43c4100b10 msgr2=0x7f43c4197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c259c640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43c4100b10 0x7f43c4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c2d9d640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43c4100b10 0x7f43c4197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c2d9d640 1 -- 172.21.15.115:0/450696381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c419fb10 con 0x7f43c41026d0 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c1d9b640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43c4101d10 0x7f43c4198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:45.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.911+0000 7f43c259c640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43c4100b10 0x7f43c4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.912+0000 7f43c2d9d640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c41026d0 0x7f43c419f4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00eaa0 tx=0x7f43bc00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.912+0000 7f43b37fe640 1 -- 172.21.15.115:0/450696381 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00cdb0 con 0x7f43c41026d0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.912+0000 7f43c8fcd640 1 -- 172.21.15.115:0/450696381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c419fe00 con 0x7f43c41026d0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.912+0000 7f43b37fe640 1 -- 172.21.15.115:0/450696381 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc005020 con 0x7f43c41026d0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.912+0000 7f43b37fe640 1 -- 172.21.15.115:0/450696381 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc0116d0 con 0x7f43c41026d0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.912+0000 7f43c8fcd640 1 -- 172.21.15.115:0/450696381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c41a0300 con 0x7f43c41026d0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.913+0000 7f43c8fcd640 1 -- 172.21.15.115:0/450696381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c4107eb0 con 0x7f43c41026d0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.914+0000 7f43b37fe640 1 -- 172.21.15.115:0/450696381 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f43bc0054d0 con 0x7f43c41026d0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.914+0000 7f43b37fe640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f43ac0a3270 0x7f43ac0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.915+0000 7f43b37fe640 1 -- 172.21.15.115:0/450696381 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7f43bc0ce9a0 con 0x7f43c41026d0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.915+0000 7f43c259c640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f43ac0a3270 0x7f43ac0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.915+0000 7f43c259c640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f43ac0a3270 0x7f43ac0a5730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43c4101b70 tx=0x7f43b4005e70 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:45.919+0000 7f43b37fe640 1 -- 172.21.15.115:0/450696381 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f43bc016030 con 0x7f43c41026d0 2024-09-19T04:13:45.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:45 smithi145 ceph-mon[35096]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:13:45.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:45 smithi145 ceph-mon[35096]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:13:45.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:45 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:45.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:45 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:45.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:45 smithi145 ceph-mon[35096]: from='osd.3 ' entity='osd.3' 2024-09-19T04:13:46.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:13:46.099+0000 7f43c8fcd640 1 -- 172.21.15.115:0/450696381 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f43c4062630 con 0x7f43ac0a3270 2024-09-19T04:13:46.817 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:13:46.817 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:13:46.817 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: osd.3 [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] boot 2024-09-19T04:13:46.817 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T04:13:46.819 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:46.819 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:13:46.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:13:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:13:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: osd.3 [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] boot 2024-09-19T04:13:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:13:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:46.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:13:46.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:13:46.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: osd.3 [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] boot 2024-09-19T04:13:46.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T04:13:46.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:13:46.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:13:46.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:13:46.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:13:46.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:46.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:13:46.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:13:46.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:13:46.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:13:46.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:13:46.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:47.754 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:47 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi107 2024-09-19T04:13:47.755 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:47 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi107 to 8544M 2024-09-19T04:13:47.755 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:47 smithi115 ceph-mon[35477]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:47.755 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:47 smithi115 ceph-mon[35477]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:13:47.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:47 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi107 2024-09-19T04:13:47.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:47 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi107 to 8544M 2024-09-19T04:13:47.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:47 smithi145 ceph-mon[35096]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:47.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:47 smithi145 ceph-mon[35096]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:13:47.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:47 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi107 2024-09-19T04:13:47.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:47 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi107 to 8544M 2024-09-19T04:13:47.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:47 smithi107 ceph-mon[30659]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:13:47.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:47 smithi107 ceph-mon[30659]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:13:48.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:48 smithi145 ceph-mon[35096]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:13:48.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:48 smithi145 ceph-mon[35096]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:13:48.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:48 smithi145 ceph-mon[35096]: from='client.? 172.21.15.115:0/3331899322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86388823-3a04-4923-8ece-d0e955be08e1"}]: dispatch 2024-09-19T04:13:48.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:48 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86388823-3a04-4923-8ece-d0e955be08e1"}]: dispatch 2024-09-19T04:13:48.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:48 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86388823-3a04-4923-8ece-d0e955be08e1"}]': finished 2024-09-19T04:13:48.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:48 smithi145 ceph-mon[35096]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:13:48.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:48 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:13:48.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:48 smithi107 ceph-mon[30659]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:13:48.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:48 smithi107 ceph-mon[30659]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:13:48.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:48 smithi107 ceph-mon[30659]: from='client.? 172.21.15.115:0/3331899322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86388823-3a04-4923-8ece-d0e955be08e1"}]: dispatch 2024-09-19T04:13:48.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:48 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86388823-3a04-4923-8ece-d0e955be08e1"}]: dispatch 2024-09-19T04:13:48.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:48 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86388823-3a04-4923-8ece-d0e955be08e1"}]': finished 2024-09-19T04:13:48.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:48 smithi107 ceph-mon[30659]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:13:48.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:48 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:13:49.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:48 smithi115 ceph-mon[35477]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:13:49.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:48 smithi115 ceph-mon[35477]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:13:49.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:48 smithi115 ceph-mon[35477]: from='client.? 172.21.15.115:0/3331899322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86388823-3a04-4923-8ece-d0e955be08e1"}]: dispatch 2024-09-19T04:13:49.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:48 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86388823-3a04-4923-8ece-d0e955be08e1"}]: dispatch 2024-09-19T04:13:49.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:48 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86388823-3a04-4923-8ece-d0e955be08e1"}]': finished 2024-09-19T04:13:49.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:48 smithi115 ceph-mon[35477]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:13:49.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:48 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:13:49.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:49 smithi145 ceph-mon[35096]: from='client.? 172.21.15.115:0/4039142797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:13:49.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:49 smithi107 ceph-mon[30659]: from='client.? 172.21.15.115:0/4039142797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:13:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:49 smithi115 ceph-mon[35477]: from='client.? 172.21.15.115:0/4039142797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:13:50.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:50 smithi145 ceph-mon[35096]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:13:50.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:50 smithi107 ceph-mon[30659]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:13:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:50 smithi115 ceph-mon[35477]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:13:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:52 smithi145 ceph-mon[35096]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T04:13:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:52 smithi115 ceph-mon[35477]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T04:13:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:52 smithi107 ceph-mon[30659]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T04:13:54.963 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:54 smithi115 ceph-mon[35477]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T04:13:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:54 smithi145 ceph-mon[35096]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T04:13:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:54 smithi107 ceph-mon[30659]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T04:13:56.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:56 smithi115 ceph-mon[35477]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:13:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:56 smithi145 ceph-mon[35096]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:13:57.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:56 smithi107 ceph-mon[30659]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:13:58.121 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:13:58.121 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:13:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:13:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:13:59.006 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:58 smithi115 ceph-mon[35477]: Deploying daemon osd.4 on smithi115 2024-09-19T04:13:59.006 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:13:58 smithi115 ceph-mon[35477]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T04:13:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:58 smithi145 ceph-mon[35096]: Deploying daemon osd.4 on smithi115 2024-09-19T04:13:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:13:58 smithi145 ceph-mon[35096]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T04:13:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:58 smithi107 ceph-mon[30659]: Deploying daemon osd.4 on smithi115 2024-09-19T04:13:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:13:58 smithi107 ceph-mon[30659]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T04:14:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:00 smithi115 ceph-mon[35477]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T04:14:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:00 smithi145 ceph-mon[35096]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T04:14:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:00 smithi107 ceph-mon[30659]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T04:14:02.895 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:02 smithi115 ceph-mon[35477]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:02 smithi145 ceph-mon[35096]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:02 smithi107 ceph-mon[30659]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:04.076 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:04.076 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:04.076 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:03 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:04.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:04.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:04.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:03 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:04.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:04.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:04.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:03 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:05.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:04 smithi107 ceph-mon[30659]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:05.191 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:04 smithi115 ceph-mon[35477]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:04 smithi145 ceph-mon[35096]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:06 smithi115 ceph-mon[35477]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:06 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:07.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:06 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:07.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:06 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:07.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:06 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:07.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:06 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:06 smithi145 ceph-mon[35096]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:06 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:06 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:06 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:06 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:06 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:06 smithi107 ceph-mon[30659]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:06 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:07.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:06 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:07.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:06 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:07.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:06 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:07.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:06 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:07.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.673+0000 7f43b37fe640 1 -- 172.21.15.115:0/450696381 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f43c4062630 con 0x7f43ac0a3270 2024-09-19T04:14:07.675 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 4 on host 'smithi115' 2024-09-19T04:14:07.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.675+0000 7f43c8fcd640 1 -- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f43ac0a3270 msgr2=0x7f43ac0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.675+0000 7f43c8fcd640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f43ac0a3270 0x7f43ac0a5730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43c4101b70 tx=0x7f43b4005e70 comp rx=0 tx=0).stop 2024-09-19T04:14:07.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.675+0000 7f43c8fcd640 1 -- 172.21.15.115:0/450696381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c41026d0 msgr2=0x7f43c419f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.675+0000 7f43c8fcd640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c41026d0 0x7f43c419f4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00eaa0 tx=0x7f43bc00ef60 comp rx=0 tx=0).stop 2024-09-19T04:14:07.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.675+0000 7f43c8fcd640 1 -- 172.21.15.115:0/450696381 shutdown_connections 2024-09-19T04:14:07.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.675+0000 7f43c8fcd640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f43ac0a3270 0x7f43ac0a5730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.675+0000 7f43c8fcd640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c41026d0 0x7f43c419f4a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.675+0000 7f43c8fcd640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43c4101d10 0x7f43c4198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.676+0000 7f43c8fcd640 1 --2- 172.21.15.115:0/450696381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43c4100b10 0x7f43c4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.676+0000 7f43c8fcd640 1 -- 172.21.15.115:0/450696381 >> 172.21.15.115:0/450696381 conn(0x7f43c40fc2a0 msgr2=0x7f43c4105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:07.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.676+0000 7f43c8fcd640 1 -- 172.21.15.115:0/450696381 shutdown_connections 2024-09-19T04:14:07.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:14:07.676+0000 7f43c8fcd640 1 -- 172.21.15.115:0/450696381 wait complete. 2024-09-19T04:14:07.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:07.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:07.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:08.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:08.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:08.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:08.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:08.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:08.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:08.348 DEBUG:teuthology.orchestra.run.smithi115:osd.4> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T04:14:08.350 INFO:tasks.cephadm:Deploying osd.5 on smithi145 with /dev/vg_nvme/lv_3... 2024-09-19T04:14:08.350 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:14:08.603 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:14:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:08 smithi115 ceph-mon[35477]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:08 smithi145 ceph-mon[35096]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:09.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:08 smithi107 ceph-mon[30659]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:09.446 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:14:09 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4[46844]: 2024-09-19T04:14:09.121+0000 7f4781c93640 -1 osd.4 0 log_to_monitors true 2024-09-19T04:14:09.928 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:09 smithi145 ceph-mon[35096]: from='osd.4 [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:14:09.928 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:09 smithi145 ceph-mon[35096]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:14:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:09 smithi115 ceph-mon[35477]: from='osd.4 [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:14:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:09 smithi115 ceph-mon[35477]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:14:10.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:09 smithi107 ceph-mon[30659]: from='osd.4 [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:14:10.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:09 smithi107 ceph-mon[30659]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:14:10.860 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:10 smithi115 ceph-mon[35477]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:10.860 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:10 smithi115 ceph-mon[35477]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:14:10.860 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:10 smithi115 ceph-mon[35477]: from='osd.4 [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:14:10.860 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:10 smithi115 ceph-mon[35477]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:14:10.860 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:10.860 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:10 smithi115 ceph-mon[35477]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:14:11.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:10 smithi145 ceph-mon[35096]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:11.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:10 smithi145 ceph-mon[35096]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:14:11.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:10 smithi145 ceph-mon[35096]: from='osd.4 [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:14:11.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:10 smithi145 ceph-mon[35096]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:14:11.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:11.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:10 smithi145 ceph-mon[35096]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:14:11.112 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:14:11.138 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_3 2024-09-19T04:14:11.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:10 smithi107 ceph-mon[30659]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:10 smithi107 ceph-mon[30659]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:14:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:10 smithi107 ceph-mon[30659]: from='osd.4 [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:14:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:10 smithi107 ceph-mon[30659]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:14:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:10 smithi107 ceph-mon[30659]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:14:11.392 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:14:11.696 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4[46844]: 2024-09-19T04:14:11.254+0000 7f477dc16640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T04:14:11.696 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4[46844]: 2024-09-19T04:14:11.261+0000 7f4775a36640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:14:11.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 -- 172.21.15.145:0/870760165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf4102660 msgr2=0x7fdbf4104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 --2- 172.21.15.145:0/870760165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf4102660 0x7fdbf4104a50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009900 tx=0x7fdbf002a410 comp rx=0 tx=0).stop 2024-09-19T04:14:11.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 -- 172.21.15.145:0/870760165 shutdown_connections 2024-09-19T04:14:11.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 --2- 172.21.15.145:0/870760165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf41078c0 0x7fdbf4109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 --2- 172.21.15.145:0/870760165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbf4104f90 0x7fdbf4107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 --2- 172.21.15.145:0/870760165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf4102660 0x7fdbf4104a50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 -- 172.21.15.145:0/870760165 >> 172.21.15.145:0/870760165 conn(0x7fdbf40fc280 msgr2=0x7fdbf40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 -- 172.21.15.145:0/870760165 shutdown_connections 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 -- 172.21.15.145:0/870760165 wait complete. 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 Processor -- start 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 -- start start 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4102660 0x7fdbf4101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf4104f90 0x7fdbf4100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbf41078c0 0x7fdbf4100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.887+0000 7fdbfb162640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410d900 con 0x7fdbf41078c0 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbfb162640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410d780 con 0x7fdbf4102660 2024-09-19T04:14:11.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbfb162640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410da80 con 0x7fdbf4104f90 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbebfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf4104f90 0x7fdbf4100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbebfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf4104f90 0x7fdbf4100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50950/0 (socket says 172.21.15.145:50950) 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbebfff640 1 -- 172.21.15.145:0/770970735 learned_addr learned my addr 172.21.15.145:0/770970735 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbf8efe640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4102660 0x7fdbf4101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbf96ff640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbf41078c0 0x7fdbf4100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbebfff640 1 -- 172.21.15.145:0/770970735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4102660 msgr2=0x7fdbf4101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbebfff640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4102660 0x7fdbf4101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbebfff640 1 -- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbf41078c0 msgr2=0x7fdbf4100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbebfff640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbf41078c0 0x7fdbf4100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbebfff640 1 -- 172.21.15.145:0/770970735 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf4101410 con 0x7fdbf4104f90 2024-09-19T04:14:11.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbf96ff640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbf41078c0 0x7fdbf4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:11.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbebfff640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf4104f90 0x7fdbf4100610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400b450 tx=0x7fdbe400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:11.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbe9ffb640 1 -- 172.21.15.145:0/770970735 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4015020 con 0x7fdbf4104f90 2024-09-19T04:14:11.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbfb162640 1 -- 172.21.15.145:0/770970735 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf41a2110 con 0x7fdbf4104f90 2024-09-19T04:14:11.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbe9ffb640 1 -- 172.21.15.145:0/770970735 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe4011d70 con 0x7fdbf4104f90 2024-09-19T04:14:11.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbe9ffb640 1 -- 172.21.15.145:0/770970735 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe400f340 con 0x7fdbf4104f90 2024-09-19T04:14:11.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.888+0000 7fdbfb162640 1 -- 172.21.15.145:0/770970735 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf41a26f0 con 0x7fdbf4104f90 2024-09-19T04:14:11.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.890+0000 7fdbfb162640 1 -- 172.21.15.145:0/770970735 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf40fdd70 con 0x7fdbf4104f90 2024-09-19T04:14:11.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.891+0000 7fdbe9ffb640 1 -- 172.21.15.145:0/770970735 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fdbe4018020 con 0x7fdbf4104f90 2024-09-19T04:14:11.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.891+0000 7fdbe9ffb640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdbdc0a3240 0x7fdbdc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.891+0000 7fdbe9ffb640 1 -- 172.21.15.145:0/770970735 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7fdbe40ce200 con 0x7fdbf4104f90 2024-09-19T04:14:11.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.892+0000 7fdbf8efe640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdbdc0a3240 0x7fdbdc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.892+0000 7fdbf8efe640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdbdc0a3240 0x7fdbdc0a5700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00314e0 tx=0x7fdbf0000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:11.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:11.896+0000 7fdbe9ffb640 1 -- 172.21.15.145:0/770970735 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdbe401f020 con 0x7fdbf4104f90 2024-09-19T04:14:12.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:12.070+0000 7fdbfb162640 1 -- 172.21.15.145:0/770970735 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdbf4062630 con 0x7fdbdc0a3240 2024-09-19T04:14:12.072 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:14:12.072 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:14:12.072 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:12.072 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:12.072 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.072 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.072 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:12.072 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='osd.4 ' entity='osd.4' 2024-09-19T04:14:12.072 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:12.073 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.073 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:12.073 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:12.073 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:11 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='osd.4 ' entity='osd.4' 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:12.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:11 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:14:12.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:14:12.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:12.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:12.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:12.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='osd.4 ' entity='osd.4' 2024-09-19T04:14:12.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:12.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:12.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:12.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:12.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:11 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:13.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:14:13.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:14:13.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi115 2024-09-19T04:14:13.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi115 to 8544M 2024-09-19T04:14:13.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:13.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:13.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:14:13.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:14:13.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:13.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: osd.4 [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] boot 2024-09-19T04:14:13.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T04:14:13.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:13.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:14:13.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:14:13.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi115 2024-09-19T04:14:13.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi115 to 8544M 2024-09-19T04:14:13.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:13.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:13.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:14:13.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:14:13.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:13.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: osd.4 [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] boot 2024-09-19T04:14:13.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T04:14:13.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:13.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi115 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi115 to 8544M 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: osd.4 [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] boot 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T04:14:13.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:14:14.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:13 smithi145 ceph-mon[35096]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:14.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:13 smithi115 ceph-mon[35477]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:14.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:13 smithi107 ceph-mon[30659]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:15.077 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:14 smithi145 ceph-mon[35096]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:15.077 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:14 smithi145 ceph-mon[35096]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:14:15.077 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:14 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/2788993127' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47753716-2254-426c-b279-28ca4388d41f"}]: dispatch 2024-09-19T04:14:15.077 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:14 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47753716-2254-426c-b279-28ca4388d41f"}]: dispatch 2024-09-19T04:14:15.077 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:14 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47753716-2254-426c-b279-28ca4388d41f"}]': finished 2024-09-19T04:14:15.078 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:14 smithi145 ceph-mon[35096]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T04:14:15.078 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:14 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:14 smithi115 ceph-mon[35477]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:14 smithi115 ceph-mon[35477]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:14:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:14 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/2788993127' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47753716-2254-426c-b279-28ca4388d41f"}]: dispatch 2024-09-19T04:14:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:14 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47753716-2254-426c-b279-28ca4388d41f"}]: dispatch 2024-09-19T04:14:15.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:14 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47753716-2254-426c-b279-28ca4388d41f"}]': finished 2024-09-19T04:14:15.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:14 smithi115 ceph-mon[35477]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T04:14:15.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:14 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:15.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:14 smithi107 ceph-mon[30659]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:15.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:14 smithi107 ceph-mon[30659]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:14:15.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:14 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/2788993127' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47753716-2254-426c-b279-28ca4388d41f"}]: dispatch 2024-09-19T04:14:15.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:14 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47753716-2254-426c-b279-28ca4388d41f"}]: dispatch 2024-09-19T04:14:15.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:14 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47753716-2254-426c-b279-28ca4388d41f"}]': finished 2024-09-19T04:14:15.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:14 smithi107 ceph-mon[30659]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T04:14:15.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:14 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:16.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:15 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/3472300268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:14:16.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:15 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/3472300268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:14:16.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:15 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/3472300268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:14:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:16 smithi115 ceph-mon[35477]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:16 smithi145 ceph-mon[35096]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:17.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:16 smithi107 ceph-mon[30659]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:18 smithi115 ceph-mon[35477]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:18 smithi145 ceph-mon[35096]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:18 smithi107 ceph-mon[30659]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:20.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:20 smithi145 ceph-mon[35096]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:20 smithi115 ceph-mon[35477]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:21.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:20 smithi107 ceph-mon[30659]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:22 smithi115 ceph-mon[35477]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:14:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:22 smithi145 ceph-mon[35096]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:14:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:23.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:22 smithi107 ceph-mon[30659]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:23.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:14:23.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:24.164 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:23 smithi145 ceph-mon[35096]: Deploying daemon osd.5 on smithi145 2024-09-19T04:14:24.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:23 smithi115 ceph-mon[35477]: Deploying daemon osd.5 on smithi145 2024-09-19T04:14:24.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:23 smithi107 ceph-mon[30659]: Deploying daemon osd.5 on smithi145 2024-09-19T04:14:25.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:24 smithi115 ceph-mon[35477]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:24 smithi145 ceph-mon[35096]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:24 smithi107 ceph-mon[30659]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:26.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:26 smithi145 ceph-mon[35096]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:26 smithi115 ceph-mon[35477]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:27.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:26 smithi107 ceph-mon[30659]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:28 smithi115 ceph-mon[35477]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:28 smithi145 ceph-mon[35096]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:29.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:28 smithi107 ceph-mon[30659]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:30.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:29 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:30.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:29 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:30.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:29 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:30.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:29 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:30.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:29 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:30.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:29 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:30.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:29 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:30.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:29 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:30.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:29 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:30 smithi115 ceph-mon[35477]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:30 smithi145 ceph-mon[35096]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:31.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:30 smithi107 ceph-mon[30659]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:33.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:32 smithi145 ceph-mon[35096]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:33.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:33.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:33.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:33.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:33.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:33.128 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.126+0000 7fdbe9ffb640 1 -- 172.21.15.145:0/770970735 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdbf4062630 con 0x7fdbdc0a3240 2024-09-19T04:14:33.129 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 5 on host 'smithi145' 2024-09-19T04:14:33.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 -- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdbdc0a3240 msgr2=0x7fdbdc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdbdc0a3240 0x7fdbdc0a5700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00314e0 tx=0x7fdbf0000bc0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 -- 172.21.15.145:0/770970735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf4104f90 msgr2=0x7fdbf4100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf4104f90 0x7fdbf4100610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400b450 tx=0x7fdbe400b910 comp rx=0 tx=0).stop 2024-09-19T04:14:33.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 -- 172.21.15.145:0/770970735 shutdown_connections 2024-09-19T04:14:33.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdbdc0a3240 0x7fdbdc0a5700 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbf41078c0 0x7fdbf4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf4104f90 0x7fdbf4100610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 --2- 172.21.15.145:0/770970735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4102660 0x7fdbf4101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 -- 172.21.15.145:0/770970735 >> 172.21.15.145:0/770970735 conn(0x7fdbf40fc280 msgr2=0x7fdbf41030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.132 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.128+0000 7fdbfb162640 1 -- 172.21.15.145:0/770970735 shutdown_connections 2024-09-19T04:14:33.132 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:14:33.129+0000 7fdbfb162640 1 -- 172.21.15.145:0/770970735 wait complete. 2024-09-19T04:14:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:32 smithi115 ceph-mon[35477]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:33.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:33.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:33.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:33.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:32 smithi107 ceph-mon[30659]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:33.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:33.782 DEBUG:teuthology.orchestra.run.smithi145:osd.5> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T04:14:33.785 INFO:tasks.cephadm:Deploying osd.6 on smithi107 with /dev/vg_nvme/lv_2... 2024-09-19T04:14:33.785 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:14:34.069 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:33 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:34.069 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:33 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:34.069 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:33 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:34.157 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:34.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:33 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:34.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:33 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:34.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:33 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:34.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:33 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:34.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:33 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:34.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:33 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:35.076 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:34 smithi145 ceph-mon[35096]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:34 smithi115 ceph-mon[35477]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:35.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:34 smithi107 ceph-mon[30659]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:35.474 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:14:35 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:14:35.220+0000 7efe9a21a640 -1 osd.5 0 log_to_monitors true 2024-09-19T04:14:35.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:35 smithi107 ceph-mon[30659]: from='osd.5 [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:14:35.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:35 smithi107 ceph-mon[30659]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:14:36.050 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:35 smithi145 ceph-mon[35096]: from='osd.5 [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:14:36.051 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:35 smithi145 ceph-mon[35096]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:14:36.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:35 smithi115 ceph-mon[35477]: from='osd.5 [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:14:36.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:35 smithi115 ceph-mon[35477]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:14:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='osd.5 [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:36 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.224 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:14:36.872+0000 7efe9619d640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T04:14:37.224 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:14:36.878+0000 7efe91fc5640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:14:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:14:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='osd.5 [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:36 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:37.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:14:37.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='osd.5 [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:14:37.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:14:37.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:37.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:14:37.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:37.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:14:37.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:37.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:14:37.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:36 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:37.234 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:14:37.266 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-09-19T04:14:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:14:38.048 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:37 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi145 2024-09-19T04:14:38.048 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:37 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-19T04:14:38.048 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:37 smithi107 ceph-mon[30659]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:14:38.048 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:37 smithi107 ceph-mon[30659]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:14:38.049 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:37 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:38.049 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:37 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.160+0000 7f339fc00640 1 -- 172.21.15.107:0/3223463512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3398109d80 msgr2=0x7f33981106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.160+0000 7f339fc00640 1 --2- 172.21.15.107:0/3223463512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3398109d80 0x7f33981106a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3390009ca0 tx=0x7f339002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.160+0000 7f339fc00640 1 -- 172.21.15.107:0/3223463512 shutdown_connections 2024-09-19T04:14:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.160+0000 7f339fc00640 1 --2- 172.21.15.107:0/3223463512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3398109d80 0x7f33981106a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.160+0000 7f339fc00640 1 --2- 172.21.15.107:0/3223463512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33981093c0 0x7f3398109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.160+0000 7f339fc00640 1 --2- 172.21.15.107:0/3223463512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3398108400 0x7f3398108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.160+0000 7f339fc00640 1 -- 172.21.15.107:0/3223463512 >> 172.21.15.107:0/3223463512 conn(0x7f33980773d0 msgr2=0x7f3398079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.161+0000 7f339fc00640 1 -- 172.21.15.107:0/3223463512 shutdown_connections 2024-09-19T04:14:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.161+0000 7f339fc00640 1 -- 172.21.15.107:0/3223463512 wait complete. 2024-09-19T04:14:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.161+0000 7f339fc00640 1 Processor -- start 2024-09-19T04:14:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339fc00640 1 -- start start 2024-09-19T04:14:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339fc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3398108400 0x7f339819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339fc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33981093c0 0x7f339819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339fc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3398109d80 0x7f33981a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339fc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3398113f20 con 0x7f3398108400 2024-09-19T04:14:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339fc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3398113da0 con 0x7f33981093c0 2024-09-19T04:14:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339fc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f33981140a0 con 0x7f3398109d80 2024-09-19T04:14:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3398108400 0x7f339819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3398108400 0x7f339819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45802/0 (socket says 172.21.15.107:45802) 2024-09-19T04:14:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339d200640 1 -- 172.21.15.107:0/503828706 learned_addr learned my addr 172.21.15.107:0/503828706 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:14:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f338fe00640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33981093c0 0x7f339819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.162+0000 7f339dc00640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3398109d80 0x7f33981a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f339d200640 1 -- 172.21.15.107:0/503828706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3398109d80 msgr2=0x7f33981a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f339d200640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3398109d80 0x7f33981a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f339d200640 1 -- 172.21.15.107:0/503828706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33981093c0 msgr2=0x7f339819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f339d200640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33981093c0 0x7f339819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f339d200640 1 -- 172.21.15.107:0/503828706 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33981a60a0 con 0x7f3398108400 2024-09-19T04:14:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f339d200640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3398108400 0x7f339819e470 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f338800e9f0 tx=0x7f338800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f3396000640 1 -- 172.21.15.107:0/503828706 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338800cd80 con 0x7f3398108400 2024-09-19T04:14:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f3396000640 1 -- 172.21.15.107:0/503828706 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3388005500 con 0x7f3398108400 2024-09-19T04:14:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f339fc00640 1 -- 172.21.15.107:0/503828706 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33981a6390 con 0x7f3398108400 2024-09-19T04:14:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f3396000640 1 -- 172.21.15.107:0/503828706 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388010690 con 0x7f3398108400 2024-09-19T04:14:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.163+0000 7f339fc00640 1 -- 172.21.15.107:0/503828706 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33981a6880 con 0x7f3398108400 2024-09-19T04:14:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.165+0000 7f339fc00640 1 -- 172.21.15.107:0/503828706 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339810f580 con 0x7f3398108400 2024-09-19T04:14:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.167+0000 7f3396000640 1 -- 172.21.15.107:0/503828706 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f33880046c0 con 0x7f3398108400 2024-09-19T04:14:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.168+0000 7f3396000640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f337c0a3270 0x7f337c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:38.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.168+0000 7f3396000640 1 -- 172.21.15.107:0/503828706 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4566+0+0 (secure 0 0 0) 0x7f33880cf0c0 con 0x7f3398108400 2024-09-19T04:14:38.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.168+0000 7f338fe00640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f337c0a3270 0x7f337c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:38.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.169+0000 7f338fe00640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f337c0a3270 0x7f337c0a5730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f339819f990 tx=0x7f338000b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:38.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.171+0000 7f3396000640 1 -- 172.21.15.107:0/503828706 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3388054210 con 0x7f3398108400 2024-09-19T04:14:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:37 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi145 2024-09-19T04:14:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:37 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-19T04:14:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:37 smithi115 ceph-mon[35477]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:14:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:37 smithi115 ceph-mon[35477]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:14:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:37 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:37 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:37 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi145 2024-09-19T04:14:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:37 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-19T04:14:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:37 smithi145 ceph-mon[35096]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:14:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:37 smithi145 ceph-mon[35096]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:14:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:37 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:37 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:38.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:14:38.358+0000 7f339fc00640 1 -- 172.21.15.107:0/503828706 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3398062630 con 0x7f337c0a3270 2024-09-19T04:14:39.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:38 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:14:39.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:38 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:14:39.193 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:38 smithi107 ceph-mon[30659]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:39.193 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:39.193 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:38 smithi107 ceph-mon[30659]: osd.5 [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] boot 2024-09-19T04:14:39.193 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:38 smithi107 ceph-mon[30659]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T04:14:39.193 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:39.193 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:14:39.193 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:14:39.193 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:38 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:14:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:38 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:14:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:38 smithi115 ceph-mon[35477]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:38 smithi115 ceph-mon[35477]: osd.5 [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] boot 2024-09-19T04:14:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:38 smithi115 ceph-mon[35477]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T04:14:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:14:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:14:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:38 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:14:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:38 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:14:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:38 smithi145 ceph-mon[35096]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:14:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:38 smithi145 ceph-mon[35096]: osd.5 [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] boot 2024-09-19T04:14:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:38 smithi145 ceph-mon[35096]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T04:14:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:14:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:14:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:14:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:39.955 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:39 smithi107 ceph-mon[30659]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:39.955 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:39 smithi107 ceph-mon[30659]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:14:40.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:39 smithi115 ceph-mon[35477]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:40.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:39 smithi115 ceph-mon[35477]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:14:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:39 smithi145 ceph-mon[35096]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:14:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:39 smithi145 ceph-mon[35096]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:14:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:40 smithi115 ceph-mon[35477]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:40 smithi115 ceph-mon[35477]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:14:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:40 smithi145 ceph-mon[35096]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:40 smithi145 ceph-mon[35096]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:14:41.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:40 smithi107 ceph-mon[30659]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:41.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:40 smithi107 ceph-mon[30659]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:14:42.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:41 smithi115 ceph-mon[35477]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:14:42.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:41 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1150353532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23e84324-77d0-460a-ae79-96ce61c8d0c0"}]: dispatch 2024-09-19T04:14:42.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:41 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1150353532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23e84324-77d0-460a-ae79-96ce61c8d0c0"}]': finished 2024-09-19T04:14:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:41 smithi115 ceph-mon[35477]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T04:14:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:41 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:14:42.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:41 smithi145 ceph-mon[35096]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:14:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:41 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1150353532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23e84324-77d0-460a-ae79-96ce61c8d0c0"}]: dispatch 2024-09-19T04:14:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:41 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1150353532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23e84324-77d0-460a-ae79-96ce61c8d0c0"}]': finished 2024-09-19T04:14:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:41 smithi145 ceph-mon[35096]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T04:14:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:41 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:14:42.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:41 smithi107 ceph-mon[30659]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:14:42.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:41 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1150353532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23e84324-77d0-460a-ae79-96ce61c8d0c0"}]: dispatch 2024-09-19T04:14:42.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:41 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1150353532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23e84324-77d0-460a-ae79-96ce61c8d0c0"}]': finished 2024-09-19T04:14:42.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:41 smithi107 ceph-mon[30659]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T04:14:42.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:41 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:14:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:42 smithi115 ceph-mon[35477]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:42 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3781509520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:14:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:42 smithi145 ceph-mon[35096]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:42 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3781509520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:14:43.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:42 smithi107 ceph-mon[30659]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:43.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:42 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3781509520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:14:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:44 smithi115 ceph-mon[35477]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:44 smithi145 ceph-mon[35096]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:45.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:44 smithi107 ceph-mon[30659]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:46.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:46 smithi107 ceph-mon[30659]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:14:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:46 smithi115 ceph-mon[35477]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:14:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:46 smithi145 ceph-mon[35096]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:14:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:48 smithi115 ceph-mon[35477]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-19T04:14:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:48 smithi145 ceph-mon[35096]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-19T04:14:49.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:48 smithi107 ceph-mon[30659]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-19T04:14:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:50 smithi115 ceph-mon[35477]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T04:14:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:14:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:50 smithi145 ceph-mon[35096]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T04:14:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:14:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:50 smithi107 ceph-mon[30659]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T04:14:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:14:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:14:51.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:51 smithi107 ceph-mon[30659]: Deploying daemon osd.6 on smithi107 2024-09-19T04:14:52.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:51 smithi115 ceph-mon[35477]: Deploying daemon osd.6 on smithi107 2024-09-19T04:14:52.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:51 smithi145 ceph-mon[35096]: Deploying daemon osd.6 on smithi107 2024-09-19T04:14:53.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:52 smithi107 ceph-mon[30659]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-19T04:14:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:52 smithi115 ceph-mon[35477]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-19T04:14:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:52 smithi145 ceph-mon[35096]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-19T04:14:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:54 smithi115 ceph-mon[35477]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:14:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:54 smithi145 ceph-mon[35096]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:14:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:54 smithi107 ceph-mon[30659]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:14:57.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:56 smithi107 ceph-mon[30659]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:14:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:56 smithi115 ceph-mon[35477]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:14:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:56 smithi145 ceph-mon[35096]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:14:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:14:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:14:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:58 smithi115 ceph-mon[35477]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:58 smithi145 ceph-mon[35096]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:58 smithi107 ceph-mon[30659]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:14:59.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:14:59 smithi107 ceph-mon[30659]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:00.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:14:59 smithi115 ceph-mon[35477]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:00.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:14:59 smithi145 ceph-mon[35096]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:01.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:01 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:01.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:01 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:01.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:01 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:01.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:01 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:01.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:01 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:01.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:01 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:01.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:01 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:01.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:01 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:01.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:01 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:01.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:01 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:01.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:01 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:01.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:01 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:01.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:01 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:01.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:01 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:01.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:01 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:02.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.310+0000 7f3396000640 1 -- 172.21.15.107:0/503828706 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3398062630 con 0x7f337c0a3270 2024-09-19T04:15:02.313 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 6 on host 'smithi107' 2024-09-19T04:15:02.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.312+0000 7f339fc00640 1 -- 172.21.15.107:0/503828706 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f337c0a3270 msgr2=0x7f337c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:02.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.312+0000 7f339fc00640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f337c0a3270 0x7f337c0a5730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f339819f990 tx=0x7f338000b410 comp rx=0 tx=0).stop 2024-09-19T04:15:02.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.312+0000 7f339fc00640 1 -- 172.21.15.107:0/503828706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3398108400 msgr2=0x7f339819e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:02.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.312+0000 7f339fc00640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3398108400 0x7f339819e470 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f338800e9f0 tx=0x7f338800edf0 comp rx=0 tx=0).stop 2024-09-19T04:15:02.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.312+0000 7f339fc00640 1 -- 172.21.15.107:0/503828706 shutdown_connections 2024-09-19T04:15:02.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.312+0000 7f339fc00640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f337c0a3270 0x7f337c0a5730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:02.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.312+0000 7f339fc00640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3398109d80 0x7f33981a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:02.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.312+0000 7f339fc00640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33981093c0 0x7f339819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:02.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.312+0000 7f339fc00640 1 --2- 172.21.15.107:0/503828706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3398108400 0x7f339819e470 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:02.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.313+0000 7f339fc00640 1 -- 172.21.15.107:0/503828706 >> 172.21.15.107:0/503828706 conn(0x7f33980773d0 msgr2=0x7f339810c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:02.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.313+0000 7f339fc00640 1 -- 172.21.15.107:0/503828706 shutdown_connections 2024-09-19T04:15:02.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:02.313+0000 7f339fc00640 1 -- 172.21.15.107:0/503828706 wait complete. 2024-09-19T04:15:02.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:02 smithi115 ceph-mon[35477]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:02.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:02.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:02 smithi145 ceph-mon[35096]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:02.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:02.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:02.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:02.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:02 smithi107 ceph-mon[30659]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:02.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:02.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:02.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:03.193 DEBUG:teuthology.orchestra.run.smithi107:osd.6> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T04:15:03.197 INFO:tasks.cephadm:Deploying osd.7 on smithi115 with /dev/vg_nvme/lv_2... 2024-09-19T04:15:03.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:15:03.422 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:15:03 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:15:03.385+0000 7f640b21e640 -1 osd.6 0 log_to_monitors true 2024-09-19T04:15:03.452 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:15:03.674 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:03 smithi107 ceph-mon[30659]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:15:03.895 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:03 smithi115 ceph-mon[35477]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:15:03.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:03 smithi145 ceph-mon[35096]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:15:04.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:04 smithi115 ceph-mon[35477]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:15:04.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:04 smithi115 ceph-mon[35477]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:15:04.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:04 smithi115 ceph-mon[35477]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:15:04.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:04 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:04.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:04 smithi115 ceph-mon[35477]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:04.935 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:15:04 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:15:04.603+0000 7f6405e00640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T04:15:04.935 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:15:04 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:15:04.617+0000 7f6400e00640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:15:04.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:04 smithi107 ceph-mon[30659]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:15:04.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:04 smithi107 ceph-mon[30659]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:15:04.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:04 smithi107 ceph-mon[30659]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:15:04.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:04 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:04.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:04 smithi107 ceph-mon[30659]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:04.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:04 smithi145 ceph-mon[35096]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:15:04.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:04 smithi145 ceph-mon[35096]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:15:04.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:04 smithi145 ceph-mon[35096]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:15:04.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:04 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:04.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:04 smithi145 ceph-mon[35096]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:05.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:05 smithi115 ceph-mon[35477]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:15:05.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:05 smithi115 ceph-mon[35477]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:15:05.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:05 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:05.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:05 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:05.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:05 smithi145 ceph-mon[35096]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:15:05.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:05 smithi145 ceph-mon[35096]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:15:05.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:05 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:05.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:05 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:05.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:05 smithi107 ceph-mon[30659]: from='osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:15:05.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:05 smithi107 ceph-mon[30659]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:15:05.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:05.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:06.177 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:15:06.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2024-09-19T04:15:06.450 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:15:06.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:06 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:15:06.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:06 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:15:06.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:06 smithi107 ceph-mon[30659]: osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] boot 2024-09-19T04:15:06.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:06 smithi107 ceph-mon[30659]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T04:15:06.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:06 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:06.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:06 smithi107 ceph-mon[30659]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:06.836 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:06 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:15:06.836 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:06 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:15:06.836 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:06 smithi115 ceph-mon[35477]: osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] boot 2024-09-19T04:15:06.836 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:06 smithi115 ceph-mon[35477]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T04:15:06.836 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:06 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:06.836 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:06 smithi115 ceph-mon[35477]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:06.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.918+0000 7fb77750f640 1 -- 172.21.15.115:0/2940383118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7701037c0 msgr2=0x7fb77010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.918+0000 7fb77750f640 1 --2- 172.21.15.115:0/2940383118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7701037c0 0x7fb77010a050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb768009ca0 tx=0x7fb76802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.918+0000 7fb77750f640 1 -- 172.21.15.115:0/2940383118 shutdown_connections 2024-09-19T04:15:06.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.918+0000 7fb77750f640 1 --2- 172.21.15.115:0/2940383118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7701037c0 0x7fb77010a050 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.918+0000 7fb77750f640 1 --2- 172.21.15.115:0/2940383118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770102e00 0x7fb770103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.918+0000 7fb77750f640 1 --2- 172.21.15.115:0/2940383118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770101c00 0x7fb770102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.918+0000 7fb77750f640 1 -- 172.21.15.115:0/2940383118 >> 172.21.15.115:0/2940383118 conn(0x7fb7700fd390 msgr2=0x7fb7700ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:06.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.919+0000 7fb77750f640 1 -- 172.21.15.115:0/2940383118 shutdown_connections 2024-09-19T04:15:06.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.919+0000 7fb77750f640 1 -- 172.21.15.115:0/2940383118 wait complete. 2024-09-19T04:15:06.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.919+0000 7fb77750f640 1 Processor -- start 2024-09-19T04:15:06.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.919+0000 7fb77750f640 1 -- start start 2024-09-19T04:15:06.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.919+0000 7fb77750f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770101c00 0x7fb770197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.919+0000 7fb77750f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770102e00 0x7fb770198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.919+0000 7fb77750f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7701037c0 0x7fb77019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.919+0000 7fb77750f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010d900 con 0x7fb770102e00 2024-09-19T04:15:06.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.919+0000 7fb77750f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010d780 con 0x7fb7701037c0 2024-09-19T04:15:06.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.919+0000 7fb77750f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010da80 con 0x7fb770101c00 2024-09-19T04:15:06.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb775aac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7701037c0 0x7fb77019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb775aac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7701037c0 0x7fb77019f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36706/0 (socket says 172.21.15.115:36706) 2024-09-19T04:15:06.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb775aac640 1 -- 172.21.15.115:0/3644651667 learned_addr learned my addr 172.21.15.115:0/3644651667 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-19T04:15:06.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb7752ab640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770101c00 0x7fb770197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb774aaa640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770102e00 0x7fb770198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb775aac640 1 -- 172.21.15.115:0/3644651667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770101c00 msgr2=0x7fb770197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb775aac640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770101c00 0x7fb770197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb775aac640 1 -- 172.21.15.115:0/3644651667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770102e00 msgr2=0x7fb770198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:06.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb775aac640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770102e00 0x7fb770198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:06.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb775aac640 1 -- 172.21.15.115:0/3644651667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77019fb00 con 0x7fb7701037c0 2024-09-19T04:15:06.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb775aac640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7701037c0 0x7fb77019f490 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb7680098d0 tx=0x7fb76802f840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:06.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb7667fc640 1 -- 172.21.15.115:0/3644651667 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76802fc30 con 0x7fb7701037c0 2024-09-19T04:15:06.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb7667fc640 1 -- 172.21.15.115:0/3644651667 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb768003e50 con 0x7fb7701037c0 2024-09-19T04:15:06.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb7667fc640 1 -- 172.21.15.115:0/3644651667 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680305c0 con 0x7fb7701037c0 2024-09-19T04:15:06.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb77750f640 1 -- 172.21.15.115:0/3644651667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77019fd90 con 0x7fb7701037c0 2024-09-19T04:15:06.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.920+0000 7fb77750f640 1 -- 172.21.15.115:0/3644651667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7701a02f0 con 0x7fb7701037c0 2024-09-19T04:15:06.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.921+0000 7fb77750f640 1 -- 172.21.15.115:0/3644651667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb770108fb0 con 0x7fb7701037c0 2024-09-19T04:15:06.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.922+0000 7fb7667fc640 1 -- 172.21.15.115:0/3644651667 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb768003970 con 0x7fb7701037c0 2024-09-19T04:15:06.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.923+0000 7fb7667fc640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb75c0a3270 0x7fb75c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:06.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.923+0000 7fb7667fc640 1 -- 172.21.15.115:0/3644651667 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7fb7680f0010 con 0x7fb7701037c0 2024-09-19T04:15:06.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.923+0000 7fb7752ab640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb75c0a3270 0x7fb75c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:06.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.924+0000 7fb7752ab640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb75c0a3270 0x7fb75c0a5730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb770102c60 tx=0x7fb76c008040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:06.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:06.927+0000 7fb7667fc640 1 -- 172.21.15.115:0/3644651667 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb7680bdb60 con 0x7fb7701037c0 2024-09-19T04:15:06.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:06 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:15:06.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:06 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:15:06.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:06 smithi145 ceph-mon[35096]: osd.6 [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] boot 2024-09-19T04:15:06.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:06 smithi145 ceph-mon[35096]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T04:15:06.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:06 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:15:06.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:06 smithi145 ceph-mon[35096]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:15:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:07.106+0000 7fb77750f640 1 -- 172.21.15.115:0/3644651667 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb770062630 con 0x7fb75c0a3270 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi107 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi107 to 5696M 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:07.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:07.824 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.824 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:15:07.824 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:15:07.824 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:07.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T04:15:07.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi107 2024-09-19T04:15:07.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi107 to 5696M 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:15:07.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:07.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T04:15:07.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi107 2024-09-19T04:15:07.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi107 to 5696M 2024-09-19T04:15:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:15:07.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:15:07.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:08.812 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:08 smithi115 ceph-mon[35477]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:08.813 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:08 smithi115 ceph-mon[35477]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:08.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:08 smithi107 ceph-mon[30659]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:08.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:08 smithi107 ceph-mon[30659]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:08 smithi145 ceph-mon[35096]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:08 smithi145 ceph-mon[35096]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:09.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:09 smithi115 ceph-mon[35477]: from='client.? 172.21.15.115:0/2294850311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0c2813-44ea-4a41-9a51-28e0e327a400"}]: dispatch 2024-09-19T04:15:09.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:09 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0c2813-44ea-4a41-9a51-28e0e327a400"}]: dispatch 2024-09-19T04:15:09.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:09 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb0c2813-44ea-4a41-9a51-28e0e327a400"}]': finished 2024-09-19T04:15:09.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:09 smithi115 ceph-mon[35477]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T04:15:09.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:09 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:09.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:09 smithi145 ceph-mon[35096]: from='client.? 172.21.15.115:0/2294850311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0c2813-44ea-4a41-9a51-28e0e327a400"}]: dispatch 2024-09-19T04:15:09.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:09 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0c2813-44ea-4a41-9a51-28e0e327a400"}]: dispatch 2024-09-19T04:15:09.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:09 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb0c2813-44ea-4a41-9a51-28e0e327a400"}]': finished 2024-09-19T04:15:09.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:09 smithi145 ceph-mon[35096]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T04:15:09.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:09 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:09.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:09 smithi107 ceph-mon[30659]: from='client.? 172.21.15.115:0/2294850311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0c2813-44ea-4a41-9a51-28e0e327a400"}]: dispatch 2024-09-19T04:15:09.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:09 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0c2813-44ea-4a41-9a51-28e0e327a400"}]: dispatch 2024-09-19T04:15:09.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:09 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb0c2813-44ea-4a41-9a51-28e0e327a400"}]': finished 2024-09-19T04:15:09.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:09 smithi107 ceph-mon[30659]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T04:15:09.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:09 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:10.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:10 smithi115 ceph-mon[35477]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:10.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:10 smithi115 ceph-mon[35477]: from='client.? 172.21.15.115:0/2639521269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:15:10.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:10 smithi145 ceph-mon[35096]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:10.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:10 smithi145 ceph-mon[35096]: from='client.? 172.21.15.115:0/2639521269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:15:10.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:10 smithi107 ceph-mon[30659]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:10.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:10 smithi107 ceph-mon[30659]: from='client.? 172.21.15.115:0/2639521269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:15:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:12 smithi115 ceph-mon[35477]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:12 smithi145 ceph-mon[35096]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:13.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:12 smithi107 ceph-mon[30659]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:15.118 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:14 smithi115 ceph-mon[35477]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:14 smithi145 ceph-mon[35096]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:15.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:14 smithi107 ceph-mon[30659]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:16.845 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:16 smithi115 ceph-mon[35477]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:16 smithi145 ceph-mon[35096]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:17.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:16 smithi107 ceph-mon[30659]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:18 smithi115 ceph-mon[35477]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:18 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:15:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:18 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:18 smithi145 ceph-mon[35096]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:18 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:15:19.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:18 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:18 smithi107 ceph-mon[30659]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:15:19.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:18 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:20.117 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:19 smithi115 ceph-mon[35477]: Deploying daemon osd.7 on smithi115 2024-09-19T04:15:20.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:19 smithi145 ceph-mon[35096]: Deploying daemon osd.7 on smithi115 2024-09-19T04:15:20.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:19 smithi107 ceph-mon[30659]: Deploying daemon osd.7 on smithi115 2024-09-19T04:15:21.161 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:20 smithi115 ceph-mon[35477]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:20 smithi145 ceph-mon[35096]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:21.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:20 smithi107 ceph-mon[30659]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:22.932 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:22 smithi115 ceph-mon[35477]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:22 smithi145 ceph-mon[35096]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:23.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:22 smithi107 ceph-mon[30659]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:24.905 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:24 smithi115 ceph-mon[35477]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:24.905 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:24 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:24.905 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:24 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:24.906 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:24 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:24 smithi145 ceph-mon[35096]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:24 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:24 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:24 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:24 smithi107 ceph-mon[30659]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:24 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:25.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:24 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:25.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:24 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:26.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:26 smithi107 ceph-mon[30659]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:26 smithi115 ceph-mon[35477]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:26 smithi145 ceph-mon[35096]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:27.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.973+0000 7fb7667fc640 1 -- 172.21.15.115:0/3644651667 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb770062630 con 0x7fb75c0a3270 2024-09-19T04:15:27.976 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 7 on host 'smithi115' 2024-09-19T04:15:27.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 -- 172.21.15.115:0/3644651667 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb75c0a3270 msgr2=0x7fb75c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb75c0a3270 0x7fb75c0a5730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb770102c60 tx=0x7fb76c008040 comp rx=0 tx=0).stop 2024-09-19T04:15:27.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 -- 172.21.15.115:0/3644651667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7701037c0 msgr2=0x7fb77019f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7701037c0 0x7fb77019f490 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb7680098d0 tx=0x7fb76802f840 comp rx=0 tx=0).stop 2024-09-19T04:15:27.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 -- 172.21.15.115:0/3644651667 shutdown_connections 2024-09-19T04:15:27.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb75c0a3270 0x7fb75c0a5730 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7701037c0 0x7fb77019f490 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770102e00 0x7fb770198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 --2- 172.21.15.115:0/3644651667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770101c00 0x7fb770197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 -- 172.21.15.115:0/3644651667 >> 172.21.15.115:0/3644651667 conn(0x7fb7700fd390 msgr2=0x7fb770106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:27.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 -- 172.21.15.115:0/3644651667 shutdown_connections 2024-09-19T04:15:27.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:15:27.976+0000 7fb77750f640 1 -- 172.21.15.115:0/3644651667 wait complete. 2024-09-19T04:15:28.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:27 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:28.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:27 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:28.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:27 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:28.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:27 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:28.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:27 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:28.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:27 smithi115 ceph-mon[35477]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:28.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:27 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:27 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:27 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:27 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:27 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:27 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:27 smithi145 ceph-mon[35096]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:28.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:27 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:27 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:27 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:27 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:27 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:27 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:28.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:27 smithi107 ceph-mon[30659]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:28.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:27 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:28.743 DEBUG:teuthology.orchestra.run.smithi115:osd.7> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T04:15:28.746 INFO:tasks.cephadm:Deploying osd.8 on smithi145 with /dev/vg_nvme/lv_2... 2024-09-19T04:15:28.746 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:15:29.001 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:15:29.070 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:29.070 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:29.122 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:29.122 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:29.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:29.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:30.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:29 smithi145 ceph-mon[35096]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:30.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:29 smithi107 ceph-mon[30659]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:30.275 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:29 smithi115 ceph-mon[35477]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:30.276 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:15:29 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:15:29.961+0000 7f827770f640 -1 osd.7 0 log_to_monitors true 2024-09-19T04:15:31.149 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:30 smithi145 ceph-mon[35096]: from='osd.7 [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:15:31.149 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:30 smithi145 ceph-mon[35096]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:15:31.150 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:30 smithi145 ceph-mon[35096]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:15:31.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:30 smithi107 ceph-mon[30659]: from='osd.7 [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:15:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:30 smithi107 ceph-mon[30659]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:15:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:30 smithi107 ceph-mon[30659]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:15:31.372 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:30 smithi115 ceph-mon[35477]: from='osd.7 [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:15:31.372 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:30 smithi115 ceph-mon[35477]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:15:31.373 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:30 smithi115 ceph-mon[35477]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:15:31.413 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:15:31.439 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_2 2024-09-19T04:15:31.686 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:15:32.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.166+0000 7fda25c4e640 1 -- 172.21.15.145:0/4281828753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda20102690 msgr2=0x7fda20108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.166+0000 7fda25c4e640 1 --2- 172.21.15.145:0/4281828753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda20102690 0x7fda20108f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda18009ca0 tx=0x7fda1802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.166+0000 7fda25c4e640 1 -- 172.21.15.145:0/4281828753 shutdown_connections 2024-09-19T04:15:32.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.166+0000 7fda25c4e640 1 --2- 172.21.15.145:0/4281828753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda20102690 0x7fda20108f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.166+0000 7fda25c4e640 1 --2- 172.21.15.145:0/4281828753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda20101cd0 0x7fda20102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.166+0000 7fda25c4e640 1 --2- 172.21.15.145:0/4281828753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda20100ad0 0x7fda20100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.166+0000 7fda25c4e640 1 -- 172.21.15.145:0/4281828753 >> 172.21.15.145:0/4281828753 conn(0x7fda200fc280 msgr2=0x7fda200fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.166+0000 7fda25c4e640 1 -- 172.21.15.145:0/4281828753 shutdown_connections 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.166+0000 7fda25c4e640 1 -- 172.21.15.145:0/4281828753 wait complete. 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.166+0000 7fda25c4e640 1 Processor -- start 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda25c4e640 1 -- start start 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda25c4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda20100ad0 0x7fda20197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda25c4e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda20101cd0 0x7fda20198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda25c4e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda20102690 0x7fda2019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda25c4e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fda2010c750 con 0x7fda20102690 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda25c4e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fda2010c5d0 con 0x7fda20100ad0 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda25c4e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fda2010c8d0 con 0x7fda20101cd0 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1effd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda20101cd0 0x7fda20198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1effd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda20101cd0 0x7fda20198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:39542/0 (socket says 172.21.15.145:39542) 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1effd640 1 -- 172.21.15.145:0/392697034 learned_addr learned my addr 172.21.15.145:0/392697034 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1f7fe640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda20100ad0 0x7fda20197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1ffff640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda20102690 0x7fda2019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1effd640 1 -- 172.21.15.145:0/392697034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda20100ad0 msgr2=0x7fda20197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1effd640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda20100ad0 0x7fda20197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1effd640 1 -- 172.21.15.145:0/392697034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda20102690 msgr2=0x7fda2019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1effd640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda20102690 0x7fda2019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1effd640 1 -- 172.21.15.145:0/392697034 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2019fa20 con 0x7fda20101cd0 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.167+0000 7fda1f7fe640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda20100ad0 0x7fda20197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.168+0000 7fda1effd640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda20101cd0 0x7fda20198330 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fda1000c920 tx=0x7fda1000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.168+0000 7fda1cff9640 1 -- 172.21.15.145:0/392697034 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10007c70 con 0x7fda20101cd0 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.168+0000 7fda25c4e640 1 -- 172.21.15.145:0/392697034 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2019fd10 con 0x7fda20101cd0 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.168+0000 7fda1cff9640 1 -- 172.21.15.145:0/392697034 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda10007e10 con 0x7fda20101cd0 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.168+0000 7fda1cff9640 1 -- 172.21.15.145:0/392697034 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10018890 con 0x7fda20101cd0 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.168+0000 7fda25c4e640 1 -- 172.21.15.145:0/392697034 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda201a0240 con 0x7fda20101cd0 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.169+0000 7fda25c4e640 1 -- 172.21.15.145:0/392697034 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda201a04d0 con 0x7fda20101cd0 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.170+0000 7fda1cff9640 1 -- 172.21.15.145:0/392697034 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fda10018af0 con 0x7fda20101cd0 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.171+0000 7fda1cff9640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda080a3270 0x7fda080a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.171+0000 7fda1cff9640 1 -- 172.21.15.145:0/392697034 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5117+0+0 (secure 0 0 0) 0x7fda100cede0 con 0x7fda20101cd0 2024-09-19T04:15:32.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.171+0000 7fda1f7fe640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda080a3270 0x7fda080a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.172+0000 7fda1f7fe640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda080a3270 0x7fda080a5730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fda14005fd0 tx=0x7fda1400b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:32.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.179+0000 7fda1cff9640 1 -- 172.21.15.145:0/392697034 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fda10014030 con 0x7fda20101cd0 2024-09-19T04:15:32.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T04:15:32.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:32.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='osd.7 [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:15:32.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:15:32.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi115 2024-09-19T04:15:32.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:32.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:32.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:32.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi115 to 5696M 2024-09-19T04:15:32.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:32.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:32.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:31 smithi145 ceph-mon[35096]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:32.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:32.360+0000 7fda25c4e640 1 -- 172.21.15.145:0/392697034 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fda201a06a0 con 0x7fda080a3270 2024-09-19T04:15:32.446 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:15:31.982+0000 7f8273692640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T04:15:32.446 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:15:31.994+0000 7f826f4ba640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:15:32.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T04:15:32.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:32.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='osd.7 [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:15:32.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:15:32.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi115 2024-09-19T04:15:32.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:32.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:32.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:32.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi115 to 5696M 2024-09-19T04:15:32.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:32.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:32.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:31 smithi115 ceph-mon[35477]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:32.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='osd.7 [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi115 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi115 to 5696M 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:32.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:32.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:32.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:31 smithi107 ceph-mon[30659]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:33.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:32 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:15:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:32 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:15:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:32 smithi115 ceph-mon[35477]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:15:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:32 smithi115 ceph-mon[35477]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:15:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:32 smithi115 ceph-mon[35477]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:15:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:15:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:33.471 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:32 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:15:33.471 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:32 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:15:33.471 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:32 smithi145 ceph-mon[35096]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:15:33.471 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:32 smithi145 ceph-mon[35096]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:15:33.471 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:33.471 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:33.471 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:32 smithi145 ceph-mon[35096]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:33.471 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:15:33.471 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:15:33.472 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:33.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:32 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:15:33.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:32 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:15:33.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:32 smithi107 ceph-mon[30659]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:15:33.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:32 smithi107 ceph-mon[30659]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:15:33.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:33.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:33.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:32 smithi107 ceph-mon[30659]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:33.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:15:33.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:15:33.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:33 smithi145 ceph-mon[35096]: osd.7 [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] boot 2024-09-19T04:15:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:33 smithi145 ceph-mon[35096]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T04:15:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:33 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:33 smithi145 ceph-mon[35096]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:34.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:33 smithi115 ceph-mon[35477]: osd.7 [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] boot 2024-09-19T04:15:34.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:33 smithi115 ceph-mon[35477]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T04:15:34.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:33 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:34.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:33 smithi115 ceph-mon[35477]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:34.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:33 smithi107 ceph-mon[30659]: osd.7 [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] boot 2024-09-19T04:15:34.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:33 smithi107 ceph-mon[30659]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T04:15:34.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:33 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:15:34.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:33 smithi107 ceph-mon[30659]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:15:35.329 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:35 smithi145 ceph-mon[35096]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T04:15:35.329 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:35 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/1073207248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8262ec4a-c805-449a-9e9c-b55b5b36ffed"}]: dispatch 2024-09-19T04:15:35.329 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:35 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8262ec4a-c805-449a-9e9c-b55b5b36ffed"}]: dispatch 2024-09-19T04:15:35.329 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:35 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8262ec4a-c805-449a-9e9c-b55b5b36ffed"}]': finished 2024-09-19T04:15:35.329 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:35 smithi145 ceph-mon[35096]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T04:15:35.329 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:35 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:35 smithi115 ceph-mon[35477]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T04:15:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:35 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/1073207248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8262ec4a-c805-449a-9e9c-b55b5b36ffed"}]: dispatch 2024-09-19T04:15:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:35 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8262ec4a-c805-449a-9e9c-b55b5b36ffed"}]: dispatch 2024-09-19T04:15:35.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:35 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8262ec4a-c805-449a-9e9c-b55b5b36ffed"}]': finished 2024-09-19T04:15:35.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:35 smithi115 ceph-mon[35477]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T04:15:35.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:35 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:35.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:34 smithi107 ceph-mon[30659]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T04:15:35.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:34 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/1073207248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8262ec4a-c805-449a-9e9c-b55b5b36ffed"}]: dispatch 2024-09-19T04:15:35.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:34 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8262ec4a-c805-449a-9e9c-b55b5b36ffed"}]: dispatch 2024-09-19T04:15:35.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:34 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8262ec4a-c805-449a-9e9c-b55b5b36ffed"}]': finished 2024-09-19T04:15:35.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:34 smithi107 ceph-mon[30659]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T04:15:35.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:35 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:36.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:36 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/1685267550' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:15:36.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:36 smithi115 ceph-mon[35477]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:36.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:36 smithi115 ceph-mon[35477]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T04:15:36.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:36 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:36.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:36 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/1685267550' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:15:36.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:36 smithi145 ceph-mon[35096]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:36.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:36 smithi145 ceph-mon[35096]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T04:15:36.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:36 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:36.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:36 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/1685267550' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:15:36.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:36 smithi107 ceph-mon[30659]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:36.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:36 smithi107 ceph-mon[30659]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T04:15:36.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:36 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:37.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:37 smithi107 ceph-mon[30659]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T04:15:37.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:37 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:37 smithi115 ceph-mon[35477]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T04:15:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:37 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:37 smithi145 ceph-mon[35096]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T04:15:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:37 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:38 smithi115 ceph-mon[35477]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:38 smithi115 ceph-mon[35477]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-19T04:15:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:38 smithi145 ceph-mon[35096]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:38 smithi145 ceph-mon[35096]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-19T04:15:39.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:38 smithi107 ceph-mon[30659]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:39.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:38 smithi107 ceph-mon[30659]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-19T04:15:40.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:39 smithi115 ceph-mon[35477]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-19T04:15:40.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:39 smithi115 ceph-mon[35477]: Cluster is now healthy 2024-09-19T04:15:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:39 smithi145 ceph-mon[35096]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-19T04:15:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:39 smithi145 ceph-mon[35096]: Cluster is now healthy 2024-09-19T04:15:40.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:39 smithi107 ceph-mon[30659]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-19T04:15:40.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:39 smithi107 ceph-mon[30659]: Cluster is now healthy 2024-09-19T04:15:41.113 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:40 smithi145 ceph-mon[35096]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:40 smithi115 ceph-mon[35477]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:41.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:40 smithi107 ceph-mon[30659]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:42 smithi115 ceph-mon[35477]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:43.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:42 smithi107 ceph-mon[30659]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:43.252 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:42 smithi145 ceph-mon[35096]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:43.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:43 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:15:43.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:43 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:44.114 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:15:44.114 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:44.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:43 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:15:44.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:43 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:44.927 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:44 smithi145 ceph-mon[35096]: Deploying daemon osd.8 on smithi145 2024-09-19T04:15:44.927 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:44 smithi145 ceph-mon[35096]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:44 smithi115 ceph-mon[35477]: Deploying daemon osd.8 on smithi145 2024-09-19T04:15:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:44 smithi115 ceph-mon[35477]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:45.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:44 smithi107 ceph-mon[30659]: Deploying daemon osd.8 on smithi145 2024-09-19T04:15:45.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:44 smithi107 ceph-mon[30659]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:47.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:46 smithi145 ceph-mon[35096]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:46 smithi115 ceph-mon[35477]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:46 smithi107 ceph-mon[30659]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:49.086 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:48 smithi145 ceph-mon[35096]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:48 smithi115 ceph-mon[35477]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:48 smithi107 ceph-mon[30659]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:50.167 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:49 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:50.167 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:49 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:50.168 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:49 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:49 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:49 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:49 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:50.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:49 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:50.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:49 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:50.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:49 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:50 smithi115 ceph-mon[35477]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:50 smithi145 ceph-mon[35096]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:50 smithi107 ceph-mon[30659]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:52 smithi145 ceph-mon[35096]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:52 smithi115 ceph-mon[35477]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:52 smithi107 ceph-mon[30659]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:53.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.398+0000 7fda1cff9640 1 -- 172.21.15.145:0/392697034 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fda201a06a0 con 0x7fda080a3270 2024-09-19T04:15:53.400 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 8 on host 'smithi145' 2024-09-19T04:15:53.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.400+0000 7fda25c4e640 1 -- 172.21.15.145:0/392697034 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda080a3270 msgr2=0x7fda080a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.400+0000 7fda25c4e640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda080a3270 0x7fda080a5730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fda14005fd0 tx=0x7fda1400b390 comp rx=0 tx=0).stop 2024-09-19T04:15:53.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.400+0000 7fda25c4e640 1 -- 172.21.15.145:0/392697034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda20101cd0 msgr2=0x7fda20198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:53.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.400+0000 7fda25c4e640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda20101cd0 0x7fda20198330 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fda1000c920 tx=0x7fda1000cde0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.400+0000 7fda25c4e640 1 -- 172.21.15.145:0/392697034 shutdown_connections 2024-09-19T04:15:53.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.400+0000 7fda25c4e640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda080a3270 0x7fda080a5730 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.400+0000 7fda25c4e640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda20102690 0x7fda2019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.400+0000 7fda25c4e640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda20101cd0 0x7fda20198330 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.401+0000 7fda25c4e640 1 --2- 172.21.15.145:0/392697034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda20100ad0 0x7fda20197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:53.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.401+0000 7fda25c4e640 1 -- 172.21.15.145:0/392697034 >> 172.21.15.145:0/392697034 conn(0x7fda200fc280 msgr2=0x7fda200fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:53.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.401+0000 7fda25c4e640 1 -- 172.21.15.145:0/392697034 shutdown_connections 2024-09-19T04:15:53.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:15:53.401+0000 7fda25c4e640 1 -- 172.21.15.145:0/392697034 wait complete. 2024-09-19T04:15:54.081 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:54.081 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:54.081 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:54.109 DEBUG:teuthology.orchestra.run.smithi145:osd.8> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T04:15:54.111 INFO:tasks.cephadm:Deploying osd.9 on smithi107 with /dev/vg_nvme/lv_1... 2024-09-19T04:15:54.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:15:54.166 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:54.166 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:54.166 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:15:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:54.482 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:54 smithi115 ceph-mon[35477]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:54 smithi145 ceph-mon[35096]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:55.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:54 smithi107 ceph-mon[30659]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:55.505 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:15:55 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:15:55.228+0000 7f9c9bd17640 -1 osd.8 0 log_to_monitors true 2024-09-19T04:15:56.159 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:55 smithi107 ceph-mon[30659]: from='osd.8 [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:15:56.159 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:55 smithi107 ceph-mon[30659]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:15:56.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:55 smithi115 ceph-mon[35477]: from='osd.8 [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:15:56.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:55 smithi115 ceph-mon[35477]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:15:56.221 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:55 smithi145 ceph-mon[35096]: from='osd.8 [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:15:56.221 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:55 smithi145 ceph-mon[35096]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:15:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='osd.8 [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:57.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:57.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:15:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T04:15:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='osd.8 [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:15:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:15:57.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:57.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:57.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:57.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:57.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:57.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:57.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:57.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:57.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:57.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:15:57.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='osd.8 [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:15:57.703 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:15:57.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:15:57 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:15:57.304+0000 7f9c97c9a640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T04:15:57.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:15:57 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:15:57.314+0000 7f9c8faba640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:15:57.734 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-09-19T04:15:57.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:57 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi145 2024-09-19T04:15:57.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:57 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-19T04:15:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:57 smithi107 ceph-mon[30659]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:15:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:57 smithi107 ceph-mon[30659]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T04:15:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:57.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:57 smithi107 ceph-mon[30659]: from='osd.8 ' entity='osd.8' 2024-09-19T04:15:58.104 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:15:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:57 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi145 2024-09-19T04:15:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:57 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-19T04:15:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:57 smithi115 ceph-mon[35477]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:15:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:57 smithi115 ceph-mon[35477]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T04:15:58.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:58.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:58.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:57 smithi115 ceph-mon[35477]: from='osd.8 ' entity='osd.8' 2024-09-19T04:15:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:57 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi145 2024-09-19T04:15:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:57 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-19T04:15:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:15:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:57 smithi145 ceph-mon[35096]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:15:58.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:57 smithi145 ceph-mon[35096]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T04:15:58.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:58.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:58.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:57 smithi145 ceph-mon[35096]: from='osd.8 ' entity='osd.8' 2024-09-19T04:15:58.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.616+0000 7f66bc000640 1 -- 172.21.15.107:0/3787528665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66b4104f60 msgr2=0x7f66b41053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.616+0000 7f66bc000640 1 --2- 172.21.15.107:0/3787528665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66b4104f60 0x7f66b41053e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009900 tx=0x7f66a002a410 comp rx=0 tx=0).stop 2024-09-19T04:15:58.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.616+0000 7f66bc000640 1 -- 172.21.15.107:0/3787528665 shutdown_connections 2024-09-19T04:15:58.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.616+0000 7f66bc000640 1 --2- 172.21.15.107:0/3787528665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b4105920 0x7f66b410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.616+0000 7f66bc000640 1 --2- 172.21.15.107:0/3787528665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66b4104f60 0x7f66b41053e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.616+0000 7f66bc000640 1 --2- 172.21.15.107:0/3787528665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b4103e00 0x7f66b4104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.616+0000 7f66bc000640 1 -- 172.21.15.107:0/3787528665 >> 172.21.15.107:0/3787528665 conn(0x7f66b40ff5b0 msgr2=0x7f66b41019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:58.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.616+0000 7f66bc000640 1 -- 172.21.15.107:0/3787528665 shutdown_connections 2024-09-19T04:15:58.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.616+0000 7f66bc000640 1 -- 172.21.15.107:0/3787528665 wait complete. 2024-09-19T04:15:58.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.617+0000 7f66bc000640 1 Processor -- start 2024-09-19T04:15:58.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.617+0000 7f66bc000640 1 -- start start 2024-09-19T04:15:58.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.617+0000 7f66bc000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b4103e00 0x7f66b419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.617+0000 7f66bc000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b4104f60 0x7f66b419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.617+0000 7f66bc000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66b4105920 0x7f66b41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.617+0000 7f66bc000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66b410fa90 con 0x7f66b4105920 2024-09-19T04:15:58.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.617+0000 7f66bc000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f66b410f910 con 0x7f66b4103e00 2024-09-19T04:15:58.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.617+0000 7f66bc000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f66b410fc10 con 0x7f66b4104f60 2024-09-19T04:15:58.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.617+0000 7f66ba000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66b4105920 0x7f66b41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b4104f60 0x7f66b419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b4104f60 0x7f66b419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:36570/0 (socket says 172.21.15.107:36570) 2024-09-19T04:15:58.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b8c00640 1 -- 172.21.15.107:0/4189765975 learned_addr learned my addr 172.21.15.107:0/4189765975 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b9600640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b4103e00 0x7f66b419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b8c00640 1 -- 172.21.15.107:0/4189765975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b4103e00 msgr2=0x7f66b419a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b8c00640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b4103e00 0x7f66b419a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b8c00640 1 -- 172.21.15.107:0/4189765975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66b4105920 msgr2=0x7f66b41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b8c00640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66b4105920 0x7f66b41a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b8c00640 1 -- 172.21.15.107:0/4189765975 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b41a1d00 con 0x7f66b4104f60 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b9600640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b4103e00 0x7f66b419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66ba000640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66b4105920 0x7f66b41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.618+0000 7f66b8c00640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b4104f60 0x7f66b419a610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66a0031610 tx=0x7f66a0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.619+0000 7f66b2000640 1 -- 172.21.15.107:0/4189765975 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a003f030 con 0x7f66b4104f60 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.619+0000 7f66b2000640 1 -- 172.21.15.107:0/4189765975 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a0003ad0 con 0x7f66b4104f60 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.619+0000 7f66bc000640 1 -- 172.21.15.107:0/4189765975 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b41a1f90 con 0x7f66b4104f60 2024-09-19T04:15:58.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.619+0000 7f66b2000640 1 -- 172.21.15.107:0/4189765975 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0035070 con 0x7f66b4104f60 2024-09-19T04:15:58.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.619+0000 7f66bc000640 1 -- 172.21.15.107:0/4189765975 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b41a23e0 con 0x7f66b4104f60 2024-09-19T04:15:58.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.620+0000 7f66bc000640 1 -- 172.21.15.107:0/4189765975 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b410b290 con 0x7f66b4104f60 2024-09-19T04:15:58.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.622+0000 7f66b2000640 1 -- 172.21.15.107:0/4189765975 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f66a0042030 con 0x7f66b4104f60 2024-09-19T04:15:58.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.623+0000 7f66b2000640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f669c0a32f0 0x7f669c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.623+0000 7f66b2000640 1 -- 172.21.15.107:0/4189765975 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5862+0+0 (secure 0 0 0) 0x7f66a00ef7e0 con 0x7f66b4104f60 2024-09-19T04:15:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.623+0000 7f66b9600640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f669c0a32f0 0x7f669c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.624+0000 7f66b9600640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f669c0a32f0 0x7f669c0a57b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009fc0 tx=0x7f66a8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.625+0000 7f66b2000640 1 -- 172.21.15.107:0/4189765975 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f66a00bc7a0 con 0x7f66b4104f60 2024-09-19T04:15:58.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:15:58.811+0000 7f66bc000640 1 -- 172.21.15.107:0/4189765975 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f66b4062630 con 0x7f669c0a32f0 2024-09-19T04:15:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:58 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:15:59.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:58 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:15:59.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:58 smithi115 ceph-mon[35477]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:59.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:58 smithi115 ceph-mon[35477]: osd.8 [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] boot 2024-09-19T04:15:59.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:58 smithi115 ceph-mon[35477]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T04:15:59.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:58 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:59.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:58 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:15:59.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:58 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:15:59.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:58 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:58 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:15:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:58 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:15:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:58 smithi145 ceph-mon[35096]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:58 smithi145 ceph-mon[35096]: osd.8 [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] boot 2024-09-19T04:15:59.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:58 smithi145 ceph-mon[35096]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T04:15:59.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:58 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:59.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:58 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:15:59.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:58 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:15:59.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:58 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:58 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:15:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:58 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:15:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:58 smithi107 ceph-mon[30659]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:15:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:58 smithi107 ceph-mon[30659]: osd.8 [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] boot 2024-09-19T04:15:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:58 smithi107 ceph-mon[30659]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T04:15:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:58 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:15:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:58 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:15:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:58 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:15:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:58 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:15:59.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:59 smithi107 ceph-mon[30659]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:15:59.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:15:59 smithi107 ceph-mon[30659]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T04:16:00.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:59 smithi115 ceph-mon[35477]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:16:00.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:15:59 smithi115 ceph-mon[35477]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T04:16:00.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:59 smithi145 ceph-mon[35096]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:16:00.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:15:59 smithi145 ceph-mon[35096]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T04:16:01.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:00 smithi107 ceph-mon[30659]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:01.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:00 smithi107 ceph-mon[30659]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T04:16:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:00 smithi115 ceph-mon[35477]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:00 smithi115 ceph-mon[35477]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T04:16:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:00 smithi145 ceph-mon[35096]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:00 smithi145 ceph-mon[35096]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T04:16:02.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:01 smithi115 ceph-mon[35477]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T04:16:02.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:01 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/957046373' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8277cde7-50c0-4a5b-9283-24f3813de2c5"}]: dispatch 2024-09-19T04:16:02.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:01 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/957046373' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8277cde7-50c0-4a5b-9283-24f3813de2c5"}]': finished 2024-09-19T04:16:02.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:01 smithi115 ceph-mon[35477]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T04:16:02.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:01 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:02.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:01 smithi145 ceph-mon[35096]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T04:16:02.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:01 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/957046373' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8277cde7-50c0-4a5b-9283-24f3813de2c5"}]: dispatch 2024-09-19T04:16:02.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:01 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/957046373' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8277cde7-50c0-4a5b-9283-24f3813de2c5"}]': finished 2024-09-19T04:16:02.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:01 smithi145 ceph-mon[35096]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T04:16:02.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:01 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:02.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:01 smithi107 ceph-mon[30659]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T04:16:02.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:01 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/957046373' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8277cde7-50c0-4a5b-9283-24f3813de2c5"}]: dispatch 2024-09-19T04:16:02.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:01 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/957046373' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8277cde7-50c0-4a5b-9283-24f3813de2c5"}]': finished 2024-09-19T04:16:02.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:01 smithi107 ceph-mon[30659]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T04:16:02.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:01 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:02 smithi115 ceph-mon[35477]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:02 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/10533307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:16:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:02 smithi145 ceph-mon[35096]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:03.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:02 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/10533307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:16:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:02 smithi107 ceph-mon[30659]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:02 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/10533307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:16:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:04 smithi115 ceph-mon[35477]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:04 smithi145 ceph-mon[35096]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:05.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:04 smithi107 ceph-mon[30659]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:06 smithi115 ceph-mon[35477]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:16:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:06 smithi145 ceph-mon[35096]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:16:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:06 smithi107 ceph-mon[30659]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:16:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:08 smithi115 ceph-mon[35477]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-19T04:16:09.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:08 smithi107 ceph-mon[30659]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-19T04:16:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:08 smithi145 ceph-mon[35096]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-19T04:16:10.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:09 smithi115 ceph-mon[35477]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T04:16:10.461 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:09 smithi107 ceph-mon[30659]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T04:16:10.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:09 smithi145 ceph-mon[35096]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T04:16:11.347 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:16:11.348 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:11.348 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:10 smithi107 ceph-mon[30659]: Deploying daemon osd.9 on smithi107 2024-09-19T04:16:11.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:16:11.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:11.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:10 smithi115 ceph-mon[35477]: Deploying daemon osd.9 on smithi107 2024-09-19T04:16:11.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:16:11.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:11.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:10 smithi145 ceph-mon[35096]: Deploying daemon osd.9 on smithi107 2024-09-19T04:16:12.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:12 smithi107 ceph-mon[30659]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:16:12.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:12 smithi115 ceph-mon[35477]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:16:12.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:12 smithi145 ceph-mon[35096]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:16:15.030 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:14 smithi107 ceph-mon[30659]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:16:15.149 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:14 smithi115 ceph-mon[35477]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:16:15.177 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:14 smithi145 ceph-mon[35096]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:16:17.106 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:16 smithi107 ceph-mon[30659]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:16:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:16 smithi115 ceph-mon[35477]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:16:17.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:16 smithi145 ceph-mon[35096]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:16:18.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:17 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:18.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:17 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:17 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:17 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:17 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:17 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:18.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:17 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:18.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:17 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:18.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:17 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:18 smithi115 ceph-mon[35477]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:18 smithi145 ceph-mon[35096]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:18 smithi107 ceph-mon[30659]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:20 smithi115 ceph-mon[35477]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:20 smithi145 ceph-mon[35096]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:21.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:20 smithi107 ceph-mon[30659]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:21.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.684+0000 7f66b2000640 1 -- 172.21.15.107:0/4189765975 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f66b4062630 con 0x7f669c0a32f0 2024-09-19T04:16:21.686 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 9 on host 'smithi107' 2024-09-19T04:16:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.686+0000 7f66bc000640 1 -- 172.21.15.107:0/4189765975 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f669c0a32f0 msgr2=0x7f669c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.686+0000 7f66bc000640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f669c0a32f0 0x7f669c0a57b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009fc0 tx=0x7f66a8005f90 comp rx=0 tx=0).stop 2024-09-19T04:16:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.686+0000 7f66bc000640 1 -- 172.21.15.107:0/4189765975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b4104f60 msgr2=0x7f66b419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.686+0000 7f66bc000640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b4104f60 0x7f66b419a610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66a0031610 tx=0x7f66a0004270 comp rx=0 tx=0).stop 2024-09-19T04:16:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.687+0000 7f66bc000640 1 -- 172.21.15.107:0/4189765975 shutdown_connections 2024-09-19T04:16:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.687+0000 7f66bc000640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f669c0a32f0 0x7f669c0a57b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.687+0000 7f66bc000640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66b4105920 0x7f66b41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.687+0000 7f66bc000640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b4104f60 0x7f66b419a610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.687+0000 7f66bc000640 1 --2- 172.21.15.107:0/4189765975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b4103e00 0x7f66b419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.687+0000 7f66bc000640 1 -- 172.21.15.107:0/4189765975 >> 172.21.15.107:0/4189765975 conn(0x7f66b40ff5b0 msgr2=0x7f66b4108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:21.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.687+0000 7f66bc000640 1 -- 172.21.15.107:0/4189765975 shutdown_connections 2024-09-19T04:16:21.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:16:21.687+0000 7f66bc000640 1 -- 172.21.15.107:0/4189765975 wait complete. 2024-09-19T04:16:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.371 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.371 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:22 smithi107 ceph-mon[30659]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:22.408 DEBUG:teuthology.orchestra.run.smithi107:osd.9> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T04:16:22.413 INFO:tasks.cephadm:Deploying osd.10 on smithi115 with /dev/vg_nvme/lv_1... 2024-09-19T04:16:22.413 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:16:22.442 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.442 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.442 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:22.442 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:22.442 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.442 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:22.442 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.442 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.442 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:22 smithi115 ceph-mon[35477]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:22.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:22.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:22.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:22.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:22.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:22 smithi145 ceph-mon[35096]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:22.669 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:16:23.728 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:23 smithi107 ceph-mon[30659]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:16:23.728 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:16:23 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:16:23.588+0000 7f8118841640 -1 osd.9 0 log_to_monitors true 2024-09-19T04:16:23.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:23 smithi115 ceph-mon[35477]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:16:23.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:23 smithi145 ceph-mon[35096]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:16:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:24 smithi145 ceph-mon[35096]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:16:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:24 smithi145 ceph-mon[35096]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T04:16:24.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:24 smithi145 ceph-mon[35096]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:16:24.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:24 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:24.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:24 smithi145 ceph-mon[35096]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:24.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:24 smithi107 ceph-mon[30659]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:16:24.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:24 smithi107 ceph-mon[30659]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T04:16:24.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:24 smithi107 ceph-mon[30659]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:16:24.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:24 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:24.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:24 smithi107 ceph-mon[30659]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:25.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:24 smithi115 ceph-mon[35477]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:16:25.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:24 smithi115 ceph-mon[35477]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T04:16:25.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:24 smithi115 ceph-mon[35477]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-19T04:16:25.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:24 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:25.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:24 smithi115 ceph-mon[35477]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:25.595 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-19T04:16:25.624 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_1 2024-09-19T04:16:25.700 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:16:25 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:16:25.252+0000 7f8113400640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T04:16:25.700 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:16:25 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:16:25.277+0000 7f8109400640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:16:25.902 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:25 smithi115 ceph-mon[35477]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:16:25.902 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:25 smithi115 ceph-mon[35477]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T04:16:25.903 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:25 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:25.903 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:25 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:25.903 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:25 smithi115 ceph-mon[35477]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' 2024-09-19T04:16:25.923 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:16:25.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:25 smithi145 ceph-mon[35096]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:16:25.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:25 smithi145 ceph-mon[35096]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T04:16:25.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:25 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:25.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:25 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:25.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:25 smithi145 ceph-mon[35096]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' 2024-09-19T04:16:25.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:25 smithi107 ceph-mon[30659]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-19T04:16:25.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:25 smithi107 ceph-mon[30659]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T04:16:25.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:25 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:25.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:25 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:25.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:25 smithi107 ceph-mon[30659]: from='osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099]' entity='osd.9' 2024-09-19T04:16:26.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.430+0000 7f1615e97640 1 -- 172.21.15.115:0/2231306113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1610101cd0 msgr2=0x7f1610102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:26.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.430+0000 7f1615e97640 1 --2- 172.21.15.115:0/2231306113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1610101cd0 0x7f1610102150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1604009900 tx=0x7f160402a410 comp rx=0 tx=0).stop 2024-09-19T04:16:26.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.430+0000 7f1615e97640 1 -- 172.21.15.115:0/2231306113 shutdown_connections 2024-09-19T04:16:26.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.430+0000 7f1615e97640 1 --2- 172.21.15.115:0/2231306113 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1610102690 0x7f1610108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.430+0000 7f1615e97640 1 --2- 172.21.15.115:0/2231306113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1610101cd0 0x7f1610102150 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.430+0000 7f1615e97640 1 --2- 172.21.15.115:0/2231306113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1610100ad0 0x7f1610100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.430+0000 7f1615e97640 1 -- 172.21.15.115:0/2231306113 >> 172.21.15.115:0/2231306113 conn(0x7f16100fc280 msgr2=0x7f16100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.430+0000 7f1615e97640 1 -- 172.21.15.115:0/2231306113 shutdown_connections 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.430+0000 7f1615e97640 1 -- 172.21.15.115:0/2231306113 wait complete. 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f1615e97640 1 Processor -- start 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f1615e97640 1 -- start start 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f1615e97640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1610100ad0 0x7f1610196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f1615e97640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1610101cd0 0x7f1610197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f1615e97640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1610102690 0x7f161019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f1615e97640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f161010c750 con 0x7f1610100ad0 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f1615e97640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f161010c5d0 con 0x7f1610101cd0 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f1615e97640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f161010c8d0 con 0x7f1610102690 2024-09-19T04:16:26.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f160effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1610101cd0 0x7f1610197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f160effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1610101cd0 0x7f1610197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47218/0 (socket says 172.21.15.115:47218) 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f160effd640 1 -- 172.21.15.115:0/4022363575 learned_addr learned my addr 172.21.15.115:0/4022363575 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f160f7fe640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1610100ad0 0x7f1610196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.431+0000 7f160ffff640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1610102690 0x7f161019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f160effd640 1 -- 172.21.15.115:0/4022363575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1610102690 msgr2=0x7f161019e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f160effd640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1610102690 0x7f161019e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f160effd640 1 -- 172.21.15.115:0/4022363575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1610100ad0 msgr2=0x7f1610196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f160effd640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1610100ad0 0x7f1610196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f160effd640 1 -- 172.21.15.115:0/4022363575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161019ea10 con 0x7f1610101cd0 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f160effd640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1610101cd0 0x7f1610197320 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1604009230 tx=0x7f1604031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f160cff9640 1 -- 172.21.15.115:0/4022363575 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160403f030 con 0x7f1610101cd0 2024-09-19T04:16:26.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f160cff9640 1 -- 172.21.15.115:0/4022363575 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1604003ad0 con 0x7f1610101cd0 2024-09-19T04:16:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f1615e97640 1 -- 172.21.15.115:0/4022363575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161019eca0 con 0x7f1610101cd0 2024-09-19T04:16:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f160cff9640 1 -- 172.21.15.115:0/4022363575 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604035070 con 0x7f1610101cd0 2024-09-19T04:16:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.432+0000 7f1615e97640 1 -- 172.21.15.115:0/4022363575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f161019f180 con 0x7f1610101cd0 2024-09-19T04:16:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.433+0000 7f1615e97640 1 -- 172.21.15.115:0/4022363575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1610107f20 con 0x7f1610101cd0 2024-09-19T04:16:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.435+0000 7f160cff9640 1 -- 172.21.15.115:0/4022363575 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1604042030 con 0x7f1610101cd0 2024-09-19T04:16:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.435+0000 7f160cff9640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f15f80a3270 0x7f15f80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.435+0000 7f160cff9640 1 -- 172.21.15.115:0/4022363575 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f16040ef990 con 0x7f1610101cd0 2024-09-19T04:16:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.436+0000 7f160f7fe640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f15f80a3270 0x7f15f80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.436+0000 7f160f7fe640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f15f80a3270 0x7f15f80a5730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1610101b30 tx=0x7f160000b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.438+0000 7f160cff9640 1 -- 172.21.15.115:0/4022363575 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f16040bc860 con 0x7f1610101cd0 2024-09-19T04:16:26.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:26.617+0000 7f1615e97640 1 -- 172.21.15.115:0/4022363575 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1610062630 con 0x7f15f80a3270 2024-09-19T04:16:26.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:16:26.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:16:26.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:26.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:26.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:26.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:26.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:26.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] boot 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:16:26.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:26.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:26.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] boot 2024-09-19T04:16:26.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T04:16:26.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:26.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:16:26.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:16:26.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:27.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:16:27.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:16:27.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:27.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:16:27.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: osd.9 [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] boot 2024-09-19T04:16:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T04:16:27.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:16:27.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:16:27.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:16:27.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:27.895 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:27 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi107 2024-09-19T04:16:27.896 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:27 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi107 to 4272M 2024-09-19T04:16:27.896 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:27 smithi107 ceph-mon[30659]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:16:27.896 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:27 smithi107 ceph-mon[30659]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T04:16:27.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:27 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi107 2024-09-19T04:16:27.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:27 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi107 to 4272M 2024-09-19T04:16:27.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:27 smithi145 ceph-mon[35096]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:16:27.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:27 smithi145 ceph-mon[35096]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T04:16:28.128 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:27 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi107 2024-09-19T04:16:28.128 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:27 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi107 to 4272M 2024-09-19T04:16:28.129 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:27 smithi115 ceph-mon[35477]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:16:28.129 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:27 smithi115 ceph-mon[35477]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T04:16:28.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:28 smithi145 ceph-mon[35096]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:28.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:28 smithi107 ceph-mon[30659]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:28 smithi115 ceph-mon[35477]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:29.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:29 smithi115 ceph-mon[35477]: from='client.? 172.21.15.115:0/3058639074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df95b38-68a9-426e-83e4-7283c68a2ead"}]: dispatch 2024-09-19T04:16:29.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:29 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df95b38-68a9-426e-83e4-7283c68a2ead"}]: dispatch 2024-09-19T04:16:29.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:29 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3df95b38-68a9-426e-83e4-7283c68a2ead"}]': finished 2024-09-19T04:16:29.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:29 smithi115 ceph-mon[35477]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T04:16:29.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:29 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:29.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:29 smithi115 ceph-mon[35477]: from='client.? 172.21.15.115:0/2313158717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:16:29.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:29 smithi145 ceph-mon[35096]: from='client.? 172.21.15.115:0/3058639074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df95b38-68a9-426e-83e4-7283c68a2ead"}]: dispatch 2024-09-19T04:16:29.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:29 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df95b38-68a9-426e-83e4-7283c68a2ead"}]: dispatch 2024-09-19T04:16:29.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:29 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3df95b38-68a9-426e-83e4-7283c68a2ead"}]': finished 2024-09-19T04:16:29.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:29 smithi145 ceph-mon[35096]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T04:16:29.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:29 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:29.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:29 smithi145 ceph-mon[35096]: from='client.? 172.21.15.115:0/2313158717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:16:29.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:29 smithi107 ceph-mon[30659]: from='client.? 172.21.15.115:0/3058639074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df95b38-68a9-426e-83e4-7283c68a2ead"}]: dispatch 2024-09-19T04:16:29.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:29 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df95b38-68a9-426e-83e4-7283c68a2ead"}]: dispatch 2024-09-19T04:16:29.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:29 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3df95b38-68a9-426e-83e4-7283c68a2ead"}]': finished 2024-09-19T04:16:29.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:29 smithi107 ceph-mon[30659]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T04:16:29.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:29 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:29.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:29 smithi107 ceph-mon[30659]: from='client.? 172.21.15.115:0/2313158717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:16:30.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:30 smithi107 ceph-mon[30659]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:30 smithi115 ceph-mon[35477]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:30 smithi145 ceph-mon[35096]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:32 smithi115 ceph-mon[35477]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:32 smithi145 ceph-mon[35096]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:33.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:32 smithi107 ceph-mon[30659]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:35.173 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:34 smithi115 ceph-mon[35477]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:34 smithi145 ceph-mon[35096]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:35.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:34 smithi107 ceph-mon[30659]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:37.176 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:36 smithi115 ceph-mon[35477]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:36 smithi145 ceph-mon[35096]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:36 smithi107 ceph-mon[30659]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:37.925 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:37 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:16:37.925 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:37 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:38.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:37 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:16:38.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:37 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:37 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:16:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:37 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:39.082 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:38 smithi115 ceph-mon[35477]: Deploying daemon osd.10 on smithi115 2024-09-19T04:16:39.083 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:38 smithi115 ceph-mon[35477]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:38 smithi145 ceph-mon[35096]: Deploying daemon osd.10 on smithi115 2024-09-19T04:16:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:38 smithi145 ceph-mon[35096]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:39.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:38 smithi107 ceph-mon[30659]: Deploying daemon osd.10 on smithi115 2024-09-19T04:16:39.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:38 smithi107 ceph-mon[30659]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:40.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:40 smithi115 ceph-mon[35477]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:40 smithi145 ceph-mon[35096]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:41.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:40 smithi107 ceph-mon[30659]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:43.176 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:42 smithi115 ceph-mon[35477]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:42 smithi145 ceph-mon[35096]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:43.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:42 smithi107 ceph-mon[30659]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:43.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:43.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:43.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:43 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:44.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:43 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:44.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:43 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:44.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:43 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:44.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:43 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:44.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:43 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:44.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:43 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:45.063 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:44 smithi115 ceph-mon[35477]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:45.125 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:44 smithi145 ceph-mon[35096]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:45.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:44 smithi107 ceph-mon[30659]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:46.887 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 10 on host 'smithi115' 2024-09-19T04:16:46.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.885+0000 7f160cff9640 1 -- 172.21.15.115:0/4022363575 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1610062630 con 0x7f15f80a3270 2024-09-19T04:16:46.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 -- 172.21.15.115:0/4022363575 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f15f80a3270 msgr2=0x7f15f80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f15f80a3270 0x7f15f80a5730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1610101b30 tx=0x7f160000b390 comp rx=0 tx=0).stop 2024-09-19T04:16:46.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 -- 172.21.15.115:0/4022363575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1610101cd0 msgr2=0x7f1610197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1610101cd0 0x7f1610197320 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1604009230 tx=0x7f1604031b60 comp rx=0 tx=0).stop 2024-09-19T04:16:46.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 -- 172.21.15.115:0/4022363575 shutdown_connections 2024-09-19T04:16:46.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f15f80a3270 0x7f15f80a5730 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1610102690 0x7f161019e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1610101cd0 0x7f1610197320 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 --2- 172.21.15.115:0/4022363575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1610100ad0 0x7f1610196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 -- 172.21.15.115:0/4022363575 >> 172.21.15.115:0/4022363575 conn(0x7f16100fc280 msgr2=0x7f16101050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:46.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 -- 172.21.15.115:0/4022363575 shutdown_connections 2024-09-19T04:16:46.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-19T04:16:46.888+0000 7f1615e97640 1 -- 172.21.15.115:0/4022363575 wait complete. 2024-09-19T04:16:46.900 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:46 smithi115 ceph-mon[35477]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:46.900 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:46.900 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:46.900 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:46.900 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:46.901 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:46 smithi145 ceph-mon[35096]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:47.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:47.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:47.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:47.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:47.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:46 smithi107 ceph-mon[30659]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:47.579 DEBUG:teuthology.orchestra.run.smithi115:osd.10> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T04:16:47.581 INFO:tasks.cephadm:Deploying osd.11 on smithi145 with /dev/vg_nvme/lv_1... 2024-09-19T04:16:47.581 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:16:47.837 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:16:48.130 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:47 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:48.131 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:47 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:48.131 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:47 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:48.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:47 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:48.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:47 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:48.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:47 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:48.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:47 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:16:48.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:47 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:48.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:47 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:49.013 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:48 smithi115 ceph-mon[35477]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:48 smithi145 ceph-mon[35096]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:48 smithi107 ceph-mon[30659]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:49.263 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:16:49 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:16:49.259+0000 7f168e247640 -1 osd.10 0 log_to_monitors true 2024-09-19T04:16:50.157 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:49 smithi145 ceph-mon[35096]: from='osd.10 [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:16:50.157 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:49 smithi145 ceph-mon[35096]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:16:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:49 smithi115 ceph-mon[35477]: from='osd.10 [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:16:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:49 smithi115 ceph-mon[35477]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:16:50.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:49 smithi107 ceph-mon[30659]: from='osd.10 [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:16:50.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:49 smithi107 ceph-mon[30659]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:16:50.419 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-19T04:16:50.446 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_1 2024-09-19T04:16:50.701 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:16:50.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:50.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:16:50.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T04:16:50.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:50.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='osd.10 [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:50.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:51.021 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:51.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:16:51.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T04:16:51.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:51.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='osd.10 [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:16:51.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:16:51.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:51.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:51.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:51.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:51.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:51.023 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:51.023 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:51.023 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:51.023 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:51.023 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:51.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 -- 172.21.15.145:0/1431739680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda5c1078c0 msgr2=0x7fda5c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 --2- 172.21.15.145:0/1431739680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda5c1078c0 0x7fda5c109d10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009900 tx=0x7fda4c02a410 comp rx=0 tx=0).stop 2024-09-19T04:16:51.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 -- 172.21.15.145:0/1431739680 shutdown_connections 2024-09-19T04:16:51.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 --2- 172.21.15.145:0/1431739680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda5c1078c0 0x7fda5c109d10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 --2- 172.21.15.145:0/1431739680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda5c104f90 0x7fda5c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 --2- 172.21.15.145:0/1431739680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c102660 0x7fda5c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 -- 172.21.15.145:0/1431739680 >> 172.21.15.145:0/1431739680 conn(0x7fda5c0fc280 msgr2=0x7fda5c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:51.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 -- 172.21.15.145:0/1431739680 shutdown_connections 2024-09-19T04:16:51.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 -- 172.21.15.145:0/1431739680 wait complete. 2024-09-19T04:16:51.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 Processor -- start 2024-09-19T04:16:51.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.197+0000 7fda60d97640 1 -- start start 2024-09-19T04:16:51.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda60d97640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c102660 0x7fda5c101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda60d97640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda5c104f90 0x7fda5c100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda60d97640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda5c1078c0 0x7fda5c100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda60d97640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c10d8e0 con 0x7fda5c102660 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda60d97640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c10d760 con 0x7fda5c1078c0 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda60d97640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c10da60 con 0x7fda5c104f90 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda59d9b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda5c104f90 0x7fda5c100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda59d9b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda5c104f90 0x7fda5c100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40152/0 (socket says 172.21.15.145:40152) 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda59d9b640 1 -- 172.21.15.145:0/870516580 learned_addr learned my addr 172.21.15.145:0/870516580 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda5ad9d640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda5c1078c0 0x7fda5c100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda59d9b640 1 -- 172.21.15.145:0/870516580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda5c1078c0 msgr2=0x7fda5c100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda5a59c640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c102660 0x7fda5c101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda59d9b640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda5c1078c0 0x7fda5c100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda59d9b640 1 -- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c102660 msgr2=0x7fda5c101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda59d9b640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c102660 0x7fda5c101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda59d9b640 1 -- 172.21.15.145:0/870516580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c101410 con 0x7fda5c104f90 2024-09-19T04:16:51.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda5a59c640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c102660 0x7fda5c101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:51.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda59d9b640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda5c104f90 0x7fda5c100610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fda4400d770 tx=0x7fda4400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:51.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.198+0000 7fda537fe640 1 -- 172.21.15.145:0/870516580 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44017070 con 0x7fda5c104f90 2024-09-19T04:16:51.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.199+0000 7fda60d97640 1 -- 172.21.15.145:0/870516580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5c1a20d0 con 0x7fda5c104f90 2024-09-19T04:16:51.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.199+0000 7fda537fe640 1 -- 172.21.15.145:0/870516580 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44009c30 con 0x7fda5c104f90 2024-09-19T04:16:51.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.199+0000 7fda537fe640 1 -- 172.21.15.145:0/870516580 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4400f620 con 0x7fda5c104f90 2024-09-19T04:16:51.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.199+0000 7fda60d97640 1 -- 172.21.15.145:0/870516580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5c1a26b0 con 0x7fda5c104f90 2024-09-19T04:16:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.200+0000 7fda60d97640 1 -- 172.21.15.145:0/870516580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda5c0fdd70 con 0x7fda5c104f90 2024-09-19T04:16:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.202+0000 7fda537fe640 1 -- 172.21.15.145:0/870516580 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fda4401a070 con 0x7fda5c104f90 2024-09-19T04:16:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.202+0000 7fda537fe640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda480a3240 0x7fda480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.203+0000 7fda537fe640 1 -- 172.21.15.145:0/870516580 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6447+0+0 (secure 0 0 0) 0x7fda440ceff0 con 0x7fda5c104f90 2024-09-19T04:16:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.203+0000 7fda5a59c640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda480a3240 0x7fda480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.203+0000 7fda5a59c640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda480a3240 0x7fda480a5700 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda54007eb0 tx=0x7fda540073c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.205+0000 7fda537fe640 1 -- 172.21.15.145:0/870516580 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fda44009de0 con 0x7fda5c104f90 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='osd.10 [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:51.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:16:51.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:51.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:51.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:16:51.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:16:51.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:16:51.390+0000 7fda60d97640 1 -- 172.21.15.145:0/870516580 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fda5c062630 con 0x7fda480a3240 2024-09-19T04:16:51.696 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:16:51.266+0000 7f168a1ca640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T04:16:51.696 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:16:51.276+0000 7f1681fea640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:16:52.057 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi115 2024-09-19T04:16:52.057 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi115 to 4272M 2024-09-19T04:16:52.057 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:16:52.057 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T04:16:52.057 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:52.057 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:52.057 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T04:16:52.058 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:52.058 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: from='osd.10 ' entity='osd.10' 2024-09-19T04:16:52.058 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:16:52.058 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:16:52.058 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:51 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:52.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi115 2024-09-19T04:16:52.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi115 to 4272M 2024-09-19T04:16:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:16:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T04:16:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T04:16:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: from='osd.10 ' entity='osd.10' 2024-09-19T04:16:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:16:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:16:52.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:51 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:52.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi115 2024-09-19T04:16:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi115 to 4272M 2024-09-19T04:16:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-19T04:16:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T04:16:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T04:16:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: from='osd.10 ' entity='osd.10' 2024-09-19T04:16:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:16:52.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:16:52.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:51 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:16:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:52 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:16:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:52 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:16:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:52 smithi115 ceph-mon[35477]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:16:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:52 smithi115 ceph-mon[35477]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:52 smithi115 ceph-mon[35477]: osd.10 [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] boot 2024-09-19T04:16:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:52 smithi115 ceph-mon[35477]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T04:16:53.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:52 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:16:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:52 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:16:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:52 smithi145 ceph-mon[35096]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:16:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:52 smithi145 ceph-mon[35096]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:52 smithi145 ceph-mon[35096]: osd.10 [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] boot 2024-09-19T04:16:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:52 smithi145 ceph-mon[35096]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T04:16:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:52 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:16:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:52 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:16:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:52 smithi107 ceph-mon[30659]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:16:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:52 smithi107 ceph-mon[30659]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:16:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:52 smithi107 ceph-mon[30659]: osd.10 [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] boot 2024-09-19T04:16:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:52 smithi107 ceph-mon[30659]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T04:16:53.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:16:54.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:54 smithi115 ceph-mon[35477]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T04:16:54.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:54 smithi115 ceph-mon[35477]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:16:54.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:54 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/565771897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fa0931c-684b-448d-8d63-2bd10881d88f"}]: dispatch 2024-09-19T04:16:54.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:54 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fa0931c-684b-448d-8d63-2bd10881d88f"}]: dispatch 2024-09-19T04:16:54.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:54 smithi115 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fa0931c-684b-448d-8d63-2bd10881d88f"}]': finished 2024-09-19T04:16:54.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:54 smithi115 ceph-mon[35477]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T04:16:54.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:54 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:16:54.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:54 smithi145 ceph-mon[35096]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T04:16:54.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:54 smithi145 ceph-mon[35096]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:16:54.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:54 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/565771897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fa0931c-684b-448d-8d63-2bd10881d88f"}]: dispatch 2024-09-19T04:16:54.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:54 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fa0931c-684b-448d-8d63-2bd10881d88f"}]: dispatch 2024-09-19T04:16:54.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:54 smithi145 ceph-mon[35096]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fa0931c-684b-448d-8d63-2bd10881d88f"}]': finished 2024-09-19T04:16:54.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:54 smithi145 ceph-mon[35096]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T04:16:54.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:54 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:16:54.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:54 smithi107 ceph-mon[30659]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T04:16:54.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:54 smithi107 ceph-mon[30659]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:16:54.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:54 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/565771897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fa0931c-684b-448d-8d63-2bd10881d88f"}]: dispatch 2024-09-19T04:16:54.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:54 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fa0931c-684b-448d-8d63-2bd10881d88f"}]: dispatch 2024-09-19T04:16:54.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:54 smithi107 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fa0931c-684b-448d-8d63-2bd10881d88f"}]': finished 2024-09-19T04:16:54.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:54 smithi107 ceph-mon[30659]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T04:16:54.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:54 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:16:55.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:55 smithi107 ceph-mon[30659]: from='client.? 172.21.15.145:0/4037261539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:16:55.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:55 smithi115 ceph-mon[35477]: from='client.? 172.21.15.145:0/4037261539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:16:55.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:55 smithi145 ceph-mon[35096]: from='client.? 172.21.15.145:0/4037261539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:16:56.436 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:56 smithi115 ceph-mon[35477]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T04:16:56.436 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:16:56.436 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:56 smithi115 ceph-mon[35477]: pgmap v239: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T04:16:56.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:56 smithi145 ceph-mon[35096]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T04:16:56.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:16:56.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:56 smithi145 ceph-mon[35096]: pgmap v239: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T04:16:56.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:56 smithi107 ceph-mon[30659]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T04:16:56.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:16:56.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:56 smithi107 ceph-mon[30659]: pgmap v239: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T04:16:57.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:57 smithi115 ceph-mon[35477]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-19T04:16:57.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:57 smithi115 ceph-mon[35477]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T04:16:57.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:57 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:16:57.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:57 smithi145 ceph-mon[35096]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-19T04:16:57.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:57 smithi145 ceph-mon[35096]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T04:16:57.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:57 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:16:57.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:57 smithi107 ceph-mon[30659]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-19T04:16:57.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:57 smithi107 ceph-mon[30659]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T04:16:57.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:57 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:16:58.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:58 smithi107 ceph-mon[30659]: pgmap v241: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T04:16:58.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:58 smithi115 ceph-mon[35477]: pgmap v241: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T04:16:58.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:58 smithi145 ceph-mon[35096]: pgmap v241: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T04:17:00.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:59 smithi115 ceph-mon[35477]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-19T04:17:00.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:16:59 smithi115 ceph-mon[35477]: Cluster is now healthy 2024-09-19T04:17:00.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:59 smithi145 ceph-mon[35096]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-19T04:17:00.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:16:59 smithi145 ceph-mon[35096]: Cluster is now healthy 2024-09-19T04:17:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:59 smithi107 ceph-mon[30659]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-19T04:17:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:16:59 smithi107 ceph-mon[30659]: Cluster is now healthy 2024-09-19T04:17:00.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:00 smithi145 ceph-mon[35096]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:17:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:00 smithi115 ceph-mon[35477]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:17:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:00 smithi107 ceph-mon[30659]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:17:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:02 smithi115 ceph-mon[35477]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T04:17:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:17:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:03.211 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:02 smithi145 ceph-mon[35096]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T04:17:03.211 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:17:03.212 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:02 smithi107 ceph-mon[30659]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T04:17:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:17:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:04.104 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:03 smithi145 ceph-mon[35096]: Deploying daemon osd.11 on smithi145 2024-09-19T04:17:04.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:03 smithi115 ceph-mon[35477]: Deploying daemon osd.11 on smithi145 2024-09-19T04:17:04.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:03 smithi107 ceph-mon[30659]: Deploying daemon osd.11 on smithi145 2024-09-19T04:17:04.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:04 smithi145 ceph-mon[35096]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T04:17:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:04 smithi115 ceph-mon[35477]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T04:17:05.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:04 smithi107 ceph-mon[30659]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T04:17:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:06 smithi115 ceph-mon[35477]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:17:07.199 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:06 smithi145 ceph-mon[35096]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:17:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:06 smithi107 ceph-mon[30659]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:17:09.146 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:08 smithi145 ceph-mon[35096]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T04:17:09.147 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:08 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:09.147 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:08 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:09.147 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:08 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:08 smithi115 ceph-mon[35477]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T04:17:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:08 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:08 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:09.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:08 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:09.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:08 smithi107 ceph-mon[30659]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T04:17:09.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:08 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:09.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:08 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:09.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:08 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:10.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:10 smithi145 ceph-mon[35096]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:17:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:10 smithi115 ceph-mon[35477]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:17:11.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:10 smithi107 ceph-mon[30659]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:17:11.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.934+0000 7fda537fe640 1 -- 172.21.15.145:0/870516580 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fda5c062630 con 0x7fda480a3240 2024-09-19T04:17:11.936 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 11 on host 'smithi145' 2024-09-19T04:17:11.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.936+0000 7fda60d97640 1 -- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda480a3240 msgr2=0x7fda480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:11.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.936+0000 7fda60d97640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda480a3240 0x7fda480a5700 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda54007eb0 tx=0x7fda540073c0 comp rx=0 tx=0).stop 2024-09-19T04:17:11.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.936+0000 7fda60d97640 1 -- 172.21.15.145:0/870516580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda5c104f90 msgr2=0x7fda5c100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:11.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.936+0000 7fda60d97640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda5c104f90 0x7fda5c100610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fda4400d770 tx=0x7fda4400dc30 comp rx=0 tx=0).stop 2024-09-19T04:17:11.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.937+0000 7fda60d97640 1 -- 172.21.15.145:0/870516580 shutdown_connections 2024-09-19T04:17:11.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.937+0000 7fda60d97640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fda480a3240 0x7fda480a5700 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:11.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.937+0000 7fda60d97640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda5c1078c0 0x7fda5c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:11.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.937+0000 7fda60d97640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda5c104f90 0x7fda5c100610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:11.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.937+0000 7fda60d97640 1 --2- 172.21.15.145:0/870516580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c102660 0x7fda5c101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:11.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.937+0000 7fda60d97640 1 -- 172.21.15.145:0/870516580 >> 172.21.15.145:0/870516580 conn(0x7fda5c0fc280 msgr2=0x7fda5c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:11.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.937+0000 7fda60d97640 1 -- 172.21.15.145:0/870516580 shutdown_connections 2024-09-19T04:17:11.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:11.937+0000 7fda60d97640 1 -- 172.21.15.145:0/870516580 wait complete. 2024-09-19T04:17:12.550 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.550 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.550 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:12.550 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:12.550 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.550 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:12 smithi145 ceph-mon[35096]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:17:12.550 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:12.551 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.551 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:12 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.577 DEBUG:teuthology.orchestra.run.smithi145:osd.11> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T04:17:12.580 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T04:17:12.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:17:12.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:12.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:12.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:12 smithi107 ceph-mon[30659]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:17:12.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:12.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:12 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:12 smithi115 ceph-mon[35477]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:17:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:12 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:12.958 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:13.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.518+0000 7ff641600640 1 -- 172.21.15.107:0/3456795195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff63c105920 msgr2=0x7ff63c10c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:13.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.518+0000 7ff641600640 1 --2- 172.21.15.107:0/3456795195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff63c105920 0x7ff63c10c240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff634009ca0 tx=0x7ff63402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.518+0000 7ff641600640 1 -- 172.21.15.107:0/3456795195 shutdown_connections 2024-09-19T04:17:13.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.518+0000 7ff641600640 1 --2- 172.21.15.107:0/3456795195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff63c105920 0x7ff63c10c240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.518+0000 7ff641600640 1 --2- 172.21.15.107:0/3456795195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c104f60 0x7ff63c1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.518+0000 7ff641600640 1 --2- 172.21.15.107:0/3456795195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff63c103e00 0x7ff63c104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.518+0000 7ff641600640 1 -- 172.21.15.107:0/3456795195 >> 172.21.15.107:0/3456795195 conn(0x7ff63c0ff5b0 msgr2=0x7ff63c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:13.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.518+0000 7ff641600640 1 -- 172.21.15.107:0/3456795195 shutdown_connections 2024-09-19T04:17:13.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.518+0000 7ff641600640 1 -- 172.21.15.107:0/3456795195 wait complete. 2024-09-19T04:17:13.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.519+0000 7ff641600640 1 Processor -- start 2024-09-19T04:17:13.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.519+0000 7ff641600640 1 -- start start 2024-09-19T04:17:13.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff641600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c103e00 0x7ff63c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:13.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff641600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff63c104f60 0x7ff63c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:13.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff641600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff63c105920 0x7ff63c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:13.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff641600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c10fab0 con 0x7ff63c105920 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff641600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c10f930 con 0x7ff63c104f60 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff641600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c10fc30 con 0x7ff63c103e00 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff63b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff63c105920 0x7ff63c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff63b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff63c105920 0x7ff63c1a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60902/0 (socket says 172.21.15.107:60902) 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff63b400640 1 -- 172.21.15.107:0/3273118331 learned_addr learned my addr 172.21.15.107:0/3273118331 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff633e00640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff63c104f60 0x7ff63c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff63aa00640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c103e00 0x7ff63c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff63b400640 1 -- 172.21.15.107:0/3273118331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c103e00 msgr2=0x7ff63c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff63b400640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c103e00 0x7ff63c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff63b400640 1 -- 172.21.15.107:0/3273118331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff63c104f60 msgr2=0x7ff63c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff63b400640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff63c104f60 0x7ff63c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff63b400640 1 -- 172.21.15.107:0/3273118331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c1a1ca0 con 0x7ff63c105920 2024-09-19T04:17:13.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.520+0000 7ff63aa00640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c103e00 0x7ff63c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:13.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.521+0000 7ff63b400640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff63c105920 0x7ff63c1a1630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff634031ae0 tx=0x7ff634031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:13.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.521+0000 7ff633400640 1 -- 172.21.15.107:0/3273118331 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634003b70 con 0x7ff63c105920 2024-09-19T04:17:13.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.521+0000 7ff633400640 1 -- 172.21.15.107:0/3273118331 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff634003d10 con 0x7ff63c105920 2024-09-19T04:17:13.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.521+0000 7ff641600640 1 -- 172.21.15.107:0/3273118331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c1a1f30 con 0x7ff63c105920 2024-09-19T04:17:13.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.521+0000 7ff633400640 1 -- 172.21.15.107:0/3273118331 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63402fc40 con 0x7ff63c105920 2024-09-19T04:17:13.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.521+0000 7ff641600640 1 -- 172.21.15.107:0/3273118331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c1a2430 con 0x7ff63c105920 2024-09-19T04:17:13.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.522+0000 7ff641600640 1 -- 172.21.15.107:0/3273118331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63c10b120 con 0x7ff63c105920 2024-09-19T04:17:13.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.524+0000 7ff633400640 1 -- 172.21.15.107:0/3273118331 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff634004050 con 0x7ff63c105920 2024-09-19T04:17:13.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.526+0000 7ff633400640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff6200a32f0 0x7ff6200a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:13.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.526+0000 7ff633400640 1 -- 172.21.15.107:0/3273118331 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7ff6340f0620 con 0x7ff63c105920 2024-09-19T04:17:13.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.527+0000 7ff63aa00640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff6200a32f0 0x7ff6200a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:13.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.527+0000 7ff63aa00640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff6200a32f0 0x7ff6200a57b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009fc0 tx=0x7ff62c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:13.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.529+0000 7ff633400640 1 -- 172.21.15.107:0/3273118331 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff6340bd880 con 0x7ff63c105920 2024-09-19T04:17:13.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.705+0000 7ff641600640 1 -- 172.21.15.107:0/3273118331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff63c02da10 con 0x7ff63c105920 2024-09-19T04:17:13.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.705+0000 7ff633400640 1 -- 172.21.15.107:0/3273118331 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7ff6340bd180 con 0x7ff63c105920 2024-09-19T04:17:13.707 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 -- 172.21.15.107:0/3273118331 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff6200a32f0 msgr2=0x7ff6200a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff6200a32f0 0x7ff6200a57b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009fc0 tx=0x7ff62c005f90 comp rx=0 tx=0).stop 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 -- 172.21.15.107:0/3273118331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff63c105920 msgr2=0x7ff63c1a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff63c105920 0x7ff63c1a1630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff634031ae0 tx=0x7ff634031ee0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 -- 172.21.15.107:0/3273118331 shutdown_connections 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff6200a32f0 0x7ff6200a57b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff63c105920 0x7ff63c1a1630 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff63c104f60 0x7ff63c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 --2- 172.21.15.107:0/3273118331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c103e00 0x7ff63c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 -- 172.21.15.107:0/3273118331 >> 172.21.15.107:0/3273118331 conn(0x7ff63c0ff5b0 msgr2=0x7ff63c1082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 -- 172.21.15.107:0/3273118331 shutdown_connections 2024-09-19T04:17:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:13.708+0000 7ff641600640 1 -- 172.21.15.107:0/3273118331 wait complete. 2024-09-19T04:17:13.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:13 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3273118331' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:17:14.103 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:13 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3273118331' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:17:14.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:13 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3273118331' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:17:14.592 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726719412,"num_in_osds":12,"osd_in_since":1726719413,"num_remapped_pgs":0} 2024-09-19T04:17:14.724 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:17:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:17:14.454+0000 7f546d262640 -1 osd.11 0 log_to_monitors true 2024-09-19T04:17:15.097 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:14 smithi115 ceph-mon[35477]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:17:15.098 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:14 smithi115 ceph-mon[35477]: from='osd.11 [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:17:15.098 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:14 smithi115 ceph-mon[35477]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:17:15.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:14 smithi145 ceph-mon[35096]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:17:15.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:14 smithi145 ceph-mon[35096]: from='osd.11 [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:17:15.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:14 smithi145 ceph-mon[35096]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:17:15.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:14 smithi107 ceph-mon[30659]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:17:15.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:14 smithi107 ceph-mon[30659]: from='osd.11 [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:17:15.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:14 smithi107 ceph-mon[30659]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:17:15.593 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:17:15.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:17:15.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='osd.11 [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:17:15.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T04:17:15.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:15.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:17:15.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:15.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:15.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:15.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:15.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:15.892 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:15.892 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:15.892 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:15.892 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:15.892 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:15 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:15.975 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:16.157 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:17:16.157 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='osd.11 [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:17:16.157 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T04:17:16.157 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:16.157 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:17:16.157 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:16.158 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:16.158 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:16.158 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:16.158 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:16.158 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:16.158 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:16.158 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:16.158 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:16.158 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:15 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:16.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:17:16.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='osd.11 [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:17:16.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T04:17:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-19T04:17:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:17:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:16.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:16.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:16.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:15 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:16.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:17:16.155+0000 7f54691e5640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T04:17:16.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:17:16.166+0000 7f5461005640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:17:16.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.505+0000 7fe663200640 1 -- 172.21.15.107:0/1380369424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe664102c40 msgr2=0x7fe6641030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.505+0000 7fe663200640 1 --2- 172.21.15.107:0/1380369424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe664102c40 0x7fe6641030c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009860 tx=0x7fe64c02a340 comp rx=0 tx=0).stop 2024-09-19T04:17:16.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.505+0000 7fe663200640 1 -- 172.21.15.107:0/1380369424 shutdown_connections 2024-09-19T04:17:16.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.505+0000 7fe663200640 1 --2- 172.21.15.107:0/1380369424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe664103600 0x7fe664109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.505+0000 7fe663200640 1 --2- 172.21.15.107:0/1380369424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe664102c40 0x7fe6641030c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.505+0000 7fe663200640 1 --2- 172.21.15.107:0/1380369424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe664101a40 0x7fe664101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.505+0000 7fe663200640 1 -- 172.21.15.107:0/1380369424 >> 172.21.15.107:0/1380369424 conn(0x7fe6640fd1f0 msgr2=0x7fe6640ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.505+0000 7fe663200640 1 -- 172.21.15.107:0/1380369424 shutdown_connections 2024-09-19T04:17:16.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.505+0000 7fe663200640 1 -- 172.21.15.107:0/1380369424 wait complete. 2024-09-19T04:17:16.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.506+0000 7fe663200640 1 Processor -- start 2024-09-19T04:17:16.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.506+0000 7fe663200640 1 -- start start 2024-09-19T04:17:16.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.506+0000 7fe663200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe664101a40 0x7fe664199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.506+0000 7fe663200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe664102c40 0x7fe66419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.506+0000 7fe663200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe664103600 0x7fe6641a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.506+0000 7fe663200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe66410d6a0 con 0x7fe664102c40 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.506+0000 7fe663200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe66410d520 con 0x7fe664101a40 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.506+0000 7fe663200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe66410d820 con 0x7fe664103600 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe659400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe664102c40 0x7fe66419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe661e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe664101a40 0x7fe664199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe662800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe664103600 0x7fe6641a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe661e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe664101a40 0x7fe664199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59058/0 (socket says 172.21.15.107:59058) 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe662800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe664103600 0x7fe6641a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55858/0 (socket says 172.21.15.107:55858) 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe661e00640 1 -- 172.21.15.107:0/1795426263 learned_addr learned my addr 172.21.15.107:0/1795426263 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:16.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe661e00640 1 -- 172.21.15.107:0/1795426263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe664103600 msgr2=0x7fe6641a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe661e00640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe664103600 0x7fe6641a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe661e00640 1 -- 172.21.15.107:0/1795426263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe664102c40 msgr2=0x7fe66419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe661e00640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe664102c40 0x7fe66419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe661e00640 1 -- 172.21.15.107:0/1795426263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6641a1b70 con 0x7fe664101a40 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe659400640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe664102c40 0x7fe66419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.507+0000 7fe662800640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe664103600 0x7fe6641a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.508+0000 7fe661e00640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe664101a40 0x7fe664199f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e9c0 tx=0x7fe65400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.508+0000 7fe65b400640 1 -- 172.21.15.107:0/1795426263 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65400cde0 con 0x7fe664101a40 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.508+0000 7fe65b400640 1 -- 172.21.15.107:0/1795426263 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe654010910 con 0x7fe664101a40 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.508+0000 7fe663200640 1 -- 172.21.15.107:0/1795426263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6641a1e60 con 0x7fe664101a40 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.508+0000 7fe65b400640 1 -- 172.21.15.107:0/1795426263 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654018a00 con 0x7fe664101a40 2024-09-19T04:17:16.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.508+0000 7fe663200640 1 -- 172.21.15.107:0/1795426263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6641a22c0 con 0x7fe664101a40 2024-09-19T04:17:16.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.509+0000 7fe663200640 1 -- 172.21.15.107:0/1795426263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6641a24f0 con 0x7fe664101a40 2024-09-19T04:17:16.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.511+0000 7fe65b400640 1 -- 172.21.15.107:0/1795426263 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe65401f020 con 0x7fe664101a40 2024-09-19T04:17:16.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.512+0000 7fe65b400640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe6480a32f0 0x7fe6480a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.512+0000 7fe65b400640 1 -- 172.21.15.107:0/1795426263 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6847+0+0 (secure 0 0 0) 0x7fe6540cf520 con 0x7fe664101a40 2024-09-19T04:17:16.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.512+0000 7fe659400640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe6480a32f0 0x7fe6480a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.513+0000 7fe659400640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe6480a32f0 0x7fe6480a57b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009830 tx=0x7fe64c030550 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.516+0000 7fe65b400640 1 -- 172.21.15.107:0/1795426263 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe65409c760 con 0x7fe664101a40 2024-09-19T04:17:16.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.697+0000 7fe663200640 1 -- 172.21.15.107:0/1795426263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe6641a2cc0 con 0x7fe664101a40 2024-09-19T04:17:16.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.697+0000 7fe65b400640 1 -- 172.21.15.107:0/1795426263 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fe65409c060 con 0x7fe664101a40 2024-09-19T04:17:16.699 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 -- 172.21.15.107:0/1795426263 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe6480a32f0 msgr2=0x7fe6480a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe6480a32f0 0x7fe6480a57b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009830 tx=0x7fe64c030550 comp rx=0 tx=0).stop 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 -- 172.21.15.107:0/1795426263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe664101a40 msgr2=0x7fe664199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe664101a40 0x7fe664199f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e9c0 tx=0x7fe65400ee80 comp rx=0 tx=0).stop 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 -- 172.21.15.107:0/1795426263 shutdown_connections 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe6480a32f0 0x7fe6480a57b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe664103600 0x7fe6641a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe664102c40 0x7fe66419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 --2- 172.21.15.107:0/1795426263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe664101a40 0x7fe664199f40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 -- 172.21.15.107:0/1795426263 >> 172.21.15.107:0/1795426263 conn(0x7fe6640fd1f0 msgr2=0x7fe664074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 -- 172.21.15.107:0/1795426263 shutdown_connections 2024-09-19T04:17:16.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:16.700+0000 7fe663200640 1 -- 172.21.15.107:0/1795426263 wait complete. 2024-09-19T04:17:16.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-mon[35096]: Detected new or changed devices on smithi145 2024-09-19T04:17:16.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-19T04:17:16.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-mon[35096]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:17:16.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-mon[35096]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T04:17:16.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:16.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:16.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-mon[35096]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:17:16.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-mon[35096]: from='osd.11 ' entity='osd.11' 2024-09-19T04:17:16.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:16 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1795426263' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:17:17.082 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:16 smithi107 ceph-mon[30659]: Detected new or changed devices on smithi145 2024-09-19T04:17:17.082 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:16 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-19T04:17:17.082 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:16 smithi107 ceph-mon[30659]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:17:17.082 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:16 smithi107 ceph-mon[30659]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T04:17:17.082 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:16 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:17.082 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:16 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:17.082 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:16 smithi107 ceph-mon[30659]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:17:17.082 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:16 smithi107 ceph-mon[30659]: from='osd.11 ' entity='osd.11' 2024-09-19T04:17:17.083 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:16 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1795426263' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:17:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:16 smithi115 ceph-mon[35477]: Detected new or changed devices on smithi145 2024-09-19T04:17:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:16 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-19T04:17:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:16 smithi115 ceph-mon[35477]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-19T04:17:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:16 smithi115 ceph-mon[35477]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T04:17:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:16 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:16 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:16 smithi115 ceph-mon[35477]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:17:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:16 smithi115 ceph-mon[35477]: from='osd.11 ' entity='osd.11' 2024-09-19T04:17:17.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:16 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1795426263' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:17:17.552 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726719412,"num_in_osds":12,"osd_in_since":1726719413,"num_remapped_pgs":0} 2024-09-19T04:17:18.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:17 smithi115 ceph-mon[35477]: purged_snaps scrub starts 2024-09-19T04:17:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:17 smithi115 ceph-mon[35477]: purged_snaps scrub ok 2024-09-19T04:17:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:17 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:17 smithi115 ceph-mon[35477]: osd.11 [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] boot 2024-09-19T04:17:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:17 smithi115 ceph-mon[35477]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T04:17:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:17 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:17 smithi145 ceph-mon[35096]: purged_snaps scrub starts 2024-09-19T04:17:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:17 smithi145 ceph-mon[35096]: purged_snaps scrub ok 2024-09-19T04:17:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:17 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:17 smithi145 ceph-mon[35096]: osd.11 [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] boot 2024-09-19T04:17:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:17 smithi145 ceph-mon[35096]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T04:17:18.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:17 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:18.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:17 smithi107 ceph-mon[30659]: purged_snaps scrub starts 2024-09-19T04:17:18.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:17 smithi107 ceph-mon[30659]: purged_snaps scrub ok 2024-09-19T04:17:18.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:17 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:18.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:17 smithi107 ceph-mon[30659]: osd.11 [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] boot 2024-09-19T04:17:18.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:17 smithi107 ceph-mon[30659]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T04:17:18.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:17 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:17:18.557 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:17:18.925 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:18.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:18 smithi107 ceph-mon[30659]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:18 smithi107 ceph-mon[30659]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T04:17:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:18 smithi115 ceph-mon[35477]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:18 smithi115 ceph-mon[35477]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T04:17:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:18 smithi145 ceph-mon[35096]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:18 smithi145 ceph-mon[35096]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T04:17:19.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.451+0000 7fbb19000640 1 -- 172.21.15.107:0/2731784416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb14103600 msgr2=0x7fbb14109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:19.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.451+0000 7fbb19000640 1 --2- 172.21.15.107:0/2731784416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb14103600 0x7fbb14109e90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009ca0 tx=0x7fbb0c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.451+0000 7fbb19000640 1 -- 172.21.15.107:0/2731784416 shutdown_connections 2024-09-19T04:17:19.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.451+0000 7fbb19000640 1 --2- 172.21.15.107:0/2731784416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb14103600 0x7fbb14109e90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.451+0000 7fbb19000640 1 --2- 172.21.15.107:0/2731784416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb14102c40 0x7fbb141030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.451+0000 7fbb19000640 1 --2- 172.21.15.107:0/2731784416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb14101a40 0x7fbb14101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.451+0000 7fbb19000640 1 -- 172.21.15.107:0/2731784416 >> 172.21.15.107:0/2731784416 conn(0x7fbb140fd1f0 msgr2=0x7fbb140ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:19.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.451+0000 7fbb19000640 1 -- 172.21.15.107:0/2731784416 shutdown_connections 2024-09-19T04:17:19.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.451+0000 7fbb19000640 1 -- 172.21.15.107:0/2731784416 wait complete. 2024-09-19T04:17:19.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.452+0000 7fbb19000640 1 Processor -- start 2024-09-19T04:17:19.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.452+0000 7fbb19000640 1 -- start start 2024-09-19T04:17:19.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb19000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb14101a40 0x7fbb14199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:19.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb19000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb14102c40 0x7fbb1419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb19000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb14103600 0x7fbb141a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb19000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1410d710 con 0x7fbb14103600 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb19000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1410d590 con 0x7fbb14101a40 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb19000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1410d890 con 0x7fbb14102c40 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb13e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb14103600 0x7fbb141a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb13e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb14103600 0x7fbb141a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57222/0 (socket says 172.21.15.107:57222) 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb13400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb14101a40 0x7fbb14199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb13e00640 1 -- 172.21.15.107:0/516633408 learned_addr learned my addr 172.21.15.107:0/516633408 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb12a00640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb14102c40 0x7fbb1419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb13e00640 1 -- 172.21.15.107:0/516633408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb14102c40 msgr2=0x7fbb1419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.453+0000 7fbb13e00640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb14102c40 0x7fbb1419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.454+0000 7fbb13e00640 1 -- 172.21.15.107:0/516633408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb14101a40 msgr2=0x7fbb14199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:19.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.454+0000 7fbb13e00640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb14101a40 0x7fbb14199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.454+0000 7fbb13e00640 1 -- 172.21.15.107:0/516633408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb141a1b40 con 0x7fbb14103600 2024-09-19T04:17:19.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.454+0000 7fbb13400640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb14101a40 0x7fbb14199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:19.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.454+0000 7fbb13e00640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb14103600 0x7fbb141a14d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0319f0 tx=0x7fbb0c02f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:19.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.454+0000 7fbb0be00640 1 -- 172.21.15.107:0/516633408 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c003c70 con 0x7fbb14103600 2024-09-19T04:17:19.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.454+0000 7fbb0be00640 1 -- 172.21.15.107:0/516633408 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb0c003e10 con 0x7fbb14103600 2024-09-19T04:17:19.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.454+0000 7fbb0be00640 1 -- 172.21.15.107:0/516633408 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c02fd30 con 0x7fbb14103600 2024-09-19T04:17:19.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.454+0000 7fbb19000640 1 -- 172.21.15.107:0/516633408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb141a1dd0 con 0x7fbb14103600 2024-09-19T04:17:19.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.454+0000 7fbb19000640 1 -- 172.21.15.107:0/516633408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb141a2280 con 0x7fbb14103600 2024-09-19T04:17:19.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.455+0000 7fbb19000640 1 -- 172.21.15.107:0/516633408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb141a24e0 con 0x7fbb14103600 2024-09-19T04:17:19.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.457+0000 7fbb0be00640 1 -- 172.21.15.107:0/516633408 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fbb0c004b40 con 0x7fbb14103600 2024-09-19T04:17:19.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.459+0000 7fbb0be00640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbaf80a3270 0x7fbaf80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:19.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.459+0000 7fbb0be00640 1 -- 172.21.15.107:0/516633408 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fbb0c0f0b20 con 0x7fbb14103600 2024-09-19T04:17:19.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.459+0000 7fbb13400640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbaf80a3270 0x7fbaf80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:19.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.459+0000 7fbb13400640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbaf80a3270 0x7fbaf80a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbb04005fd0 tx=0x7fbb0400b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:19.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.461+0000 7fbb0be00640 1 -- 172.21.15.107:0/516633408 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbb0c0bdc40 con 0x7fbb14103600 2024-09-19T04:17:19.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.636+0000 7fbb19000640 1 -- 172.21.15.107:0/516633408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbb141a2870 con 0x7fbb14103600 2024-09-19T04:17:19.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.636+0000 7fbb0be00640 1 -- 172.21.15.107:0/516633408 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7fbb0c0bd540 con 0x7fbb14103600 2024-09-19T04:17:19.638 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:19.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.638+0000 7fbb19000640 1 -- 172.21.15.107:0/516633408 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbaf80a3270 msgr2=0x7fbaf80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:19.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.638+0000 7fbb19000640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbaf80a3270 0x7fbaf80a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbb04005fd0 tx=0x7fbb0400b410 comp rx=0 tx=0).stop 2024-09-19T04:17:19.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.638+0000 7fbb19000640 1 -- 172.21.15.107:0/516633408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb14103600 msgr2=0x7fbb141a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:19.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.638+0000 7fbb19000640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb14103600 0x7fbb141a14d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0319f0 tx=0x7fbb0c02f450 comp rx=0 tx=0).stop 2024-09-19T04:17:19.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.639+0000 7fbb19000640 1 -- 172.21.15.107:0/516633408 shutdown_connections 2024-09-19T04:17:19.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.639+0000 7fbb19000640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbaf80a3270 0x7fbaf80a5730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.639+0000 7fbb19000640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb14103600 0x7fbb141a14d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.639+0000 7fbb19000640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb14102c40 0x7fbb1419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.639+0000 7fbb19000640 1 --2- 172.21.15.107:0/516633408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb14101a40 0x7fbb14199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:19.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.639+0000 7fbb19000640 1 -- 172.21.15.107:0/516633408 >> 172.21.15.107:0/516633408 conn(0x7fbb140fd1f0 msgr2=0x7fbb14074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:19.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.639+0000 7fbb19000640 1 -- 172.21.15.107:0/516633408 shutdown_connections 2024-09-19T04:17:19.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:19.639+0000 7fbb19000640 1 -- 172.21.15.107:0/516633408 wait complete. 2024-09-19T04:17:19.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:19 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/516633408' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:17:20.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:19 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/516633408' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:17:20.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:19 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/516633408' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:17:20.445 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726719437,"num_in_osds":12,"osd_in_since":1726719413,"num_remapped_pgs":0} 2024-09-19T04:17:20.446 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:17:20.809 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:21.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:20 smithi107 ceph-mon[30659]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:20 smithi115 ceph-mon[35477]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:20 smithi145 ceph-mon[35096]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:21.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.342+0000 7fe186200640 1 -- 172.21.15.107:0/4257297264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe180112a30 msgr2=0x7fe180114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.342+0000 7fe186200640 1 --2- 172.21.15.107:0/4257297264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe180112a30 0x7fe180114e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe178009ca0 tx=0x7fe17802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.342+0000 7fe186200640 1 -- 172.21.15.107:0/4257297264 shutdown_connections 2024-09-19T04:17:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.342+0000 7fe186200640 1 --2- 172.21.15.107:0/4257297264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe180112a30 0x7fe180114e20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.342+0000 7fe186200640 1 --2- 172.21.15.107:0/4257297264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe180077040 0x7fe1801124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.342+0000 7fe186200640 1 --2- 172.21.15.107:0/4257297264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe180076720 0x7fe180076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.342+0000 7fe186200640 1 -- 172.21.15.107:0/4257297264 >> 172.21.15.107:0/4257297264 conn(0x7fe1800ffa90 msgr2=0x7fe180101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.342+0000 7fe186200640 1 -- 172.21.15.107:0/4257297264 shutdown_connections 2024-09-19T04:17:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.342+0000 7fe186200640 1 -- 172.21.15.107:0/4257297264 wait complete. 2024-09-19T04:17:21.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.343+0000 7fe186200640 1 Processor -- start 2024-09-19T04:17:21.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.343+0000 7fe186200640 1 -- start start 2024-09-19T04:17:21.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe186200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe180076720 0x7fe1801a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe186200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe180077040 0x7fe1801a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe186200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe180112a30 0x7fe18019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe186200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1801185d0 con 0x7fe180076720 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe186200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe180118450 con 0x7fe180112a30 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe186200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe180118750 con 0x7fe180077040 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe17f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe180076720 0x7fe1801a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe17ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe180077040 0x7fe1801a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe17ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe180077040 0x7fe1801a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:48680/0 (socket says 172.21.15.107:48680) 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe17ea00640 1 -- 172.21.15.107:0/694124380 learned_addr learned my addr 172.21.15.107:0/694124380 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.344+0000 7fe17fe00640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe180112a30 0x7fe18019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.345+0000 7fe17ea00640 1 -- 172.21.15.107:0/694124380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe180112a30 msgr2=0x7fe18019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.345+0000 7fe17ea00640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe180112a30 0x7fe18019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.345+0000 7fe17ea00640 1 -- 172.21.15.107:0/694124380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe180076720 msgr2=0x7fe1801a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.345+0000 7fe17ea00640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe180076720 0x7fe1801a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.345+0000 7fe17ea00640 1 -- 172.21.15.107:0/694124380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18019efd0 con 0x7fe180077040 2024-09-19T04:17:21.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.345+0000 7fe17ea00640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe180077040 0x7fe1801a4c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00e9f0 tx=0x7fe16c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.345+0000 7fe177e00640 1 -- 172.21.15.107:0/694124380 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00cd80 con 0x7fe180077040 2024-09-19T04:17:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.345+0000 7fe186200640 1 -- 172.21.15.107:0/694124380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18019f2c0 con 0x7fe180077040 2024-09-19T04:17:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.346+0000 7fe186200640 1 -- 172.21.15.107:0/694124380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1801ab5b0 con 0x7fe180077040 2024-09-19T04:17:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.346+0000 7fe177e00640 1 -- 172.21.15.107:0/694124380 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe16c010910 con 0x7fe180077040 2024-09-19T04:17:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.346+0000 7fe177e00640 1 -- 172.21.15.107:0/694124380 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c018980 con 0x7fe180077040 2024-09-19T04:17:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.347+0000 7fe186200640 1 -- 172.21.15.107:0/694124380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe160005290 con 0x7fe180077040 2024-09-19T04:17:21.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.350+0000 7fe177e00640 1 -- 172.21.15.107:0/694124380 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe16c010ac0 con 0x7fe180077040 2024-09-19T04:17:21.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.351+0000 7fe177e00640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1640a3240 0x7fe1640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:21.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.351+0000 7fe177e00640 1 -- 172.21.15.107:0/694124380 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fe16c0cf880 con 0x7fe180077040 2024-09-19T04:17:21.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.351+0000 7fe17f400640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1640a3240 0x7fe1640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:21.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.352+0000 7fe17f400640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1640a3240 0x7fe1640a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe18019f9e0 tx=0x7fe170005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:21.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.352+0000 7fe177e00640 1 -- 172.21.15.107:0/694124380 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe16c09c350 con 0x7fe180077040 2024-09-19T04:17:21.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.530+0000 7fe186200640 1 -- 172.21.15.107:0/694124380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe160005850 con 0x7fe180077040 2024-09-19T04:17:21.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.532+0000 7fe177e00640 1 -- 172.21.15.107:0/694124380 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18537 (secure 0 0 0) 0x7fe16c0a0650 con 0x7fe180077040 2024-09-19T04:17:21.534 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:21.534 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":76,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:17:18.160142+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:17:21.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 -- 172.21.15.107:0/694124380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1640a3240 msgr2=0x7fe1640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1640a3240 0x7fe1640a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe18019f9e0 tx=0x7fe170005f90 comp rx=0 tx=0).stop 2024-09-19T04:17:21.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 -- 172.21.15.107:0/694124380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe180077040 msgr2=0x7fe1801a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:21.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe180077040 0x7fe1801a4c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00e9f0 tx=0x7fe16c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 -- 172.21.15.107:0/694124380 shutdown_connections 2024-09-19T04:17:21.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1640a3240 0x7fe1640a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe180112a30 0x7fe18019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe180077040 0x7fe1801a4c20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 --2- 172.21.15.107:0/694124380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe180076720 0x7fe1801a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:21.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 -- 172.21.15.107:0/694124380 >> 172.21.15.107:0/694124380 conn(0x7fe1800ffa90 msgr2=0x7fe180101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:21.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 -- 172.21.15.107:0/694124380 shutdown_connections 2024-09-19T04:17:21.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:21.534+0000 7fe186200640 1 -- 172.21.15.107:0/694124380 wait complete. 2024-09-19T04:17:22.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:21 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/694124380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:17:22.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:21 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/694124380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:17:22.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:21 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/694124380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:17:22.388 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:13:18.308041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:17:22.388 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T04:17:22.750 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:23.051 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:22 smithi107 ceph-mon[30659]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:22 smithi115 ceph-mon[35477]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:22 smithi145 ceph-mon[35096]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:23.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.313+0000 7fb7c4800640 1 -- 172.21.15.107:0/2764041166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7bc1049a0 msgr2=0x7fb7bc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:23.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.313+0000 7fb7c4800640 1 --2- 172.21.15.107:0/2764041166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7bc1049a0 0x7fb7bc1089f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009900 tx=0x7fb7ac02a410 comp rx=0 tx=0).stop 2024-09-19T04:17:23.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.313+0000 7fb7c4800640 1 -- 172.21.15.107:0/2764041166 shutdown_connections 2024-09-19T04:17:23.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.313+0000 7fb7c4800640 1 --2- 172.21.15.107:0/2764041166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7bc109120 0x7fb7bc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:23.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.313+0000 7fb7c4800640 1 --2- 172.21.15.107:0/2764041166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7bc1049a0 0x7fb7bc1089f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:23.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.313+0000 7fb7c4800640 1 --2- 172.21.15.107:0/2764041166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7bc103ff0 0x7fb7bc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:23.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.313+0000 7fb7c4800640 1 -- 172.21.15.107:0/2764041166 >> 172.21.15.107:0/2764041166 conn(0x7fb7bc0ffa90 msgr2=0x7fb7bc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:23.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.313+0000 7fb7c4800640 1 -- 172.21.15.107:0/2764041166 shutdown_connections 2024-09-19T04:17:23.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.313+0000 7fb7c4800640 1 -- 172.21.15.107:0/2764041166 wait complete. 2024-09-19T04:17:23.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c4800640 1 Processor -- start 2024-09-19T04:17:23.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c4800640 1 -- start start 2024-09-19T04:17:23.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c4800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7bc103ff0 0x7fb7bc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:23.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c4800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7bc1049a0 0x7fb7bc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:23.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c4800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7bc109120 0x7fb7bc19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:23.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c4800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc114190 con 0x7fb7bc109120 2024-09-19T04:17:23.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c4800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc114010 con 0x7fb7bc1049a0 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c4800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc114310 con 0x7fb7bc103ff0 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7bc103ff0 0x7fb7bc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7bc109120 0x7fb7bc19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7bc103ff0 0x7fb7bc19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:48700/0 (socket says 172.21.15.107:48700) 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.315+0000 7fb7c1e00640 1 -- 172.21.15.107:0/3235244400 learned_addr learned my addr 172.21.15.107:0/3235244400 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.316+0000 7fb7c1400640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7bc1049a0 0x7fb7bc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.316+0000 7fb7c1e00640 1 -- 172.21.15.107:0/3235244400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7bc1049a0 msgr2=0x7fb7bc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.316+0000 7fb7c1e00640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7bc1049a0 0x7fb7bc19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.316+0000 7fb7c1e00640 1 -- 172.21.15.107:0/3235244400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7bc109120 msgr2=0x7fb7bc19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.316+0000 7fb7c1e00640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7bc109120 0x7fb7bc19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.316+0000 7fb7c1e00640 1 -- 172.21.15.107:0/3235244400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc19f5c0 con 0x7fb7bc103ff0 2024-09-19T04:17:23.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.316+0000 7fb7c2800640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7bc109120 0x7fb7bc19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:23.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.316+0000 7fb7c1400640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7bc1049a0 0x7fb7bc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:23.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.316+0000 7fb7c1e00640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7bc103ff0 0x7fb7bc19a610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e9f0 tx=0x7fb7b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:23.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.316+0000 7fb7b6a00640 1 -- 172.21.15.107:0/3235244400 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000cd80 con 0x7fb7bc103ff0 2024-09-19T04:17:23.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.317+0000 7fb7c4800640 1 -- 172.21.15.107:0/3235244400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc19f8b0 con 0x7fb7bc103ff0 2024-09-19T04:17:23.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.317+0000 7fb7b6a00640 1 -- 172.21.15.107:0/3235244400 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b0005500 con 0x7fb7bc103ff0 2024-09-19T04:17:23.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.317+0000 7fb7b6a00640 1 -- 172.21.15.107:0/3235244400 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0010690 con 0x7fb7bc103ff0 2024-09-19T04:17:23.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.317+0000 7fb7c4800640 1 -- 172.21.15.107:0/3235244400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc1a7190 con 0x7fb7bc103ff0 2024-09-19T04:17:23.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.318+0000 7fb7c4800640 1 -- 172.21.15.107:0/3235244400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7bc1043d0 con 0x7fb7bc103ff0 2024-09-19T04:17:23.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.320+0000 7fb7b6a00640 1 -- 172.21.15.107:0/3235244400 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb7b00046c0 con 0x7fb7bc103ff0 2024-09-19T04:17:23.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.321+0000 7fb7b6a00640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7a40a32c0 0x7fb7a40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:23.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.321+0000 7fb7b6a00640 1 -- 172.21.15.107:0/3235244400 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fb7b00cfb20 con 0x7fb7bc103ff0 2024-09-19T04:17:23.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.321+0000 7fb7c1400640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7a40a32c0 0x7fb7a40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:23.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.322+0000 7fb7c1400640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7a40a32c0 0x7fb7a40a5780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac000c00 tx=0x7fb7ac005a40 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:23.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.323+0000 7fb7b6a00640 1 -- 172.21.15.107:0/3235244400 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb7b0054210 con 0x7fb7bc103ff0 2024-09-19T04:17:23.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.498+0000 7fb7c4800640 1 -- 172.21.15.107:0/3235244400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb7bc10c3c0 con 0x7fb7bc103ff0 2024-09-19T04:17:23.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.499+0000 7fb7b6a00640 1 -- 172.21.15.107:0/3235244400 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7fb7b009cc40 con 0x7fb7bc103ff0 2024-09-19T04:17:23.502 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-09-19T04:17:23.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.501+0000 7fb7c4800640 1 -- 172.21.15.107:0/3235244400 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7a40a32c0 msgr2=0x7fb7a40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:23.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.501+0000 7fb7c4800640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7a40a32c0 0x7fb7a40a5780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac000c00 tx=0x7fb7ac005a40 comp rx=0 tx=0).stop 2024-09-19T04:17:23.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.501+0000 7fb7c4800640 1 -- 172.21.15.107:0/3235244400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7bc103ff0 msgr2=0x7fb7bc19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:23.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.501+0000 7fb7c4800640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7bc103ff0 0x7fb7bc19a610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e9f0 tx=0x7fb7b000edf0 comp rx=0 tx=0).stop 2024-09-19T04:17:23.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.502+0000 7fb7c4800640 1 -- 172.21.15.107:0/3235244400 shutdown_connections 2024-09-19T04:17:23.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.502+0000 7fb7c4800640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb7a40a32c0 0x7fb7a40a5780 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:23.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.502+0000 7fb7c4800640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7bc109120 0x7fb7bc19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:23.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.502+0000 7fb7c4800640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7bc1049a0 0x7fb7bc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:23.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.502+0000 7fb7c4800640 1 --2- 172.21.15.107:0/3235244400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7bc103ff0 0x7fb7bc19a610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:23.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.502+0000 7fb7c4800640 1 -- 172.21.15.107:0/3235244400 >> 172.21.15.107:0/3235244400 conn(0x7fb7bc0ffa90 msgr2=0x7fb7bc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:23.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.502+0000 7fb7c4800640 1 -- 172.21.15.107:0/3235244400 shutdown_connections 2024-09-19T04:17:23.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:23.502+0000 7fb7c4800640 1 -- 172.21.15.107:0/3235244400 wait complete. 2024-09-19T04:17:24.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:23 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3235244400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:17:24.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:23 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3235244400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:17:24.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:23 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3235244400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:17:24.345 INFO:tasks.cephadm:Adding mds.a on smithi107 2024-09-19T04:17:24.345 INFO:tasks.cephadm:Adding mds.d on smithi107 2024-09-19T04:17:24.345 INFO:tasks.cephadm:Adding mds.g on smithi107 2024-09-19T04:17:24.345 INFO:tasks.cephadm:Adding mds.j on smithi107 2024-09-19T04:17:24.346 INFO:tasks.cephadm:Adding mds.b on smithi115 2024-09-19T04:17:24.346 INFO:tasks.cephadm:Adding mds.e on smithi115 2024-09-19T04:17:24.346 INFO:tasks.cephadm:Adding mds.h on smithi115 2024-09-19T04:17:24.346 INFO:tasks.cephadm:Adding mds.k on smithi115 2024-09-19T04:17:24.346 INFO:tasks.cephadm:Adding mds.c on smithi145 2024-09-19T04:17:24.346 INFO:tasks.cephadm:Adding mds.f on smithi145 2024-09-19T04:17:24.346 INFO:tasks.cephadm:Adding mds.i on smithi145 2024-09-19T04:17:24.346 INFO:tasks.cephadm:Adding mds.l on smithi145 2024-09-19T04:17:24.346 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l' 2024-09-19T04:17:24.614 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:17:24.995 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:24 smithi145 ceph-mon[35096]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:25.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.077+0000 7f2753ab8640 1 -- 172.21.15.145:0/860373107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c1016b0 msgr2=0x7f274c108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:25.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.077+0000 7f2753ab8640 1 --2- 172.21.15.145:0/860373107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c1016b0 0x7f274c108f70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2744009ca0 tx=0x7f274402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:17:25.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.077+0000 7f2753ab8640 1 -- 172.21.15.145:0/860373107 shutdown_connections 2024-09-19T04:17:25.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.077+0000 7f2753ab8640 1 --2- 172.21.15.145:0/860373107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c1016b0 0x7f274c108f70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:25.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.077+0000 7f2753ab8640 1 --2- 172.21.15.145:0/860373107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f274c100d10 0x7f274c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:25.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.077+0000 7f2753ab8640 1 --2- 172.21.15.145:0/860373107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f274c1020b0 0x7f274c1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:25.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.077+0000 7f2753ab8640 1 -- 172.21.15.145:0/860373107 >> 172.21.15.145:0/860373107 conn(0x7f274c0fba30 msgr2=0x7f274c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:25.098 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.077+0000 7f2753ab8640 1 -- 172.21.15.145:0/860373107 shutdown_connections 2024-09-19T04:17:25.098 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.077+0000 7f2753ab8640 1 -- 172.21.15.145:0/860373107 wait complete. 2024-09-19T04:17:25.098 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2753ab8640 1 Processor -- start 2024-09-19T04:17:25.098 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2753ab8640 1 -- start start 2024-09-19T04:17:25.098 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2753ab8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f274c100d10 0x7f274c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:25.098 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2753ab8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f274c1016b0 0x7f274c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:25.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2753ab8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c1020b0 0x7f274c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:25.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2753ab8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10c860 con 0x7f274c1016b0 2024-09-19T04:17:25.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2753ab8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10c6e0 con 0x7f274c100d10 2024-09-19T04:17:25.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2753ab8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10c9e0 con 0x7f274c1020b0 2024-09-19T04:17:25.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2752055640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c1020b0 0x7f274c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:25.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2752055640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c1020b0 0x7f274c19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35108/0 (socket says 172.21.15.145:35108) 2024-09-19T04:17:25.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2752055640 1 -- 172.21.15.145:0/4079805785 learned_addr learned my addr 172.21.15.145:0/4079805785 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-19T04:17:25.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2751854640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f274c100d10 0x7f274c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:25.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2752055640 1 -- 172.21.15.145:0/4079805785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f274c100d10 msgr2=0x7f274c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:25.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2752055640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f274c100d10 0x7f274c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:25.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2752055640 1 -- 172.21.15.145:0/4079805785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f274c1016b0 msgr2=0x7f274c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:25.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2752055640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f274c1016b0 0x7f274c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:25.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2752055640 1 -- 172.21.15.145:0/4079805785 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c19fc30 con 0x7f274c1020b0 2024-09-19T04:17:25.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.078+0000 7f2751854640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f274c100d10 0x7f274c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:25.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.079+0000 7f2752055640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c1020b0 0x7f274c19f5c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27440098d0 tx=0x7f274402f840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:25.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.079+0000 7f2742ffd640 1 -- 172.21.15.145:0/4079805785 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274402fc30 con 0x7f274c1020b0 2024-09-19T04:17:25.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.079+0000 7f2742ffd640 1 -- 172.21.15.145:0/4079805785 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2744003e50 con 0x7f274c1020b0 2024-09-19T04:17:25.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.079+0000 7f2753ab8640 1 -- 172.21.15.145:0/4079805785 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c19fec0 con 0x7f274c1020b0 2024-09-19T04:17:25.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.079+0000 7f2742ffd640 1 -- 172.21.15.145:0/4079805785 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440305c0 con 0x7f274c1020b0 2024-09-19T04:17:25.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.079+0000 7f2753ab8640 1 -- 172.21.15.145:0/4079805785 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c1a0450 con 0x7f274c1020b0 2024-09-19T04:17:25.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.080+0000 7f2753ab8640 1 -- 172.21.15.145:0/4079805785 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c1a0680 con 0x7f274c1020b0 2024-09-19T04:17:25.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.081+0000 7f2742ffd640 1 -- 172.21.15.145:0/4079805785 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2744003970 con 0x7f274c1020b0 2024-09-19T04:17:25.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.081+0000 7f2742ffd640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f27380a3270 0x7f27380a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:25.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.082+0000 7f2742ffd640 1 -- 172.21.15.145:0/4079805785 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f27440f0870 con 0x7f274c1020b0 2024-09-19T04:17:25.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.082+0000 7f2751854640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f27380a3270 0x7f27380a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:25.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.082+0000 7f2751854640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f27380a3270 0x7f27380a5730 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2748009930 tx=0x7f2748008040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:25.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.086+0000 7f2742ffd640 1 -- 172.21.15.145:0/4079805785 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f27440bda30 con 0x7f274c1020b0 2024-09-19T04:17:25.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:24 smithi115 ceph-mon[35477]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:24 smithi107 ceph-mon[30659]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:25.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.262+0000 7f2753ab8640 1 -- 172.21.15.145:0/4079805785 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}) -- 0x7f274c062630 con 0x7f27380a3270 2024-09-19T04:17:25.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.271+0000 7f2742ffd640 1 -- 172.21.15.145:0/4079805785 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f274c062630 con 0x7f27380a3270 2024-09-19T04:17:25.273 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mds.all update... 2024-09-19T04:17:25.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 -- 172.21.15.145:0/4079805785 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f27380a3270 msgr2=0x7f27380a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:25.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f27380a3270 0x7f27380a5730 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2748009930 tx=0x7f2748008040 comp rx=0 tx=0).stop 2024-09-19T04:17:25.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 -- 172.21.15.145:0/4079805785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c1020b0 msgr2=0x7f274c19f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:25.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c1020b0 0x7f274c19f5c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27440098d0 tx=0x7f274402f840 comp rx=0 tx=0).stop 2024-09-19T04:17:25.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 -- 172.21.15.145:0/4079805785 shutdown_connections 2024-09-19T04:17:25.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f27380a3270 0x7f27380a5730 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:25.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c1020b0 0x7f274c19f5c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:25.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f274c1016b0 0x7f274c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:25.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 --2- 172.21.15.145:0/4079805785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f274c100d10 0x7f274c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:25.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 -- 172.21.15.145:0/4079805785 >> 172.21.15.145:0/4079805785 conn(0x7f274c0fba30 msgr2=0x7f274c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:25.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.273+0000 7f2753ab8640 1 -- 172.21.15.145:0/4079805785 shutdown_connections 2024-09-19T04:17:25.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-19T04:17:25.274+0000 7f2753ab8640 1 -- 172.21.15.145:0/4079805785 wait complete. 2024-09-19T04:17:26.053 DEBUG:teuthology.orchestra.run.smithi107:mds.a> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T04:17:26.057 DEBUG:teuthology.orchestra.run.smithi107:mds.d> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T04:17:26.102 DEBUG:teuthology.orchestra.run.smithi107:mds.g> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T04:17:26.148 DEBUG:teuthology.orchestra.run.smithi107:mds.j> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T04:17:26.193 DEBUG:teuthology.orchestra.run.smithi115:mds.b> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T04:17:26.195 DEBUG:teuthology.orchestra.run.smithi115:mds.e> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T04:17:26.238 DEBUG:teuthology.orchestra.run.smithi115:mds.h> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T04:17:26.281 DEBUG:teuthology.orchestra.run.smithi115:mds.k> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T04:17:26.324 DEBUG:teuthology.orchestra.run.smithi145:mds.c> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T04:17:26.326 DEBUG:teuthology.orchestra.run.smithi145:mds.f> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T04:17:26.347 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:17:26.347 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-09-19T04:17:26.347 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:26.347 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:26.347 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:26.347 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:26.347 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:26.347 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:26.348 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:26.348 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:26.348 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: Deploying daemon mds.l on smithi145 2024-09-19T04:17:26.348 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:26 smithi145 ceph-mon[35096]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:26.349 DEBUG:teuthology.orchestra.run.smithi145:mds.i> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T04:17:26.391 DEBUG:teuthology.orchestra.run.smithi145:mds.l> sudo journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T04:17:26.434 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T04:17:26.434 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:17:26.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:17:26.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-09-19T04:17:26.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:26.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:26.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:26.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:26.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:26.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:26.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:26.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:26.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: Deploying daemon mds.l on smithi145 2024-09-19T04:17:26.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:26 smithi107 ceph-mon[30659]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:26.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:17:26.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-09-19T04:17:26.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:17:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:17:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: Deploying daemon mds.l on smithi145 2024-09-19T04:17:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:26 smithi115 ceph-mon[35477]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:26.795 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:27.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.327+0000 7fa694400640 1 -- 172.21.15.107:0/2003826673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa68c102790 msgr2=0x7fa68c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.327+0000 7fa694400640 1 --2- 172.21.15.107:0/2003826673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa68c102790 0x7fa68c1067e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa678009900 tx=0x7fa67802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.327+0000 7fa694400640 1 -- 172.21.15.107:0/2003826673 shutdown_connections 2024-09-19T04:17:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.327+0000 7fa694400640 1 --2- 172.21.15.107:0/2003826673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa68c106f10 0x7fa68c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.327+0000 7fa694400640 1 --2- 172.21.15.107:0/2003826673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa68c102790 0x7fa68c1067e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.327+0000 7fa694400640 1 --2- 172.21.15.107:0/2003826673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa68c101de0 0x7fa68c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.327+0000 7fa694400640 1 -- 172.21.15.107:0/2003826673 >> 172.21.15.107:0/2003826673 conn(0x7fa68c0fd8c0 msgr2=0x7fa68c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.328+0000 7fa694400640 1 -- 172.21.15.107:0/2003826673 shutdown_connections 2024-09-19T04:17:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.328+0000 7fa694400640 1 -- 172.21.15.107:0/2003826673 wait complete. 2024-09-19T04:17:27.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.328+0000 7fa694400640 1 Processor -- start 2024-09-19T04:17:27.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa694400640 1 -- start start 2024-09-19T04:17:27.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa694400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa68c101de0 0x7fa68c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa694400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa68c102790 0x7fa68c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa694400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa68c106f10 0x7fa68c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa694400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c111fc0 con 0x7fa68c106f10 2024-09-19T04:17:27.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa694400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c111e40 con 0x7fa68c101de0 2024-09-19T04:17:27.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa694400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c112140 con 0x7fa68c102790 2024-09-19T04:17:27.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa691a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa68c101de0 0x7fa68c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa691000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa68c102790 0x7fa68c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa692400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa68c106f10 0x7fa68c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa691a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa68c101de0 0x7fa68c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57332/0 (socket says 172.21.15.107:57332) 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa691000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa68c102790 0x7fa68c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:48716/0 (socket says 172.21.15.107:48716) 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.329+0000 7fa691a00640 1 -- 172.21.15.107:0/781589238 learned_addr learned my addr 172.21.15.107:0/781589238 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa692400640 1 -- 172.21.15.107:0/781589238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa68c102790 msgr2=0x7fa68c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa692400640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa68c102790 0x7fa68c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa692400640 1 -- 172.21.15.107:0/781589238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa68c101de0 msgr2=0x7fa68c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa692400640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa68c101de0 0x7fa68c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa692400640 1 -- 172.21.15.107:0/781589238 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68c19d450 con 0x7fa68c106f10 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa691000640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa68c102790 0x7fa68c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa691a00640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa68c101de0 0x7fa68c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:27.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa692400640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa68c106f10 0x7fa68c19cd70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa68400e940 tx=0x7fa68400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:27.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa68a000640 1 -- 172.21.15.107:0/781589238 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa684004d20 con 0x7fa68c106f10 2024-09-19T04:17:27.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa68a000640 1 -- 172.21.15.107:0/781589238 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa684005020 con 0x7fa68c106f10 2024-09-19T04:17:27.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa694400640 1 -- 172.21.15.107:0/781589238 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68c19d740 con 0x7fa68c106f10 2024-09-19T04:17:27.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.330+0000 7fa68a000640 1 -- 172.21.15.107:0/781589238 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6840116b0 con 0x7fa68c106f10 2024-09-19T04:17:27.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.331+0000 7fa694400640 1 -- 172.21.15.107:0/781589238 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa68c1a5020 con 0x7fa68c106f10 2024-09-19T04:17:27.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.332+0000 7fa694400640 1 -- 172.21.15.107:0/781589238 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa68c1032c0 con 0x7fa68c106f10 2024-09-19T04:17:27.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.334+0000 7fa68a000640 1 -- 172.21.15.107:0/781589238 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa6840054d0 con 0x7fa68c106f10 2024-09-19T04:17:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.335+0000 7fa68a000640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa6740a32c0 0x7fa6740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.335+0000 7fa68a000640 1 -- 172.21.15.107:0/781589238 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fa6840cf640 con 0x7fa68c106f10 2024-09-19T04:17:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.335+0000 7fa691a00640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa6740a32c0 0x7fa6740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.336+0000 7fa691a00640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa6740a32c0 0x7fa6740a5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa680005fd0 tx=0x7fa68000b480 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.338+0000 7fa68a000640 1 -- 172.21.15.107:0/781589238 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa684016030 con 0x7fa68c106f10 2024-09-19T04:17:27.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.548+0000 7fa694400640 1 -- 172.21.15.107:0/781589238 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa68c1021c0 con 0x7fa68c106f10 2024-09-19T04:17:27.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.548+0000 7fa68a000640 1 -- 172.21.15.107:0/781589238 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fa68c1021c0 con 0x7fa68c106f10 2024-09-19T04:17:27.550 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:27.550 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":1,"btime":"2024-09-19T04:10:05:820337+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T04:17:27.551 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 1 2024-09-19T04:17:27.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.550+0000 7fa694400640 1 -- 172.21.15.107:0/781589238 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa6740a32c0 msgr2=0x7fa6740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.550+0000 7fa694400640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa6740a32c0 0x7fa6740a5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa680005fd0 tx=0x7fa68000b480 comp rx=0 tx=0).stop 2024-09-19T04:17:27.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.551+0000 7fa694400640 1 -- 172.21.15.107:0/781589238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa68c106f10 msgr2=0x7fa68c19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:27.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.551+0000 7fa694400640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa68c106f10 0x7fa68c19cd70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa68400e940 tx=0x7fa68400ee00 comp rx=0 tx=0).stop 2024-09-19T04:17:27.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.551+0000 7fa694400640 1 -- 172.21.15.107:0/781589238 shutdown_connections 2024-09-19T04:17:27.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.551+0000 7fa694400640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa6740a32c0 0x7fa6740a5780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.551+0000 7fa694400640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa68c106f10 0x7fa68c19cd70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.551+0000 7fa694400640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa68c102790 0x7fa68c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.551+0000 7fa694400640 1 --2- 172.21.15.107:0/781589238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa68c101de0 0x7fa68c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:27.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.551+0000 7fa694400640 1 -- 172.21.15.107:0/781589238 >> 172.21.15.107:0/781589238 conn(0x7fa68c0fd8c0 msgr2=0x7fa68c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:27.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.551+0000 7fa694400640 1 -- 172.21.15.107:0/781589238 shutdown_connections 2024-09-19T04:17:27.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:27.551+0000 7fa694400640 1 -- 172.21.15.107:0/781589238 wait complete. 2024-09-19T04:17:27.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:27 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/781589238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:27.906 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:27 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/781589238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:27.906 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:17:27 smithi145 systemd[1]: Starting Ceph mds.l for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:17:27.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:27 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/781589238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:28.171 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:17:27 smithi145 podman[64562]: 2024-09-19 04:17:27.807554251 +0000 UTC m=+0.015685634 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:17:28.172 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:17:27 smithi145 podman[64562]: 2024-09-19 04:17:27.908256658 +0000 UTC m=+0.116388032 container create 59f8105a5daa102163b90fb0d2b3653ad4e6aa053faf174b43f2029921cd73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T04:17:28.172 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:17:28 smithi145 podman[64562]: 2024-09-19 04:17:28.14420225 +0000 UTC m=+0.352333624 container init 59f8105a5daa102163b90fb0d2b3653ad4e6aa053faf174b43f2029921cd73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:17:28.172 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:17:28 smithi145 podman[64562]: 2024-09-19 04:17:28.147301306 +0000 UTC m=+0.355432681 container start 59f8105a5daa102163b90fb0d2b3653ad4e6aa053faf174b43f2029921cd73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T04:17:28.474 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mds[64581]: -- 172.21.15.145:0/4271318317 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c52793c20 con 0x556c52770800 2024-09-19T04:17:28.474 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l[64577]: starting mds.l at 2024-09-19T04:17:28.475 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:17:28 smithi145 bash[64562]: 59f8105a5daa102163b90fb0d2b3653ad4e6aa053faf174b43f2029921cd73f1 2024-09-19T04:17:28.475 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:17:28 smithi145 systemd[1]: Started Ceph mds.l for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:28.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:28 smithi115 ceph-mon[35477]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:28.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:28 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6832/2248561847,v1:172.21.15.145:6833/2248561847] up:boot 2024-09-19T04:17:28.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:28 smithi115 ceph-mon[35477]: fsmap 1 up:standby 2024-09-19T04:17:28.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:17:28.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:28.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:28.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:28.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:28.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:28.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:28 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:28.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mon[35096]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:28.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6832/2248561847,v1:172.21.15.145:6833/2248561847] up:boot 2024-09-19T04:17:28.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mon[35096]: fsmap 1 up:standby 2024-09-19T04:17:28.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:17:28.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:28.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:28.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:28.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:28.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:28.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:28 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:28.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:28 smithi107 ceph-mon[30659]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:28.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:28 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6832/2248561847,v1:172.21.15.145:6833/2248561847] up:boot 2024-09-19T04:17:28.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:28 smithi107 ceph-mon[30659]: fsmap 1 up:standby 2024-09-19T04:17:28.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:17:28.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:28.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:28.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:28.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:28.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:28.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:28 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:29.900 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:29 smithi115 ceph-mon[35477]: Deploying daemon mds.k on smithi115 2024-09-19T04:17:29.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:29 smithi145 ceph-mon[35096]: Deploying daemon mds.k on smithi115 2024-09-19T04:17:29.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:29 smithi107 ceph-mon[30659]: Deploying daemon mds.k on smithi115 2024-09-19T04:17:30.322 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:17:30.446 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:17:30 smithi115 systemd[1]: Starting Ceph mds.k for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:17:30.698 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:30.831 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:30 smithi115 ceph-mon[35477]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:30.832 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:17:30 smithi115 podman[64629]: 2024-09-19 04:17:30.467645858 +0000 UTC m=+0.015965459 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:17:30.832 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:17:30 smithi115 podman[64629]: 2024-09-19 04:17:30.603647328 +0000 UTC m=+0.151966924 container create f6e1855e54d27240a15cca636d2406415a8f64c60cff6384ffdd353497f48b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-19T04:17:30.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:30 smithi145 ceph-mon[35096]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:30.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:30 smithi107 ceph-mon[30659]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:31.196 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:17:30 smithi115 podman[64629]: 2024-09-19 04:17:30.969742137 +0000 UTC m=+0.518061734 container init f6e1855e54d27240a15cca636d2406415a8f64c60cff6384ffdd353497f48b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:17:31.197 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:17:30 smithi115 podman[64629]: 2024-09-19 04:17:30.972844771 +0000 UTC m=+0.521164367 container start f6e1855e54d27240a15cca636d2406415a8f64c60cff6384ffdd353497f48b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T04:17:31.197 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:17:30 smithi115 ceph-mds[64648]: -- 172.21.15.115:0/2809928682 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c418f9d860 con 0x55c418f7b400 2024-09-19T04:17:31.197 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:17:30 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k[64644]: starting mds.k at 2024-09-19T04:17:31.197 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:17:31 smithi115 bash[64629]: f6e1855e54d27240a15cca636d2406415a8f64c60cff6384ffdd353497f48b11 2024-09-19T04:17:31.197 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:17:31 smithi115 systemd[1]: Started Ceph mds.k for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:31.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.243+0000 7f3f88400640 1 -- 172.21.15.107:0/1512462928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80101de0 msgr2=0x7f3f801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:31.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.243+0000 7f3f88400640 1 --2- 172.21.15.107:0/1512462928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80101de0 0x7f3f801021c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009900 tx=0x7f3f7402a410 comp rx=0 tx=0).stop 2024-09-19T04:17:31.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.243+0000 7f3f88400640 1 -- 172.21.15.107:0/1512462928 shutdown_connections 2024-09-19T04:17:31.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.243+0000 7f3f88400640 1 --2- 172.21.15.107:0/1512462928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f80106f10 0x7f3f8010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:31.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.243+0000 7f3f88400640 1 --2- 172.21.15.107:0/1512462928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f80102790 0x7f3f801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:31.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.243+0000 7f3f88400640 1 --2- 172.21.15.107:0/1512462928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80101de0 0x7f3f801021c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:31.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.243+0000 7f3f88400640 1 -- 172.21.15.107:0/1512462928 >> 172.21.15.107:0/1512462928 conn(0x7f3f800fd8c0 msgr2=0x7f3f800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:31.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.244+0000 7f3f88400640 1 -- 172.21.15.107:0/1512462928 shutdown_connections 2024-09-19T04:17:31.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.244+0000 7f3f88400640 1 -- 172.21.15.107:0/1512462928 wait complete. 2024-09-19T04:17:31.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.245+0000 7f3f88400640 1 Processor -- start 2024-09-19T04:17:31.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.245+0000 7f3f88400640 1 -- start start 2024-09-19T04:17:31.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.245+0000 7f3f88400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80101de0 0x7f3f8019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:31.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.245+0000 7f3f88400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f80102790 0x7f3f8019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:31.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f88400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f80106f10 0x7f3f8019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:31.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f88400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80111f80 con 0x7f3f80101de0 2024-09-19T04:17:31.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f88400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80111e00 con 0x7f3f80106f10 2024-09-19T04:17:31.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f88400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80112100 con 0x7f3f80102790 2024-09-19T04:17:31.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f85a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80101de0 0x7f3f8019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:31.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f85a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80101de0 0x7f3f8019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48512/0 (socket says 172.21.15.107:48512) 2024-09-19T04:17:31.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f85a00640 1 -- 172.21.15.107:0/94436607 learned_addr learned my addr 172.21.15.107:0/94436607 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:31.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f85000640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f80102790 0x7f3f8019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:31.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f86400640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f80106f10 0x7f3f8019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:31.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f85a00640 1 -- 172.21.15.107:0/94436607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f80102790 msgr2=0x7f3f8019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:31.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f85a00640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f80102790 0x7f3f8019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:31.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f85a00640 1 -- 172.21.15.107:0/94436607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f80106f10 msgr2=0x7f3f8019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:31.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f85a00640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f80106f10 0x7f3f8019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:31.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.246+0000 7f3f85a00640 1 -- 172.21.15.107:0/94436607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8019f620 con 0x7f3f80101de0 2024-09-19T04:17:31.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.247+0000 7f3f85a00640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80101de0 0x7f3f8019a670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3f74004770 tx=0x7f3f74003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:31.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.247+0000 7f3f7e000640 1 -- 172.21.15.107:0/94436607 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7403f070 con 0x7f3f80101de0 2024-09-19T04:17:31.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.247+0000 7f3f7e000640 1 -- 172.21.15.107:0/94436607 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f74031ca0 con 0x7f3f80101de0 2024-09-19T04:17:31.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.247+0000 7f3f7e000640 1 -- 172.21.15.107:0/94436607 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74035030 con 0x7f3f80101de0 2024-09-19T04:17:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.248+0000 7f3f88400640 1 -- 172.21.15.107:0/94436607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8019f8b0 con 0x7f3f80101de0 2024-09-19T04:17:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.248+0000 7f3f88400640 1 -- 172.21.15.107:0/94436607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8019fd40 con 0x7f3f80101de0 2024-09-19T04:17:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.249+0000 7f3f88400640 1 -- 172.21.15.107:0/94436607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f801021c0 con 0x7f3f80101de0 2024-09-19T04:17:31.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.251+0000 7f3f7e000640 1 -- 172.21.15.107:0/94436607 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3f74042070 con 0x7f3f80101de0 2024-09-19T04:17:31.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.252+0000 7f3f7e000640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3f680a3240 0x7f3f680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:31.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.253+0000 7f3f7e000640 1 -- 172.21.15.107:0/94436607 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f3f740f00b0 con 0x7f3f80101de0 2024-09-19T04:17:31.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.253+0000 7f3f85000640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3f680a3240 0x7f3f680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:31.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.254+0000 7f3f85000640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3f680a3240 0x7f3f680a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3f800ff740 tx=0x7f3f6c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:31.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.255+0000 7f3f7e000640 1 -- 172.21.15.107:0/94436607 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f740bcc40 con 0x7f3f80101de0 2024-09-19T04:17:31.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.470+0000 7f3f88400640 1 -- 172.21.15.107:0/94436607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f801a7320 con 0x7f3f80101de0 2024-09-19T04:17:31.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.470+0000 7f3f7e000640 1 -- 172.21.15.107:0/94436607 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f3f740c0f40 con 0x7f3f80101de0 2024-09-19T04:17:31.472 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:31.472 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":2,"btime":"2024-09-19T04:17:28:175593+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:17:31.472 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 2 2024-09-19T04:17:31.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 -- 172.21.15.107:0/94436607 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3f680a3240 msgr2=0x7f3f680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3f680a3240 0x7f3f680a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3f800ff740 tx=0x7f3f6c005f90 comp rx=0 tx=0).stop 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 -- 172.21.15.107:0/94436607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80101de0 msgr2=0x7f3f8019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80101de0 0x7f3f8019a670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3f74004770 tx=0x7f3f74003c10 comp rx=0 tx=0).stop 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 -- 172.21.15.107:0/94436607 shutdown_connections 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3f680a3240 0x7f3f680a5700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f80106f10 0x7f3f8019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f80102790 0x7f3f8019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 --2- 172.21.15.107:0/94436607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80101de0 0x7f3f8019a670 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 -- 172.21.15.107:0/94436607 >> 172.21.15.107:0/94436607 conn(0x7f3f800fd8c0 msgr2=0x7f3f800feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 -- 172.21.15.107:0/94436607 shutdown_connections 2024-09-19T04:17:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:31.474+0000 7f3f88400640 1 -- 172.21.15.107:0/94436607 wait complete. 2024-09-19T04:17:32.383 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:32.383 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:32.383 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:32.383 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:32.383 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:32.383 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:32.383 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: Deploying daemon mds.j on smithi107 2024-09-19T04:17:32.383 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/94436607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:32.383 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.115:6832/1661283120,v1:172.21.15.115:6833/1661283120] up:boot 2024-09-19T04:17:32.384 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: fsmap 2 up:standby 2024-09-19T04:17:32.384 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:17:32.384 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:32 smithi115 ceph-mon[35477]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:32.412 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:32.412 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:32.412 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:32.412 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:32.412 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:32.412 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:32.412 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: Deploying daemon mds.j on smithi107 2024-09-19T04:17:32.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/94436607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:32.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.115:6832/1661283120,v1:172.21.15.115:6833/1661283120] up:boot 2024-09-19T04:17:32.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: fsmap 2 up:standby 2024-09-19T04:17:32.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:17:32.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:32 smithi107 ceph-mon[30659]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:32.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:32.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:32.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:32.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:32.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:32.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:32.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: Deploying daemon mds.j on smithi107 2024-09-19T04:17:32.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/94436607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:32.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.115:6832/1661283120,v1:172.21.15.115:6833/1661283120] up:boot 2024-09-19T04:17:32.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: fsmap 2 up:standby 2024-09-19T04:17:32.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:17:32.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:32 smithi145 ceph-mon[35096]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:34.257 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:17:34.719 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:17:34 smithi107 podman[70236]: 2024-09-19 04:17:34.711038704 +0000 UTC m=+0.372984068 container init 2b706c582c8015970dda86c97f6f6ec40e36fbc3cfa6999f03f71765ee1f555b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T04:17:34.719 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:17:34 smithi107 podman[70236]: 2024-09-19 04:17:34.71748729 +0000 UTC m=+0.379432654 container start 2b706c582c8015970dda86c97f6f6ec40e36fbc3cfa6999f03f71765ee1f555b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T04:17:34.906 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:34.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:34 smithi107 ceph-mon[30659]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:34.981 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:17:34 smithi107 ceph-mds[70278]: -- 172.21.15.107:0/107189286 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569e9f35c20 con 0x5569e9f13000 2024-09-19T04:17:34.981 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:17:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-j[70262]: starting mds.j at 2024-09-19T04:17:34.981 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:17:34 smithi107 bash[70236]: 2b706c582c8015970dda86c97f6f6ec40e36fbc3cfa6999f03f71765ee1f555b 2024-09-19T04:17:34.981 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:17:34 smithi107 systemd[1]: Started Ceph mds.j for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:34 smithi115 ceph-mon[35477]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:34 smithi145 ceph-mon[35096]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.462+0000 7f2c88e00640 1 -- 172.21.15.107:0/633678786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c84106f20 msgr2=0x7f2c8410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.462+0000 7f2c88e00640 1 --2- 172.21.15.107:0/633678786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c84106f20 0x7f2c8410eaa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009ca0 tx=0x7f2c7c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.462+0000 7f2c88e00640 1 -- 172.21.15.107:0/633678786 shutdown_connections 2024-09-19T04:17:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.462+0000 7f2c88e00640 1 --2- 172.21.15.107:0/633678786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c84106f20 0x7f2c8410eaa0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.462+0000 7f2c88e00640 1 --2- 172.21.15.107:0/633678786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c841027a0 0x7f2c841067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.462+0000 7f2c88e00640 1 --2- 172.21.15.107:0/633678786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84101df0 0x7f2c841021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.462+0000 7f2c88e00640 1 -- 172.21.15.107:0/633678786 >> 172.21.15.107:0/633678786 conn(0x7f2c840fd8b0 msgr2=0x7f2c840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.462+0000 7f2c88e00640 1 -- 172.21.15.107:0/633678786 shutdown_connections 2024-09-19T04:17:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.462+0000 7f2c88e00640 1 -- 172.21.15.107:0/633678786 wait complete. 2024-09-19T04:17:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.463+0000 7f2c88e00640 1 Processor -- start 2024-09-19T04:17:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.463+0000 7f2c88e00640 1 -- start start 2024-09-19T04:17:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.463+0000 7f2c88e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84101df0 0x7f2c841983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.463+0000 7f2c88e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c841027a0 0x7f2c84198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.463+0000 7f2c88e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c84106f20 0x7f2c8419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.463+0000 7f2c88e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84111fd0 con 0x7f2c84106f20 2024-09-19T04:17:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.463+0000 7f2c88e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84111e50 con 0x7f2c84101df0 2024-09-19T04:17:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.463+0000 7f2c88e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84112150 con 0x7f2c841027a0 2024-09-19T04:17:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c81400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c841027a0 0x7f2c84198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c81e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84101df0 0x7f2c841983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c82800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c84106f20 0x7f2c8419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c81400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c841027a0 0x7f2c84198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38352/0 (socket says 172.21.15.107:38352) 2024-09-19T04:17:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c81400640 1 -- 172.21.15.107:0/1091098205 learned_addr learned my addr 172.21.15.107:0/1091098205 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c81400640 1 -- 172.21.15.107:0/1091098205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84101df0 msgr2=0x7f2c841983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c81400640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84101df0 0x7f2c841983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c81400640 1 -- 172.21.15.107:0/1091098205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c84106f20 msgr2=0x7f2c8419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c81400640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c84106f20 0x7f2c8419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c81400640 1 -- 172.21.15.107:0/1091098205 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8419d380 con 0x7f2c841027a0 2024-09-19T04:17:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.464+0000 7f2c81e00640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84101df0 0x7f2c841983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.465+0000 7f2c81400640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c841027a0 0x7f2c84198910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000e9f0 tx=0x7f2c7000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.465+0000 7f2c7aa00640 1 -- 172.21.15.107:0/1091098205 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000cd80 con 0x7f2c841027a0 2024-09-19T04:17:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.465+0000 7f2c88e00640 1 -- 172.21.15.107:0/1091098205 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8419d670 con 0x7f2c841027a0 2024-09-19T04:17:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.465+0000 7f2c7aa00640 1 -- 172.21.15.107:0/1091098205 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c70010910 con 0x7f2c841027a0 2024-09-19T04:17:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.465+0000 7f2c7aa00640 1 -- 172.21.15.107:0/1091098205 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c700188f0 con 0x7f2c841027a0 2024-09-19T04:17:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.466+0000 7f2c88e00640 1 -- 172.21.15.107:0/1091098205 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841a4f50 con 0x7f2c841027a0 2024-09-19T04:17:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.467+0000 7f2c88e00640 1 -- 172.21.15.107:0/1091098205 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c841021d0 con 0x7f2c841027a0 2024-09-19T04:17:35.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.468+0000 7f2c7aa00640 1 -- 172.21.15.107:0/1091098205 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2c70010ac0 con 0x7f2c841027a0 2024-09-19T04:17:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.469+0000 7f2c7aa00640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2c680a3240 0x7f2c680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.470+0000 7f2c7aa00640 1 -- 172.21.15.107:0/1091098205 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f2c700cf830 con 0x7f2c841027a0 2024-09-19T04:17:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.470+0000 7f2c81e00640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2c680a3240 0x7f2c680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.470+0000 7f2c81e00640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2c680a3240 0x7f2c680a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009fc0 tx=0x7f2c74005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.473+0000 7f2c7aa00640 1 -- 172.21.15.107:0/1091098205 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2c7009c300 con 0x7f2c841027a0 2024-09-19T04:17:35.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.703+0000 7f2c88e00640 1 -- 172.21.15.107:0/1091098205 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c841a5120 con 0x7f2c841027a0 2024-09-19T04:17:35.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.704+0000 7f2c7aa00640 1 -- 172.21.15.107:0/1091098205 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f2c700a0600 con 0x7f2c841027a0 2024-09-19T04:17:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:35.706 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":4,"btime":"2024-09-19T04:17:34:851449+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:17:35.707 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 4 2024-09-19T04:17:35.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 -- 172.21.15.107:0/1091098205 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2c680a3240 msgr2=0x7f2c680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2c680a3240 0x7f2c680a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009fc0 tx=0x7f2c74005f90 comp rx=0 tx=0).stop 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 -- 172.21.15.107:0/1091098205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c841027a0 msgr2=0x7f2c84198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c841027a0 0x7f2c84198910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000e9f0 tx=0x7f2c7000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 -- 172.21.15.107:0/1091098205 shutdown_connections 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2c680a3240 0x7f2c680a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c84106f20 0x7f2c8419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c841027a0 0x7f2c84198910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 --2- 172.21.15.107:0/1091098205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84101df0 0x7f2c841983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 -- 172.21.15.107:0/1091098205 >> 172.21.15.107:0/1091098205 conn(0x7f2c840fd8b0 msgr2=0x7f2c840fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 -- 172.21.15.107:0/1091098205 shutdown_connections 2024-09-19T04:17:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:35.707+0000 7f2c88e00640 1 -- 172.21.15.107:0/1091098205 wait complete. 2024-09-19T04:17:35.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.107:6834/3986650541,v1:172.21.15.107:6835/3986650541] up:boot 2024-09-19T04:17:35.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: fsmap 3 up:standby 2024-09-19T04:17:35.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:17:35.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:35.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:35.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:35.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:35.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:35.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:35.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: Deploying daemon mds.d on smithi107 2024-09-19T04:17:35.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:35 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1091098205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:36.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.107:6834/3986650541,v1:172.21.15.107:6835/3986650541] up:boot 2024-09-19T04:17:36.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: fsmap 3 up:standby 2024-09-19T04:17:36.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:17:36.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:36.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:36.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:36.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:36.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:36.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:36.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: Deploying daemon mds.d on smithi107 2024-09-19T04:17:36.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:35 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1091098205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:36.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.107:6834/3986650541,v1:172.21.15.107:6835/3986650541] up:boot 2024-09-19T04:17:36.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: fsmap 3 up:standby 2024-09-19T04:17:36.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:17:36.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:36.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:36.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:36.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:36.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:36.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:36.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: Deploying daemon mds.d on smithi107 2024-09-19T04:17:36.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:35 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1091098205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:37.137 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:36 smithi107 ceph-mon[30659]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:36 smithi115 ceph-mon[35477]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:36 smithi145 ceph-mon[35096]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:38.419 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:17:38 smithi107 systemd[1]: Starting Ceph mds.d for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:17:38.535 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:17:38.676 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:17:38 smithi107 podman[70630]: 2024-09-19 04:17:38.322771004 +0000 UTC m=+0.026423613 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:17:38.676 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:17:38 smithi107 podman[70630]: 2024-09-19 04:17:38.438632838 +0000 UTC m=+0.142285431 container create 937c61de8fef775e232ebc5b794a9c8cbd27c4f25e167fde361900839d77fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T04:17:38.676 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:17:38 smithi107 podman[70630]: 2024-09-19 04:17:38.666025577 +0000 UTC m=+0.369678190 container init 937c61de8fef775e232ebc5b794a9c8cbd27c4f25e167fde361900839d77fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T04:17:38.676 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:17:38 smithi107 podman[70630]: 2024-09-19 04:17:38.674461545 +0000 UTC m=+0.378114134 container start 937c61de8fef775e232ebc5b794a9c8cbd27c4f25e167fde361900839d77fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T04:17:38.929 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:38 smithi107 ceph-mon[30659]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:38 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:38.981 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:17:38 smithi107 ceph-mds[70667]: -- 172.21.15.107:0/3409098729 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dfbfdda680 con 0x55dfbfdae800 2024-09-19T04:17:38.981 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:17:38 smithi107 ceph-mds[70667]: -- 172.21.15.107:0/3409098729 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dfbfdd1c20 con 0x55dfbfdae800 2024-09-19T04:17:38.982 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:17:38 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-d[70649]: starting mds.d at 2024-09-19T04:17:38.982 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:17:38 smithi107 bash[70630]: 937c61de8fef775e232ebc5b794a9c8cbd27c4f25e167fde361900839d77fba1 2024-09-19T04:17:38.982 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:17:38 smithi107 systemd[1]: Started Ceph mds.d for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:38 smithi115 ceph-mon[35477]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:38 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:38 smithi145 ceph-mon[35096]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:38 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.489+0000 7f1256c00640 1 -- 172.21.15.107:0/2948256272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1250102790 msgr2=0x7f12501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.489+0000 7f1256c00640 1 --2- 172.21.15.107:0/2948256272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1250102790 0x7f12501067e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f123c009900 tx=0x7f123c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.489+0000 7f1256c00640 1 -- 172.21.15.107:0/2948256272 shutdown_connections 2024-09-19T04:17:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.489+0000 7f1256c00640 1 --2- 172.21.15.107:0/2948256272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1250106f10 0x7f125010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.489+0000 7f1256c00640 1 --2- 172.21.15.107:0/2948256272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1250102790 0x7f12501067e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.489+0000 7f1256c00640 1 --2- 172.21.15.107:0/2948256272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1250101de0 0x7f12501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.489+0000 7f1256c00640 1 -- 172.21.15.107:0/2948256272 >> 172.21.15.107:0/2948256272 conn(0x7f12500fd8c0 msgr2=0x7f12500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.489+0000 7f1256c00640 1 -- 172.21.15.107:0/2948256272 shutdown_connections 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.489+0000 7f1256c00640 1 -- 172.21.15.107:0/2948256272 wait complete. 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.490+0000 7f1256c00640 1 Processor -- start 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.490+0000 7f1256c00640 1 -- start start 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.490+0000 7f1256c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1250101de0 0x7f1250198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.490+0000 7f1256c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1250102790 0x7f1250198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.490+0000 7f1256c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1250106f10 0x7f125019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.490+0000 7f1256c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1250111fc0 con 0x7f1250106f10 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.490+0000 7f1256c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1250111e40 con 0x7f1250101de0 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.490+0000 7f1256c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1250112140 con 0x7f1250102790 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f1254c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1250106f10 0x7f125019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f124fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1250101de0 0x7f1250198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f1254c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1250106f10 0x7f125019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57988/0 (socket says 172.21.15.107:57988) 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f1254c00640 1 -- 172.21.15.107:0/1908830749 learned_addr learned my addr 172.21.15.107:0/1908830749 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f124fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1250101de0 0x7f1250198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44652/0 (socket says 172.21.15.107:44652) 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f124f400640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1250102790 0x7f1250198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f1254c00640 1 -- 172.21.15.107:0/1908830749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1250102790 msgr2=0x7f1250198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f1254c00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1250102790 0x7f1250198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f1254c00640 1 -- 172.21.15.107:0/1908830749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1250101de0 msgr2=0x7f1250198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f1254c00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1250101de0 0x7f1250198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f1254c00640 1 -- 172.21.15.107:0/1908830749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125019d3d0 con 0x7f1250106f10 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f124fe00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1250101de0 0x7f1250198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f1254c00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1250106f10 0x7f125019ccf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f124800eb10 tx=0x7f124800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f124cc00640 1 -- 172.21.15.107:0/1908830749 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248004b20 con 0x7f1250106f10 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f124cc00640 1 -- 172.21.15.107:0/1908830749 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1248004cc0 con 0x7f1250106f10 2024-09-19T04:17:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.491+0000 7f1256c00640 1 -- 172.21.15.107:0/1908830749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125019d6c0 con 0x7f1250106f10 2024-09-19T04:17:39.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.492+0000 7f124cc00640 1 -- 172.21.15.107:0/1908830749 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248010760 con 0x7f1250106f10 2024-09-19T04:17:39.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.492+0000 7f1256c00640 1 -- 172.21.15.107:0/1908830749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1250074420 con 0x7f1250106f10 2024-09-19T04:17:39.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.493+0000 7f1256c00640 1 -- 172.21.15.107:0/1908830749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12501032c0 con 0x7f1250106f10 2024-09-19T04:17:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.495+0000 7f124cc00640 1 -- 172.21.15.107:0/1908830749 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1248005020 con 0x7f1250106f10 2024-09-19T04:17:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.496+0000 7f124cc00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f12340a3240 0x7f12340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.496+0000 7f124cc00640 1 -- 172.21.15.107:0/1908830749 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f12480cfb50 con 0x7f1250106f10 2024-09-19T04:17:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.497+0000 7f124fe00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f12340a3240 0x7f12340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.497+0000 7f124fe00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f12340a3240 0x7f12340a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f12400096a0 tx=0x7f1240012040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:39.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.500+0000 7f124cc00640 1 -- 172.21.15.107:0/1908830749 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f124809cc70 con 0x7f1250106f10 2024-09-19T04:17:39.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.708+0000 7f1256c00640 1 -- 172.21.15.107:0/1908830749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f125010a0f0 con 0x7f1250106f10 2024-09-19T04:17:39.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.709+0000 7f124cc00640 1 -- 172.21.15.107:0/1908830749 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7f12480bc030 con 0x7f1250106f10 2024-09-19T04:17:39.711 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:39.711 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":5,"btime":"2024-09-19T04:17:38:910610+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:17:39.711 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 5 2024-09-19T04:17:39.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.711+0000 7f1256c00640 1 -- 172.21.15.107:0/1908830749 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f12340a3240 msgr2=0x7f12340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.711+0000 7f1256c00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f12340a3240 0x7f12340a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f12400096a0 tx=0x7f1240012040 comp rx=0 tx=0).stop 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.711+0000 7f1256c00640 1 -- 172.21.15.107:0/1908830749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1250106f10 msgr2=0x7f125019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.711+0000 7f1256c00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1250106f10 0x7f125019ccf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f124800eb10 tx=0x7f124800c2d0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.711+0000 7f1256c00640 1 -- 172.21.15.107:0/1908830749 shutdown_connections 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.711+0000 7f1256c00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f12340a3240 0x7f12340a5700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.711+0000 7f1256c00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1250106f10 0x7f125019ccf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.711+0000 7f1256c00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1250102790 0x7f1250198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.711+0000 7f1256c00640 1 --2- 172.21.15.107:0/1908830749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1250101de0 0x7f1250198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.711+0000 7f1256c00640 1 -- 172.21.15.107:0/1908830749 >> 172.21.15.107:0/1908830749 conn(0x7f12500fd8c0 msgr2=0x7f12500fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.712+0000 7f1256c00640 1 -- 172.21.15.107:0/1908830749 shutdown_connections 2024-09-19T04:17:39.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:39.712+0000 7f1256c00640 1 -- 172.21.15.107:0/1908830749 wait complete. 2024-09-19T04:17:39.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:39 smithi107 ceph-mon[30659]: Deploying daemon mds.e on smithi115 2024-09-19T04:17:39.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:39 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.107:6836/2692745433,v1:172.21.15.107:6837/2692745433] up:boot 2024-09-19T04:17:39.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:39 smithi107 ceph-mon[30659]: fsmap 4 up:standby 2024-09-19T04:17:39.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:39 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:17:39.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:39 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1908830749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:40.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:39 smithi115 ceph-mon[35477]: Deploying daemon mds.e on smithi115 2024-09-19T04:17:40.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:39 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.107:6836/2692745433,v1:172.21.15.107:6837/2692745433] up:boot 2024-09-19T04:17:40.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:39 smithi115 ceph-mon[35477]: fsmap 4 up:standby 2024-09-19T04:17:40.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:39 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:17:40.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:39 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1908830749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:39 smithi145 ceph-mon[35096]: Deploying daemon mds.e on smithi115 2024-09-19T04:17:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:39 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.107:6836/2692745433,v1:172.21.15.107:6837/2692745433] up:boot 2024-09-19T04:17:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:39 smithi145 ceph-mon[35096]: fsmap 4 up:standby 2024-09-19T04:17:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:39 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:17:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:39 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1908830749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:41.162 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:40 smithi115 ceph-mon[35477]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:41.162 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:17:40 smithi115 systemd[1]: Starting Ceph mds.e for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:17:41.162 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:17:41 smithi115 podman[64940]: 2024-09-19 04:17:41.062910537 +0000 UTC m=+0.015855267 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:17:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:40 smithi145 ceph-mon[35096]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:41.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:40 smithi107 ceph-mon[30659]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:41.433 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:17:41 smithi115 podman[64940]: 2024-09-19 04:17:41.188352216 +0000 UTC m=+0.141296934 container create 2e28e55b62ee6afc50789f74f93b2e31066df47283c84c1fc0ca8fbac19d36ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T04:17:41.696 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:17:41 smithi115 podman[64940]: 2024-09-19 04:17:41.433088666 +0000 UTC m=+0.386033386 container init 2e28e55b62ee6afc50789f74f93b2e31066df47283c84c1fc0ca8fbac19d36ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T04:17:41.696 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:17:41 smithi115 podman[64940]: 2024-09-19 04:17:41.435885672 +0000 UTC m=+0.388830401 container start 2e28e55b62ee6afc50789f74f93b2e31066df47283c84c1fc0ca8fbac19d36ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T04:17:41.696 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:17:41 smithi115 ceph-mds[64959]: -- 172.21.15.115:0/2872179037 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603ae3b9860 con 0x5603ae397000 2024-09-19T04:17:41.696 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:17:41 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e[64955]: starting mds.e at 2024-09-19T04:17:41.697 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:17:41 smithi115 bash[64940]: 2e28e55b62ee6afc50789f74f93b2e31066df47283c84c1fc0ca8fbac19d36ce 2024-09-19T04:17:41.697 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:17:41 smithi115 systemd[1]: Started Ceph mds.e for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:42.447 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:17:42.718 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:42.718 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:42.719 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:42.719 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:42.719 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:42.719 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:42.719 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: Deploying daemon mds.f on smithi145 2024-09-19T04:17:42.719 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:42.719 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.115:6834/687464127,v1:172.21.15.115:6835/687464127] up:boot 2024-09-19T04:17:42.719 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: fsmap 5 up:standby 2024-09-19T04:17:42.719 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:42 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:17:42.828 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:42.845 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:42.845 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:42.845 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:42.845 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:42.845 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:42.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:42.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: Deploying daemon mds.f on smithi145 2024-09-19T04:17:42.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:42.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.115:6834/687464127,v1:172.21.15.115:6835/687464127] up:boot 2024-09-19T04:17:42.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: fsmap 5 up:standby 2024-09-19T04:17:42.846 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:42 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: Deploying daemon mds.f on smithi145 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.115:6834/687464127,v1:172.21.15.115:6835/687464127] up:boot 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: fsmap 5 up:standby 2024-09-19T04:17:42.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:42 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:17:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.384+0000 7fec3c800640 1 -- 172.21.15.107:0/821716582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3410e3c0 msgr2=0x7fec34110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.384+0000 7fec3c800640 1 --2- 172.21.15.107:0/821716582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3410e3c0 0x7fec34110810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fec30009ca0 tx=0x7fec3002a940 comp rx=0 tx=0).stop 2024-09-19T04:17:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.385+0000 7fec3c800640 1 -- 172.21.15.107:0/821716582 shutdown_connections 2024-09-19T04:17:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.385+0000 7fec3c800640 1 --2- 172.21.15.107:0/821716582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec3410e3c0 0x7fec34110810 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.385+0000 7fec3c800640 1 --2- 172.21.15.107:0/821716582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec34074f40 0x7fec3410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.385+0000 7fec3c800640 1 --2- 172.21.15.107:0/821716582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec34074620 0x7fec34074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.385+0000 7fec3c800640 1 -- 172.21.15.107:0/821716582 >> 172.21.15.107:0/821716582 conn(0x7fec340fd8c0 msgr2=0x7fec340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.385+0000 7fec3c800640 1 -- 172.21.15.107:0/821716582 shutdown_connections 2024-09-19T04:17:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.385+0000 7fec3c800640 1 -- 172.21.15.107:0/821716582 wait complete. 2024-09-19T04:17:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec3c800640 1 Processor -- start 2024-09-19T04:17:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec3c800640 1 -- start start 2024-09-19T04:17:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec3c800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec34074620 0x7fec3419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec3c800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec34074f40 0x7fec3419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec3c800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec3410e3c0 0x7fec3419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec3c800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec341141a0 con 0x7fec34074f40 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec3c800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fec34114020 con 0x7fec34074620 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec3c800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fec34114320 con 0x7fec3410e3c0 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec39400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec34074f40 0x7fec3419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec39e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec34074620 0x7fec3419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec3a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec3410e3c0 0x7fec3419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec39e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec34074620 0x7fec3419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44668/0 (socket says 172.21.15.107:44668) 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.386+0000 7fec39e00640 1 -- 172.21.15.107:0/4165057684 learned_addr learned my addr 172.21.15.107:0/4165057684 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec39e00640 1 -- 172.21.15.107:0/4165057684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec3410e3c0 msgr2=0x7fec3419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec39e00640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec3410e3c0 0x7fec3419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec39e00640 1 -- 172.21.15.107:0/4165057684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec34074f40 msgr2=0x7fec3419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec39e00640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec34074f40 0x7fec3419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec39e00640 1 -- 172.21.15.107:0/4165057684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3419f5d0 con 0x7fec34074620 2024-09-19T04:17:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec3a800640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec3410e3c0 0x7fec3419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec39e00640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec34074620 0x7fec3419a620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec2800ea60 tx=0x7fec2800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec2ea00640 1 -- 172.21.15.107:0/4165057684 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2800cde0 con 0x7fec34074620 2024-09-19T04:17:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec2ea00640 1 -- 172.21.15.107:0/4165057684 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec280035f0 con 0x7fec34074620 2024-09-19T04:17:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec2ea00640 1 -- 172.21.15.107:0/4165057684 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28010670 con 0x7fec34074620 2024-09-19T04:17:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.387+0000 7fec3c800640 1 -- 172.21.15.107:0/4165057684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3419f8c0 con 0x7fec34074620 2024-09-19T04:17:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.388+0000 7fec3c800640 1 -- 172.21.15.107:0/4165057684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec341a71a0 con 0x7fec34074620 2024-09-19T04:17:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.388+0000 7fec3c800640 1 -- 172.21.15.107:0/4165057684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec34074a00 con 0x7fec34074620 2024-09-19T04:17:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.390+0000 7fec2ea00640 1 -- 172.21.15.107:0/4165057684 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fec280037a0 con 0x7fec34074620 2024-09-19T04:17:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.392+0000 7fec2ea00640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec1c0a3240 0x7fec1c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.392+0000 7fec2ea00640 1 -- 172.21.15.107:0/4165057684 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fec280cfae0 con 0x7fec34074620 2024-09-19T04:17:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.392+0000 7fec39400640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec1c0a3240 0x7fec1c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.393+0000 7fec39400640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec1c0a3240 0x7fec1c0a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec3410f6e0 tx=0x7fec24005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.394+0000 7fec2ea00640 1 -- 172.21.15.107:0/4165057684 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fec2809cc00 con 0x7fec34074620 2024-09-19T04:17:43.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.605+0000 7fec3c800640 1 -- 172.21.15.107:0/4165057684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec3419b980 con 0x7fec34074620 2024-09-19T04:17:43.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.606+0000 7fec2ea00640 1 -- 172.21.15.107:0/4165057684 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4747 (secure 0 0 0) 0x7fec2809c500 con 0x7fec34074620 2024-09-19T04:17:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:43.608 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":6,"btime":"2024-09-19T04:17:41:924864+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T04:17:43.608 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 6 2024-09-19T04:17:43.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 -- 172.21.15.107:0/4165057684 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec1c0a3240 msgr2=0x7fec1c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:43.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec1c0a3240 0x7fec1c0a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec3410f6e0 tx=0x7fec24005f90 comp rx=0 tx=0).stop 2024-09-19T04:17:43.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 -- 172.21.15.107:0/4165057684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec34074620 msgr2=0x7fec3419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:43.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec34074620 0x7fec3419a620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec2800ea60 tx=0x7fec2800ef20 comp rx=0 tx=0).stop 2024-09-19T04:17:43.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 -- 172.21.15.107:0/4165057684 shutdown_connections 2024-09-19T04:17:43.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fec1c0a3240 0x7fec1c0a5700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:43.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec3410e3c0 0x7fec3419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:43.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec34074f40 0x7fec3419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:43.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 --2- 172.21.15.107:0/4165057684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec34074620 0x7fec3419a620 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:43.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 -- 172.21.15.107:0/4165057684 >> 172.21.15.107:0/4165057684 conn(0x7fec340fd8c0 msgr2=0x7fec340fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:43.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 -- 172.21.15.107:0/4165057684 shutdown_connections 2024-09-19T04:17:43.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:43.609+0000 7fec3c800640 1 -- 172.21.15.107:0/4165057684 wait complete. 2024-09-19T04:17:43.660 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:17:43 smithi145 systemd[1]: Starting Ceph mds.f for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:17:43.925 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:43 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/4165057684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:43.925 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:17:43 smithi145 podman[64862]: 2024-09-19 04:17:43.575566594 +0000 UTC m=+0.015855557 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:17:43.925 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:17:43 smithi145 podman[64862]: 2024-09-19 04:17:43.679310419 +0000 UTC m=+0.119599379 container create 8fe700ea027eaa3af1cb476e5ec541d1c606b1f63fccf024629387aee9ba99b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:17:43.925 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:17:43 smithi145 podman[64862]: 2024-09-19 04:17:43.898671596 +0000 UTC m=+0.338960563 container init 8fe700ea027eaa3af1cb476e5ec541d1c606b1f63fccf024629387aee9ba99b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:17:43.925 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:17:43 smithi145 podman[64862]: 2024-09-19 04:17:43.901342322 +0000 UTC m=+0.341631286 container start 8fe700ea027eaa3af1cb476e5ec541d1c606b1f63fccf024629387aee9ba99b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T04:17:43.925 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:17:43 smithi145 ceph-mds[64882]: -- 172.21.15.145:0/1826997561 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af32185860 con 0x55af32163400 2024-09-19T04:17:43.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:43 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/4165057684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:43.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:43 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/4165057684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:44.224 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:17:43 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f[64878]: starting mds.f at 2024-09-19T04:17:44.224 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:17:43 smithi145 bash[64862]: 8fe700ea027eaa3af1cb476e5ec541d1c606b1f63fccf024629387aee9ba99b4 2024-09-19T04:17:44.225 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:17:43 smithi145 systemd[1]: Started Ceph mds.f for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:44.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:44 smithi145 ceph-mon[35096]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:44.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:44 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:44.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:44 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:44.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:44 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:44.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:44 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:44.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:44 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:44.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:44 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:44.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:44 smithi115 ceph-mon[35477]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:44.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:44 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:44.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:44 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:44.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:44 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:44.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:44 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:44.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:44 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:44.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:44 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:44.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:44 smithi107 ceph-mon[30659]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:44.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:44 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:44.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:44 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:44.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:44 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:44.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:44 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:44.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:44 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:44.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:44 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:45.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:45 smithi145 ceph-mon[35096]: Deploying daemon mds.c on smithi145 2024-09-19T04:17:45.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:45 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] up:boot 2024-09-19T04:17:45.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:45 smithi145 ceph-mon[35096]: fsmap 6 up:standby 2024-09-19T04:17:45.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:45 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:17:45.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:45 smithi115 ceph-mon[35477]: Deploying daemon mds.c on smithi145 2024-09-19T04:17:45.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:45 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] up:boot 2024-09-19T04:17:45.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:45 smithi115 ceph-mon[35477]: fsmap 6 up:standby 2024-09-19T04:17:45.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:45 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:17:45.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:45 smithi107 ceph-mon[30659]: Deploying daemon mds.c on smithi145 2024-09-19T04:17:45.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:45 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] up:boot 2024-09-19T04:17:45.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:45 smithi107 ceph-mon[30659]: fsmap 6 up:standby 2024-09-19T04:17:45.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:45 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:17:46.167 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:17:45 smithi145 systemd[1]: Starting Ceph mds.c for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:17:46.378 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:17:46.421 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:17:46 smithi145 podman[65165]: 2024-09-19 04:17:46.069088005 +0000 UTC m=+0.015798727 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:17:46.421 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:17:46 smithi145 podman[65165]: 2024-09-19 04:17:46.18454206 +0000 UTC m=+0.131252818 container create 1e3bf76d9ee6e563a0e3bb3a798405900868d2ec46aa42384ed30768453725b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T04:17:46.672 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:17:46 smithi145 podman[65165]: 2024-09-19 04:17:46.42047536 +0000 UTC m=+0.367186074 container init 1e3bf76d9ee6e563a0e3bb3a798405900868d2ec46aa42384ed30768453725b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T04:17:46.672 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:17:46 smithi145 podman[65165]: 2024-09-19 04:17:46.42372184 +0000 UTC m=+0.370432563 container start 1e3bf76d9ee6e563a0e3bb3a798405900868d2ec46aa42384ed30768453725b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T04:17:46.672 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:17:46 smithi145 ceph-mds[65185]: -- 172.21.15.145:0/1968656221 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa73f43860 con 0x55fa73f20800 2024-09-19T04:17:46.672 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:17:46 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c[65180]: starting mds.c at 2024-09-19T04:17:46.672 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:17:46 smithi145 bash[65165]: 1e3bf76d9ee6e563a0e3bb3a798405900868d2ec46aa42384ed30768453725b9 2024-09-19T04:17:46.672 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:17:46 smithi145 systemd[1]: Started Ceph mds.c for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:46.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:46 smithi107 ceph-mon[30659]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:46.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:46.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:46.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:46.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:46.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:46.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:46 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:46.761 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:46.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:46 smithi115 ceph-mon[35477]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:46.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:46.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:46.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:46.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:46.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:46.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:46 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:46 smithi145 ceph-mon[35096]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:46.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:46 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:47.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.347+0000 7f6173e00640 1 -- 172.21.15.107:0/1345588829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61741129d0 msgr2=0x7f6174114dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.347+0000 7f6173e00640 1 --2- 172.21.15.107:0/1345588829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61741129d0 0x7f6174114dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f616c009be0 tx=0x7f616c02a750 comp rx=0 tx=0).stop 2024-09-19T04:17:47.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.347+0000 7f6173e00640 1 -- 172.21.15.107:0/1345588829 shutdown_connections 2024-09-19T04:17:47.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.347+0000 7f6173e00640 1 --2- 172.21.15.107:0/1345588829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61741129d0 0x7f6174114dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.347+0000 7f6173e00640 1 --2- 172.21.15.107:0/1345588829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6174074eb0 0x7f6174112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.347+0000 7f6173e00640 1 --2- 172.21.15.107:0/1345588829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174074590 0x7f6174074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.347+0000 7f6173e00640 1 -- 172.21.15.107:0/1345588829 >> 172.21.15.107:0/1345588829 conn(0x7f61740fd6e0 msgr2=0x7f61740ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:47.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.347+0000 7f6173e00640 1 -- 172.21.15.107:0/1345588829 shutdown_connections 2024-09-19T04:17:47.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.347+0000 7f6173e00640 1 -- 172.21.15.107:0/1345588829 wait complete. 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.348+0000 7f6173e00640 1 Processor -- start 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.348+0000 7f6173e00640 1 -- start start 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f6173e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6174074590 0x7f61741a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f6173e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174074eb0 0x7f61741a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f6173e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61741129d0 0x7f617419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f6173e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6174118390 con 0x7f61741129d0 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f6173e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6174118210 con 0x7f6174074eb0 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f6173e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6174118510 con 0x7f6174074590 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f6172a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6174074590 0x7f61741a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f6172a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6174074590 0x7f61741a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51296/0 (socket says 172.21.15.107:51296) 2024-09-19T04:17:47.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f6172a00640 1 -- 172.21.15.107:0/3846859181 learned_addr learned my addr 172.21.15.107:0/3846859181 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:47.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f6173400640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61741129d0 0x7f617419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.349+0000 7f616a000640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174074eb0 0x7f61741a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f6172a00640 1 -- 172.21.15.107:0/3846859181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174074eb0 msgr2=0x7f61741a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f6172a00640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174074eb0 0x7f61741a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f6172a00640 1 -- 172.21.15.107:0/3846859181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61741129d0 msgr2=0x7f617419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f6172a00640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61741129d0 0x7f617419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f6172a00640 1 -- 172.21.15.107:0/3846859181 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617419ef10 con 0x7f6174074590 2024-09-19T04:17:47.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f6172a00640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6174074590 0x7f61741a45d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f616400d950 tx=0x7f616400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f616be00640 1 -- 172.21.15.107:0/3846859181 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164017070 con 0x7f6174074590 2024-09-19T04:17:47.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f616be00640 1 -- 172.21.15.107:0/3846859181 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6164009c30 con 0x7f6174074590 2024-09-19T04:17:47.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f6173e00640 1 -- 172.21.15.107:0/3846859181 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617419f200 con 0x7f6174074590 2024-09-19T04:17:47.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f616be00640 1 -- 172.21.15.107:0/3846859181 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616400f6b0 con 0x7f6174074590 2024-09-19T04:17:47.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.350+0000 7f6173e00640 1 -- 172.21.15.107:0/3846859181 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61741ab550 con 0x7f6174074590 2024-09-19T04:17:47.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.352+0000 7f6173e00640 1 -- 172.21.15.107:0/3846859181 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6174075cd0 con 0x7f6174074590 2024-09-19T04:17:47.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.353+0000 7f616be00640 1 -- 172.21.15.107:0/3846859181 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f61640109d0 con 0x7f6174074590 2024-09-19T04:17:47.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.354+0000 7f616be00640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f61580a32c0 0x7f61580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.355+0000 7f616be00640 1 -- 172.21.15.107:0/3846859181 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f61640ceff0 con 0x7f6174074590 2024-09-19T04:17:47.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.355+0000 7f616a000640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f61580a32c0 0x7f61580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.355+0000 7f616a000640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f61580a32c0 0x7f61580a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f615c0096c0 tx=0x7f615c009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.358+0000 7f616be00640 1 -- 172.21.15.107:0/3846859181 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6164009de0 con 0x7f6174074590 2024-09-19T04:17:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.567+0000 7f6173e00640 1 -- 172.21.15.107:0/3846859181 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6174104e10 con 0x7f6174074590 2024-09-19T04:17:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.567+0000 7f616be00640 1 -- 172.21.15.107:0/3846859181 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6403 (secure 0 0 0) 0x7f616409bb20 con 0x7f6174074590 2024-09-19T04:17:47.570 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:47.570 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":8,"btime":"2024-09-19T04:17:46:666635+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T04:17:47.570 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 8 2024-09-19T04:17:47.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 -- 172.21.15.107:0/3846859181 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f61580a32c0 msgr2=0x7f61580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f61580a32c0 0x7f61580a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f615c0096c0 tx=0x7f615c009260 comp rx=0 tx=0).stop 2024-09-19T04:17:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 -- 172.21.15.107:0/3846859181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6174074590 msgr2=0x7f61741a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6174074590 0x7f61741a45d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f616400d950 tx=0x7f616400de10 comp rx=0 tx=0).stop 2024-09-19T04:17:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 -- 172.21.15.107:0/3846859181 shutdown_connections 2024-09-19T04:17:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f61580a32c0 0x7f61580a5780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61741129d0 0x7f617419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174074eb0 0x7f61741a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 --2- 172.21.15.107:0/3846859181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6174074590 0x7f61741a45d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 -- 172.21.15.107:0/3846859181 >> 172.21.15.107:0/3846859181 conn(0x7f61740fd6e0 msgr2=0x7f61740fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 -- 172.21.15.107:0/3846859181 shutdown_connections 2024-09-19T04:17:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:47.570+0000 7f6173e00640 1 -- 172.21.15.107:0/3846859181 wait complete. 2024-09-19T04:17:47.673 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:47 smithi115 ceph-mon[35477]: Deploying daemon mds.h on smithi115 2024-09-19T04:17:47.674 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:47 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] up:boot 2024-09-19T04:17:47.674 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:47 smithi115 ceph-mon[35477]: fsmap 7 up:standby 2024-09-19T04:17:47.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:47 smithi107 ceph-mon[30659]: Deploying daemon mds.h on smithi115 2024-09-19T04:17:47.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:47 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] up:boot 2024-09-19T04:17:47.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:47 smithi107 ceph-mon[30659]: fsmap 7 up:standby 2024-09-19T04:17:47.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:47 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:17:47.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:47 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3846859181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:47.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:47 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:17:47.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:47 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3846859181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:47.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:47 smithi145 ceph-mon[35096]: Deploying daemon mds.h on smithi115 2024-09-19T04:17:47.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:47 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] up:boot 2024-09-19T04:17:47.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:47 smithi145 ceph-mon[35096]: fsmap 7 up:standby 2024-09-19T04:17:47.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:47 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:17:47.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:47 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3846859181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:48.910 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:48 smithi115 ceph-mon[35477]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:48.911 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:17:48 smithi115 systemd[1]: Starting Ceph mds.h for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:17:48.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:48 smithi145 ceph-mon[35096]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:48.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:48 smithi107 ceph-mon[30659]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:49.165 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:17:48 smithi115 podman[65259]: 2024-09-19 04:17:48.812332184 +0000 UTC m=+0.016291253 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:17:49.165 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:17:48 smithi115 podman[65259]: 2024-09-19 04:17:48.936425215 +0000 UTC m=+0.140384283 container create 00340df01aeec6a96c6d7454c70c04f497c8cff70ddf0b47afd9ca3c35b94d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:17:49.165 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:17:49 smithi115 podman[65259]: 2024-09-19 04:17:49.164192677 +0000 UTC m=+0.368151742 container init 00340df01aeec6a96c6d7454c70c04f497c8cff70ddf0b47afd9ca3c35b94d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T04:17:49.447 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:17:49 smithi115 podman[65259]: 2024-09-19 04:17:49.166916207 +0000 UTC m=+0.370875271 container start 00340df01aeec6a96c6d7454c70c04f497c8cff70ddf0b47afd9ca3c35b94d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T04:17:49.447 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:17:49 smithi115 ceph-mds[65278]: -- 172.21.15.115:0/3267370407 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558084793860 con 0x558084770800 2024-09-19T04:17:49.447 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:17:49 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h[65274]: starting mds.h at 2024-09-19T04:17:49.447 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:17:49 smithi115 bash[65259]: 00340df01aeec6a96c6d7454c70c04f497c8cff70ddf0b47afd9ca3c35b94d75 2024-09-19T04:17:49.448 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:17:49 smithi115 systemd[1]: Started Ceph mds.h for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:50.270 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:17:50.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:50.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:50.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:50.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:50.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:50.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:50.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: Deploying daemon mds.i on smithi145 2024-09-19T04:17:50.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] up:boot 2024-09-19T04:17:50.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: fsmap 8 up:standby 2024-09-19T04:17:50.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:17:50.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:50 smithi145 ceph-mon[35096]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:50.649 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:50.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:50.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:50.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:50.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:50.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:50.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:50.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: Deploying daemon mds.i on smithi145 2024-09-19T04:17:50.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] up:boot 2024-09-19T04:17:50.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: fsmap 8 up:standby 2024-09-19T04:17:50.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:17:50.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:50 smithi115 ceph-mon[35477]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:50.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:50.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:50.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:50.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:50.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:50.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:50.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: Deploying daemon mds.i on smithi145 2024-09-19T04:17:50.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] up:boot 2024-09-19T04:17:50.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: fsmap 8 up:standby 2024-09-19T04:17:50.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:17:50.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:50 smithi107 ceph-mon[30659]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:51.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.195+0000 7fc7da600640 1 -- 172.21.15.107:0/706497988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d4106f10 msgr2=0x7fc7d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:51.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.195+0000 7fc7da600640 1 --2- 172.21.15.107:0/706497988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d4106f10 0x7fc7d410ea90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009ca0 tx=0x7fc7cc02a8b0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.195+0000 7fc7da600640 1 -- 172.21.15.107:0/706497988 shutdown_connections 2024-09-19T04:17:51.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.195+0000 7fc7da600640 1 --2- 172.21.15.107:0/706497988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d4106f10 0x7fc7d410ea90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.195+0000 7fc7da600640 1 --2- 172.21.15.107:0/706497988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d4102790 0x7fc7d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.195+0000 7fc7da600640 1 --2- 172.21.15.107:0/706497988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4101de0 0x7fc7d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.195+0000 7fc7da600640 1 -- 172.21.15.107:0/706497988 >> 172.21.15.107:0/706497988 conn(0x7fc7d40fd8c0 msgr2=0x7fc7d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.195+0000 7fc7da600640 1 -- 172.21.15.107:0/706497988 shutdown_connections 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.195+0000 7fc7da600640 1 -- 172.21.15.107:0/706497988 wait complete. 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.196+0000 7fc7da600640 1 Processor -- start 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.196+0000 7fc7da600640 1 -- start start 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7da600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d4101de0 0x7fc7d40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7da600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4102790 0x7fc7d4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7da600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d4106f10 0x7fc7d4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7da600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d4111f70 con 0x7fc7d4101de0 2024-09-19T04:17:51.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7da600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d4111df0 con 0x7fc7d4102790 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7da600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41120f0 con 0x7fc7d4106f10 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7d3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d4101de0 0x7fc7d40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7d3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d4101de0 0x7fc7d40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45260/0 (socket says 172.21.15.107:45260) 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7d3400640 1 -- 172.21.15.107:0/589287150 learned_addr learned my addr 172.21.15.107:0/589287150 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7d3e00640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d4106f10 0x7fc7d4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7caa00640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4102790 0x7fc7d4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7d3400640 1 -- 172.21.15.107:0/589287150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d4106f10 msgr2=0x7fc7d4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7d3400640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d4106f10 0x7fc7d4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7d3400640 1 -- 172.21.15.107:0/589287150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4102790 msgr2=0x7fc7d4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7d3400640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4102790 0x7fc7d4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.197+0000 7fc7d3400640 1 -- 172.21.15.107:0/589287150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d41a68b0 con 0x7fc7d4101de0 2024-09-19T04:17:51.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.198+0000 7fc7d3400640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d4101de0 0x7fc7d40744f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400e9f0 tx=0x7fc7c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:51.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.198+0000 7fc7d0c00640 1 -- 172.21.15.107:0/589287150 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400cd80 con 0x7fc7d4101de0 2024-09-19T04:17:51.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.198+0000 7fc7da600640 1 -- 172.21.15.107:0/589287150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d41a6ba0 con 0x7fc7d4101de0 2024-09-19T04:17:51.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.198+0000 7fc7d0c00640 1 -- 172.21.15.107:0/589287150 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c4010910 con 0x7fc7d4101de0 2024-09-19T04:17:51.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.198+0000 7fc7d0c00640 1 -- 172.21.15.107:0/589287150 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40188f0 con 0x7fc7d4101de0 2024-09-19T04:17:51.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.198+0000 7fc7da600640 1 -- 172.21.15.107:0/589287150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d41a7130 con 0x7fc7d4101de0 2024-09-19T04:17:51.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.199+0000 7fc7da600640 1 -- 172.21.15.107:0/589287150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d41021c0 con 0x7fc7d4101de0 2024-09-19T04:17:51.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.201+0000 7fc7d0c00640 1 -- 172.21.15.107:0/589287150 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc7c4010ac0 con 0x7fc7d4101de0 2024-09-19T04:17:51.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.203+0000 7fc7d0c00640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc7b80a32c0 0x7fc7b80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:51.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.203+0000 7fc7d0c00640 1 -- 172.21.15.107:0/589287150 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fc7c40cf830 con 0x7fc7d4101de0 2024-09-19T04:17:51.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.203+0000 7fc7caa00640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc7b80a32c0 0x7fc7b80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:51.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.204+0000 7fc7caa00640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc7b80a32c0 0x7fc7b80a5780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0095f0 tx=0x7fc7bc005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:51.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.207+0000 7fc7d0c00640 1 -- 172.21.15.107:0/589287150 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc7c409c300 con 0x7fc7d4101de0 2024-09-19T04:17:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.418+0000 7fc7da600640 1 -- 172.21.15.107:0/589287150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7d4075a20 con 0x7fc7d4101de0 2024-09-19T04:17:51.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.418+0000 7fc7d0c00640 1 -- 172.21.15.107:0/589287150 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7228 (secure 0 0 0) 0x7fc7c40a0600 con 0x7fc7d4101de0 2024-09-19T04:17:51.420 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:51.420 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":9,"btime":"2024-09-19T04:17:49:678199+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T04:17:51.420 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 9 2024-09-19T04:17:51.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 -- 172.21.15.107:0/589287150 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc7b80a32c0 msgr2=0x7fc7b80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:51.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc7b80a32c0 0x7fc7b80a5780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0095f0 tx=0x7fc7bc005f90 comp rx=0 tx=0).stop 2024-09-19T04:17:51.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 -- 172.21.15.107:0/589287150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d4101de0 msgr2=0x7fc7d40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:51.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d4101de0 0x7fc7d40744f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400e9f0 tx=0x7fc7c400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 -- 172.21.15.107:0/589287150 shutdown_connections 2024-09-19T04:17:51.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc7b80a32c0 0x7fc7b80a5780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d4106f10 0x7fc7d4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4102790 0x7fc7d4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 --2- 172.21.15.107:0/589287150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d4101de0 0x7fc7d40744f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 -- 172.21.15.107:0/589287150 >> 172.21.15.107:0/589287150 conn(0x7fc7d40fd8c0 msgr2=0x7fc7d40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:51.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 -- 172.21.15.107:0/589287150 shutdown_connections 2024-09-19T04:17:51.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:51.421+0000 7fc7da600640 1 -- 172.21.15.107:0/589287150 wait complete. 2024-09-19T04:17:51.502 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:17:51 smithi145 systemd[1]: Starting Ceph mds.i for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:17:51.810 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:51 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/589287150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:51.811 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:17:51 smithi145 podman[65478]: 2024-09-19 04:17:51.451703557 +0000 UTC m=+0.015907666 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:17:51.811 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:17:51 smithi145 podman[65478]: 2024-09-19 04:17:51.565523107 +0000 UTC m=+0.129727242 container create ce9d8167bf21a30d43f441b1fc7acad21ec28c731f75dc1697212f9832da6af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:17:51.811 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:17:51 smithi145 podman[65478]: 2024-09-19 04:17:51.810259977 +0000 UTC m=+0.374464080 container init ce9d8167bf21a30d43f441b1fc7acad21ec28c731f75dc1697212f9832da6af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T04:17:51.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:51 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/589287150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:51.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:51 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/589287150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:52.224 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:17:51 smithi145 podman[65478]: 2024-09-19 04:17:51.813216916 +0000 UTC m=+0.377421040 container start ce9d8167bf21a30d43f441b1fc7acad21ec28c731f75dc1697212f9832da6af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T04:17:52.224 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:17:51 smithi145 ceph-mds[65497]: -- 172.21.15.145:0/3860326945 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564384001860 con 0x564383fde800 2024-09-19T04:17:52.224 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:17:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i[65493]: starting mds.i at 2024-09-19T04:17:52.224 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:17:51 smithi145 bash[65478]: ce9d8167bf21a30d43f441b1fc7acad21ec28c731f75dc1697212f9832da6af5 2024-09-19T04:17:52.225 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:17:51 smithi145 systemd[1]: Started Ceph mds.i for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:52.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:52 smithi107 ceph-mon[30659]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:52.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:52.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:52.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:52.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:52.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:52.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:52 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:52.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:52 smithi115 ceph-mon[35477]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:52.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:52 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:52 smithi145 ceph-mon[35096]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:52 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:53.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:53 smithi115 ceph-mon[35477]: Deploying daemon mds.g on smithi107 2024-09-19T04:17:53.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:53 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] up:boot 2024-09-19T04:17:53.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:53 smithi115 ceph-mon[35477]: fsmap 9 up:standby 2024-09-19T04:17:53.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:53 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:17:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:53 smithi145 ceph-mon[35096]: Deploying daemon mds.g on smithi107 2024-09-19T04:17:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:53 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] up:boot 2024-09-19T04:17:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:53 smithi145 ceph-mon[35096]: fsmap 9 up:standby 2024-09-19T04:17:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:53 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:17:53.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:53 smithi107 ceph-mon[30659]: Deploying daemon mds.g on smithi107 2024-09-19T04:17:53.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:53 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] up:boot 2024-09-19T04:17:53.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:53 smithi107 ceph-mon[30659]: fsmap 9 up:standby 2024-09-19T04:17:53.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:53 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:17:54.224 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:17:54.697 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:17:54 smithi107 systemd[1]: Starting Ceph mds.g for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:17:54.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:54 smithi145 ceph-mon[35096]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:54.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:54 smithi107 ceph-mon[30659]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:54.981 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:17:54 smithi107 podman[71419]: 2024-09-19 04:17:54.646401244 +0000 UTC m=+0.028494772 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:17:54.981 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:17:54 smithi107 podman[71419]: 2024-09-19 04:17:54.775207872 +0000 UTC m=+0.157301397 container create 64d4ddb63aeaec62f4342985a40426da308e43e9edaae814242f11a3a0dc4813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T04:17:55.133 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:54 smithi115 ceph-mon[35477]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:55.333 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:17:55 smithi107 podman[71419]: 2024-09-19 04:17:55.022325169 +0000 UTC m=+0.404418698 container init 64d4ddb63aeaec62f4342985a40426da308e43e9edaae814242f11a3a0dc4813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T04:17:55.333 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:17:55 smithi107 podman[71419]: 2024-09-19 04:17:55.029198739 +0000 UTC m=+0.411292277 container start 64d4ddb63aeaec62f4342985a40426da308e43e9edaae814242f11a3a0dc4813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T04:17:55.333 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:17:55 smithi107 ceph-mds[71467]: -- 172.21.15.107:0/2150264642 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55acdc4ab860 con 0x55acdc489400 2024-09-19T04:17:55.333 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:17:55 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g[71463]: starting mds.g at 2024-09-19T04:17:55.333 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:17:55 smithi107 bash[71419]: 64d4ddb63aeaec62f4342985a40426da308e43e9edaae814242f11a3a0dc4813 2024-09-19T04:17:55.334 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:17:55 smithi107 systemd[1]: Started Ceph mds.g for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:55.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.710+0000 7fab9a400640 1 -- 172.21.15.107:0/2456219114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab94109140 msgr2=0x7fab94110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:55.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.710+0000 7fab9a400640 1 --2- 172.21.15.107:0/2456219114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab94109140 0x7fab94110cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009be0 tx=0x7fab8c02a6c0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.710+0000 7fab9a400640 1 -- 172.21.15.107:0/2456219114 shutdown_connections 2024-09-19T04:17:55.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.710+0000 7fab9a400640 1 --2- 172.21.15.107:0/2456219114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab94109140 0x7fab94110cc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.710+0000 7fab9a400640 1 --2- 172.21.15.107:0/2456219114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab941049c0 0x7fab94108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.710+0000 7fab9a400640 1 --2- 172.21.15.107:0/2456219114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab94104010 0x7fab941043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.710+0000 7fab9a400640 1 -- 172.21.15.107:0/2456219114 >> 172.21.15.107:0/2456219114 conn(0x7fab940ffa50 msgr2=0x7fab94101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:55.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.710+0000 7fab9a400640 1 -- 172.21.15.107:0/2456219114 shutdown_connections 2024-09-19T04:17:55.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.710+0000 7fab9a400640 1 -- 172.21.15.107:0/2456219114 wait complete. 2024-09-19T04:17:55.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.711+0000 7fab9a400640 1 Processor -- start 2024-09-19T04:17:55.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.711+0000 7fab9a400640 1 -- start start 2024-09-19T04:17:55.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.711+0000 7fab9a400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab94104010 0x7fab9419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:55.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.711+0000 7fab9a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab941049c0 0x7fab9419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:55.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab9a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab94109140 0x7fab9419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:55.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab9a400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fab94114170 con 0x7fab941049c0 2024-09-19T04:17:55.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab9a400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fab94113ff0 con 0x7fab94104010 2024-09-19T04:17:55.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab9a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fab941142f0 con 0x7fab94109140 2024-09-19T04:17:55.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab93400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab94104010 0x7fab9419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:55.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab93400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab94104010 0x7fab9419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:41096/0 (socket says 172.21.15.107:41096) 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab93400640 1 -- 172.21.15.107:0/938653385 learned_addr learned my addr 172.21.15.107:0/938653385 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab92a00640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab941049c0 0x7fab9419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab93e00640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab94109140 0x7fab9419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab93400640 1 -- 172.21.15.107:0/938653385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab94109140 msgr2=0x7fab9419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab93400640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab94109140 0x7fab9419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab93400640 1 -- 172.21.15.107:0/938653385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab941049c0 msgr2=0x7fab9419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab93400640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab941049c0 0x7fab9419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab93400640 1 -- 172.21.15.107:0/938653385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9419f5e0 con 0x7fab94104010 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.712+0000 7fab93e00640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab94109140 0x7fab9419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.713+0000 7fab93400640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab94104010 0x7fab9419a630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab8400e9f0 tx=0x7fab8400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.713+0000 7fab8be00640 1 -- 172.21.15.107:0/938653385 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8400cd80 con 0x7fab94104010 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.713+0000 7fab8be00640 1 -- 172.21.15.107:0/938653385 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab84005500 con 0x7fab94104010 2024-09-19T04:17:55.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.713+0000 7fab9a400640 1 -- 172.21.15.107:0/938653385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9419f8d0 con 0x7fab94104010 2024-09-19T04:17:55.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.713+0000 7fab8be00640 1 -- 172.21.15.107:0/938653385 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84010690 con 0x7fab94104010 2024-09-19T04:17:55.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.713+0000 7fab9a400640 1 -- 172.21.15.107:0/938653385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab941a71b0 con 0x7fab94104010 2024-09-19T04:17:55.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.714+0000 7fab9a400640 1 -- 172.21.15.107:0/938653385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab941043f0 con 0x7fab94104010 2024-09-19T04:17:55.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.716+0000 7fab8be00640 1 -- 172.21.15.107:0/938653385 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fab840046c0 con 0x7fab94104010 2024-09-19T04:17:55.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.717+0000 7fab8be00640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fab780a32c0 0x7fab780a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:55.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.717+0000 7fab8be00640 1 -- 172.21.15.107:0/938653385 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fab840cf8b0 con 0x7fab94104010 2024-09-19T04:17:55.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.718+0000 7fab92a00640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fab780a32c0 0x7fab780a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:55.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.718+0000 7fab92a00640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fab780a32c0 0x7fab780a5780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fab800096c0 tx=0x7fab80009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:55.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.721+0000 7fab8be00640 1 -- 172.21.15.107:0/938653385 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fab8409c9d0 con 0x7fab94104010 2024-09-19T04:17:55.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.947+0000 7fab9a400640 1 -- 172.21.15.107:0/938653385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab9419b990 con 0x7fab94104010 2024-09-19T04:17:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.948+0000 7fab8be00640 1 -- 172.21.15.107:0/938653385 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8887 (secure 0 0 0) 0x7fab8409c2d0 con 0x7fab94104010 2024-09-19T04:17:55.949 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:17:55.949 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":11,"btime":"2024-09-19T04:17:55:695741+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14499,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T04:17:55.950 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 11 2024-09-19T04:17:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.950+0000 7fab9a400640 1 -- 172.21.15.107:0/938653385 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fab780a32c0 msgr2=0x7fab780a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.950+0000 7fab9a400640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fab780a32c0 0x7fab780a5780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fab800096c0 tx=0x7fab80009260 comp rx=0 tx=0).stop 2024-09-19T04:17:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.950+0000 7fab9a400640 1 -- 172.21.15.107:0/938653385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab94104010 msgr2=0x7fab9419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.950+0000 7fab9a400640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab94104010 0x7fab9419a630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab8400e9f0 tx=0x7fab8400edf0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.950+0000 7fab9a400640 1 -- 172.21.15.107:0/938653385 shutdown_connections 2024-09-19T04:17:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.950+0000 7fab9a400640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fab780a32c0 0x7fab780a5780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.950+0000 7fab9a400640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab94109140 0x7fab9419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.950+0000 7fab9a400640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab941049c0 0x7fab9419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.950+0000 7fab9a400640 1 --2- 172.21.15.107:0/938653385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab94104010 0x7fab9419a630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.950+0000 7fab9a400640 1 -- 172.21.15.107:0/938653385 >> 172.21.15.107:0/938653385 conn(0x7fab940ffa50 msgr2=0x7fab94100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.951+0000 7fab9a400640 1 -- 172.21.15.107:0/938653385 shutdown_connections 2024-09-19T04:17:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:55.951+0000 7fab9a400640 1 -- 172.21.15.107:0/938653385 wait complete. 2024-09-19T04:17:56.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: Deploying daemon mds.a on smithi107 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.107:6838/2243096646,v1:172.21.15.107:6839/2243096646] up:boot 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: fsmap 10 up:standby 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:56.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:56 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/938653385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:56.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:56.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:56.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:56.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:56.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:56.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:56.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: Deploying daemon mds.a on smithi107 2024-09-19T04:17:56.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.107:6838/2243096646,v1:172.21.15.107:6839/2243096646] up:boot 2024-09-19T04:17:56.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: fsmap 10 up:standby 2024-09-19T04:17:56.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:17:56.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:56.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:56 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/938653385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:56.583 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:56.583 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:56.583 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:17:56.583 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:17:56.583 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:17:56.583 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:17:56.584 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: Deploying daemon mds.a on smithi107 2024-09-19T04:17:56.584 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.107:6838/2243096646,v1:172.21.15.107:6839/2243096646] up:boot 2024-09-19T04:17:56.584 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: fsmap 10 up:standby 2024-09-19T04:17:56.584 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:17:56.584 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:56.584 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:56 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/938653385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:17:58.643 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:17:58 smithi107 systemd[1]: Starting Ceph mds.a for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:17:58.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:17:58.917 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:17:58 smithi107 ceph-mon[30659]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:58.917 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:17:58 smithi107 podman[71802]: 2024-09-19 04:17:58.546555405 +0000 UTC m=+0.027876080 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:17:58.918 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:17:58 smithi107 podman[71802]: 2024-09-19 04:17:58.693711132 +0000 UTC m=+0.175031785 container create d992b5b44b7c72fbd26ab3023cd88a2752dbecc2e78c9de52656bf47d6ff7d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T04:17:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:17:58 smithi115 ceph-mon[35477]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:17:58 smithi145 ceph-mon[35096]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:17:59.230 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:17:58 smithi107 podman[71802]: 2024-09-19 04:17:58.922147516 +0000 UTC m=+0.403468191 container init d992b5b44b7c72fbd26ab3023cd88a2752dbecc2e78c9de52656bf47d6ff7d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T04:17:59.231 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:17:58 smithi107 podman[71802]: 2024-09-19 04:17:58.928754132 +0000 UTC m=+0.410074786 container start d992b5b44b7c72fbd26ab3023cd88a2752dbecc2e78c9de52656bf47d6ff7d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T04:17:59.231 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:17:58 smithi107 ceph-mds[71839]: -- 172.21.15.107:0/4145218803 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579843abc20 con 0x557984388800 2024-09-19T04:17:59.231 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:17:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a[71817]: starting mds.a at 2024-09-19T04:17:59.231 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:17:59 smithi107 bash[71802]: d992b5b44b7c72fbd26ab3023cd88a2752dbecc2e78c9de52656bf47d6ff7d5c 2024-09-19T04:17:59.231 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:17:59 smithi107 systemd[1]: Started Ceph mds.a for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:17:59.254 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:17:59.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.822+0000 7fe09de00640 1 -- 172.21.15.107:0/1374575504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe098106f10 msgr2=0x7fe09810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:59.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.822+0000 7fe09de00640 1 --2- 172.21.15.107:0/1374575504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe098106f10 0x7fe09810ea90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe090009ca0 tx=0x7fe09002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.823+0000 7fe09de00640 1 -- 172.21.15.107:0/1374575504 shutdown_connections 2024-09-19T04:17:59.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.823+0000 7fe09de00640 1 --2- 172.21.15.107:0/1374575504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe098106f10 0x7fe09810ea90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.823+0000 7fe09de00640 1 --2- 172.21.15.107:0/1374575504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe098102790 0x7fe0981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.823+0000 7fe09de00640 1 --2- 172.21.15.107:0/1374575504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098101de0 0x7fe0981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.823+0000 7fe09de00640 1 -- 172.21.15.107:0/1374575504 >> 172.21.15.107:0/1374575504 conn(0x7fe0980fd8c0 msgr2=0x7fe0980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.823+0000 7fe09de00640 1 -- 172.21.15.107:0/1374575504 shutdown_connections 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.823+0000 7fe09de00640 1 -- 172.21.15.107:0/1374575504 wait complete. 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe09de00640 1 Processor -- start 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe09de00640 1 -- start start 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe09de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe09de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe098102790 0x7fe098074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe09de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe09de00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111f70 con 0x7fe098106f10 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe09de00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111df0 con 0x7fe098101de0 2024-09-19T04:17:59.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe09de00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981120f0 con 0x7fe098102790 2024-09-19T04:17:59.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe097400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe096a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe096a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39006/0 (socket says 172.21.15.107:39006) 2024-09-19T04:17:59.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe097400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37642/0 (socket says 172.21.15.107:37642) 2024-09-19T04:17:59.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe096a00640 1 -- 172.21.15.107:0/2822876314 learned_addr learned my addr 172.21.15.107:0/2822876314 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:17:59.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.824+0000 7fe08e000640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe098102790 0x7fe098074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe096a00640 1 -- 172.21.15.107:0/2822876314 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe098102790 msgr2=0x7fe098074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe096a00640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe098102790 0x7fe098074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe096a00640 1 -- 172.21.15.107:0/2822876314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe098106f10 msgr2=0x7fe098074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe096a00640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe096a00640 1 -- 172.21.15.107:0/2822876314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0981a68b0 con 0x7fe098101de0 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe097400640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe096a00640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe08800ea60 tx=0x7fe08800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe09ca00640 1 -- 172.21.15.107:0/2822876314 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08800cde0 con 0x7fe098101de0 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe09de00640 1 -- 172.21.15.107:0/2822876314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0981a6ba0 con 0x7fe098101de0 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe09ca00640 1 -- 172.21.15.107:0/2822876314 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0880035f0 con 0x7fe098101de0 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe09ca00640 1 -- 172.21.15.107:0/2822876314 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088010670 con 0x7fe098101de0 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.825+0000 7fe09de00640 1 -- 172.21.15.107:0/2822876314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0981a7130 con 0x7fe098101de0 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.826+0000 7fe09de00640 1 -- 172.21.15.107:0/2822876314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0981021c0 con 0x7fe098101de0 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.828+0000 7fe09ca00640 1 -- 172.21.15.107:0/2822876314 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe0880037a0 con 0x7fe098101de0 2024-09-19T04:17:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.830+0000 7fe09ca00640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe07c0a3240 0x7fe07c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.830+0000 7fe09ca00640 1 -- 172.21.15.107:0/2822876314 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fe0880cfaf0 con 0x7fe098101de0 2024-09-19T04:17:59.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.830+0000 7fe08e000640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe07c0a3240 0x7fe07c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.830+0000 7fe08e000640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe07c0a3240 0x7fe07c0a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe098075f80 tx=0x7fe080005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:59.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:17:59.836+0000 7fe09ca00640 1 -- 172.21.15.107:0/2822876314 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe08809cc10 con 0x7fe098101de0 2024-09-19T04:18:00.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.049+0000 7fe09de00640 1 -- 172.21.15.107:0/2822876314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe09810a150 con 0x7fe098101de0 2024-09-19T04:18:00.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.050+0000 7fe09ca00640 1 -- 172.21.15.107:0/2822876314 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9713 (secure 0 0 0) 0x7fe08809c510 con 0x7fe098101de0 2024-09-19T04:18:00.052 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:00.052 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":12,"btime":"2024-09-19T04:17:59:857887+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14499,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T04:18:00.052 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 12 2024-09-19T04:18:00.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.052+0000 7fe09de00640 1 -- 172.21.15.107:0/2822876314 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe07c0a3240 msgr2=0x7fe07c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:00.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.052+0000 7fe09de00640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe07c0a3240 0x7fe07c0a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe098075f80 tx=0x7fe080005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:00.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.052+0000 7fe09de00640 1 -- 172.21.15.107:0/2822876314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098101de0 msgr2=0x7fe0980744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:00.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.052+0000 7fe09de00640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe08800ea60 tx=0x7fe08800ef20 comp rx=0 tx=0).stop 2024-09-19T04:18:00.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.052+0000 7fe09de00640 1 -- 172.21.15.107:0/2822876314 shutdown_connections 2024-09-19T04:18:00.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.052+0000 7fe09de00640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe07c0a3240 0x7fe07c0a5700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.052+0000 7fe09de00640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.052+0000 7fe09de00640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe098102790 0x7fe098074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.052+0000 7fe09de00640 1 --2- 172.21.15.107:0/2822876314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.052+0000 7fe09de00640 1 -- 172.21.15.107:0/2822876314 >> 172.21.15.107:0/2822876314 conn(0x7fe0980fd8c0 msgr2=0x7fe0980feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:00.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.053+0000 7fe09de00640 1 -- 172.21.15.107:0/2822876314 shutdown_connections 2024-09-19T04:18:00.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:00.053+0000 7fe09de00640 1 -- 172.21.15.107:0/2822876314 wait complete. 2024-09-19T04:18:00.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:00.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:00.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:00.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:18:00.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:18:00.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:00.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: Deploying daemon mds.b on smithi115 2024-09-19T04:18:00.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:00.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.107:6840/305932896,v1:172.21.15.107:6841/305932896] up:boot 2024-09-19T04:18:00.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: fsmap 11 up:standby 2024-09-19T04:18:00.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:18:00.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:00 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2822876314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:18:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:18:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: Deploying daemon mds.b on smithi115 2024-09-19T04:18:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.107:6840/305932896,v1:172.21.15.107:6841/305932896] up:boot 2024-09-19T04:18:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: fsmap 11 up:standby 2024-09-19T04:18:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:18:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:00 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2822876314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:00.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:00.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:00.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:18:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:18:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: Deploying daemon mds.b on smithi115 2024-09-19T04:18:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.107:6840/305932896,v1:172.21.15.107:6841/305932896] up:boot 2024-09-19T04:18:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: fsmap 11 up:standby 2024-09-19T04:18:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:18:00.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:00 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2822876314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:01.385 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:01 smithi115 systemd[1]: Starting Ceph mds.b for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:18:01.661 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:01 smithi115 podman[65560]: 2024-09-19 04:18:01.287104338 +0000 UTC m=+0.015893995 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:18:01.661 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:01 smithi115 podman[65560]: 2024-09-19 04:18:01.398981588 +0000 UTC m=+0.127771278 container create 9d1512338f7a6d07e0370e84d348593cba05d85872bc4eb9f75885047a54c8d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T04:18:01.661 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:01 smithi115 podman[65560]: 2024-09-19 04:18:01.66018938 +0000 UTC m=+0.388979026 container init 9d1512338f7a6d07e0370e84d348593cba05d85872bc4eb9f75885047a54c8d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T04:18:01.946 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:01 smithi115 podman[65560]: 2024-09-19 04:18:01.663209853 +0000 UTC m=+0.391999495 container start 9d1512338f7a6d07e0370e84d348593cba05d85872bc4eb9f75885047a54c8d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:18:01.946 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:01 smithi115 ceph-mds[65578]: -- 172.21.15.115:0/2615490578 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599a6823860 con 0x5599a6801400 2024-09-19T04:18:01.946 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:01 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b[65574]: starting mds.b at 2024-09-19T04:18:01.946 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:01 smithi115 bash[65560]: 9d1512338f7a6d07e0370e84d348593cba05d85872bc4eb9f75885047a54c8d1 2024-09-19T04:18:01.947 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:01 smithi115 systemd[1]: Started Ceph mds.b for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:18:02.745 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:18:02.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] up:boot 2024-09-19T04:18:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:02 smithi115 ceph-mon[35477]: fsmap 12 up:standby 2024-09-19T04:18:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:18:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:02 smithi115 ceph-mon[35477]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:02.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:02 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:18:02.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] up:boot 2024-09-19T04:18:02.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:02 smithi145 ceph-mon[35096]: fsmap 12 up:standby 2024-09-19T04:18:02.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:18:02.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:02.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:02 smithi145 ceph-mon[35096]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:02.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:02.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:02.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:02.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:02.976 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:02 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:18:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] up:boot 2024-09-19T04:18:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:02 smithi107 ceph-mon[30659]: fsmap 12 up:standby 2024-09-19T04:18:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:18:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:02 smithi107 ceph-mon[30659]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:02 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:18:03.363 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.024+0000 7fa8d2000640 1 -- 172.21.15.107:0/4255969649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc108f60 msgr2=0x7fa8cc110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.024+0000 7fa8d2000640 1 --2- 172.21.15.107:0/4255969649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc108f60 0x7fa8cc110ae0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009ca0 tx=0x7fa8c402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.024+0000 7fa8d2000640 1 -- 172.21.15.107:0/4255969649 shutdown_connections 2024-09-19T04:18:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.024+0000 7fa8d2000640 1 --2- 172.21.15.107:0/4255969649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc108f60 0x7fa8cc110ae0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.024+0000 7fa8d2000640 1 --2- 172.21.15.107:0/4255969649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc1047e0 0x7fa8cc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.024+0000 7fa8d2000640 1 --2- 172.21.15.107:0/4255969649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8cc103e30 0x7fa8cc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.024+0000 7fa8d2000640 1 -- 172.21.15.107:0/4255969649 >> 172.21.15.107:0/4255969649 conn(0x7fa8cc0ff8d0 msgr2=0x7fa8cc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.024+0000 7fa8d2000640 1 -- 172.21.15.107:0/4255969649 shutdown_connections 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.024+0000 7fa8d2000640 1 -- 172.21.15.107:0/4255969649 wait complete. 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.025+0000 7fa8d2000640 1 Processor -- start 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.025+0000 7fa8d2000640 1 -- start start 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.025+0000 7fa8d2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc103e30 0x7fa8cc19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.025+0000 7fa8d2000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8cc1047e0 0x7fa8cc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.025+0000 7fa8d2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc108f60 0x7fa8cc19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.025+0000 7fa8d2000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc114010 con 0x7fa8cc103e30 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.025+0000 7fa8d2000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc113e90 con 0x7fa8cc1047e0 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.025+0000 7fa8d2000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc114190 con 0x7fa8cc108f60 2024-09-19T04:18:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8d1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc108f60 0x7fa8cc19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8cc1047e0 0x7fa8cc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8cc1047e0 0x7fa8cc19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39026/0 (socket says 172.21.15.107:39026) 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8d1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc108f60 0x7fa8cc19eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46700/0 (socket says 172.21.15.107:46700) 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8cbe00640 1 -- 172.21.15.107:0/3426718659 learned_addr learned my addr 172.21.15.107:0/3426718659 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8d0c00640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc103e30 0x7fa8cc19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8cbe00640 1 -- 172.21.15.107:0/3426718659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc108f60 msgr2=0x7fa8cc19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8cbe00640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc108f60 0x7fa8cc19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8cbe00640 1 -- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc103e30 msgr2=0x7fa8cc19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8cbe00640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc103e30 0x7fa8cc19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8cbe00640 1 -- 172.21.15.107:0/3426718659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc19f380 con 0x7fa8cc1047e0 2024-09-19T04:18:04.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8d0c00640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc103e30 0x7fa8cc19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8cbe00640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8cc1047e0 0x7fa8cc19a910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800e9f0 tx=0x7fa8b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8c9600640 1 -- 172.21.15.107:0/3426718659 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b800cd80 con 0x7fa8cc1047e0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8c9600640 1 -- 172.21.15.107:0/3426718659 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b8010910 con 0x7fa8cc1047e0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.026+0000 7fa8d2000640 1 -- 172.21.15.107:0/3426718659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc19f670 con 0x7fa8cc1047e0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.027+0000 7fa8c9600640 1 -- 172.21.15.107:0/3426718659 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8018990 con 0x7fa8cc1047e0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.027+0000 7fa8d2000640 1 -- 172.21.15.107:0/3426718659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc1a6f50 con 0x7fa8cc1047e0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.028+0000 7fa8d2000640 1 -- 172.21.15.107:0/3426718659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8cc104210 con 0x7fa8cc1047e0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.030+0000 7fa8c9600640 1 -- 172.21.15.107:0/3426718659 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa8b8010ac0 con 0x7fa8cc1047e0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.031+0000 7fa8c9600640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa8b40a3240 0x7fa8b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.031+0000 7fa8c9600640 1 -- 172.21.15.107:0/3426718659 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fa8b80cf850 con 0x7fa8cc1047e0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.031+0000 7fa8d0c00640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa8b40a3240 0x7fa8b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.032+0000 7fa8d0c00640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa8b40a3240 0x7fa8b40a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009c30 tx=0x7fa8c0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:04.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.035+0000 7fa8c9600640 1 -- 172.21.15.107:0/3426718659 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8b809c970 con 0x7fa8cc1047e0 2024-09-19T04:18:04.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.304+0000 7fa8d2000640 1 -- 172.21.15.107:0/3426718659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8cc10c1a0 con 0x7fa8cc1047e0 2024-09-19T04:18:04.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.304+0000 7fa8c9600640 1 -- 172.21.15.107:0/3426718659 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7fa8b809c270 con 0x7fa8cc1047e0 2024-09-19T04:18:04.306 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:04.306 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 13 2024-09-19T04:18:04.307 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":13,"btime":"2024-09-19T04:18:01:741629+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14499,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T04:18:04.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 -- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa8b40a3240 msgr2=0x7fa8b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa8b40a3240 0x7fa8b40a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009c30 tx=0x7fa8c0005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:04.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 -- 172.21.15.107:0/3426718659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8cc1047e0 msgr2=0x7fa8cc19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8cc1047e0 0x7fa8cc19a910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800e9f0 tx=0x7fa8b800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 -- 172.21.15.107:0/3426718659 shutdown_connections 2024-09-19T04:18:04.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa8b40a3240 0x7fa8b40a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc108f60 0x7fa8cc19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8cc1047e0 0x7fa8cc19a910 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 --2- 172.21.15.107:0/3426718659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc103e30 0x7fa8cc19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 -- 172.21.15.107:0/3426718659 >> 172.21.15.107:0/3426718659 conn(0x7fa8cc0ff8d0 msgr2=0x7fa8cc100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:04.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 -- 172.21.15.107:0/3426718659 shutdown_connections 2024-09-19T04:18:04.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:04.307+0000 7fa8d2000640 1 -- 172.21.15.107:0/3426718659 wait complete. 2024-09-19T04:18:04.653 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:04 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:04 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:04 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:04 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:04 smithi107 ceph-mon[30659]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:04.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:04 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3426718659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:04.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:04 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:04 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:04 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:04 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:04 smithi115 ceph-mon[35477]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:04 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3426718659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:04.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:04 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:04 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:04 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:04 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:04 smithi145 ceph-mon[35096]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:04 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3426718659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:05.033 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T04:18:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:05.884 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:05.884 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:05.885 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:05.885 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:05.885 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:18:05.885 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:05.885 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:05 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:18:06.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.990+0000 7f0c36c00640 1 -- 172.21.15.107:0/4169790535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c30101de0 msgr2=0x7f0c301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.990+0000 7f0c36c00640 1 --2- 172.21.15.107:0/4169790535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c30101de0 0x7f0c301021c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009900 tx=0x7f0c2002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.990+0000 7f0c36c00640 1 -- 172.21.15.107:0/4169790535 shutdown_connections 2024-09-19T04:18:06.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.990+0000 7f0c36c00640 1 --2- 172.21.15.107:0/4169790535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c30106f10 0x7f0c3010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.990+0000 7f0c36c00640 1 --2- 172.21.15.107:0/4169790535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c30102790 0x7f0c301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.990+0000 7f0c36c00640 1 --2- 172.21.15.107:0/4169790535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c30101de0 0x7f0c301021c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.990+0000 7f0c36c00640 1 -- 172.21.15.107:0/4169790535 >> 172.21.15.107:0/4169790535 conn(0x7f0c300fd8c0 msgr2=0x7f0c300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:06.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.990+0000 7f0c36c00640 1 -- 172.21.15.107:0/4169790535 shutdown_connections 2024-09-19T04:18:06.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.990+0000 7f0c36c00640 1 -- 172.21.15.107:0/4169790535 wait complete. 2024-09-19T04:18:06.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.991+0000 7f0c36c00640 1 Processor -- start 2024-09-19T04:18:06.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.991+0000 7f0c36c00640 1 -- start start 2024-09-19T04:18:06.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c36c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c30101de0 0x7f0c3019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c36c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c30102790 0x7f0c3019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c36c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c30106f10 0x7f0c3019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c36c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30111f70 con 0x7f0c30106f10 2024-09-19T04:18:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c36c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30111df0 con 0x7f0c30101de0 2024-09-19T04:18:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c36c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301120f0 con 0x7f0c30102790 2024-09-19T04:18:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c27400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c30102790 0x7f0c3019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c27400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c30102790 0x7f0c3019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46716/0 (socket says 172.21.15.107:46716) 2024-09-19T04:18:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c27400640 1 -- 172.21.15.107:0/3044399543 learned_addr learned my addr 172.21.15.107:0/3044399543 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c2fe00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c30101de0 0x7f0c3019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c34c00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c30106f10 0x7f0c3019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c27400640 1 -- 172.21.15.107:0/3044399543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c30101de0 msgr2=0x7f0c3019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c27400640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c30101de0 0x7f0c3019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c27400640 1 -- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c30106f10 msgr2=0x7f0c3019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c27400640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c30106f10 0x7f0c3019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c27400640 1 -- 172.21.15.107:0/3044399543 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3019f5d0 con 0x7f0c30102790 2024-09-19T04:18:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c34c00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c30106f10 0x7f0c3019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.992+0000 7f0c27400640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c30102790 0x7f0c3019ab60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800e9f0 tx=0x7f0c1800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.993+0000 7f0c2d600640 1 -- 172.21.15.107:0/3044399543 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800cd80 con 0x7f0c30102790 2024-09-19T04:18:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.993+0000 7f0c2d600640 1 -- 172.21.15.107:0/3044399543 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c18010910 con 0x7f0c30102790 2024-09-19T04:18:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.993+0000 7f0c36c00640 1 -- 172.21.15.107:0/3044399543 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3019f8c0 con 0x7f0c30102790 2024-09-19T04:18:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.993+0000 7f0c2d600640 1 -- 172.21.15.107:0/3044399543 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18018990 con 0x7f0c30102790 2024-09-19T04:18:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.993+0000 7f0c36c00640 1 -- 172.21.15.107:0/3044399543 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c301a71a0 con 0x7f0c30102790 2024-09-19T04:18:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.994+0000 7f0c36c00640 1 -- 172.21.15.107:0/3044399543 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c301021c0 con 0x7f0c30102790 2024-09-19T04:18:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.996+0000 7f0c2d600640 1 -- 172.21.15.107:0/3044399543 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0c18010ac0 con 0x7f0c30102790 2024-09-19T04:18:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.997+0000 7f0c2d600640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0c140a32c0 0x7f0c140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.997+0000 7f0c2d600640 1 -- 172.21.15.107:0/3044399543 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f0c180cfbd0 con 0x7f0c30102790 2024-09-19T04:18:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.997+0000 7f0c2fe00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0c140a32c0 0x7f0c140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.998+0000 7f0c2fe00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0c140a32c0 0x7f0c140a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009230 tx=0x7f0c200058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:05.999+0000 7f0c2d600640 1 -- 172.21.15.107:0/3044399543 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0c1809ccf0 con 0x7f0c30102790 2024-09-19T04:18:06.047 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:05 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:06.048 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:05 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:06.048 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:05 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:06.048 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:05 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:06.048 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:05 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:18:06.048 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:05 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:06.049 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:05 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:18:06.106 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:05 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:06.106 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:05 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:06.106 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:05 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:06.106 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:05 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:06.107 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:05 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:18:06.107 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:05 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:06.107 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:05 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:18:06.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.196+0000 7f0c36c00640 1 -- 172.21.15.107:0/3044399543 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0c301a74d0 con 0x7f0c30102790 2024-09-19T04:18:06.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.197+0000 7f0c2d600640 1 -- 172.21.15.107:0/3044399543 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f0c1809c5f0 con 0x7f0c30102790 2024-09-19T04:18:06.198 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:06.198 INFO:teuthology.orchestra.run.smithi107.stdout:[ 2024-09-19T04:18:06.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:18:06.199 INFO:teuthology.orchestra.run.smithi107.stdout: "poolnum": 1, 2024-09-19T04:18:06.199 INFO:teuthology.orchestra.run.smithi107.stdout: "poolname": ".mgr" 2024-09-19T04:18:06.199 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:18:06.199 INFO:teuthology.orchestra.run.smithi107.stdout:] 2024-09-19T04:18:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 -- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0c140a32c0 msgr2=0x7f0c140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0c140a32c0 0x7f0c140a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009230 tx=0x7f0c200058d0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 -- 172.21.15.107:0/3044399543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c30102790 msgr2=0x7f0c3019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c30102790 0x7f0c3019ab60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800e9f0 tx=0x7f0c1800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 -- 172.21.15.107:0/3044399543 shutdown_connections 2024-09-19T04:18:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0c140a32c0 0x7f0c140a5780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c30106f10 0x7f0c3019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c30102790 0x7f0c3019ab60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 --2- 172.21.15.107:0/3044399543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c30101de0 0x7f0c3019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 -- 172.21.15.107:0/3044399543 >> 172.21.15.107:0/3044399543 conn(0x7f0c300fd8c0 msgr2=0x7f0c300feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:06.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 -- 172.21.15.107:0/3044399543 shutdown_connections 2024-09-19T04:18:06.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:06.199+0000 7f0c36c00640 1 -- 172.21.15.107:0/3044399543 wait complete. 2024-09-19T04:18:06.914 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:06 smithi107 ceph-mon[30659]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:06.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:06 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3044399543' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:18:06.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:06 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:06.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:06 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:06.964 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:18:06.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:18:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:06 smithi115 ceph-mon[35477]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:07.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:06 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3044399543' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:18:07.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:06 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:07.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:06 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:07.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:06 smithi145 ceph-mon[35096]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:07.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:06 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3044399543' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:18:07.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:06 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:07.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:06 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:07.722 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:08.072 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:08.073 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:07 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:08.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:08.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:07 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:08.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:08.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:07 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:08.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.283+0000 7f31ecc00640 1 -- 172.21.15.107:0/3685418062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8108f60 msgr2=0x7f31e8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.283+0000 7f31ecc00640 1 --2- 172.21.15.107:0/3685418062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8108f60 0x7f31e8110ae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009ca0 tx=0x7f31e002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.283+0000 7f31ecc00640 1 -- 172.21.15.107:0/3685418062 shutdown_connections 2024-09-19T04:18:08.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.283+0000 7f31ecc00640 1 --2- 172.21.15.107:0/3685418062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8108f60 0x7f31e8110ae0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.283+0000 7f31ecc00640 1 --2- 172.21.15.107:0/3685418062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f31e81047e0 0x7f31e8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.283+0000 7f31ecc00640 1 --2- 172.21.15.107:0/3685418062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e8103e30 0x7f31e8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.283+0000 7f31ecc00640 1 -- 172.21.15.107:0/3685418062 >> 172.21.15.107:0/3685418062 conn(0x7f31e80ff910 msgr2=0x7f31e8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:08.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.283+0000 7f31ecc00640 1 -- 172.21.15.107:0/3685418062 shutdown_connections 2024-09-19T04:18:08.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.284+0000 7f31ecc00640 1 -- 172.21.15.107:0/3685418062 wait complete. 2024-09-19T04:18:08.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.285+0000 7f31ecc00640 1 Processor -- start 2024-09-19T04:18:08.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.285+0000 7f31ecc00640 1 -- start start 2024-09-19T04:18:08.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31ecc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f31e8103e30 0x7f31e819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31ecc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81047e0 0x7f31e819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31ecc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8108f60 0x7f31e819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31ecc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f31e8114010 con 0x7f31e8108f60 2024-09-19T04:18:08.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31ecc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f31e8113e90 con 0x7f31e81047e0 2024-09-19T04:18:08.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31ecc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f31e8114190 con 0x7f31e8103e30 2024-09-19T04:18:08.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31e7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8108f60 0x7f31e819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31dfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81047e0 0x7f31e819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31dfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81047e0 0x7f31e819a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39050/0 (socket says 172.21.15.107:39050) 2024-09-19T04:18:08.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31dfe00640 1 -- 172.21.15.107:0/3971226411 learned_addr learned my addr 172.21.15.107:0/3971226411 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:08.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31e7400640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f31e8103e30 0x7f31e819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31dfe00640 1 -- 172.21.15.107:0/3971226411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f31e8103e30 msgr2=0x7f31e819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31dfe00640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f31e8103e30 0x7f31e819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31dfe00640 1 -- 172.21.15.107:0/3971226411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8108f60 msgr2=0x7f31e819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31dfe00640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8108f60 0x7f31e819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31dfe00640 1 -- 172.21.15.107:0/3971226411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e819f3a0 con 0x7f31e81047e0 2024-09-19T04:18:08.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.286+0000 7f31e7400640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f31e8103e30 0x7f31e819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:08.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.287+0000 7f31dfe00640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81047e0 0x7f31e819a930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f31d000e9f0 tx=0x7f31d000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.287+0000 7f31e4c00640 1 -- 172.21.15.107:0/3971226411 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d000cd80 con 0x7f31e81047e0 2024-09-19T04:18:08.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.287+0000 7f31e4c00640 1 -- 172.21.15.107:0/3971226411 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d0005500 con 0x7f31e81047e0 2024-09-19T04:18:08.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.287+0000 7f31ecc00640 1 -- 172.21.15.107:0/3971226411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e819f690 con 0x7f31e81047e0 2024-09-19T04:18:08.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.287+0000 7f31e4c00640 1 -- 172.21.15.107:0/3971226411 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0010690 con 0x7f31e81047e0 2024-09-19T04:18:08.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.287+0000 7f31ecc00640 1 -- 172.21.15.107:0/3971226411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e81a6f70 con 0x7f31e81047e0 2024-09-19T04:18:08.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.289+0000 7f31ecc00640 1 -- 172.21.15.107:0/3971226411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e8105310 con 0x7f31e81047e0 2024-09-19T04:18:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.291+0000 7f31e4c00640 1 -- 172.21.15.107:0/3971226411 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f31d00046c0 con 0x7f31e81047e0 2024-09-19T04:18:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.293+0000 7f31e4c00640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f31cc0a3240 0x7f31cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.293+0000 7f31e4c00640 1 -- 172.21.15.107:0/3971226411 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f31d00cf940 con 0x7f31e81047e0 2024-09-19T04:18:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.293+0000 7f31e7400640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f31cc0a3240 0x7f31cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.294+0000 7f31e7400640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f31cc0a3240 0x7f31cc0a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009b50 tx=0x7f31d8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.296+0000 7f31e4c00640 1 -- 172.21.15.107:0/3971226411 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f31d009ca60 con 0x7f31e81047e0 2024-09-19T04:18:08.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:08.487+0000 7f31ecc00640 1 -- 172.21.15.107:0/3971226411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f31e802d7c0 con 0x7f31e81047e0 2024-09-19T04:18:09.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.086+0000 7f31e4c00640 1 -- 172.21.15.107:0/3971226411 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7f31d009c360 con 0x7f31e81047e0 2024-09-19T04:18:09.088 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' created 2024-09-19T04:18:09.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.090+0000 7f31ecc00640 1 -- 172.21.15.107:0/3971226411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f31cc0a3240 msgr2=0x7f31cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:09.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.090+0000 7f31ecc00640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f31cc0a3240 0x7f31cc0a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009b50 tx=0x7f31d8005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:09.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.090+0000 7f31ecc00640 1 -- 172.21.15.107:0/3971226411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81047e0 msgr2=0x7f31e819a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:09.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.090+0000 7f31ecc00640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81047e0 0x7f31e819a930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f31d000e9f0 tx=0x7f31d000edf0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.090+0000 7f31ecc00640 1 -- 172.21.15.107:0/3971226411 shutdown_connections 2024-09-19T04:18:09.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.090+0000 7f31ecc00640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f31cc0a3240 0x7f31cc0a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.090+0000 7f31ecc00640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8108f60 0x7f31e819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.090+0000 7f31ecc00640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81047e0 0x7f31e819a930 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.090+0000 7f31ecc00640 1 --2- 172.21.15.107:0/3971226411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f31e8103e30 0x7f31e819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:09.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.090+0000 7f31ecc00640 1 -- 172.21.15.107:0/3971226411 >> 172.21.15.107:0/3971226411 conn(0x7f31e80ff910 msgr2=0x7f31e8100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:09.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.091+0000 7f31ecc00640 1 -- 172.21.15.107:0/3971226411 shutdown_connections 2024-09-19T04:18:09.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:09.091+0000 7f31ecc00640 1 -- 172.21.15.107:0/3971226411 wait complete. 2024-09-19T04:18:09.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:09 smithi115 ceph-mon[35477]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:09.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:09 smithi145 ceph-mon[35096]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:09.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:09 smithi107 ceph-mon[30659]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:09.807 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:18:10.182 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:10.320 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:10 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3971226411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:18:10.321 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:10 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:18:10.321 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:10.321 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:10.321 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:10.321 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:18:10.321 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:10.321 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:10 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:18:10.321 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:10 smithi107 ceph-mon[30659]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:18:10.321 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:10 smithi107 ceph-mon[30659]: pgmap v282: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:10.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:10 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3971226411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:18:10.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:10 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:18:10.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:10.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:10.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:10.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:18:10.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:10.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:10 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:18:10.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:10 smithi115 ceph-mon[35477]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:18:10.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:10 smithi115 ceph-mon[35477]: pgmap v282: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:10.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:10 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3971226411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:18:10.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:10 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:18:10.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:10.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:10.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:18:10.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:18:10.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:18:10.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:10 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:18:10.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:10 smithi145 ceph-mon[35096]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:18:10.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:10 smithi145 ceph-mon[35096]: pgmap v282: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.764+0000 7f9290800640 1 -- 172.21.15.107:0/2223268871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928810e3c0 msgr2=0x7f9288110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.764+0000 7f9290800640 1 --2- 172.21.15.107:0/2223268871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928810e3c0 0x7f9288110810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9284009ca0 tx=0x7f928402a950 comp rx=0 tx=0).stop 2024-09-19T04:18:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.764+0000 7f9290800640 1 -- 172.21.15.107:0/2223268871 shutdown_connections 2024-09-19T04:18:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.764+0000 7f9290800640 1 --2- 172.21.15.107:0/2223268871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928810e3c0 0x7f9288110810 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.764+0000 7f9290800640 1 --2- 172.21.15.107:0/2223268871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9288074f40 0x7f928810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.764+0000 7f9290800640 1 --2- 172.21.15.107:0/2223268871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9288074620 0x7f9288074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.764+0000 7f9290800640 1 -- 172.21.15.107:0/2223268871 >> 172.21.15.107:0/2223268871 conn(0x7f92880fd8c0 msgr2=0x7f92880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.764+0000 7f9290800640 1 -- 172.21.15.107:0/2223268871 shutdown_connections 2024-09-19T04:18:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.764+0000 7f9290800640 1 -- 172.21.15.107:0/2223268871 wait complete. 2024-09-19T04:18:10.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.765+0000 7f9290800640 1 Processor -- start 2024-09-19T04:18:10.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.765+0000 7f9290800640 1 -- start start 2024-09-19T04:18:10.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f9290800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288074620 0x7f928819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:10.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f9290800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9288074f40 0x7f928819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:10.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f9290800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928810e3c0 0x7f928819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:10.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f9290800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9288114150 con 0x7f9288074620 2024-09-19T04:18:10.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f9290800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9288113fd0 con 0x7f928810e3c0 2024-09-19T04:18:10.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f9290800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f92881142d0 con 0x7f9288074f40 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f928de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288074620 0x7f928819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f928d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9288074f40 0x7f928819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f928d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9288074f40 0x7f928819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51820/0 (socket says 172.21.15.107:51820) 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f928d400640 1 -- 172.21.15.107:0/42237256 learned_addr learned my addr 172.21.15.107:0/42237256 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f928e800640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928810e3c0 0x7f928819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f928d400640 1 -- 172.21.15.107:0/42237256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928810e3c0 msgr2=0x7f928819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f928d400640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928810e3c0 0x7f928819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f928d400640 1 -- 172.21.15.107:0/42237256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288074620 msgr2=0x7f928819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f928d400640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288074620 0x7f928819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.766+0000 7f928d400640 1 -- 172.21.15.107:0/42237256 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928819f620 con 0x7f9288074f40 2024-09-19T04:18:10.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.767+0000 7f928d400640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9288074f40 0x7f928819abb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f927800d950 tx=0x7f927800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:10.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.767+0000 7f9282a00640 1 -- 172.21.15.107:0/42237256 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278017030 con 0x7f9288074f40 2024-09-19T04:18:10.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.767+0000 7f9290800640 1 -- 172.21.15.107:0/42237256 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928819f910 con 0x7f9288074f40 2024-09-19T04:18:10.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.767+0000 7f9282a00640 1 -- 172.21.15.107:0/42237256 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9278004020 con 0x7f9288074f40 2024-09-19T04:18:10.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.767+0000 7f9282a00640 1 -- 172.21.15.107:0/42237256 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92780047a0 con 0x7f9288074f40 2024-09-19T04:18:10.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.767+0000 7f9290800640 1 -- 172.21.15.107:0/42237256 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92881a71f0 con 0x7f9288074f40 2024-09-19T04:18:10.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.768+0000 7f9290800640 1 -- 172.21.15.107:0/42237256 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9288074a00 con 0x7f9288074f40 2024-09-19T04:18:10.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.770+0000 7f9282a00640 1 -- 172.21.15.107:0/42237256 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f92780041d0 con 0x7f9288074f40 2024-09-19T04:18:10.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.771+0000 7f9282a00640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92700a3240 0x7f92700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:10.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.771+0000 7f9282a00640 1 -- 172.21.15.107:0/42237256 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7f92780cef50 con 0x7f9288074f40 2024-09-19T04:18:10.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.772+0000 7f928de00640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92700a3240 0x7f92700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:10.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.772+0000 7f928de00640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92700a3240 0x7f92700a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f927c005d00 tx=0x7f927c00b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:10.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.774+0000 7f9282a00640 1 -- 172.21.15.107:0/42237256 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f927809bf90 con 0x7f9288074f40 2024-09-19T04:18:10.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:10.950+0000 7f9290800640 1 -- 172.21.15.107:0/42237256 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f928819b9d0 con 0x7f9288074f40 2024-09-19T04:18:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.473+0000 7f9282a00640 1 -- 172.21.15.107:0/42237256 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f927809b890 con 0x7f9288074f40 2024-09-19T04:18:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' created 2024-09-19T04:18:11.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.478+0000 7f9290800640 1 -- 172.21.15.107:0/42237256 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92700a3240 msgr2=0x7f92700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.478+0000 7f9290800640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92700a3240 0x7f92700a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f927c005d00 tx=0x7f927c00b410 comp rx=0 tx=0).stop 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.478+0000 7f9290800640 1 -- 172.21.15.107:0/42237256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9288074f40 msgr2=0x7f928819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.478+0000 7f9290800640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9288074f40 0x7f928819abb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f927800d950 tx=0x7f927800de10 comp rx=0 tx=0).stop 2024-09-19T04:18:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.479+0000 7f9290800640 1 -- 172.21.15.107:0/42237256 shutdown_connections 2024-09-19T04:18:11.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.479+0000 7f9290800640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92700a3240 0x7f92700a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.479+0000 7f9290800640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928810e3c0 0x7f928819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.479+0000 7f9290800640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9288074f40 0x7f928819abb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.479+0000 7f9290800640 1 --2- 172.21.15.107:0/42237256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288074620 0x7f928819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.479+0000 7f9290800640 1 -- 172.21.15.107:0/42237256 >> 172.21.15.107:0/42237256 conn(0x7f92880fd8c0 msgr2=0x7f92880fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.479+0000 7f9290800640 1 -- 172.21.15.107:0/42237256 shutdown_connections 2024-09-19T04:18:11.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:11.480+0000 7f9290800640 1 -- 172.21.15.107:0/42237256 wait complete. 2024-09-19T04:18:11.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:11 smithi145 ceph-mon[35096]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:18:11.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:11 smithi145 ceph-mon[35096]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:18:11.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:11 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/42237256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:18:11.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:11 smithi107 ceph-mon[30659]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:18:11.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:11 smithi107 ceph-mon[30659]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:18:11.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:11 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/42237256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:18:11.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:11 smithi115 ceph-mon[35477]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:18:11.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:11 smithi115 ceph-mon[35477]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:18:11.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:11 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/42237256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:18:12.183 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:18:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:12.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:12 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:18:12.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:12 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:18:12.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:12 smithi145 ceph-mon[35096]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:18:12.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:12 smithi145 ceph-mon[35096]: pgmap v285: 97 pgs: 8 creating+peering, 80 unknown, 9 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:12.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:12 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:18:12.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:12 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:18:12.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:12 smithi107 ceph-mon[30659]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:18:12.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:12 smithi107 ceph-mon[30659]: pgmap v285: 97 pgs: 8 creating+peering, 80 unknown, 9 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:12 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:18:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:12 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:18:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:12 smithi115 ceph-mon[35477]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:18:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:12 smithi115 ceph-mon[35477]: pgmap v285: 97 pgs: 8 creating+peering, 80 unknown, 9 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:13.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.127+0000 7fd0e7200640 1 -- 172.21.15.107:0/219082039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e81047e0 msgr2=0x7fd0e8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.127+0000 7fd0e7200640 1 --2- 172.21.15.107:0/219082039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e81047e0 0x7fd0e8108830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009900 tx=0x7fd0d402a410 comp rx=0 tx=0).stop 2024-09-19T04:18:13.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.127+0000 7fd0e7200640 1 -- 172.21.15.107:0/219082039 shutdown_connections 2024-09-19T04:18:13.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.127+0000 7fd0e7200640 1 --2- 172.21.15.107:0/219082039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e8108f60 0x7fd0e8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.127+0000 7fd0e7200640 1 --2- 172.21.15.107:0/219082039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e81047e0 0x7fd0e8108830 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.127+0000 7fd0e7200640 1 --2- 172.21.15.107:0/219082039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0e8103e30 0x7fd0e8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.127+0000 7fd0e7200640 1 -- 172.21.15.107:0/219082039 >> 172.21.15.107:0/219082039 conn(0x7fd0e80ff910 msgr2=0x7fd0e8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:13.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.127+0000 7fd0e7200640 1 -- 172.21.15.107:0/219082039 shutdown_connections 2024-09-19T04:18:13.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.127+0000 7fd0e7200640 1 -- 172.21.15.107:0/219082039 wait complete. 2024-09-19T04:18:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.128+0000 7fd0e7200640 1 Processor -- start 2024-09-19T04:18:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.128+0000 7fd0e7200640 1 -- start start 2024-09-19T04:18:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e8103e30 0x7fd0e819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e7200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0e81047e0 0x7fd0e819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e8108f60 0x7fd0e819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8114010 con 0x7fd0e8108f60 2024-09-19T04:18:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e7200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8113e90 con 0x7fd0e81047e0 2024-09-19T04:18:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8114190 con 0x7fd0e8103e30 2024-09-19T04:18:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0e81047e0 0x7fd0e819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e8108f60 0x7fd0e819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0e81047e0 0x7fd0e819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45034/0 (socket says 172.21.15.107:45034) 2024-09-19T04:18:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e5400640 1 -- 172.21.15.107:0/2334884506 learned_addr learned my addr 172.21.15.107:0/2334884506 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e5e00640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e8103e30 0x7fd0e819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e6800640 1 -- 172.21.15.107:0/2334884506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e8103e30 msgr2=0x7fd0e819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e6800640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e8103e30 0x7fd0e819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e6800640 1 -- 172.21.15.107:0/2334884506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0e81047e0 msgr2=0x7fd0e819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e6800640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0e81047e0 0x7fd0e819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e6800640 1 -- 172.21.15.107:0/2334884506 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e819f420 con 0x7fd0e8108f60 2024-09-19T04:18:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.129+0000 7fd0e5400640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0e81047e0 0x7fd0e819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:13.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.130+0000 7fd0e6800640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e8108f60 0x7fd0e819ed40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009fd0 tx=0x7fd0e000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.130+0000 7fd0dea00640 1 -- 172.21.15.107:0/2334884506 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e000ce20 con 0x7fd0e8108f60 2024-09-19T04:18:13.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.130+0000 7fd0dea00640 1 -- 172.21.15.107:0/2334884506 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0e0005020 con 0x7fd0e8108f60 2024-09-19T04:18:13.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.130+0000 7fd0e7200640 1 -- 172.21.15.107:0/2334884506 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e819f710 con 0x7fd0e8108f60 2024-09-19T04:18:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.130+0000 7fd0dea00640 1 -- 172.21.15.107:0/2334884506 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0010720 con 0x7fd0e8108f60 2024-09-19T04:18:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.130+0000 7fd0e7200640 1 -- 172.21.15.107:0/2334884506 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e81a6ff0 con 0x7fd0e8108f60 2024-09-19T04:18:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.131+0000 7fd0e7200640 1 -- 172.21.15.107:0/2334884506 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e8105310 con 0x7fd0e8108f60 2024-09-19T04:18:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.133+0000 7fd0dea00640 1 -- 172.21.15.107:0/2334884506 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd0e0005540 con 0x7fd0e8108f60 2024-09-19T04:18:13.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.134+0000 7fd0dea00640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd0cc0a32c0 0x7fd0cc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.135+0000 7fd0dea00640 1 -- 172.21.15.107:0/2334884506 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7fd0e00cfd20 con 0x7fd0e8108f60 2024-09-19T04:18:13.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.135+0000 7fd0e5e00640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd0cc0a32c0 0x7fd0cc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.135+0000 7fd0e5e00640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd0cc0a32c0 0x7fd0cc0a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8009bd0 tx=0x7fd0d8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.139+0000 7fd0dea00640 1 -- 172.21.15.107:0/2334884506 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0e009cb50 con 0x7fd0e8108f60 2024-09-19T04:18:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.347+0000 7fd0e7200640 1 -- 172.21.15.107:0/2334884506 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd0e802d7c0 con 0x7fd0e8108f60 2024-09-19T04:18:13.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.474+0000 7fd0dea00640 1 -- 172.21.15.107:0/2334884506 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd0e009c450 con 0x7fd0e8108f60 2024-09-19T04:18:13.476 INFO:teuthology.orchestra.run.smithi107.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T04:18:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 -- 172.21.15.107:0/2334884506 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd0cc0a32c0 msgr2=0x7fd0cc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd0cc0a32c0 0x7fd0cc0a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8009bd0 tx=0x7fd0d8005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 -- 172.21.15.107:0/2334884506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e8108f60 msgr2=0x7fd0e819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e8108f60 0x7fd0e819ed40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009fd0 tx=0x7fd0e000ef10 comp rx=0 tx=0).stop 2024-09-19T04:18:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 -- 172.21.15.107:0/2334884506 shutdown_connections 2024-09-19T04:18:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd0cc0a32c0 0x7fd0cc0a5780 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e8108f60 0x7fd0e819ed40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0e81047e0 0x7fd0e819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 --2- 172.21.15.107:0/2334884506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e8103e30 0x7fd0e819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 -- 172.21.15.107:0/2334884506 >> 172.21.15.107:0/2334884506 conn(0x7fd0e80ff910 msgr2=0x7fd0e8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:13.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 -- 172.21.15.107:0/2334884506 shutdown_connections 2024-09-19T04:18:13.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:13.477+0000 7fd0e7200640 1 -- 172.21.15.107:0/2334884506 wait complete. 2024-09-19T04:18:13.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:13 smithi107 ceph-mon[30659]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:18:13.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:13 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2334884506' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:18:13.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:13 smithi107 ceph-mon[30659]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:18:13.823 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:13 smithi115 ceph-mon[35477]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:18:13.824 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:13 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2334884506' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:18:13.824 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:13 smithi115 ceph-mon[35477]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:18:13.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:13 smithi145 ceph-mon[35096]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:18:13.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:13 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2334884506' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:18:13.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:13 smithi145 ceph-mon[35096]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:18:14.310 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:18:14.680 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:14.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:14 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2334884506' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:18:14.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:14 smithi115 ceph-mon[35477]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:18:14.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:14 smithi115 ceph-mon[35477]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:18:14.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:14 smithi115 ceph-mon[35477]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:18:14.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:14 smithi115 ceph-mon[35477]: pgmap v288: 97 pgs: 8 creating+peering, 36 unknown, 53 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:14.819 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:14 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2334884506' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:18:14.819 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:14 smithi107 ceph-mon[30659]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:18:14.819 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:14 smithi107 ceph-mon[30659]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:18:14.819 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:14 smithi107 ceph-mon[30659]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:18:14.819 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:14 smithi107 ceph-mon[30659]: pgmap v288: 97 pgs: 8 creating+peering, 36 unknown, 53 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:14.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:14 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2334884506' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:18:14.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:14 smithi145 ceph-mon[35096]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:18:14.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:14 smithi145 ceph-mon[35096]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:18:14.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:14 smithi145 ceph-mon[35096]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:18:14.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:14 smithi145 ceph-mon[35096]: pgmap v288: 97 pgs: 8 creating+peering, 36 unknown, 53 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:18:15.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.216+0000 7f7924400640 1 -- 172.21.15.107:0/1036034778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f791c106f10 msgr2=0x7f791c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.216+0000 7f7924400640 1 --2- 172.21.15.107:0/1036034778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f791c106f10 0x7f791c10ea90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7914009ca0 tx=0x7f791402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.216+0000 7f7924400640 1 -- 172.21.15.107:0/1036034778 shutdown_connections 2024-09-19T04:18:15.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.216+0000 7f7924400640 1 --2- 172.21.15.107:0/1036034778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f791c106f10 0x7f791c10ea90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.216+0000 7f7924400640 1 --2- 172.21.15.107:0/1036034778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f791c102790 0x7f791c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.216+0000 7f7924400640 1 --2- 172.21.15.107:0/1036034778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f791c101de0 0x7f791c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.216+0000 7f7924400640 1 -- 172.21.15.107:0/1036034778 >> 172.21.15.107:0/1036034778 conn(0x7f791c0fd8c0 msgr2=0x7f791c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:15.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.216+0000 7f7924400640 1 -- 172.21.15.107:0/1036034778 shutdown_connections 2024-09-19T04:18:15.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.217+0000 7f7924400640 1 -- 172.21.15.107:0/1036034778 wait complete. 2024-09-19T04:18:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.217+0000 7f7924400640 1 Processor -- start 2024-09-19T04:18:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7924400640 1 -- start start 2024-09-19T04:18:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7924400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f791c101de0 0x7f791c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7924400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f791c102790 0x7f791c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7924400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f791c106f10 0x7f791c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7924400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f791c111fc0 con 0x7f791c101de0 2024-09-19T04:18:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7924400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f791c111e40 con 0x7f791c106f10 2024-09-19T04:18:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7924400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f791c112140 con 0x7f791c102790 2024-09-19T04:18:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7921a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f791c101de0 0x7f791c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7921a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f791c101de0 0x7f791c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52670/0 (socket says 172.21.15.107:52670) 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7921a00640 1 -- 172.21.15.107:0/1508215982 learned_addr learned my addr 172.21.15.107:0/1508215982 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7922400640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f791c106f10 0x7f791c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.218+0000 7f7921000640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f791c102790 0x7f791c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f7921a00640 1 -- 172.21.15.107:0/1508215982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f791c102790 msgr2=0x7f791c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f7921a00640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f791c102790 0x7f791c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f7921a00640 1 -- 172.21.15.107:0/1508215982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f791c106f10 msgr2=0x7f791c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f7921a00640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f791c106f10 0x7f791c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f7921a00640 1 -- 172.21.15.107:0/1508215982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c19d330 con 0x7f791c101de0 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f7921000640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f791c102790 0x7f791c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f7921a00640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f791c101de0 0x7f791c198380 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f791000e9f0 tx=0x7f791000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f791a000640 1 -- 172.21.15.107:0/1508215982 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791000cd80 con 0x7f791c101de0 2024-09-19T04:18:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f791a000640 1 -- 172.21.15.107:0/1508215982 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7910010910 con 0x7f791c101de0 2024-09-19T04:18:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f7924400640 1 -- 172.21.15.107:0/1508215982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c19d620 con 0x7f791c101de0 2024-09-19T04:18:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f791a000640 1 -- 172.21.15.107:0/1508215982 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910018990 con 0x7f791c101de0 2024-09-19T04:18:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.219+0000 7f7924400640 1 -- 172.21.15.107:0/1508215982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c1a4f00 con 0x7f791c101de0 2024-09-19T04:18:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.220+0000 7f7924400640 1 -- 172.21.15.107:0/1508215982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791c1021c0 con 0x7f791c101de0 2024-09-19T04:18:15.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.222+0000 7f791a000640 1 -- 172.21.15.107:0/1508215982 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7910010ac0 con 0x7f791c101de0 2024-09-19T04:18:15.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.223+0000 7f791a000640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79040a3240 0x7f79040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:15.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.224+0000 7f791a000640 1 -- 172.21.15.107:0/1508215982 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f79100cfc20 con 0x7f791c101de0 2024-09-19T04:18:15.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.224+0000 7f7921000640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79040a3240 0x7f79040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:15.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.225+0000 7f7921000640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79040a3240 0x7f79040a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f791c0fdce0 tx=0x7f7908005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:15.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.227+0000 7f791a000640 1 -- 172.21.15.107:0/1508215982 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f791009c9f0 con 0x7f791c101de0 2024-09-19T04:18:15.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.403+0000 7f7924400640 1 -- 172.21.15.107:0/1508215982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f791c1996e0 con 0x7f791c101de0 2024-09-19T04:18:15.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.406+0000 7f791a000640 1 -- 172.21.15.107:0/1508215982 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22348 (secure 0 0 0) 0x7f79100ab760 con 0x7f791c101de0 2024-09-19T04:18:15.409 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:15.409 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":81,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:18:13.349028+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:15.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.408+0000 7f7924400640 1 -- 172.21.15.107:0/1508215982 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79040a3240 msgr2=0x7f79040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.408+0000 7f7924400640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79040a3240 0x7f79040a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f791c0fdce0 tx=0x7f7908005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:15.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.408+0000 7f7924400640 1 -- 172.21.15.107:0/1508215982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f791c101de0 msgr2=0x7f791c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:15.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.408+0000 7f7924400640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f791c101de0 0x7f791c198380 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f791000e9f0 tx=0x7f791000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.408+0000 7f7924400640 1 -- 172.21.15.107:0/1508215982 shutdown_connections 2024-09-19T04:18:15.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.408+0000 7f7924400640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79040a3240 0x7f79040a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.408+0000 7f7924400640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f791c106f10 0x7f791c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.408+0000 7f7924400640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f791c102790 0x7f791c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.408+0000 7f7924400640 1 --2- 172.21.15.107:0/1508215982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f791c101de0 0x7f791c198380 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:15.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.408+0000 7f7924400640 1 -- 172.21.15.107:0/1508215982 >> 172.21.15.107:0/1508215982 conn(0x7f791c0fd8c0 msgr2=0x7f791c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:15.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.409+0000 7f7924400640 1 -- 172.21.15.107:0/1508215982 shutdown_connections 2024-09-19T04:18:15.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:15.409+0000 7f7924400640 1 -- 172.21.15.107:0/1508215982 wait complete. 2024-09-19T04:18:15.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:15 smithi107 ceph-mon[30659]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:18:15.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:15 smithi107 ceph-mon[30659]: Cluster is now healthy 2024-09-19T04:18:15.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:15 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] up:active 2024-09-19T04:18:15.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:15 smithi107 ceph-mon[30659]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:15.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:15 smithi107 ceph-mon[30659]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:15.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:15 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1508215982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:15.868 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:15 smithi115 ceph-mon[35477]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:18:15.868 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:15 smithi115 ceph-mon[35477]: Cluster is now healthy 2024-09-19T04:18:15.868 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:15 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] up:active 2024-09-19T04:18:15.869 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:15 smithi115 ceph-mon[35477]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:15.869 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:15 smithi115 ceph-mon[35477]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:15.869 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:15 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1508215982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:15.875 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:15 smithi145 ceph-mon[35096]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:18:15.875 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:15 smithi145 ceph-mon[35096]: Cluster is now healthy 2024-09-19T04:18:15.876 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:15 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] up:active 2024-09-19T04:18:15.876 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:15 smithi145 ceph-mon[35096]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:15.876 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:15 smithi145 ceph-mon[35096]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:15.876 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:15 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1508215982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:16.291 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:18:16.677 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:16.751 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:16 smithi115 ceph-mon[35477]: pgmap v289: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 358 B/s wr, 0 op/s 2024-09-19T04:18:16.817 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:16 smithi107 ceph-mon[30659]: pgmap v289: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 358 B/s wr, 0 op/s 2024-09-19T04:18:16.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:16 smithi145 ceph-mon[35096]: pgmap v289: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 358 B/s wr, 0 op/s 2024-09-19T04:18:17.196 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b[65574]: 2024-09-19T04:18:16.749+0000 7fc96b170640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:18:17.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.277+0000 7f0264400640 1 -- 172.21.15.107:0/2810626045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025c106f10 msgr2=0x7f025c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:17.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.277+0000 7f0264400640 1 --2- 172.21.15.107:0/2810626045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025c106f10 0x7f025c10ea90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0254009be0 tx=0x7f025402a6d0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.277+0000 7f0264400640 1 -- 172.21.15.107:0/2810626045 shutdown_connections 2024-09-19T04:18:17.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.277+0000 7f0264400640 1 --2- 172.21.15.107:0/2810626045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025c106f10 0x7f025c10ea90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.277+0000 7f0264400640 1 --2- 172.21.15.107:0/2810626045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f025c102790 0x7f025c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.277+0000 7f0264400640 1 --2- 172.21.15.107:0/2810626045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f025c101de0 0x7f025c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.277+0000 7f0264400640 1 -- 172.21.15.107:0/2810626045 >> 172.21.15.107:0/2810626045 conn(0x7f025c0fd8c0 msgr2=0x7f025c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:17.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.277+0000 7f0264400640 1 -- 172.21.15.107:0/2810626045 shutdown_connections 2024-09-19T04:18:17.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.277+0000 7f0264400640 1 -- 172.21.15.107:0/2810626045 wait complete. 2024-09-19T04:18:17.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.278+0000 7f0264400640 1 Processor -- start 2024-09-19T04:18:17.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.278+0000 7f0264400640 1 -- start start 2024-09-19T04:18:17.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.278+0000 7f0264400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025c101de0 0x7f025c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:17.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.278+0000 7f0264400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f025c102790 0x7f025c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:17.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.278+0000 7f0264400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f025c106f10 0x7f025c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:17.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0264400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f025c111f70 con 0x7f025c106f10 2024-09-19T04:18:17.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0264400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f025c111df0 con 0x7f025c101de0 2024-09-19T04:18:17.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0264400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1120f0 con 0x7f025c102790 2024-09-19T04:18:17.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0261a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025c101de0 0x7f025c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:17.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0261000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f025c102790 0x7f025c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0261a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025c101de0 0x7f025c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45074/0 (socket says 172.21.15.107:45074) 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0261a00640 1 -- 172.21.15.107:0/2222991308 learned_addr learned my addr 172.21.15.107:0/2222991308 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0262400640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f025c106f10 0x7f025c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0261000640 1 -- 172.21.15.107:0/2222991308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025c101de0 msgr2=0x7f025c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0261000640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025c101de0 0x7f025c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0261000640 1 -- 172.21.15.107:0/2222991308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f025c106f10 msgr2=0x7f025c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0261000640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f025c106f10 0x7f025c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.279+0000 7f0261000640 1 -- 172.21.15.107:0/2222991308 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c19d3a0 con 0x7f025c102790 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.280+0000 7f0261000640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f025c102790 0x7f025c198930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0248005fd0 tx=0x7f024800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.280+0000 7f025a000640 1 -- 172.21.15.107:0/2222991308 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024800cd00 con 0x7f025c102790 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.280+0000 7f025a000640 1 -- 172.21.15.107:0/2222991308 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02480035f0 con 0x7f025c102790 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.280+0000 7f0264400640 1 -- 172.21.15.107:0/2222991308 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c19d690 con 0x7f025c102790 2024-09-19T04:18:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.280+0000 7f025a000640 1 -- 172.21.15.107:0/2222991308 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248010670 con 0x7f025c102790 2024-09-19T04:18:17.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.280+0000 7f0264400640 1 -- 172.21.15.107:0/2222991308 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c1a4f70 con 0x7f025c102790 2024-09-19T04:18:17.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.281+0000 7f0264400640 1 -- 172.21.15.107:0/2222991308 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c1021c0 con 0x7f025c102790 2024-09-19T04:18:17.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.283+0000 7f025a000640 1 -- 172.21.15.107:0/2222991308 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f02480037a0 con 0x7f025c102790 2024-09-19T04:18:17.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.285+0000 7f025a000640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f02440a3240 0x7f02440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:17.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.285+0000 7f025a000640 1 -- 172.21.15.107:0/2222991308 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f02480cf9b0 con 0x7f025c102790 2024-09-19T04:18:17.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.285+0000 7f0261a00640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f02440a3240 0x7f02440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.286+0000 7f0261a00640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f02440a3240 0x7f02440a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0250005fd0 tx=0x7f025000b4a0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.289+0000 7f025a000640 1 -- 172.21.15.107:0/2222991308 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0248009510 con 0x7f025c102790 2024-09-19T04:18:17.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.465+0000 7f0264400640 1 -- 172.21.15.107:0/2222991308 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f025c199750 con 0x7f025c102790 2024-09-19T04:18:17.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.467+0000 7f025a000640 1 -- 172.21.15.107:0/2222991308 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22348 (secure 0 0 0) 0x7f025c199750 con 0x7f025c102790 2024-09-19T04:18:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:17.469 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":81,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:18:13.349028+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.469+0000 7f0264400640 1 -- 172.21.15.107:0/2222991308 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f02440a3240 msgr2=0x7f02440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.469+0000 7f0264400640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f02440a3240 0x7f02440a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0250005fd0 tx=0x7f025000b4a0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.469+0000 7f0264400640 1 -- 172.21.15.107:0/2222991308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f025c102790 msgr2=0x7f025c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.469+0000 7f0264400640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f025c102790 0x7f025c198930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0248005fd0 tx=0x7f024800ee50 comp rx=0 tx=0).stop 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.469+0000 7f0264400640 1 -- 172.21.15.107:0/2222991308 shutdown_connections 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.469+0000 7f0264400640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f02440a3240 0x7f02440a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.469+0000 7f0264400640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f025c106f10 0x7f025c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.469+0000 7f0264400640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f025c102790 0x7f025c198930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.469+0000 7f0264400640 1 --2- 172.21.15.107:0/2222991308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025c101de0 0x7f025c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.469+0000 7f0264400640 1 -- 172.21.15.107:0/2222991308 >> 172.21.15.107:0/2222991308 conn(0x7f025c0fd8c0 msgr2=0x7f025c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.470+0000 7f0264400640 1 -- 172.21.15.107:0/2222991308 shutdown_connections 2024-09-19T04:18:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:17.470+0000 7f0264400640 1 -- 172.21.15.107:0/2222991308 wait complete. 2024-09-19T04:18:17.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:17 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2222991308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:17.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:17 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2222991308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:17.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:17 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2222991308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:18.203 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:18:18.573 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:18.849 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:18 smithi107 ceph-mon[30659]: pgmap v290: 97 pgs: 97 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 484 B/s wr, 2 op/s 2024-09-19T04:18:18.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:18 smithi115 ceph-mon[35477]: pgmap v290: 97 pgs: 97 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 484 B/s wr, 2 op/s 2024-09-19T04:18:18.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:18 smithi145 ceph-mon[35096]: pgmap v290: 97 pgs: 97 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 484 B/s wr, 2 op/s 2024-09-19T04:18:19.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.131+0000 7fc1d7000640 1 -- 172.21.15.107:0/1526937899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1d0112a50 msgr2=0x7fc1d0114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:19.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.131+0000 7fc1d7000640 1 --2- 172.21.15.107:0/1526937899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1d0112a50 0x7fc1d0114e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009ca0 tx=0x7fc1c802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.131+0000 7fc1d7000640 1 -- 172.21.15.107:0/1526937899 shutdown_connections 2024-09-19T04:18:19.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.131+0000 7fc1d7000640 1 --2- 172.21.15.107:0/1526937899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1d0112a50 0x7fc1d0114e40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.131+0000 7fc1d7000640 1 --2- 172.21.15.107:0/1526937899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d0077040 0x7fc1d0112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.131+0000 7fc1d7000640 1 --2- 172.21.15.107:0/1526937899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d0076720 0x7fc1d0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.131+0000 7fc1d7000640 1 -- 172.21.15.107:0/1526937899 >> 172.21.15.107:0/1526937899 conn(0x7fc1d00ffa50 msgr2=0x7fc1d0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:19.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.131+0000 7fc1d7000640 1 -- 172.21.15.107:0/1526937899 shutdown_connections 2024-09-19T04:18:19.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.131+0000 7fc1d7000640 1 -- 172.21.15.107:0/1526937899 wait complete. 2024-09-19T04:18:19.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.132+0000 7fc1d7000640 1 Processor -- start 2024-09-19T04:18:19.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.132+0000 7fc1d7000640 1 -- start start 2024-09-19T04:18:19.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1d7000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d0076720 0x7fc1d019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:19.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1d7000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1d0077040 0x7fc1d019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1d7000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d0112a50 0x7fc1d01a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1d7000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0118600 con 0x7fc1d0077040 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1d7000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0118480 con 0x7fc1d0076720 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1d7000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0118780 con 0x7fc1d0112a50 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1d0077040 0x7fc1d019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1d0077040 0x7fc1d019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50878/0 (socket says 172.21.15.107:50878) 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1c7e00640 1 -- 172.21.15.107:0/3461894338 learned_addr learned my addr 172.21.15.107:0/3461894338 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1cfe00640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d0076720 0x7fc1d019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.133+0000 7fc1d5000640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d0112a50 0x7fc1d01a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1c7e00640 1 -- 172.21.15.107:0/3461894338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d0112a50 msgr2=0x7fc1d01a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1c7e00640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d0112a50 0x7fc1d01a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1c7e00640 1 -- 172.21.15.107:0/3461894338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d0076720 msgr2=0x7fc1d019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:19.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1c7e00640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d0076720 0x7fc1d019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1c7e00640 1 -- 172.21.15.107:0/3461894338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d01a3b90 con 0x7fc1d0077040 2024-09-19T04:18:19.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1d5000640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d0112a50 0x7fc1d01a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:19.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1c7e00640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1d0077040 0x7fc1d019f0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800e9f0 tx=0x7fc1b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:19.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1cd600640 1 -- 172.21.15.107:0/3461894338 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800cd80 con 0x7fc1d0077040 2024-09-19T04:18:19.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1cd600640 1 -- 172.21.15.107:0/3461894338 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b8005500 con 0x7fc1d0077040 2024-09-19T04:18:19.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1d7000640 1 -- 172.21.15.107:0/3461894338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d01a3e80 con 0x7fc1d0077040 2024-09-19T04:18:19.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1cd600640 1 -- 172.21.15.107:0/3461894338 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8010690 con 0x7fc1d0077040 2024-09-19T04:18:19.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.134+0000 7fc1d7000640 1 -- 172.21.15.107:0/3461894338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d01ab710 con 0x7fc1d0077040 2024-09-19T04:18:19.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.135+0000 7fc1d7000640 1 -- 172.21.15.107:0/3461894338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d0076b00 con 0x7fc1d0077040 2024-09-19T04:18:19.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.137+0000 7fc1cd600640 1 -- 172.21.15.107:0/3461894338 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc1b80046c0 con 0x7fc1d0077040 2024-09-19T04:18:19.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.139+0000 7fc1cd600640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc1b40a32c0 0x7fc1b40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:19.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.139+0000 7fc1cd600640 1 -- 172.21.15.107:0/3461894338 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fc1b80cfc60 con 0x7fc1d0077040 2024-09-19T04:18:19.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.139+0000 7fc1cfe00640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc1b40a32c0 0x7fc1b40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:19.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.139+0000 7fc1cfe00640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc1b40a32c0 0x7fc1b40a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005d00 tx=0x7fc1c000b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:19.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.143+0000 7fc1cd600640 1 -- 172.21.15.107:0/3461894338 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc1b809ca30 con 0x7fc1d0077040 2024-09-19T04:18:19.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.353+0000 7fc1d7000640 1 -- 172.21.15.107:0/3461894338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc1d01ab990 con 0x7fc1d0077040 2024-09-19T04:18:19.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.511+0000 7fc1cd600640 1 -- 172.21.15.107:0/3461894338 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fc1b809c330 con 0x7fc1d0077040 2024-09-19T04:18:19.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 -- 172.21.15.107:0/3461894338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc1b40a32c0 msgr2=0x7fc1b40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:19.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc1b40a32c0 0x7fc1b40a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005d00 tx=0x7fc1c000b410 comp rx=0 tx=0).stop 2024-09-19T04:18:19.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 -- 172.21.15.107:0/3461894338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1d0077040 msgr2=0x7fc1d019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:19.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1d0077040 0x7fc1d019f0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800e9f0 tx=0x7fc1b800edf0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 -- 172.21.15.107:0/3461894338 shutdown_connections 2024-09-19T04:18:19.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc1b40a32c0 0x7fc1b40a5780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d0112a50 0x7fc1d01a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1d0077040 0x7fc1d019f0a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 --2- 172.21.15.107:0/3461894338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d0076720 0x7fc1d019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:19.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 -- 172.21.15.107:0/3461894338 >> 172.21.15.107:0/3461894338 conn(0x7fc1d00ffa50 msgr2=0x7fc1d0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:19.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 -- 172.21.15.107:0/3461894338 shutdown_connections 2024-09-19T04:18:19.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:19.514+0000 7fc1d7000640 1 -- 172.21.15.107:0/3461894338 wait complete. 2024-09-19T04:18:19.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:19 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3461894338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:18:19.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:19 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3461894338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:18:19.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:19 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3461894338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:18:20.304 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-19T04:18:20.305 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T04:18:20.680 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:20.820 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:20 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3461894338' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:18:20.820 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:20 smithi107 ceph-mon[30659]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:20.820 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:20 smithi107 ceph-mon[30659]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T04:18:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:20 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3461894338' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:18:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:20 smithi115 ceph-mon[35477]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:20 smithi115 ceph-mon[35477]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T04:18:20.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:20 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3461894338' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:18:20.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:20 smithi145 ceph-mon[35096]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:20.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:20 smithi145 ceph-mon[35096]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T04:18:21.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.239+0000 7ff1abe00640 1 -- 172.21.15.107:0/3249498825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1ac1047e0 msgr2=0x7ff1ac108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.239+0000 7ff1abe00640 1 --2- 172.21.15.107:0/3249498825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1ac1047e0 0x7ff1ac108830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff194009900 tx=0x7ff19402a410 comp rx=0 tx=0).stop 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.240+0000 7ff1abe00640 1 -- 172.21.15.107:0/3249498825 shutdown_connections 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.240+0000 7ff1abe00640 1 --2- 172.21.15.107:0/3249498825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1ac108f60 0x7ff1ac110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.240+0000 7ff1abe00640 1 --2- 172.21.15.107:0/3249498825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1ac1047e0 0x7ff1ac108830 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.240+0000 7ff1abe00640 1 --2- 172.21.15.107:0/3249498825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ac103e30 0x7ff1ac104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.240+0000 7ff1abe00640 1 -- 172.21.15.107:0/3249498825 >> 172.21.15.107:0/3249498825 conn(0x7ff1ac0ff8d0 msgr2=0x7ff1ac101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.240+0000 7ff1abe00640 1 -- 172.21.15.107:0/3249498825 shutdown_connections 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.240+0000 7ff1abe00640 1 -- 172.21.15.107:0/3249498825 wait complete. 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1abe00640 1 Processor -- start 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1abe00640 1 -- start start 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1abe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1ac103e30 0x7ff1ac19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1ac1047e0 0x7ff1ac19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1abe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ac108f60 0x7ff1ac19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1abe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac113fc0 con 0x7ff1ac1047e0 2024-09-19T04:18:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1abe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac113e40 con 0x7ff1ac108f60 2024-09-19T04:18:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1abe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac114140 con 0x7ff1ac103e30 2024-09-19T04:18:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1a2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1ac1047e0 0x7ff1ac19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1a2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1ac1047e0 0x7ff1ac19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50886/0 (socket says 172.21.15.107:50886) 2024-09-19T04:18:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1a2000640 1 -- 172.21.15.107:0/2941388868 learned_addr learned my addr 172.21.15.107:0/2941388868 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1aaa00640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1ac103e30 0x7ff1ac19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.241+0000 7ff1ab400640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ac108f60 0x7ff1ac19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.242+0000 7ff1a2000640 1 -- 172.21.15.107:0/2941388868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1ac103e30 msgr2=0x7ff1ac19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.242+0000 7ff1a2000640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1ac103e30 0x7ff1ac19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.242+0000 7ff1a2000640 1 -- 172.21.15.107:0/2941388868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ac108f60 msgr2=0x7ff1ac19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.242+0000 7ff1a2000640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ac108f60 0x7ff1ac19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.242+0000 7ff1a2000640 1 -- 172.21.15.107:0/2941388868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac19f410 con 0x7ff1ac1047e0 2024-09-19T04:18:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.242+0000 7ff1a2000640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1ac1047e0 0x7ff1ac19a9a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff194006010 tx=0x7ff1940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.242+0000 7ff1b0a00640 1 -- 172.21.15.107:0/2941388868 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19403f030 con 0x7ff1ac1047e0 2024-09-19T04:18:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.243+0000 7ff1b0a00640 1 -- 172.21.15.107:0/2941388868 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff194003ce0 con 0x7ff1ac1047e0 2024-09-19T04:18:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.243+0000 7ff1abe00640 1 -- 172.21.15.107:0/2941388868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac19f6a0 con 0x7ff1ac1047e0 2024-09-19T04:18:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.243+0000 7ff1b0a00640 1 -- 172.21.15.107:0/2941388868 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194035070 con 0x7ff1ac1047e0 2024-09-19T04:18:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.243+0000 7ff1abe00640 1 -- 172.21.15.107:0/2941388868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac1a6fe0 con 0x7ff1ac1047e0 2024-09-19T04:18:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.244+0000 7ff1abe00640 1 -- 172.21.15.107:0/2941388868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ac104210 con 0x7ff1ac1047e0 2024-09-19T04:18:21.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.245+0000 7ff1b0a00640 1 -- 172.21.15.107:0/2941388868 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff194042030 con 0x7ff1ac1047e0 2024-09-19T04:18:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.247+0000 7ff1b0a00640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff1900a3240 0x7ff1900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.247+0000 7ff1b0a00640 1 -- 172.21.15.107:0/2941388868 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ff1940ef990 con 0x7ff1ac1047e0 2024-09-19T04:18:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.247+0000 7ff1aaa00640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff1900a3240 0x7ff1900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.248+0000 7ff1aaa00640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff1900a3240 0x7ff1900a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff19c005d00 tx=0x7ff19c00b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:21.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.250+0000 7ff1b0a00640 1 -- 172.21.15.107:0/2941388868 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff1940bc1d0 con 0x7ff1ac1047e0 2024-09-19T04:18:21.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.460+0000 7ff1abe00640 1 -- 172.21.15.107:0/2941388868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ff1ac10c170 con 0x7ff1ac1047e0 2024-09-19T04:18:21.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.525+0000 7ff1b0a00640 1 -- 172.21.15.107:0/2941388868 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7ff1940c04d0 con 0x7ff1ac1047e0 2024-09-19T04:18:21.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.527+0000 7ff1abe00640 1 -- 172.21.15.107:0/2941388868 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff1900a3240 msgr2=0x7ff1900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.527+0000 7ff1abe00640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff1900a3240 0x7ff1900a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff19c005d00 tx=0x7ff19c00b410 comp rx=0 tx=0).stop 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.527+0000 7ff1abe00640 1 -- 172.21.15.107:0/2941388868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1ac1047e0 msgr2=0x7ff1ac19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.527+0000 7ff1abe00640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1ac1047e0 0x7ff1ac19a9a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff194006010 tx=0x7ff1940042f0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.528+0000 7ff1abe00640 1 -- 172.21.15.107:0/2941388868 shutdown_connections 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.528+0000 7ff1abe00640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff1900a3240 0x7ff1900a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.528+0000 7ff1abe00640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ac108f60 0x7ff1ac19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.528+0000 7ff1abe00640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1ac1047e0 0x7ff1ac19a9a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.528+0000 7ff1abe00640 1 --2- 172.21.15.107:0/2941388868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1ac103e30 0x7ff1ac19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.528+0000 7ff1abe00640 1 -- 172.21.15.107:0/2941388868 >> 172.21.15.107:0/2941388868 conn(0x7ff1ac0ff8d0 msgr2=0x7ff1ac100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.528+0000 7ff1abe00640 1 -- 172.21.15.107:0/2941388868 shutdown_connections 2024-09-19T04:18:21.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:21.528+0000 7ff1abe00640 1 -- 172.21.15.107:0/2941388868 wait complete. 2024-09-19T04:18:21.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:21 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2941388868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T04:18:21.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:21 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2941388868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T04:18:21.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:21 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2941388868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T04:18:22.282 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T04:18:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:22.793 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:22 smithi107 ceph-mon[30659]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:18:22.840 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:22 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2941388868' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T04:18:22.840 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:22 smithi107 ceph-mon[30659]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:18:22.840 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:22 smithi107 ceph-mon[30659]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:22.840 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:22 smithi107 ceph-mon[30659]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:18:22.841 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:22 smithi107 ceph-mon[30659]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:18:22.841 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:22 smithi107 ceph-mon[30659]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-19T04:18:22.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:22 smithi115 ceph-mon[35477]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:18:22.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:22 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2941388868' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T04:18:22.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:22 smithi115 ceph-mon[35477]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:18:22.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:22 smithi115 ceph-mon[35477]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:22.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:22 smithi115 ceph-mon[35477]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:18:22.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:22 smithi115 ceph-mon[35477]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:18:22.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:22 smithi115 ceph-mon[35477]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-19T04:18:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:22 smithi145 ceph-mon[35096]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:18:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:22 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2941388868' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T04:18:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:22 smithi145 ceph-mon[35096]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:18:22.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:22 smithi145 ceph-mon[35096]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:18:22.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:22 smithi145 ceph-mon[35096]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:18:22.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:22 smithi145 ceph-mon[35096]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:18:22.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:22 smithi145 ceph-mon[35096]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-19T04:18:23.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.225+0000 7fbe9fa00640 1 -- 172.21.15.107:0/4033790800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe98101de0 msgr2=0x7fbe981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:23.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.225+0000 7fbe9fa00640 1 --2- 172.21.15.107:0/4033790800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe98101de0 0x7fbe981021c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009900 tx=0x7fbe8802a410 comp rx=0 tx=0).stop 2024-09-19T04:18:23.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.225+0000 7fbe9fa00640 1 -- 172.21.15.107:0/4033790800 shutdown_connections 2024-09-19T04:18:23.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.225+0000 7fbe9fa00640 1 --2- 172.21.15.107:0/4033790800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe98106f10 0x7fbe9810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:23.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.225+0000 7fbe9fa00640 1 --2- 172.21.15.107:0/4033790800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe98102790 0x7fbe981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:23.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.225+0000 7fbe9fa00640 1 --2- 172.21.15.107:0/4033790800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe98101de0 0x7fbe981021c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:23.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.225+0000 7fbe9fa00640 1 -- 172.21.15.107:0/4033790800 >> 172.21.15.107:0/4033790800 conn(0x7fbe980fd8c0 msgr2=0x7fbe980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:23.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.225+0000 7fbe9fa00640 1 -- 172.21.15.107:0/4033790800 shutdown_connections 2024-09-19T04:18:23.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.225+0000 7fbe9fa00640 1 -- 172.21.15.107:0/4033790800 wait complete. 2024-09-19T04:18:23.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.226+0000 7fbe9fa00640 1 Processor -- start 2024-09-19T04:18:23.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.226+0000 7fbe9fa00640 1 -- start start 2024-09-19T04:18:23.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe9fa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe98101de0 0x7fbe9819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:23.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe9fa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe98102790 0x7fbe9819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:23.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe9fa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe98106f10 0x7fbe9819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:23.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe9fa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98111f70 con 0x7fbe98101de0 2024-09-19T04:18:23.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe9fa00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98111df0 con 0x7fbe98106f10 2024-09-19T04:18:23.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe9fa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981120f0 con 0x7fbe98102790 2024-09-19T04:18:23.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe9d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe98101de0 0x7fbe9819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:23.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe9d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe98101de0 0x7fbe9819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50900/0 (socket says 172.21.15.107:50900) 2024-09-19T04:18:23.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe9d000640 1 -- 172.21.15.107:0/1647723411 learned_addr learned my addr 172.21.15.107:0/1647723411 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:23.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe9da00640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe98106f10 0x7fbe9819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:23.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.227+0000 7fbe97e00640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe98102790 0x7fbe9819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe9d000640 1 -- 172.21.15.107:0/1647723411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe98102790 msgr2=0x7fbe9819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe9d000640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe98102790 0x7fbe9819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe9d000640 1 -- 172.21.15.107:0/1647723411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe98106f10 msgr2=0x7fbe9819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe9d000640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe98106f10 0x7fbe9819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe9d000640 1 -- 172.21.15.107:0/1647723411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9819f610 con 0x7fbe98101de0 2024-09-19T04:18:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe9d000640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe98101de0 0x7fbe9819a660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe88006010 tx=0x7fbe88004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe95600640 1 -- 172.21.15.107:0/1647723411 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8803f030 con 0x7fbe98101de0 2024-09-19T04:18:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe95600640 1 -- 172.21.15.107:0/1647723411 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe88004440 con 0x7fbe98101de0 2024-09-19T04:18:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe95600640 1 -- 172.21.15.107:0/1647723411 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88035070 con 0x7fbe98101de0 2024-09-19T04:18:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe9fa00640 1 -- 172.21.15.107:0/1647723411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9819f8a0 con 0x7fbe98101de0 2024-09-19T04:18:23.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.228+0000 7fbe9fa00640 1 -- 172.21.15.107:0/1647723411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9819fd30 con 0x7fbe98101de0 2024-09-19T04:18:23.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.229+0000 7fbe9fa00640 1 -- 172.21.15.107:0/1647723411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe981021c0 con 0x7fbe98101de0 2024-09-19T04:18:23.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.231+0000 7fbe95600640 1 -- 172.21.15.107:0/1647723411 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fbe88042030 con 0x7fbe98101de0 2024-09-19T04:18:23.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.233+0000 7fbe95600640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbe7c0a32c0 0x7fbe7c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:23.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.233+0000 7fbe95600640 1 -- 172.21.15.107:0/1647723411 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fbe880efb20 con 0x7fbe98101de0 2024-09-19T04:18:23.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.233+0000 7fbe97e00640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbe7c0a32c0 0x7fbe7c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.233+0000 7fbe97e00640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbe7c0a32c0 0x7fbe7c0a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe9810ee50 tx=0x7fbe84005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:23.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.236+0000 7fbe95600640 1 -- 172.21.15.107:0/1647723411 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbe880bc2a0 con 0x7fbe98101de0 2024-09-19T04:18:23.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.446+0000 7fbe9fa00640 1 -- 172.21.15.107:0/1647723411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fbe981a7420 con 0x7fbe98101de0 2024-09-19T04:18:23.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.548+0000 7fbe95600640 1 -- 172.21.15.107:0/1647723411 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fbe880c05a0 con 0x7fbe98101de0 2024-09-19T04:18:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 -- 172.21.15.107:0/1647723411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbe7c0a32c0 msgr2=0x7fbe7c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbe7c0a32c0 0x7fbe7c0a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe9810ee50 tx=0x7fbe84005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 -- 172.21.15.107:0/1647723411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe98101de0 msgr2=0x7fbe9819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe98101de0 0x7fbe9819a660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe88006010 tx=0x7fbe88004040 comp rx=0 tx=0).stop 2024-09-19T04:18:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 -- 172.21.15.107:0/1647723411 shutdown_connections 2024-09-19T04:18:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbe7c0a32c0 0x7fbe7c0a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe98106f10 0x7fbe9819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe98102790 0x7fbe9819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 --2- 172.21.15.107:0/1647723411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe98101de0 0x7fbe9819a660 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 -- 172.21.15.107:0/1647723411 >> 172.21.15.107:0/1647723411 conn(0x7fbe980fd8c0 msgr2=0x7fbe980ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:23.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 -- 172.21.15.107:0/1647723411 shutdown_connections 2024-09-19T04:18:23.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:23.551+0000 7fbe9fa00640 1 -- 172.21.15.107:0/1647723411 wait complete. 2024-09-19T04:18:23.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:23 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] up:active 2024-09-19T04:18:23.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:23 smithi107 ceph-mon[30659]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:18:23.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:23 smithi107 ceph-mon[30659]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:18:23.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:23 smithi107 ceph-mon[30659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T04:18:23.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:23 smithi107 ceph-mon[30659]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T04:18:23.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:23 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1647723411' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:18:23.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:23 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] up:active 2024-09-19T04:18:23.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:23 smithi115 ceph-mon[35477]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:18:23.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:23 smithi115 ceph-mon[35477]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:18:23.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:23 smithi115 ceph-mon[35477]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T04:18:23.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:23 smithi115 ceph-mon[35477]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T04:18:23.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:23 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1647723411' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:18:23.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:23 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] up:active 2024-09-19T04:18:23.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:23 smithi145 ceph-mon[35096]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:18:23.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:23 smithi145 ceph-mon[35096]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:18:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:23 smithi145 ceph-mon[35096]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T04:18:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:23 smithi145 ceph-mon[35096]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T04:18:23.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:23 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1647723411' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:18:24.322 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T04:18:24.552 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:24 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1647723411' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:18:24.552 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:24 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] up:active 2024-09-19T04:18:24.552 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:24 smithi107 ceph-mon[30659]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T04:18:24.552 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:24 smithi107 ceph-mon[30659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-19T04:18:24.695 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:24.834 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:24 smithi107 ceph-mon[30659]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-19T04:18:24.834 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:24 smithi107 ceph-mon[30659]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T04:18:24.834 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:24 smithi107 ceph-mon[30659]: pgmap v293: 97 pgs: 97 active+clean; 580 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 8 op/s 2024-09-19T04:18:24.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:24 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1647723411' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:18:24.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:24 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] up:active 2024-09-19T04:18:24.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:24 smithi115 ceph-mon[35477]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T04:18:24.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:24 smithi115 ceph-mon[35477]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-19T04:18:24.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:24 smithi115 ceph-mon[35477]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-19T04:18:24.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:24 smithi115 ceph-mon[35477]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T04:18:24.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:24 smithi115 ceph-mon[35477]: pgmap v293: 97 pgs: 97 active+clean; 580 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 8 op/s 2024-09-19T04:18:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:24 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1647723411' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:18:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:24 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] up:active 2024-09-19T04:18:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:24 smithi145 ceph-mon[35096]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T04:18:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:24 smithi145 ceph-mon[35096]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-19T04:18:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:24 smithi145 ceph-mon[35096]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-19T04:18:24.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:24 smithi145 ceph-mon[35096]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T04:18:24.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:24 smithi145 ceph-mon[35096]: pgmap v293: 97 pgs: 97 active+clean; 580 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 8 op/s 2024-09-19T04:18:25.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.278+0000 7f49ed000640 1 -- 172.21.15.107:0/76707829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e8106f10 msgr2=0x7f49e810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:25.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.278+0000 7f49ed000640 1 --2- 172.21.15.107:0/76707829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e8106f10 0x7f49e810ea90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009ca0 tx=0x7f49e002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:18:25.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.278+0000 7f49ed000640 1 -- 172.21.15.107:0/76707829 shutdown_connections 2024-09-19T04:18:25.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.278+0000 7f49ed000640 1 --2- 172.21.15.107:0/76707829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e8106f10 0x7f49e810ea90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:25.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.278+0000 7f49ed000640 1 --2- 172.21.15.107:0/76707829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e8102790 0x7f49e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:25.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.278+0000 7f49ed000640 1 --2- 172.21.15.107:0/76707829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49e8101de0 0x7f49e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:25.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.278+0000 7f49ed000640 1 -- 172.21.15.107:0/76707829 >> 172.21.15.107:0/76707829 conn(0x7f49e80fd8c0 msgr2=0x7f49e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:25.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.278+0000 7f49ed000640 1 -- 172.21.15.107:0/76707829 shutdown_connections 2024-09-19T04:18:25.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.279+0000 7f49ed000640 1 -- 172.21.15.107:0/76707829 wait complete. 2024-09-19T04:18:25.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.279+0000 7f49ed000640 1 Processor -- start 2024-09-19T04:18:25.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.279+0000 7f49ed000640 1 -- start start 2024-09-19T04:18:25.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49ed000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e8101de0 0x7f49e819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:25.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49ed000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49e8102790 0x7f49e819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:25.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49ed000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e8106f10 0x7f49e819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:25.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49ed000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8111fc0 con 0x7f49e8106f10 2024-09-19T04:18:25.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49ed000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8111e40 con 0x7f49e8101de0 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49ed000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8112140 con 0x7f49e8102790 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49e6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e8106f10 0x7f49e819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49e6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e8106f10 0x7f49e819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50922/0 (socket says 172.21.15.107:50922) 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49e6800640 1 -- 172.21.15.107:0/2209931978 learned_addr learned my addr 172.21.15.107:0/2209931978 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49e5e00640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e8101de0 0x7f49e819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.280+0000 7f49dd400640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49e8102790 0x7f49e819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49e6800640 1 -- 172.21.15.107:0/2209931978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49e8102790 msgr2=0x7f49e819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49e6800640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49e8102790 0x7f49e819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49e6800640 1 -- 172.21.15.107:0/2209931978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e8101de0 msgr2=0x7f49e819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49e6800640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e8101de0 0x7f49e819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49e6800640 1 -- 172.21.15.107:0/2209931978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e819f5f0 con 0x7f49e8106f10 2024-09-19T04:18:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49e5e00640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e8101de0 0x7f49e819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:25.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49e6800640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e8106f10 0x7f49e819ef10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f49e0033040 tx=0x7f49e0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:25.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49df400640 1 -- 172.21.15.107:0/2209931978 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0031650 con 0x7f49e8106f10 2024-09-19T04:18:25.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49ed000640 1 -- 172.21.15.107:0/2209931978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e819f880 con 0x7f49e8106f10 2024-09-19T04:18:25.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49df400640 1 -- 172.21.15.107:0/2209931978 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e00317f0 con 0x7f49e8106f10 2024-09-19T04:18:25.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49df400640 1 -- 172.21.15.107:0/2209931978 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e002fa40 con 0x7f49e8106f10 2024-09-19T04:18:25.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.281+0000 7f49ed000640 1 -- 172.21.15.107:0/2209931978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e81a71c0 con 0x7f49e8106f10 2024-09-19T04:18:25.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.282+0000 7f49ed000640 1 -- 172.21.15.107:0/2209931978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e81021c0 con 0x7f49e8106f10 2024-09-19T04:18:25.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.284+0000 7f49df400640 1 -- 172.21.15.107:0/2209931978 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f49e002fbe0 con 0x7f49e8106f10 2024-09-19T04:18:25.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.286+0000 7f49df400640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49cc0a3240 0x7f49cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:25.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.286+0000 7f49df400640 1 -- 172.21.15.107:0/2209931978 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f49e00f0e70 con 0x7f49e8106f10 2024-09-19T04:18:25.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.286+0000 7f49e5e00640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49cc0a3240 0x7f49cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:25.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.286+0000 7f49e5e00640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49cc0a3240 0x7f49cc0a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49d8005fd0 tx=0x7f49d800b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:25.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.288+0000 7f49df400640 1 -- 172.21.15.107:0/2209931978 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f49e00bdc40 con 0x7f49e8106f10 2024-09-19T04:18:25.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.499+0000 7f49ed000640 1 -- 172.21.15.107:0/2209931978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f49e81a7390 con 0x7f49e8106f10 2024-09-19T04:18:25.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.567+0000 7f49df400640 1 -- 172.21.15.107:0/2209931978 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f49e00bd540 con 0x7f49e8106f10 2024-09-19T04:18:25.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.569+0000 7f49ed000640 1 -- 172.21.15.107:0/2209931978 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49cc0a3240 msgr2=0x7f49cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:25.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.569+0000 7f49ed000640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49cc0a3240 0x7f49cc0a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49d8005fd0 tx=0x7f49d800b390 comp rx=0 tx=0).stop 2024-09-19T04:18:25.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.569+0000 7f49ed000640 1 -- 172.21.15.107:0/2209931978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e8106f10 msgr2=0x7f49e819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:25.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.569+0000 7f49ed000640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e8106f10 0x7f49e819ef10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f49e0033040 tx=0x7f49e0003f10 comp rx=0 tx=0).stop 2024-09-19T04:18:25.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.570+0000 7f49ed000640 1 -- 172.21.15.107:0/2209931978 shutdown_connections 2024-09-19T04:18:25.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.570+0000 7f49ed000640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49cc0a3240 0x7f49cc0a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:25.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.570+0000 7f49ed000640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e8106f10 0x7f49e819ef10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:25.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.570+0000 7f49ed000640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49e8102790 0x7f49e819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:25.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.570+0000 7f49ed000640 1 --2- 172.21.15.107:0/2209931978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e8101de0 0x7f49e819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:25.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.570+0000 7f49ed000640 1 -- 172.21.15.107:0/2209931978 >> 172.21.15.107:0/2209931978 conn(0x7f49e80fd8c0 msgr2=0x7f49e80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:25.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.570+0000 7f49ed000640 1 -- 172.21.15.107:0/2209931978 shutdown_connections 2024-09-19T04:18:25.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:25.570+0000 7f49ed000640 1 -- 172.21.15.107:0/2209931978 wait complete. 2024-09-19T04:18:25.918 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:25 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] up:active 2024-09-19T04:18:25.918 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:25 smithi107 ceph-mon[30659]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T04:18:25.918 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:25 smithi107 ceph-mon[30659]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:18:25.918 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:25 smithi107 ceph-mon[30659]: Cluster is now healthy 2024-09-19T04:18:25.918 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:25 smithi107 ceph-mon[30659]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-19T04:18:25.918 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:25 smithi107 ceph-mon[30659]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T04:18:25.918 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:25 smithi107 ceph-mon[30659]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T04:18:25.918 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:25 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2209931978' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:18:25.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:25 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] up:active 2024-09-19T04:18:25.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:25 smithi115 ceph-mon[35477]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T04:18:25.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:25 smithi115 ceph-mon[35477]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:18:25.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:25 smithi115 ceph-mon[35477]: Cluster is now healthy 2024-09-19T04:18:25.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:25 smithi115 ceph-mon[35477]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-19T04:18:25.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:25 smithi115 ceph-mon[35477]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T04:18:25.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:25 smithi115 ceph-mon[35477]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T04:18:25.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:25 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2209931978' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:18:25.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:25 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] up:active 2024-09-19T04:18:25.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:25 smithi145 ceph-mon[35096]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T04:18:25.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:25 smithi145 ceph-mon[35096]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:18:25.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:25 smithi145 ceph-mon[35096]: Cluster is now healthy 2024-09-19T04:18:25.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:25 smithi145 ceph-mon[35096]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-19T04:18:25.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:25 smithi145 ceph-mon[35096]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T04:18:25.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:25 smithi145 ceph-mon[35096]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T04:18:25.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:25 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2209931978' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:18:26.384 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-19T04:18:26.384 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T04:18:26.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:26 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2209931978' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:18:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:26 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] up:active 2024-09-19T04:18:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:26 smithi115 ceph-mon[35477]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T04:18:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:26 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:18:26.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:26 smithi115 ceph-mon[35477]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 9 op/s 2024-09-19T04:18:26.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:26 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2209931978' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:18:26.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:26 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] up:active 2024-09-19T04:18:26.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:26 smithi107 ceph-mon[30659]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T04:18:26.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:26 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:18:26.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:26 smithi107 ceph-mon[30659]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 9 op/s 2024-09-19T04:18:26.759 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:26.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:26 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2209931978' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:18:26.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:26 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] up:active 2024-09-19T04:18:26.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:26 smithi145 ceph-mon[35096]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T04:18:26.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:26 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:18:26.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:26 smithi145 ceph-mon[35096]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 9 op/s 2024-09-19T04:18:27.196 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:26 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b[65574]: 2024-09-19T04:18:26.749+0000 7fc96b170640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-19T04:18:27.196 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:18:26 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b[65574]: 2024-09-19T04:18:26.749+0000 7fc96b170640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-19T04:18:27.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.320+0000 7f7894a00640 1 -- 172.21.15.107:0/482287881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7890112870 msgr2=0x7f7890114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:27.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.320+0000 7f7894a00640 1 --2- 172.21.15.107:0/482287881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7890112870 0x7f7890114c60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7888009ca0 tx=0x7f788802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:18:27.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.320+0000 7f7894a00640 1 -- 172.21.15.107:0/482287881 shutdown_connections 2024-09-19T04:18:27.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.320+0000 7f7894a00640 1 --2- 172.21.15.107:0/482287881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7890112870 0x7f7890114c60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.320+0000 7f7894a00640 1 --2- 172.21.15.107:0/482287881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7890077040 0x7f7890112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.320+0000 7f7894a00640 1 --2- 172.21.15.107:0/482287881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7890076720 0x7f7890076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.320+0000 7f7894a00640 1 -- 172.21.15.107:0/482287881 >> 172.21.15.107:0/482287881 conn(0x7f78900ff910 msgr2=0x7f7890101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.320+0000 7f7894a00640 1 -- 172.21.15.107:0/482287881 shutdown_connections 2024-09-19T04:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.320+0000 7f7894a00640 1 -- 172.21.15.107:0/482287881 wait complete. 2024-09-19T04:18:27.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.321+0000 7f7894a00640 1 Processor -- start 2024-09-19T04:18:27.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.321+0000 7f7894a00640 1 -- start start 2024-09-19T04:18:27.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7894a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7890076720 0x7f78901a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:27.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7894a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7890077040 0x7f78901a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:27.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7894a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7890112870 0x7f789019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:27.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7894a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7890118400 con 0x7f7890077040 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7894a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7890118280 con 0x7f7890076720 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7894a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7890118580 con 0x7f7890112870 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7887e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7890077040 0x7f78901a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7887e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7890077040 0x7f78901a4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50936/0 (socket says 172.21.15.107:50936) 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7887e00640 1 -- 172.21.15.107:0/4060667501 learned_addr learned my addr 172.21.15.107:0/4060667501 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f788e800640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7890076720 0x7f78901a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f788f200640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7890112870 0x7f789019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7887e00640 1 -- 172.21.15.107:0/4060667501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7890112870 msgr2=0x7f789019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7887e00640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7890112870 0x7f789019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7887e00640 1 -- 172.21.15.107:0/4060667501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7890076720 msgr2=0x7f78901a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7887e00640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7890076720 0x7f78901a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:27.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.322+0000 7f7887e00640 1 -- 172.21.15.107:0/4060667501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789019ef20 con 0x7f7890077040 2024-09-19T04:18:27.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.323+0000 7f7887e00640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7890077040 0x7f78901a4b20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f787800d950 tx=0x7f787800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:27.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.323+0000 7f7887400640 1 -- 172.21.15.107:0/4060667501 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878017030 con 0x7f7890077040 2024-09-19T04:18:27.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.323+0000 7f7887400640 1 -- 172.21.15.107:0/4060667501 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7878004020 con 0x7f7890077040 2024-09-19T04:18:27.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.323+0000 7f7887400640 1 -- 172.21.15.107:0/4060667501 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78780047a0 con 0x7f7890077040 2024-09-19T04:18:27.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.323+0000 7f7894a00640 1 -- 172.21.15.107:0/4060667501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789019f210 con 0x7f7890077040 2024-09-19T04:18:27.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.323+0000 7f7894a00640 1 -- 172.21.15.107:0/4060667501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78901ab390 con 0x7f7890077040 2024-09-19T04:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.324+0000 7f7894a00640 1 -- 172.21.15.107:0/4060667501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7890077e60 con 0x7f7890077040 2024-09-19T04:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.326+0000 7f7887400640 1 -- 172.21.15.107:0/4060667501 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f78780041d0 con 0x7f7890077040 2024-09-19T04:18:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.328+0000 7f7887400640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f78740a32c0 0x7f78740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.328+0000 7f7887400640 1 -- 172.21.15.107:0/4060667501 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f78780cf0b0 con 0x7f7890077040 2024-09-19T04:18:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.328+0000 7f788e800640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f78740a32c0 0x7f78740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.329+0000 7f788e800640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f78740a32c0 0x7f78740a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7880009bd0 tx=0x7f7880005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.332+0000 7f7887400640 1 -- 172.21.15.107:0/4060667501 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f787809be80 con 0x7f7890077040 2024-09-19T04:18:27.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.531+0000 7f7894a00640 1 -- 172.21.15.107:0/4060667501 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7890062630 con 0x7f78740a32c0 2024-09-19T04:18:27.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.564+0000 7f7887400640 1 -- 172.21.15.107:0/4060667501 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7890062630 con 0x7f78740a32c0 2024-09-19T04:18:27.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.566+0000 7f7894a00640 1 -- 172.21.15.107:0/4060667501 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f78740a32c0 msgr2=0x7f78740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:27.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.566+0000 7f7894a00640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f78740a32c0 0x7f78740a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7880009bd0 tx=0x7f7880005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:27.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.566+0000 7f7894a00640 1 -- 172.21.15.107:0/4060667501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7890077040 msgr2=0x7f78901a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:27.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.566+0000 7f7894a00640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7890077040 0x7f78901a4b20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f787800d950 tx=0x7f787800de10 comp rx=0 tx=0).stop 2024-09-19T04:18:27.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.567+0000 7f7894a00640 1 -- 172.21.15.107:0/4060667501 shutdown_connections 2024-09-19T04:18:27.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.567+0000 7f7894a00640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f78740a32c0 0x7f78740a5780 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:27.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.567+0000 7f7894a00640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7890112870 0x7f789019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:27.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.567+0000 7f7894a00640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7890077040 0x7f78901a4b20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:27.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.567+0000 7f7894a00640 1 --2- 172.21.15.107:0/4060667501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7890076720 0x7f78901a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:27.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.567+0000 7f7894a00640 1 -- 172.21.15.107:0/4060667501 >> 172.21.15.107:0/4060667501 conn(0x7f78900ff910 msgr2=0x7f7890101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:27.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.567+0000 7f7894a00640 1 -- 172.21.15.107:0/4060667501 shutdown_connections 2024-09-19T04:18:27.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:27.567+0000 7f7894a00640 1 -- 172.21.15.107:0/4060667501 wait complete. 2024-09-19T04:18:27.907 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:18:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:18:27.537+0000 7f58b6e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:18:27.907 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:18:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:18:27.537+0000 7f58b6e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:18:27.907 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:18:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:18:27.537+0000 7f58b6e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:18:27.907 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:18:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:18:27.537+0000 7f58b6e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:18:27.907 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:18:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:18:27.537+0000 7f58b6e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:18:28.414 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T04:18:28.791 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:28 smithi115 ceph-mon[35477]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:28 smithi115 ceph-mon[35477]: pgmap v295: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-19T04:18:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:28 smithi145 ceph-mon[35096]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:28 smithi145 ceph-mon[35096]: pgmap v295: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-19T04:18:29.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:28 smithi107 ceph-mon[30659]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:29.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:28 smithi107 ceph-mon[30659]: pgmap v295: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-19T04:18:29.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.369+0000 7f7e68800640 1 -- 172.21.15.107:0/625533440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e60101de0 msgr2=0x7f7e601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.369+0000 7f7e68800640 1 --2- 172.21.15.107:0/625533440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e60101de0 0x7f7e601021c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009900 tx=0x7f7e5402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.369+0000 7f7e68800640 1 -- 172.21.15.107:0/625533440 shutdown_connections 2024-09-19T04:18:29.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.369+0000 7f7e68800640 1 --2- 172.21.15.107:0/625533440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e60106f10 0x7f7e6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.369+0000 7f7e68800640 1 --2- 172.21.15.107:0/625533440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e60102790 0x7f7e601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.369+0000 7f7e68800640 1 --2- 172.21.15.107:0/625533440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e60101de0 0x7f7e601021c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.369+0000 7f7e68800640 1 -- 172.21.15.107:0/625533440 >> 172.21.15.107:0/625533440 conn(0x7f7e600fd8c0 msgr2=0x7f7e600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.370+0000 7f7e68800640 1 -- 172.21.15.107:0/625533440 shutdown_connections 2024-09-19T04:18:29.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.370+0000 7f7e68800640 1 -- 172.21.15.107:0/625533440 wait complete. 2024-09-19T04:18:29.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.371+0000 7f7e68800640 1 Processor -- start 2024-09-19T04:18:29.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.371+0000 7f7e68800640 1 -- start start 2024-09-19T04:18:29.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.371+0000 7f7e68800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e60101de0 0x7f7e60198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.371+0000 7f7e68800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e60102790 0x7f7e601989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.371+0000 7f7e68800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e60106f10 0x7f7e6019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.371+0000 7f7e68800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e60111f70 con 0x7f7e60106f10 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.371+0000 7f7e68800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7e60111df0 con 0x7f7e60101de0 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.371+0000 7f7e68800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7e601120f0 con 0x7f7e60102790 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.371+0000 7f7e66800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e60106f10 0x7f7e6019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e66800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e60106f10 0x7f7e6019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44374/0 (socket says 172.21.15.107:44374) 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e66800640 1 -- 172.21.15.107:0/1496669141 learned_addr learned my addr 172.21.15.107:0/1496669141 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e65e00640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e60101de0 0x7f7e60198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e65400640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e60102790 0x7f7e601989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e66800640 1 -- 172.21.15.107:0/1496669141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e60102790 msgr2=0x7f7e601989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e66800640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e60102790 0x7f7e601989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e66800640 1 -- 172.21.15.107:0/1496669141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e60101de0 msgr2=0x7f7e60198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e66800640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e60101de0 0x7f7e60198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e66800640 1 -- 172.21.15.107:0/1496669141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6019d420 con 0x7f7e60106f10 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e65e00640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e60101de0 0x7f7e60198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.372+0000 7f7e66800640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e60106f10 0x7f7e6019cd40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00eb10 tx=0x7f7e5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.373+0000 7f7e5aa00640 1 -- 172.21.15.107:0/1496669141 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c004b20 con 0x7f7e60106f10 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.373+0000 7f7e68800640 1 -- 172.21.15.107:0/1496669141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6019d710 con 0x7f7e60106f10 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.373+0000 7f7e5aa00640 1 -- 172.21.15.107:0/1496669141 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e5c004cc0 con 0x7f7e60106f10 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.373+0000 7f7e5aa00640 1 -- 172.21.15.107:0/1496669141 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c010760 con 0x7f7e60106f10 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.373+0000 7f7e68800640 1 -- 172.21.15.107:0/1496669141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e601a4ff0 con 0x7f7e60106f10 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.374+0000 7f7e68800640 1 -- 172.21.15.107:0/1496669141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e601021c0 con 0x7f7e60106f10 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.375+0000 7f7e5aa00640 1 -- 172.21.15.107:0/1496669141 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7e5c005020 con 0x7f7e60106f10 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.377+0000 7f7e5aa00640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7e480a3350 0x7f7e480a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.377+0000 7f7e5aa00640 1 -- 172.21.15.107:0/1496669141 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f7e5c0cffc0 con 0x7f7e60106f10 2024-09-19T04:18:29.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.377+0000 7f7e65e00640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7e480a3350 0x7f7e480a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.378+0000 7f7e65e00640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7e480a3350 0x7f7e480a5810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009230 tx=0x7f7e540058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.380+0000 7f7e5aa00640 1 -- 172.21.15.107:0/1496669141 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7e5c09cc30 con 0x7f7e60106f10 2024-09-19T04:18:29.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.583+0000 7f7e68800640 1 -- 172.21.15.107:0/1496669141 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7e60062630 con 0x7f7e480a3350 2024-09-19T04:18:29.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.609+0000 7f7e5aa00640 1 -- 172.21.15.107:0/1496669141 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e60062630 con 0x7f7e480a3350 2024-09-19T04:18:29.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 -- 172.21.15.107:0/1496669141 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7e480a3350 msgr2=0x7f7e480a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7e480a3350 0x7f7e480a5810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009230 tx=0x7f7e540058d0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 -- 172.21.15.107:0/1496669141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e60106f10 msgr2=0x7f7e6019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e60106f10 0x7f7e6019cd40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00eb10 tx=0x7f7e5c00c2d0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 -- 172.21.15.107:0/1496669141 shutdown_connections 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f7e480a3350 0x7f7e480a5810 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e60106f10 0x7f7e6019cd40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e60102790 0x7f7e601989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 --2- 172.21.15.107:0/1496669141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e60101de0 0x7f7e60198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 -- 172.21.15.107:0/1496669141 >> 172.21.15.107:0/1496669141 conn(0x7f7e600fd8c0 msgr2=0x7f7e600fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 -- 172.21.15.107:0/1496669141 shutdown_connections 2024-09-19T04:18:29.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:29.612+0000 7f7e68800640 1 -- 172.21.15.107:0/1496669141 wait complete. 2024-09-19T04:18:30.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:29 smithi115 ceph-mon[35477]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:18:30.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:29 smithi145 ceph-mon[35096]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:18:30.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:29 smithi107 ceph-mon[30659]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:18:30.467 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T04:18:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:31.143 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:30 smithi107 ceph-mon[30659]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:31.143 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:30 smithi107 ceph-mon[30659]: pgmap v296: 97 pgs: 97 active+clean; 592 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-19T04:18:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:30 smithi115 ceph-mon[35477]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:30 smithi115 ceph-mon[35477]: pgmap v296: 97 pgs: 97 active+clean; 592 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-19T04:18:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:30 smithi145 ceph-mon[35096]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:30 smithi145 ceph-mon[35096]: pgmap v296: 97 pgs: 97 active+clean; 592 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-19T04:18:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.387+0000 7f7644c00640 1 -- 172.21.15.107:0/4049264837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76401008d0 msgr2=0x7f7640100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.387+0000 7f7644c00640 1 --2- 172.21.15.107:0/4049264837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76401008d0 0x7f7640100d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7628009930 tx=0x7f762802a420 comp rx=0 tx=0).stop 2024-09-19T04:18:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.387+0000 7f7644c00640 1 -- 172.21.15.107:0/4049264837 shutdown_connections 2024-09-19T04:18:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.387+0000 7f7644c00640 1 --2- 172.21.15.107:0/4049264837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7640101270 0x7f764010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.387+0000 7f7644c00640 1 --2- 172.21.15.107:0/4049264837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76401008d0 0x7f7640100d30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.387+0000 7f7644c00640 1 --2- 172.21.15.107:0/4049264837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76401068d0 0x7f7640106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.387+0000 7f7644c00640 1 -- 172.21.15.107:0/4049264837 >> 172.21.15.107:0/4049264837 conn(0x7f76400fc610 msgr2=0x7f76400fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.387+0000 7f7644c00640 1 -- 172.21.15.107:0/4049264837 shutdown_connections 2024-09-19T04:18:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.387+0000 7f7644c00640 1 -- 172.21.15.107:0/4049264837 wait complete. 2024-09-19T04:18:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.388+0000 7f7644c00640 1 Processor -- start 2024-09-19T04:18:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.388+0000 7f7644c00640 1 -- start start 2024-09-19T04:18:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f7644c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76401008d0 0x7f76401982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f7644c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7640101270 0x7f76401987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f7644c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76401068d0 0x7f764019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f7644c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7640110cd0 con 0x7f7640101270 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f7644c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7640110b50 con 0x7f76401068d0 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f7644c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7640110e50 con 0x7f76401008d0 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f7636a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7640101270 0x7f76401987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f7636a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7640101270 0x7f76401987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44378/0 (socket says 172.21.15.107:44378) 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f7636a00640 1 -- 172.21.15.107:0/3848252944 learned_addr learned my addr 172.21.15.107:0/3848252944 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f763f400640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76401008d0 0x7f76401982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.389+0000 7f763fe00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76401068d0 0x7f764019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f7636a00640 1 -- 172.21.15.107:0/3848252944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76401008d0 msgr2=0x7f76401982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f7636a00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76401008d0 0x7f76401982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f7636a00640 1 -- 172.21.15.107:0/3848252944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76401068d0 msgr2=0x7f764019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f7636a00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76401068d0 0x7f764019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f7636a00640 1 -- 172.21.15.107:0/3848252944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764019d260 con 0x7f7640101270 2024-09-19T04:18:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f7636a00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7640101270 0x7f76401987f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7628009230 tx=0x7f7628004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f763cc00640 1 -- 172.21.15.107:0/3848252944 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762803f030 con 0x7f7640101270 2024-09-19T04:18:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f763cc00640 1 -- 172.21.15.107:0/3848252944 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7628004440 con 0x7f7640101270 2024-09-19T04:18:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f763cc00640 1 -- 172.21.15.107:0/3848252944 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628035070 con 0x7f7640101270 2024-09-19T04:18:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f7644c00640 1 -- 172.21.15.107:0/3848252944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764019d4f0 con 0x7f7640101270 2024-09-19T04:18:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.390+0000 7f7644c00640 1 -- 172.21.15.107:0/3848252944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764019d980 con 0x7f7640101270 2024-09-19T04:18:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.392+0000 7f7644c00640 1 -- 172.21.15.107:0/3848252944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7640100d30 con 0x7f7640101270 2024-09-19T04:18:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.394+0000 7f763cc00640 1 -- 172.21.15.107:0/3848252944 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7628042030 con 0x7f7640101270 2024-09-19T04:18:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.395+0000 7f763cc00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76240a32d0 0x7f76240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.395+0000 7f763cc00640 1 -- 172.21.15.107:0/3848252944 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f76280efcc0 con 0x7f7640101270 2024-09-19T04:18:31.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.395+0000 7f763f400640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76240a32d0 0x7f76240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.396+0000 7f763f400640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76240a32d0 0x7f76240a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7630009b50 tx=0x7f7630005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.402+0000 7f763cc00640 1 -- 172.21.15.107:0/3848252944 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f76280f6080 con 0x7f7640101270 2024-09-19T04:18:31.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.604+0000 7f7644c00640 1 -- 172.21.15.107:0/3848252944 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f7640062630 con 0x7f76240a32d0 2024-09-19T04:18:31.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.609+0000 7f763cc00640 1 -- 172.21.15.107:0/3848252944 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7640062630 con 0x7f76240a32d0 2024-09-19T04:18:31.609 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/qa/sv_0/d36d9841-1c76-4e66-9005-37644d1a3889 2024-09-19T04:18:31.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.611+0000 7f7644c00640 1 -- 172.21.15.107:0/3848252944 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76240a32d0 msgr2=0x7f76240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.611+0000 7f7644c00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76240a32d0 0x7f76240a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7630009b50 tx=0x7f7630005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:31.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.611+0000 7f7644c00640 1 -- 172.21.15.107:0/3848252944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7640101270 msgr2=0x7f76401987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.611+0000 7f7644c00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7640101270 0x7f76401987f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7628009230 tx=0x7f7628004040 comp rx=0 tx=0).stop 2024-09-19T04:18:31.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.611+0000 7f7644c00640 1 -- 172.21.15.107:0/3848252944 shutdown_connections 2024-09-19T04:18:31.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.611+0000 7f7644c00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76240a32d0 0x7f76240a5790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.611+0000 7f7644c00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76401068d0 0x7f764019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.611+0000 7f7644c00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7640101270 0x7f76401987f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.611+0000 7f7644c00640 1 --2- 172.21.15.107:0/3848252944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76401008d0 0x7f76401982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.611+0000 7f7644c00640 1 -- 172.21.15.107:0/3848252944 >> 172.21.15.107:0/3848252944 conn(0x7f76400fc610 msgr2=0x7f764006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:31.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.612+0000 7f7644c00640 1 -- 172.21.15.107:0/3848252944 shutdown_connections 2024-09-19T04:18:31.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:31.612+0000 7f7644c00640 1 -- 172.21.15.107:0/3848252944 wait complete. 2024-09-19T04:18:32.477 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T04:18:32.846 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:33.153 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:32 smithi107 ceph-mon[30659]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:33.153 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:32 smithi107 ceph-mon[30659]: pgmap v297: 97 pgs: 97 active+clean; 592 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.0 KiB/s wr, 20 op/s 2024-09-19T04:18:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:32 smithi115 ceph-mon[35477]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:32 smithi115 ceph-mon[35477]: pgmap v297: 97 pgs: 97 active+clean; 592 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.0 KiB/s wr, 20 op/s 2024-09-19T04:18:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:32 smithi145 ceph-mon[35096]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:32 smithi145 ceph-mon[35096]: pgmap v297: 97 pgs: 97 active+clean; 592 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.0 KiB/s wr, 20 op/s 2024-09-19T04:18:33.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.382+0000 7fd8ffe00640 1 -- 172.21.15.107:0/3318504014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd900104800 msgr2=0x7fd900108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.382+0000 7fd8ffe00640 1 --2- 172.21.15.107:0/3318504014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd900104800 0x7fd900108960 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009900 tx=0x7fd8e802a410 comp rx=0 tx=0).stop 2024-09-19T04:18:33.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.382+0000 7fd8ffe00640 1 -- 172.21.15.107:0/3318504014 shutdown_connections 2024-09-19T04:18:33.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.382+0000 7fd8ffe00640 1 --2- 172.21.15.107:0/3318504014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd900109090 0x7fd900110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.382+0000 7fd8ffe00640 1 --2- 172.21.15.107:0/3318504014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd900104800 0x7fd900108960 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.382+0000 7fd8ffe00640 1 --2- 172.21.15.107:0/3318504014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd900103e50 0x7fd900104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.382+0000 7fd8ffe00640 1 -- 172.21.15.107:0/3318504014 >> 172.21.15.107:0/3318504014 conn(0x7fd9000ff8b0 msgr2=0x7fd900101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:33.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.382+0000 7fd8ffe00640 1 -- 172.21.15.107:0/3318504014 shutdown_connections 2024-09-19T04:18:33.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.382+0000 7fd8ffe00640 1 -- 172.21.15.107:0/3318504014 wait complete. 2024-09-19T04:18:33.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.383+0000 7fd8ffe00640 1 Processor -- start 2024-09-19T04:18:33.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.383+0000 7fd8ffe00640 1 -- start start 2024-09-19T04:18:33.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.383+0000 7fd8ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd900103e50 0x7fd90019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.383+0000 7fd8ffe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd900104800 0x7fd90019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.383+0000 7fd8ffe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd900109090 0x7fd90019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.383+0000 7fd8ffe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9001140b0 con 0x7fd900103e50 2024-09-19T04:18:33.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.383+0000 7fd8ffe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd900113f30 con 0x7fd900109090 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.383+0000 7fd8ffe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd900114230 con 0x7fd900104800 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8fea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd900103e50 0x7fd90019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8f7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd900104800 0x7fd90019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8fea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd900103e50 0x7fd90019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44388/0 (socket says 172.21.15.107:44388) 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8fea00640 1 -- 172.21.15.107:0/2465156261 learned_addr learned my addr 172.21.15.107:0/2465156261 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8f7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd900104800 0x7fd90019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:44524/0 (socket says 172.21.15.107:44524) 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8ff400640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd900109090 0x7fd90019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8fea00640 1 -- 172.21.15.107:0/2465156261 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd900104800 msgr2=0x7fd90019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8fea00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd900104800 0x7fd90019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8fea00640 1 -- 172.21.15.107:0/2465156261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd900109090 msgr2=0x7fd90019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8fea00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd900109090 0x7fd90019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8fea00640 1 -- 172.21.15.107:0/2465156261 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90019f540 con 0x7fd900103e50 2024-09-19T04:18:33.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8ff400640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd900109090 0x7fd90019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:33.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.384+0000 7fd8fea00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd900103e50 0x7fd90019a590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000e9f0 tx=0x7fd8f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.385+0000 7fd8f7400640 1 -- 172.21.15.107:0/2465156261 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000cd80 con 0x7fd900103e50 2024-09-19T04:18:33.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.385+0000 7fd8ffe00640 1 -- 172.21.15.107:0/2465156261 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90019f830 con 0x7fd900103e50 2024-09-19T04:18:33.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.385+0000 7fd8f7400640 1 -- 172.21.15.107:0/2465156261 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8f0010910 con 0x7fd900103e50 2024-09-19T04:18:33.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.385+0000 7fd8f7400640 1 -- 172.21.15.107:0/2465156261 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00188f0 con 0x7fd900103e50 2024-09-19T04:18:33.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.385+0000 7fd8ffe00640 1 -- 172.21.15.107:0/2465156261 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9001a7110 con 0x7fd900103e50 2024-09-19T04:18:33.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.386+0000 7fd8ffe00640 1 -- 172.21.15.107:0/2465156261 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd900105330 con 0x7fd900103e50 2024-09-19T04:18:33.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.388+0000 7fd8f7400640 1 -- 172.21.15.107:0/2465156261 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd8f0010ac0 con 0x7fd900103e50 2024-09-19T04:18:33.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.389+0000 7fd8f7400640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd8e40a3350 0x7fd8e40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.389+0000 7fd8f7400640 1 -- 172.21.15.107:0/2465156261 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fd8f00cfd30 con 0x7fd900103e50 2024-09-19T04:18:33.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.389+0000 7fd8f7e00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd8e40a3350 0x7fd8e40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.390+0000 7fd8f7e00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd8e40a3350 0x7fd8e40a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009230 tx=0x7fd8e80058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.393+0000 7fd8f7400640 1 -- 172.21.15.107:0/2465156261 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd8f009c350 con 0x7fd900103e50 2024-09-19T04:18:33.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.593+0000 7fd8ffe00640 1 -- 172.21.15.107:0/2465156261 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd900062630 con 0x7fd8e40a3350 2024-09-19T04:18:33.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.618+0000 7fd8f7400640 1 -- 172.21.15.107:0/2465156261 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd900062630 con 0x7fd8e40a3350 2024-09-19T04:18:33.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.620+0000 7fd8ffe00640 1 -- 172.21.15.107:0/2465156261 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd8e40a3350 msgr2=0x7fd8e40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.620+0000 7fd8ffe00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd8e40a3350 0x7fd8e40a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009230 tx=0x7fd8e80058d0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.620+0000 7fd8ffe00640 1 -- 172.21.15.107:0/2465156261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd900103e50 msgr2=0x7fd90019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.620+0000 7fd8ffe00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd900103e50 0x7fd90019a590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000e9f0 tx=0x7fd8f000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.621+0000 7fd8ffe00640 1 -- 172.21.15.107:0/2465156261 shutdown_connections 2024-09-19T04:18:33.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.621+0000 7fd8ffe00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd8e40a3350 0x7fd8e40a5810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.621+0000 7fd8ffe00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd900109090 0x7fd90019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.621+0000 7fd8ffe00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd900104800 0x7fd90019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.621+0000 7fd8ffe00640 1 --2- 172.21.15.107:0/2465156261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd900103e50 0x7fd90019a590 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.621+0000 7fd8ffe00640 1 -- 172.21.15.107:0/2465156261 >> 172.21.15.107:0/2465156261 conn(0x7fd9000ff8b0 msgr2=0x7fd900100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:33.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.621+0000 7fd8ffe00640 1 -- 172.21.15.107:0/2465156261 shutdown_connections 2024-09-19T04:18:33.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:33.621+0000 7fd8ffe00640 1 -- 172.21.15.107:0/2465156261 wait complete. 2024-09-19T04:18:34.370 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T04:18:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:35.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:34 smithi107 ceph-mon[30659]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:35.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:34 smithi107 ceph-mon[30659]: pgmap v298: 97 pgs: 97 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 23 op/s 2024-09-19T04:18:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:34 smithi115 ceph-mon[35477]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:34 smithi115 ceph-mon[35477]: pgmap v298: 97 pgs: 97 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 23 op/s 2024-09-19T04:18:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:34 smithi145 ceph-mon[35096]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:34 smithi145 ceph-mon[35096]: pgmap v298: 97 pgs: 97 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 23 op/s 2024-09-19T04:18:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.291+0000 7f42d7a00640 1 -- 172.21.15.107:0/3769015516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d0101de0 msgr2=0x7f42d01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.291+0000 7f42d7a00640 1 --2- 172.21.15.107:0/3769015516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d0101de0 0x7f42d01021c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009900 tx=0x7f42c402a400 comp rx=0 tx=0).stop 2024-09-19T04:18:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.291+0000 7f42d7a00640 1 -- 172.21.15.107:0/3769015516 shutdown_connections 2024-09-19T04:18:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.291+0000 7f42d7a00640 1 --2- 172.21.15.107:0/3769015516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42d0106f10 0x7f42d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.291+0000 7f42d7a00640 1 --2- 172.21.15.107:0/3769015516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42d0102790 0x7f42d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.291+0000 7f42d7a00640 1 --2- 172.21.15.107:0/3769015516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d0101de0 0x7f42d01021c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.291+0000 7f42d7a00640 1 -- 172.21.15.107:0/3769015516 >> 172.21.15.107:0/3769015516 conn(0x7f42d00fd8c0 msgr2=0x7f42d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:35.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.291+0000 7f42d7a00640 1 -- 172.21.15.107:0/3769015516 shutdown_connections 2024-09-19T04:18:35.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.291+0000 7f42d7a00640 1 -- 172.21.15.107:0/3769015516 wait complete. 2024-09-19T04:18:35.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.292+0000 7f42d7a00640 1 Processor -- start 2024-09-19T04:18:35.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42d7a00640 1 -- start start 2024-09-19T04:18:35.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42d7a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42d0101de0 0x7f42d019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42d7a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d0102790 0x7f42d019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42d7a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42d0106f10 0x7f42d019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42d7a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0111f90 con 0x7f42d0102790 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42d7a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0111e10 con 0x7f42d0106f10 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42d7a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0112110 con 0x7f42d0101de0 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42cfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d0102790 0x7f42d019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42cfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d0102790 0x7f42d019ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44410/0 (socket says 172.21.15.107:44410) 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42cfe00640 1 -- 172.21.15.107:0/2063267029 learned_addr learned my addr 172.21.15.107:0/2063267029 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42d5000640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42d0101de0 0x7f42d019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.293+0000 7f42d5a00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42d0106f10 0x7f42d019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42cfe00640 1 -- 172.21.15.107:0/2063267029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42d0101de0 msgr2=0x7f42d019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:35.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42cfe00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42d0101de0 0x7f42d019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42cfe00640 1 -- 172.21.15.107:0/2063267029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42d0106f10 msgr2=0x7f42d019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:35.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42cfe00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42d0106f10 0x7f42d019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42cfe00640 1 -- 172.21.15.107:0/2063267029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d019f700 con 0x7f42d0102790 2024-09-19T04:18:35.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42d5a00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42d0106f10 0x7f42d019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:35.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42cfe00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d0102790 0x7f42d019ac90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00d950 tx=0x7f42bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:35.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42cd600640 1 -- 172.21.15.107:0/2063267029 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc017070 con 0x7f42d0102790 2024-09-19T04:18:35.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42d7a00640 1 -- 172.21.15.107:0/2063267029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d019f9f0 con 0x7f42d0102790 2024-09-19T04:18:35.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42cd600640 1 -- 172.21.15.107:0/2063267029 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42bc009c30 con 0x7f42d0102790 2024-09-19T04:18:35.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42cd600640 1 -- 172.21.15.107:0/2063267029 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00f6b0 con 0x7f42d0102790 2024-09-19T04:18:35.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.294+0000 7f42d7a00640 1 -- 172.21.15.107:0/2063267029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d01a72d0 con 0x7f42d0102790 2024-09-19T04:18:35.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.296+0000 7f42d7a00640 1 -- 172.21.15.107:0/2063267029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d01021c0 con 0x7f42d0102790 2024-09-19T04:18:35.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.297+0000 7f42cd600640 1 -- 172.21.15.107:0/2063267029 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f42bc0109d0 con 0x7f42d0102790 2024-09-19T04:18:35.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.298+0000 7f42cd600640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f42b80a3350 0x7f42b80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:35.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.299+0000 7f42cd600640 1 -- 172.21.15.107:0/2063267029 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f42bc0cf5f0 con 0x7f42d0102790 2024-09-19T04:18:35.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.299+0000 7f42d5000640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f42b80a3350 0x7f42b80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:35.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.299+0000 7f42d5000640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f42b80a3350 0x7f42b80a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009230 tx=0x7f42c40058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:35.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.303+0000 7f42cd600640 1 -- 172.21.15.107:0/2063267029 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f42bc009de0 con 0x7f42d0102790 2024-09-19T04:18:35.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.507+0000 7f42d7a00640 1 -- 172.21.15.107:0/2063267029 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f42d0075430 con 0x7f42b80a3350 2024-09-19T04:18:35.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.513+0000 7f42cd600640 1 -- 172.21.15.107:0/2063267029 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f42d0075430 con 0x7f42b80a3350 2024-09-19T04:18:35.515 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/qa/sv_1/26bf594d-d66e-4c83-a2f3-faba049e9da5 2024-09-19T04:18:35.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 -- 172.21.15.107:0/2063267029 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f42b80a3350 msgr2=0x7f42b80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:35.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f42b80a3350 0x7f42b80a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009230 tx=0x7f42c40058d0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 -- 172.21.15.107:0/2063267029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d0102790 msgr2=0x7f42d019ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:35.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d0102790 0x7f42d019ac90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00d950 tx=0x7f42bc00de10 comp rx=0 tx=0).stop 2024-09-19T04:18:35.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 -- 172.21.15.107:0/2063267029 shutdown_connections 2024-09-19T04:18:35.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f42b80a3350 0x7f42b80a5810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42d0106f10 0x7f42d019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d0102790 0x7f42d019ac90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 --2- 172.21.15.107:0/2063267029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42d0101de0 0x7f42d019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 -- 172.21.15.107:0/2063267029 >> 172.21.15.107:0/2063267029 conn(0x7f42d00fd8c0 msgr2=0x7f42d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 -- 172.21.15.107:0/2063267029 shutdown_connections 2024-09-19T04:18:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:35.516+0000 7f42d7a00640 1 -- 172.21.15.107:0/2063267029 wait complete. 2024-09-19T04:18:36.373 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:18:36.746 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:37.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:36 smithi107 ceph-mon[30659]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:37.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:36 smithi107 ceph-mon[30659]: pgmap v299: 97 pgs: 97 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 20 op/s 2024-09-19T04:18:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:36 smithi115 ceph-mon[35477]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:36 smithi115 ceph-mon[35477]: pgmap v299: 97 pgs: 97 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 20 op/s 2024-09-19T04:18:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:36 smithi145 ceph-mon[35096]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:18:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:36 smithi145 ceph-mon[35096]: pgmap v299: 97 pgs: 97 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 20 op/s 2024-09-19T04:18:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.288+0000 7f1d53600640 1 -- 172.21.15.107:0/1420086280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d4c102790 msgr2=0x7f1d4c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.288+0000 7f1d53600640 1 --2- 172.21.15.107:0/1420086280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d4c102790 0x7f1d4c1067e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009900 tx=0x7f1d3802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.288+0000 7f1d53600640 1 -- 172.21.15.107:0/1420086280 shutdown_connections 2024-09-19T04:18:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.288+0000 7f1d53600640 1 --2- 172.21.15.107:0/1420086280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4c106f10 0x7f1d4c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.288+0000 7f1d53600640 1 --2- 172.21.15.107:0/1420086280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d4c102790 0x7f1d4c1067e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.288+0000 7f1d53600640 1 --2- 172.21.15.107:0/1420086280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d4c101de0 0x7f1d4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.288+0000 7f1d53600640 1 -- 172.21.15.107:0/1420086280 >> 172.21.15.107:0/1420086280 conn(0x7f1d4c0fd8c0 msgr2=0x7f1d4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.288+0000 7f1d53600640 1 -- 172.21.15.107:0/1420086280 shutdown_connections 2024-09-19T04:18:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.288+0000 7f1d53600640 1 -- 172.21.15.107:0/1420086280 wait complete. 2024-09-19T04:18:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.289+0000 7f1d53600640 1 Processor -- start 2024-09-19T04:18:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.289+0000 7f1d53600640 1 -- start start 2024-09-19T04:18:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.289+0000 7f1d53600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d4c101de0 0x7f1d4c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d53600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d4c102790 0x7f1d4c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d53600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4c106f10 0x7f1d4c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d53600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c111f70 con 0x7f1d4c102790 2024-09-19T04:18:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d53600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c111df0 con 0x7f1d4c101de0 2024-09-19T04:18:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d53600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c1120f0 con 0x7f1d4c106f10 2024-09-19T04:18:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d50c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d4c101de0 0x7f1d4c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d50c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d4c101de0 0x7f1d4c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38700/0 (socket says 172.21.15.107:38700) 2024-09-19T04:18:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d50c00640 1 -- 172.21.15.107:0/3362254289 learned_addr learned my addr 172.21.15.107:0/3362254289 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d4be00640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d4c102790 0x7f1d4c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d51600640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4c106f10 0x7f1d4c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d50c00640 1 -- 172.21.15.107:0/3362254289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4c106f10 msgr2=0x7f1d4c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d50c00640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4c106f10 0x7f1d4c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d50c00640 1 -- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d4c102790 msgr2=0x7f1d4c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d50c00640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d4c102790 0x7f1d4c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d50c00640 1 -- 172.21.15.107:0/3362254289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c19f620 con 0x7f1d4c101de0 2024-09-19T04:18:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d4be00640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d4c102790 0x7f1d4c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.290+0000 7f1d50c00640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d4c101de0 0x7f1d4c19a670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000d950 tx=0x7f1d4000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.291+0000 7f1d49600640 1 -- 172.21.15.107:0/3362254289 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40017070 con 0x7f1d4c101de0 2024-09-19T04:18:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.291+0000 7f1d49600640 1 -- 172.21.15.107:0/3362254289 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d40009c30 con 0x7f1d4c101de0 2024-09-19T04:18:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.291+0000 7f1d49600640 1 -- 172.21.15.107:0/3362254289 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4000f6b0 con 0x7f1d4c101de0 2024-09-19T04:18:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.291+0000 7f1d53600640 1 -- 172.21.15.107:0/3362254289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4c19f910 con 0x7f1d4c101de0 2024-09-19T04:18:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.291+0000 7f1d53600640 1 -- 172.21.15.107:0/3362254289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4c1a71f0 con 0x7f1d4c101de0 2024-09-19T04:18:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.292+0000 7f1d53600640 1 -- 172.21.15.107:0/3362254289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4c1021c0 con 0x7f1d4c101de0 2024-09-19T04:18:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.294+0000 7f1d49600640 1 -- 172.21.15.107:0/3362254289 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1d400109d0 con 0x7f1d4c101de0 2024-09-19T04:18:37.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.295+0000 7f1d49600640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d340a3350 0x7f1d340a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.295+0000 7f1d49600640 1 -- 172.21.15.107:0/3362254289 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f1d400cf670 con 0x7f1d4c101de0 2024-09-19T04:18:37.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.295+0000 7f1d4be00640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d340a3350 0x7f1d340a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.296+0000 7f1d4be00640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d340a3350 0x7f1d340a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0fddc0 tx=0x7f1d38005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.298+0000 7f1d49600640 1 -- 172.21.15.107:0/3362254289 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d40009de0 con 0x7f1d4c101de0 2024-09-19T04:18:37.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.509+0000 7f1d53600640 1 -- 172.21.15.107:0/3362254289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d4c1a73c0 con 0x7f1d4c101de0 2024-09-19T04:18:37.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.510+0000 7f1d49600640 1 -- 172.21.15.107:0/3362254289 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11992 (secure 0 0 0) 0x7f1d4009c360 con 0x7f1d4c101de0 2024-09-19T04:18:37.511 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:37.512 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"btime":"2024-09-19T04:18:25:571089+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:18:25.571069+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:18:37.512 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-09-19T04:18:37.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 -- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d340a3350 msgr2=0x7f1d340a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d340a3350 0x7f1d340a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0fddc0 tx=0x7f1d38005a20 comp rx=0 tx=0).stop 2024-09-19T04:18:37.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 -- 172.21.15.107:0/3362254289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d4c101de0 msgr2=0x7f1d4c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d4c101de0 0x7f1d4c19a670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000d950 tx=0x7f1d4000de10 comp rx=0 tx=0).stop 2024-09-19T04:18:37.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 -- 172.21.15.107:0/3362254289 shutdown_connections 2024-09-19T04:18:37.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d340a3350 0x7f1d340a5810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4c106f10 0x7f1d4c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d4c102790 0x7f1d4c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 --2- 172.21.15.107:0/3362254289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d4c101de0 0x7f1d4c19a670 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 -- 172.21.15.107:0/3362254289 >> 172.21.15.107:0/3362254289 conn(0x7f1d4c0fd8c0 msgr2=0x7f1d4c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:37.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 -- 172.21.15.107:0/3362254289 shutdown_connections 2024-09-19T04:18:37.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:37.512+0000 7f1d53600640 1 -- 172.21.15.107:0/3362254289 wait complete. 2024-09-19T04:18:38.209 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:37 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3362254289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:37 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3362254289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:38.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:37 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3362254289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:18:38.391 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:18:38.760 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:38 smithi115 ceph-mon[35477]: pgmap v300: 97 pgs: 97 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.4 KiB/s wr, 19 op/s 2024-09-19T04:18:39.208 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:38 smithi107 ceph-mon[30659]: pgmap v300: 97 pgs: 97 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.4 KiB/s wr, 19 op/s 2024-09-19T04:18:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:38 smithi145 ceph-mon[35096]: pgmap v300: 97 pgs: 97 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.4 KiB/s wr, 19 op/s 2024-09-19T04:18:39.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.315+0000 7f346f400640 1 -- 172.21.15.107:0/1269199396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3468102790 msgr2=0x7f34681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:39.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.315+0000 7f346f400640 1 --2- 172.21.15.107:0/1269199396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3468102790 0x7f34681067e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3454009900 tx=0x7f345402a410 comp rx=0 tx=0).stop 2024-09-19T04:18:39.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.315+0000 7f346f400640 1 -- 172.21.15.107:0/1269199396 shutdown_connections 2024-09-19T04:18:39.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.315+0000 7f346f400640 1 --2- 172.21.15.107:0/1269199396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3468106f10 0x7f346810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.315+0000 7f346f400640 1 --2- 172.21.15.107:0/1269199396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3468102790 0x7f34681067e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.315+0000 7f346f400640 1 --2- 172.21.15.107:0/1269199396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3468101de0 0x7f34681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.315+0000 7f346f400640 1 -- 172.21.15.107:0/1269199396 >> 172.21.15.107:0/1269199396 conn(0x7f34680fd8c0 msgr2=0x7f34680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:39.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.315+0000 7f346f400640 1 -- 172.21.15.107:0/1269199396 shutdown_connections 2024-09-19T04:18:39.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.315+0000 7f346f400640 1 -- 172.21.15.107:0/1269199396 wait complete. 2024-09-19T04:18:39.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.316+0000 7f346f400640 1 Processor -- start 2024-09-19T04:18:39.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.316+0000 7f346f400640 1 -- start start 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.316+0000 7f346f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3468101de0 0x7f346819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.316+0000 7f346f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3468102790 0x7f346819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.316+0000 7f346f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3468106f10 0x7f346819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.316+0000 7f346f400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3468111f70 con 0x7f3468102790 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.316+0000 7f346f400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3468111df0 con 0x7f3468101de0 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.316+0000 7f346f400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f34681120f0 con 0x7f3468106f10 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f3467e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3468102790 0x7f346819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f346ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3468101de0 0x7f346819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f346ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3468101de0 0x7f346819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57012/0 (socket says 172.21.15.107:57012) 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f346ca00640 1 -- 172.21.15.107:0/1016559610 learned_addr learned my addr 172.21.15.107:0/1016559610 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f3467e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3468102790 0x7f346819ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45952/0 (socket says 172.21.15.107:45952) 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f346d400640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3468106f10 0x7f346819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:39.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f346ca00640 1 -- 172.21.15.107:0/1016559610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3468106f10 msgr2=0x7f346819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:39.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f346ca00640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3468106f10 0x7f346819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f346ca00640 1 -- 172.21.15.107:0/1016559610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3468102790 msgr2=0x7f346819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:39.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f346ca00640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3468102790 0x7f346819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f346ca00640 1 -- 172.21.15.107:0/1016559610 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346819f690 con 0x7f3468101de0 2024-09-19T04:18:39.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.317+0000 7f346ca00640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3468101de0 0x7f346819a6e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f345800ebc0 tx=0x7f345800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:39.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.318+0000 7f3465600640 1 -- 172.21.15.107:0/1016559610 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3458010630 con 0x7f3468101de0 2024-09-19T04:18:39.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.318+0000 7f3465600640 1 -- 172.21.15.107:0/1016559610 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3458010cb0 con 0x7f3468101de0 2024-09-19T04:18:39.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.318+0000 7f346f400640 1 -- 172.21.15.107:0/1016559610 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346819f980 con 0x7f3468101de0 2024-09-19T04:18:39.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.318+0000 7f3465600640 1 -- 172.21.15.107:0/1016559610 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3458018bd0 con 0x7f3468101de0 2024-09-19T04:18:39.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.318+0000 7f346f400640 1 -- 172.21.15.107:0/1016559610 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34681a7260 con 0x7f3468101de0 2024-09-19T04:18:39.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.319+0000 7f346f400640 1 -- 172.21.15.107:0/1016559610 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34681021c0 con 0x7f3468101de0 2024-09-19T04:18:39.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.321+0000 7f3465600640 1 -- 172.21.15.107:0/1016559610 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f34580107d0 con 0x7f3468101de0 2024-09-19T04:18:39.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.322+0000 7f3465600640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f344c0a32d0 0x7f344c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:39.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.322+0000 7f3465600640 1 -- 172.21.15.107:0/1016559610 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f34580d00d0 con 0x7f3468101de0 2024-09-19T04:18:39.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.322+0000 7f3467e00640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f344c0a32d0 0x7f344c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:39.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.323+0000 7f3467e00640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f344c0a32d0 0x7f344c0a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3454009230 tx=0x7f34540058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:39.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.324+0000 7f3465600640 1 -- 172.21.15.107:0/1016559610 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f345809c6f0 con 0x7f3468101de0 2024-09-19T04:18:39.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.500+0000 7f346f400640 1 -- 172.21.15.107:0/1016559610 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34681a7460 con 0x7f3468101de0 2024-09-19T04:18:39.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.502+0000 7f3465600640 1 -- 172.21.15.107:0/1016559610 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22348 (secure 0 0 0) 0x7f34580a09f0 con 0x7f3468101de0 2024-09-19T04:18:39.504 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:39.504 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":81,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:18:13.349028+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 -- 172.21.15.107:0/1016559610 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f344c0a32d0 msgr2=0x7f344c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f344c0a32d0 0x7f344c0a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3454009230 tx=0x7f34540058d0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 -- 172.21.15.107:0/1016559610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3468101de0 msgr2=0x7f346819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3468101de0 0x7f346819a6e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f345800ebc0 tx=0x7f345800c6a0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 -- 172.21.15.107:0/1016559610 shutdown_connections 2024-09-19T04:18:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f344c0a32d0 0x7f344c0a5790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3468106f10 0x7f346819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3468102790 0x7f346819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 --2- 172.21.15.107:0/1016559610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3468101de0 0x7f346819a6e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 -- 172.21.15.107:0/1016559610 >> 172.21.15.107:0/1016559610 conn(0x7f34680fd8c0 msgr2=0x7f34680ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.504+0000 7f346f400640 1 -- 172.21.15.107:0/1016559610 shutdown_connections 2024-09-19T04:18:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:39.505+0000 7f346f400640 1 -- 172.21.15.107:0/1016559610 wait complete. 2024-09-19T04:18:40.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:39 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1016559610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:40.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:39 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1016559610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:39 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1016559610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:40.259 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T04:18:40.259 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:18:40.627 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:41.059 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:40 smithi107 ceph-mon[30659]: pgmap v301: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 20 op/s 2024-09-19T04:18:41.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.166+0000 7f238cc00640 1 -- 172.21.15.107:0/4062064800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2388074ee0 msgr2=0x7f238810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:41.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.166+0000 7f238cc00640 1 --2- 172.21.15.107:0/4062064800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2388074ee0 0x7f238810dd20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2374009900 tx=0x7f237402a480 comp rx=0 tx=0).stop 2024-09-19T04:18:41.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.166+0000 7f238cc00640 1 -- 172.21.15.107:0/4062064800 shutdown_connections 2024-09-19T04:18:41.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.166+0000 7f238cc00640 1 --2- 172.21.15.107:0/4062064800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f238810e260 0x7f2388110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.166+0000 7f238cc00640 1 --2- 172.21.15.107:0/4062064800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2388074ee0 0x7f238810dd20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.166+0000 7f238cc00640 1 --2- 172.21.15.107:0/4062064800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23880745c0 0x7f23880749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.166+0000 7f238cc00640 1 -- 172.21.15.107:0/4062064800 >> 172.21.15.107:0/4062064800 conn(0x7f23880fd6e0 msgr2=0x7f23880ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:41.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.166+0000 7f238cc00640 1 -- 172.21.15.107:0/4062064800 shutdown_connections 2024-09-19T04:18:41.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.166+0000 7f238cc00640 1 -- 172.21.15.107:0/4062064800 wait complete. 2024-09-19T04:18:41.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f238cc00640 1 Processor -- start 2024-09-19T04:18:41.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f238cc00640 1 -- start start 2024-09-19T04:18:41.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f238cc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23880745c0 0x7f238819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f238cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2388074ee0 0x7f238819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f238cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f238810e260 0x7f238819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f238cc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2388114000 con 0x7f2388074ee0 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f238cc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2388113e80 con 0x7f23880745c0 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f238cc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2388114180 con 0x7f238810e260 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f2386a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2388074ee0 0x7f238819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f2386a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2388074ee0 0x7f238819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45978/0 (socket says 172.21.15.107:45978) 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f2386a00640 1 -- 172.21.15.107:0/1914305703 learned_addr learned my addr 172.21.15.107:0/1914305703 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.167+0000 7f2387400640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23880745c0 0x7f238819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.168+0000 7f2387e00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f238810e260 0x7f238819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.168+0000 7f2386a00640 1 -- 172.21.15.107:0/1914305703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f238810e260 msgr2=0x7f238819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:41.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.168+0000 7f2386a00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f238810e260 0x7f238819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.168+0000 7f2386a00640 1 -- 172.21.15.107:0/1914305703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23880745c0 msgr2=0x7f238819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.168+0000 7f2386a00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23880745c0 0x7f238819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.168+0000 7f2386a00640 1 -- 172.21.15.107:0/1914305703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238819f460 con 0x7f2388074ee0 2024-09-19T04:18:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.168+0000 7f2387400640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23880745c0 0x7f238819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.168+0000 7f2386a00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2388074ee0 0x7f238819a9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2374004770 tx=0x7f2374003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.169+0000 7f237fe00640 1 -- 172.21.15.107:0/1914305703 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237403f070 con 0x7f2388074ee0 2024-09-19T04:18:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.169+0000 7f237fe00640 1 -- 172.21.15.107:0/1914305703 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2374031ca0 con 0x7f2388074ee0 2024-09-19T04:18:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.169+0000 7f237fe00640 1 -- 172.21.15.107:0/1914305703 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374035030 con 0x7f2388074ee0 2024-09-19T04:18:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.169+0000 7f238cc00640 1 -- 172.21.15.107:0/1914305703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238819f6f0 con 0x7f2388074ee0 2024-09-19T04:18:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.169+0000 7f238cc00640 1 -- 172.21.15.107:0/1914305703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238819fb80 con 0x7f2388074ee0 2024-09-19T04:18:41.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.170+0000 7f238cc00640 1 -- 172.21.15.107:0/1914305703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23880749a0 con 0x7f2388074ee0 2024-09-19T04:18:41.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.172+0000 7f237fe00640 1 -- 172.21.15.107:0/1914305703 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2374042070 con 0x7f2388074ee0 2024-09-19T04:18:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.173+0000 7f237fe00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f236c0a32d0 0x7f236c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.173+0000 7f237fe00640 1 -- 172.21.15.107:0/1914305703 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f23740f0070 con 0x7f2388074ee0 2024-09-19T04:18:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.173+0000 7f2387400640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f236c0a32d0 0x7f236c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.174+0000 7f2387400640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f236c0a32d0 0x7f236c0a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2378005fd0 tx=0x7f237800b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:41.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.177+0000 7f237fe00640 1 -- 172.21.15.107:0/1914305703 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f23740bc690 con 0x7f2388074ee0 2024-09-19T04:18:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:40 smithi115 ceph-mon[35477]: pgmap v301: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 20 op/s 2024-09-19T04:18:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:40 smithi145 ceph-mon[35096]: pgmap v301: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 20 op/s 2024-09-19T04:18:41.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.350+0000 7f238cc00640 1 -- 172.21.15.107:0/1914305703 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2388062630 con 0x7f236c0a32d0 2024-09-19T04:18:41.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.359+0000 7f237fe00640 1 -- 172.21.15.107:0/1914305703 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258914 (secure 0 0 0) 0x7f2388062630 con 0x7f236c0a32d0 2024-09-19T04:18:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:41.365 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-19T04:18:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 -- 172.21.15.107:0/1914305703 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f236c0a32d0 msgr2=0x7f236c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f236c0a32d0 0x7f236c0a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2378005fd0 tx=0x7f237800b390 comp rx=0 tx=0).stop 2024-09-19T04:18:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 -- 172.21.15.107:0/1914305703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2388074ee0 msgr2=0x7f238819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2388074ee0 0x7f238819a9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2374004770 tx=0x7f2374003c10 comp rx=0 tx=0).stop 2024-09-19T04:18:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 -- 172.21.15.107:0/1914305703 shutdown_connections 2024-09-19T04:18:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f236c0a32d0 0x7f236c0a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f238810e260 0x7f238819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2388074ee0 0x7f238819a9f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 --2- 172.21.15.107:0/1914305703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23880745c0 0x7f238819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 -- 172.21.15.107:0/1914305703 >> 172.21.15.107:0/1914305703 conn(0x7f23880fd6e0 msgr2=0x7f23880fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:41.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 -- 172.21.15.107:0/1914305703 shutdown_connections 2024-09-19T04:18:41.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:41.363+0000 7f238cc00640 1 -- 172.21.15.107:0/1914305703 wait complete. 2024-09-19T04:18:42.245 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-19T04:18:39.813703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647922,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":305,"num_read_kb":265,"num_write":417,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372648,"kb_used_data":10920,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569176,"statfs":{"total":1151940427776,"available":1151558836224,"internally_reserved":0,"allocated":11182080,"data_stored":7015278,"data_compressed":107517,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51548,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":163,"num_read_kb":140,"num_write":87,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003017"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480938+0000","last_change":"2024-09-19T04:18:12.499720+0000","last_active":"2024-09-19T04:18:13.480938+0000","last_peered":"2024-09-19T04:18:13.480938+0000","last_clean":"2024-09-19T04:18:13.480938+0000","last_became_active":"2024-09-19T04:18:12.499381+0000","last_became_peered":"2024-09-19T04:18:12.499381+0000","last_unstale":"2024-09-19T04:18:13.480938+0000","last_undegraded":"2024-09-19T04:18:13.480938+0000","last_fullsized":"2024-09-19T04:18:13.480938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:46.708870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491536+0000","last_change":"2024-09-19T04:18:12.498163+0000","last_active":"2024-09-19T04:18:13.491536+0000","last_peered":"2024-09-19T04:18:13.491536+0000","last_clean":"2024-09-19T04:18:13.491536+0000","last_became_active":"2024-09-19T04:18:12.495706+0000","last_became_peered":"2024-09-19T04:18:12.495706+0000","last_unstale":"2024-09-19T04:18:13.491536+0000","last_undegraded":"2024-09-19T04:18:13.491536+0000","last_fullsized":"2024-09-19T04:18:13.491536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:50.604410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491537+0000","last_change":"2024-09-19T04:18:12.493903+0000","last_active":"2024-09-19T04:18:13.491537+0000","last_peered":"2024-09-19T04:18:13.491537+0000","last_clean":"2024-09-19T04:18:13.491537+0000","last_became_active":"2024-09-19T04:18:12.493443+0000","last_became_peered":"2024-09-19T04:18:12.493443+0000","last_unstale":"2024-09-19T04:18:13.491537+0000","last_undegraded":"2024-09-19T04:18:13.491537+0000","last_fullsized":"2024-09-19T04:18:13.491537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:51.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487370+0000","last_change":"2024-09-19T04:18:12.500460+0000","last_active":"2024-09-19T04:18:13.487370+0000","last_peered":"2024-09-19T04:18:13.487370+0000","last_clean":"2024-09-19T04:18:13.487370+0000","last_became_active":"2024-09-19T04:18:12.494675+0000","last_became_peered":"2024-09-19T04:18:12.494675+0000","last_unstale":"2024-09-19T04:18:13.487370+0000","last_undegraded":"2024-09-19T04:18:13.487370+0000","last_fullsized":"2024-09-19T04:18:13.487370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:13:32.863156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484850+0000","last_change":"2024-09-19T04:18:12.494936+0000","last_active":"2024-09-19T04:18:13.484850+0000","last_peered":"2024-09-19T04:18:13.484850+0000","last_clean":"2024-09-19T04:18:13.484850+0000","last_became_active":"2024-09-19T04:18:12.494556+0000","last_became_peered":"2024-09-19T04:18:12.494556+0000","last_unstale":"2024-09-19T04:18:13.484850+0000","last_undegraded":"2024-09-19T04:18:13.484850+0000","last_fullsized":"2024-09-19T04:18:13.484850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:06.001529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478573+0000","last_change":"2024-09-19T04:18:12.495880+0000","last_active":"2024-09-19T04:18:13.478573+0000","last_peered":"2024-09-19T04:18:13.478573+0000","last_clean":"2024-09-19T04:18:13.478573+0000","last_became_active":"2024-09-19T04:18:12.495562+0000","last_became_peered":"2024-09-19T04:18:12.495562+0000","last_unstale":"2024-09-19T04:18:13.478573+0000","last_undegraded":"2024-09-19T04:18:13.478573+0000","last_fullsized":"2024-09-19T04:18:13.478573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:20.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490803+0000","last_change":"2024-09-19T04:18:12.497991+0000","last_active":"2024-09-19T04:18:13.490803+0000","last_peered":"2024-09-19T04:18:13.490803+0000","last_clean":"2024-09-19T04:18:13.490803+0000","last_became_active":"2024-09-19T04:18:12.497644+0000","last_became_peered":"2024-09-19T04:18:12.497644+0000","last_unstale":"2024-09-19T04:18:13.490803+0000","last_undegraded":"2024-09-19T04:18:13.490803+0000","last_fullsized":"2024-09-19T04:18:13.490803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.851930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484998+0000","last_change":"2024-09-19T04:18:12.495789+0000","last_active":"2024-09-19T04:18:13.484998+0000","last_peered":"2024-09-19T04:18:13.484998+0000","last_clean":"2024-09-19T04:18:13.484998+0000","last_became_active":"2024-09-19T04:18:12.494637+0000","last_became_peered":"2024-09-19T04:18:12.494637+0000","last_unstale":"2024-09-19T04:18:13.484998+0000","last_undegraded":"2024-09-19T04:18:13.484998+0000","last_fullsized":"2024-09-19T04:18:13.484998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:14.245180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485190+0000","last_change":"2024-09-19T04:18:12.491433+0000","last_active":"2024-09-19T04:18:13.485190+0000","last_peered":"2024-09-19T04:18:13.485190+0000","last_clean":"2024-09-19T04:18:13.485190+0000","last_became_active":"2024-09-19T04:18:12.490444+0000","last_became_peered":"2024-09-19T04:18:12.490444+0000","last_unstale":"2024-09-19T04:18:13.485190+0000","last_undegraded":"2024-09-19T04:18:13.485190+0000","last_fullsized":"2024-09-19T04:18:13.485190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:43.068355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.479183+0000","last_change":"2024-09-19T04:18:12.493188+0000","last_active":"2024-09-19T04:18:13.479183+0000","last_peered":"2024-09-19T04:18:13.479183+0000","last_clean":"2024-09-19T04:18:13.479183+0000","last_became_active":"2024-09-19T04:18:12.492788+0000","last_became_peered":"2024-09-19T04:18:12.492788+0000","last_unstale":"2024-09-19T04:18:13.479183+0000","last_undegraded":"2024-09-19T04:18:13.479183+0000","last_fullsized":"2024-09-19T04:18:13.479183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:17.912820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490852+0000","last_change":"2024-09-19T04:18:12.490616+0000","last_active":"2024-09-19T04:18:13.490852+0000","last_peered":"2024-09-19T04:18:13.490852+0000","last_clean":"2024-09-19T04:18:13.490852+0000","last_became_active":"2024-09-19T04:18:12.490155+0000","last_became_peered":"2024-09-19T04:18:12.490155+0000","last_unstale":"2024-09-19T04:18:13.490852+0000","last_undegraded":"2024-09-19T04:18:13.490852+0000","last_fullsized":"2024-09-19T04:18:13.490852+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:47.082082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.486611+0000","last_change":"2024-09-19T04:18:12.494191+0000","last_active":"2024-09-19T04:18:13.486611+0000","last_peered":"2024-09-19T04:18:13.486611+0000","last_clean":"2024-09-19T04:18:13.486611+0000","last_became_active":"2024-09-19T04:18:12.493786+0000","last_became_peered":"2024-09-19T04:18:12.493786+0000","last_unstale":"2024-09-19T04:18:13.486611+0000","last_undegraded":"2024-09-19T04:18:13.486611+0000","last_fullsized":"2024-09-19T04:18:13.486611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:29.691556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478270+0000","last_change":"2024-09-19T04:18:12.482167+0000","last_active":"2024-09-19T04:18:13.478270+0000","last_peered":"2024-09-19T04:18:13.478270+0000","last_clean":"2024-09-19T04:18:13.478270+0000","last_became_active":"2024-09-19T04:18:12.481314+0000","last_became_peered":"2024-09-19T04:18:12.481314+0000","last_unstale":"2024-09-19T04:18:13.478270+0000","last_undegraded":"2024-09-19T04:18:13.478270+0000","last_fullsized":"2024-09-19T04:18:13.478270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:28.081641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491377+0000","last_change":"2024-09-19T04:18:12.491711+0000","last_active":"2024-09-19T04:18:13.491377+0000","last_peered":"2024-09-19T04:18:13.491377+0000","last_clean":"2024-09-19T04:18:13.491377+0000","last_became_active":"2024-09-19T04:18:12.490964+0000","last_became_peered":"2024-09-19T04:18:12.490964+0000","last_unstale":"2024-09-19T04:18:13.491377+0000","last_undegraded":"2024-09-19T04:18:13.491377+0000","last_fullsized":"2024-09-19T04:18:13.491377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:16.607679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491940+0000","last_change":"2024-09-19T04:18:12.504737+0000","last_active":"2024-09-19T04:18:13.491940+0000","last_peered":"2024-09-19T04:18:13.491940+0000","last_clean":"2024-09-19T04:18:13.491940+0000","last_became_active":"2024-09-19T04:18:12.497583+0000","last_became_peered":"2024-09-19T04:18:12.497583+0000","last_unstale":"2024-09-19T04:18:13.491940+0000","last_undegraded":"2024-09-19T04:18:13.491940+0000","last_fullsized":"2024-09-19T04:18:13.491940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:48.456738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484709+0000","last_change":"2024-09-19T04:18:12.499045+0000","last_active":"2024-09-19T04:18:13.484709+0000","last_peered":"2024-09-19T04:18:13.484709+0000","last_clean":"2024-09-19T04:18:13.484709+0000","last_became_active":"2024-09-19T04:18:12.498713+0000","last_became_peered":"2024-09-19T04:18:12.498713+0000","last_unstale":"2024-09-19T04:18:13.484709+0000","last_undegraded":"2024-09-19T04:18:13.484709+0000","last_fullsized":"2024-09-19T04:18:13.484709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:54.337769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480975+0000","last_change":"2024-09-19T04:18:12.495989+0000","last_active":"2024-09-19T04:18:13.480975+0000","last_peered":"2024-09-19T04:18:13.480975+0000","last_clean":"2024-09-19T04:18:13.480975+0000","last_became_active":"2024-09-19T04:18:12.495633+0000","last_became_peered":"2024-09-19T04:18:12.495633+0000","last_unstale":"2024-09-19T04:18:13.480975+0000","last_undegraded":"2024-09-19T04:18:13.480975+0000","last_fullsized":"2024-09-19T04:18:13.480975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:17.225861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485052+0000","last_change":"2024-09-19T04:18:12.488532+0000","last_active":"2024-09-19T04:18:13.485052+0000","last_peered":"2024-09-19T04:18:13.485052+0000","last_clean":"2024-09-19T04:18:13.485052+0000","last_became_active":"2024-09-19T04:18:12.488117+0000","last_became_peered":"2024-09-19T04:18:12.488117+0000","last_unstale":"2024-09-19T04:18:13.485052+0000","last_undegraded":"2024-09-19T04:18:13.485052+0000","last_fullsized":"2024-09-19T04:18:13.485052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:45.318852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487085+0000","last_change":"2024-09-19T04:18:12.496784+0000","last_active":"2024-09-19T04:18:13.487085+0000","last_peered":"2024-09-19T04:18:13.487085+0000","last_clean":"2024-09-19T04:18:13.487085+0000","last_became_active":"2024-09-19T04:18:12.496443+0000","last_became_peered":"2024-09-19T04:18:12.496443+0000","last_unstale":"2024-09-19T04:18:13.487085+0000","last_undegraded":"2024-09-19T04:18:13.487085+0000","last_fullsized":"2024-09-19T04:18:13.487085+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:25:46.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480595+0000","last_change":"2024-09-19T04:18:12.496196+0000","last_active":"2024-09-19T04:18:13.480595+0000","last_peered":"2024-09-19T04:18:13.480595+0000","last_clean":"2024-09-19T04:18:13.480595+0000","last_became_active":"2024-09-19T04:18:12.495517+0000","last_became_peered":"2024-09-19T04:18:12.495517+0000","last_unstale":"2024-09-19T04:18:13.480595+0000","last_undegraded":"2024-09-19T04:18:13.480595+0000","last_fullsized":"2024-09-19T04:18:13.480595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:32.281885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478408+0000","last_change":"2024-09-19T04:18:12.493731+0000","last_active":"2024-09-19T04:18:13.478408+0000","last_peered":"2024-09-19T04:18:13.478408+0000","last_clean":"2024-09-19T04:18:13.478408+0000","last_became_active":"2024-09-19T04:18:12.493412+0000","last_became_peered":"2024-09-19T04:18:12.493412+0000","last_unstale":"2024-09-19T04:18:13.478408+0000","last_undegraded":"2024-09-19T04:18:13.478408+0000","last_fullsized":"2024-09-19T04:18:13.478408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:41.089637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485124+0000","last_change":"2024-09-19T04:18:12.497033+0000","last_active":"2024-09-19T04:18:13.485124+0000","last_peered":"2024-09-19T04:18:13.485124+0000","last_clean":"2024-09-19T04:18:13.485124+0000","last_became_active":"2024-09-19T04:18:12.496710+0000","last_became_peered":"2024-09-19T04:18:12.496710+0000","last_unstale":"2024-09-19T04:18:13.485124+0000","last_undegraded":"2024-09-19T04:18:13.485124+0000","last_fullsized":"2024-09-19T04:18:13.485124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:19.193869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485428+0000","last_change":"2024-09-19T04:18:12.499213+0000","last_active":"2024-09-19T04:18:13.485428+0000","last_peered":"2024-09-19T04:18:13.485428+0000","last_clean":"2024-09-19T04:18:13.485428+0000","last_became_active":"2024-09-19T04:18:12.498728+0000","last_became_peered":"2024-09-19T04:18:12.498728+0000","last_unstale":"2024-09-19T04:18:13.485428+0000","last_undegraded":"2024-09-19T04:18:13.485428+0000","last_fullsized":"2024-09-19T04:18:13.485428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:37.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487908+0000","last_change":"2024-09-19T04:18:12.498603+0000","last_active":"2024-09-19T04:18:13.487908+0000","last_peered":"2024-09-19T04:18:13.487908+0000","last_clean":"2024-09-19T04:18:13.487908+0000","last_became_active":"2024-09-19T04:18:12.497958+0000","last_became_peered":"2024-09-19T04:18:12.497958+0000","last_unstale":"2024-09-19T04:18:13.487908+0000","last_undegraded":"2024-09-19T04:18:13.487908+0000","last_fullsized":"2024-09-19T04:18:13.487908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:57.699594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487095+0000","last_change":"2024-09-19T04:18:12.506679+0000","last_active":"2024-09-19T04:18:13.487095+0000","last_peered":"2024-09-19T04:18:13.487095+0000","last_clean":"2024-09-19T04:18:13.487095+0000","last_became_active":"2024-09-19T04:18:12.505953+0000","last_became_peered":"2024-09-19T04:18:12.505953+0000","last_unstale":"2024-09-19T04:18:13.487095+0000","last_undegraded":"2024-09-19T04:18:13.487095+0000","last_fullsized":"2024-09-19T04:18:13.487095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:53.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.489669+0000","last_change":"2024-09-19T04:18:12.490511+0000","last_active":"2024-09-19T04:18:13.489669+0000","last_peered":"2024-09-19T04:18:13.489669+0000","last_clean":"2024-09-19T04:18:13.489669+0000","last_became_active":"2024-09-19T04:18:12.490051+0000","last_became_peered":"2024-09-19T04:18:12.490051+0000","last_unstale":"2024-09-19T04:18:13.489669+0000","last_undegraded":"2024-09-19T04:18:13.489669+0000","last_fullsized":"2024-09-19T04:18:13.489669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:32.283312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480846+0000","last_change":"2024-09-19T04:18:12.491465+0000","last_active":"2024-09-19T04:18:13.480846+0000","last_peered":"2024-09-19T04:18:13.480846+0000","last_clean":"2024-09-19T04:18:13.480846+0000","last_became_active":"2024-09-19T04:18:12.491136+0000","last_became_peered":"2024-09-19T04:18:12.491136+0000","last_unstale":"2024-09-19T04:18:13.480846+0000","last_undegraded":"2024-09-19T04:18:13.480846+0000","last_fullsized":"2024-09-19T04:18:13.480846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:29.498909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478381+0000","last_change":"2024-09-19T04:18:12.494879+0000","last_active":"2024-09-19T04:18:13.478381+0000","last_peered":"2024-09-19T04:18:13.478381+0000","last_clean":"2024-09-19T04:18:13.478381+0000","last_became_active":"2024-09-19T04:18:12.494534+0000","last_became_peered":"2024-09-19T04:18:12.494534+0000","last_unstale":"2024-09-19T04:18:13.478381+0000","last_undegraded":"2024-09-19T04:18:13.478381+0000","last_fullsized":"2024-09-19T04:18:13.478381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.647417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491875+0000","last_change":"2024-09-19T04:18:12.504364+0000","last_active":"2024-09-19T04:18:13.491875+0000","last_peered":"2024-09-19T04:18:13.491875+0000","last_clean":"2024-09-19T04:18:13.491875+0000","last_became_active":"2024-09-19T04:18:12.504006+0000","last_became_peered":"2024-09-19T04:18:12.504006+0000","last_unstale":"2024-09-19T04:18:13.491875+0000","last_undegraded":"2024-09-19T04:18:13.491875+0000","last_fullsized":"2024-09-19T04:18:13.491875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:53.309963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485337+0000","last_change":"2024-09-19T04:18:12.498223+0000","last_active":"2024-09-19T04:18:13.485337+0000","last_peered":"2024-09-19T04:18:13.485337+0000","last_clean":"2024-09-19T04:18:13.485337+0000","last_became_active":"2024-09-19T04:18:12.497862+0000","last_became_peered":"2024-09-19T04:18:12.497862+0000","last_unstale":"2024-09-19T04:18:13.485337+0000","last_undegraded":"2024-09-19T04:18:13.485337+0000","last_fullsized":"2024-09-19T04:18:13.485337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:28.858620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485415+0000","last_change":"2024-09-19T04:18:12.502335+0000","last_active":"2024-09-19T04:18:13.485415+0000","last_peered":"2024-09-19T04:18:13.485415+0000","last_clean":"2024-09-19T04:18:13.485415+0000","last_became_active":"2024-09-19T04:18:12.501686+0000","last_became_peered":"2024-09-19T04:18:12.501686+0000","last_unstale":"2024-09-19T04:18:13.485415+0000","last_undegraded":"2024-09-19T04:18:13.485415+0000","last_fullsized":"2024-09-19T04:18:13.485415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:22.348745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480527+0000","last_change":"2024-09-19T04:18:12.490905+0000","last_active":"2024-09-19T04:18:13.480527+0000","last_peered":"2024-09-19T04:18:13.480527+0000","last_clean":"2024-09-19T04:18:13.480527+0000","last_became_active":"2024-09-19T04:18:12.490489+0000","last_became_peered":"2024-09-19T04:18:12.490489+0000","last_unstale":"2024-09-19T04:18:13.480527+0000","last_undegraded":"2024-09-19T04:18:13.480527+0000","last_fullsized":"2024-09-19T04:18:13.480527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.946474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485197+0000","last_change":"2024-09-19T04:18:12.494607+0000","last_active":"2024-09-19T04:18:13.485197+0000","last_peered":"2024-09-19T04:18:13.485197+0000","last_clean":"2024-09-19T04:18:13.485197+0000","last_became_active":"2024-09-19T04:18:12.494288+0000","last_became_peered":"2024-09-19T04:18:12.494288+0000","last_unstale":"2024-09-19T04:18:13.485197+0000","last_undegraded":"2024-09-19T04:18:13.485197+0000","last_fullsized":"2024-09-19T04:18:13.485197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:42.824951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:37.600084+0000","last_change":"2024-09-19T04:18:10.103108+0000","last_active":"2024-09-19T04:18:37.600084+0000","last_peered":"2024-09-19T04:18:37.600084+0000","last_clean":"2024-09-19T04:18:37.600084+0000","last_became_active":"2024-09-19T04:18:10.102783+0000","last_became_peered":"2024-09-19T04:18:10.102783+0000","last_unstale":"2024-09-19T04:18:37.600084+0000","last_undegraded":"2024-09-19T04:18:37.600084+0000","last_fullsized":"2024-09-19T04:18:37.600084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:25.521953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.488677+0000","last_change":"2024-09-19T04:18:12.495453+0000","last_active":"2024-09-19T04:18:13.488677+0000","last_peered":"2024-09-19T04:18:13.488677+0000","last_clean":"2024-09-19T04:18:13.488677+0000","last_became_active":"2024-09-19T04:18:12.494744+0000","last_became_peered":"2024-09-19T04:18:12.494744+0000","last_unstale":"2024-09-19T04:18:13.488677+0000","last_undegraded":"2024-09-19T04:18:13.488677+0000","last_fullsized":"2024-09-19T04:18:13.488677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:41.071723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:23.567040+0000","last_change":"2024-09-19T04:18:10.102675+0000","last_active":"2024-09-19T04:18:23.567040+0000","last_peered":"2024-09-19T04:18:23.567040+0000","last_clean":"2024-09-19T04:18:23.567040+0000","last_became_active":"2024-09-19T04:18:10.102368+0000","last_became_peered":"2024-09-19T04:18:10.102368+0000","last_unstale":"2024-09-19T04:18:23.567040+0000","last_undegraded":"2024-09-19T04:18:23.567040+0000","last_fullsized":"2024-09-19T04:18:23.567040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:45.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:22.558114+0000","last_change":"2024-09-19T04:18:10.099932+0000","last_active":"2024-09-19T04:18:22.558114+0000","last_peered":"2024-09-19T04:18:22.558114+0000","last_clean":"2024-09-19T04:18:22.558114+0000","last_became_active":"2024-09-19T04:18:10.099637+0000","last_became_peered":"2024-09-19T04:18:10.099637+0000","last_unstale":"2024-09-19T04:18:22.558114+0000","last_undegraded":"2024-09-19T04:18:22.558114+0000","last_fullsized":"2024-09-19T04:18:22.558114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:50.594382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485265+0000","last_change":"2024-09-19T04:18:12.502229+0000","last_active":"2024-09-19T04:18:13.485265+0000","last_peered":"2024-09-19T04:18:13.485265+0000","last_clean":"2024-09-19T04:18:13.485265+0000","last_became_active":"2024-09-19T04:18:12.501921+0000","last_became_peered":"2024-09-19T04:18:12.501921+0000","last_unstale":"2024-09-19T04:18:13.485265+0000","last_undegraded":"2024-09-19T04:18:13.485265+0000","last_fullsized":"2024-09-19T04:18:13.485265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:55.412103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.576553+0000","last_change":"2024-09-19T04:18:10.103961+0000","last_active":"2024-09-19T04:18:24.576553+0000","last_peered":"2024-09-19T04:18:24.576553+0000","last_clean":"2024-09-19T04:18:24.576553+0000","last_became_active":"2024-09-19T04:18:10.102737+0000","last_became_peered":"2024-09-19T04:18:10.102737+0000","last_unstale":"2024-09-19T04:18:24.576553+0000","last_undegraded":"2024-09-19T04:18:24.576553+0000","last_fullsized":"2024-09-19T04:18:24.576553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:31:01.806124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484985+0000","last_change":"2024-09-19T04:18:12.502385+0000","last_active":"2024-09-19T04:18:13.484985+0000","last_peered":"2024-09-19T04:18:13.484985+0000","last_clean":"2024-09-19T04:18:13.484985+0000","last_became_active":"2024-09-19T04:18:12.502025+0000","last_became_peered":"2024-09-19T04:18:12.502025+0000","last_unstale":"2024-09-19T04:18:13.484985+0000","last_undegraded":"2024-09-19T04:18:13.484985+0000","last_fullsized":"2024-09-19T04:18:13.484985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:24.650911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:37.599189+0000","last_change":"2024-09-19T04:18:10.103526+0000","last_active":"2024-09-19T04:18:37.599189+0000","last_peered":"2024-09-19T04:18:37.599189+0000","last_clean":"2024-09-19T04:18:37.599189+0000","last_became_active":"2024-09-19T04:18:10.103191+0000","last_became_peered":"2024-09-19T04:18:10.103191+0000","last_unstale":"2024-09-19T04:18:37.599189+0000","last_undegraded":"2024-09-19T04:18:37.599189+0000","last_fullsized":"2024-09-19T04:18:37.599189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:07.695715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485430+0000","last_change":"2024-09-19T04:18:12.493169+0000","last_active":"2024-09-19T04:18:13.485430+0000","last_peered":"2024-09-19T04:18:13.485430+0000","last_clean":"2024-09-19T04:18:13.485430+0000","last_became_active":"2024-09-19T04:18:12.492669+0000","last_became_peered":"2024-09-19T04:18:12.492669+0000","last_unstale":"2024-09-19T04:18:13.485430+0000","last_undegraded":"2024-09-19T04:18:13.485430+0000","last_fullsized":"2024-09-19T04:18:13.485430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:04.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.574947+0000","last_change":"2024-09-19T04:18:10.102835+0000","last_active":"2024-09-19T04:18:24.574947+0000","last_peered":"2024-09-19T04:18:24.574947+0000","last_clean":"2024-09-19T04:18:24.574947+0000","last_became_active":"2024-09-19T04:18:10.102494+0000","last_became_peered":"2024-09-19T04:18:10.102494+0000","last_unstale":"2024-09-19T04:18:24.574947+0000","last_undegraded":"2024-09-19T04:18:24.574947+0000","last_fullsized":"2024-09-19T04:18:24.574947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:03:17.218245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.492030+0000","last_change":"2024-09-19T04:18:12.504348+0000","last_active":"2024-09-19T04:18:13.492030+0000","last_peered":"2024-09-19T04:18:13.492030+0000","last_clean":"2024-09-19T04:18:13.492030+0000","last_became_active":"2024-09-19T04:18:12.503180+0000","last_became_peered":"2024-09-19T04:18:12.503180+0000","last_unstale":"2024-09-19T04:18:13.492030+0000","last_undegraded":"2024-09-19T04:18:13.492030+0000","last_fullsized":"2024-09-19T04:18:13.492030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:03.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:23.566490+0000","last_change":"2024-09-19T04:18:10.365136+0000","last_active":"2024-09-19T04:18:23.566490+0000","last_peered":"2024-09-19T04:18:23.566490+0000","last_clean":"2024-09-19T04:18:23.566490+0000","last_became_active":"2024-09-19T04:18:10.364594+0000","last_became_peered":"2024-09-19T04:18:10.364594+0000","last_unstale":"2024-09-19T04:18:23.566490+0000","last_undegraded":"2024-09-19T04:18:23.566490+0000","last_fullsized":"2024-09-19T04:18:23.566490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:52.252157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484882+0000","last_change":"2024-09-19T04:18:12.502051+0000","last_active":"2024-09-19T04:18:13.484882+0000","last_peered":"2024-09-19T04:18:13.484882+0000","last_clean":"2024-09-19T04:18:13.484882+0000","last_became_active":"2024-09-19T04:18:12.501578+0000","last_became_peered":"2024-09-19T04:18:12.501578+0000","last_unstale":"2024-09-19T04:18:13.484882+0000","last_undegraded":"2024-09-19T04:18:13.484882+0000","last_fullsized":"2024-09-19T04:18:13.484882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:59.693175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'27","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:37.599800+0000","last_change":"2024-09-19T04:18:10.101507+0000","last_active":"2024-09-19T04:18:37.599800+0000","last_peered":"2024-09-19T04:18:37.599800+0000","last_clean":"2024-09-19T04:18:37.599800+0000","last_became_active":"2024-09-19T04:18:10.101152+0000","last_became_peered":"2024-09-19T04:18:10.101152+0000","last_unstale":"2024-09-19T04:18:37.599800+0000","last_undegraded":"2024-09-19T04:18:37.599800+0000","last_fullsized":"2024-09-19T04:18:37.599800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:01.397605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":33,"num_read_kb":64,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485358+0000","last_change":"2024-09-19T04:18:12.491611+0000","last_active":"2024-09-19T04:18:13.485358+0000","last_peered":"2024-09-19T04:18:13.485358+0000","last_clean":"2024-09-19T04:18:13.485358+0000","last_became_active":"2024-09-19T04:18:12.491279+0000","last_became_peered":"2024-09-19T04:18:12.491279+0000","last_unstale":"2024-09-19T04:18:13.485358+0000","last_undegraded":"2024-09-19T04:18:13.485358+0000","last_fullsized":"2024-09-19T04:18:13.485358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:01.414713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:25.578520+0000","last_change":"2024-09-19T04:18:10.105574+0000","last_active":"2024-09-19T04:18:25.578520+0000","last_peered":"2024-09-19T04:18:25.578520+0000","last_clean":"2024-09-19T04:18:25.578520+0000","last_became_active":"2024-09-19T04:18:10.105203+0000","last_became_peered":"2024-09-19T04:18:10.105203+0000","last_unstale":"2024-09-19T04:18:25.578520+0000","last_undegraded":"2024-09-19T04:18:25.578520+0000","last_fullsized":"2024-09-19T04:18:25.578520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:00.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478119+0000","last_change":"2024-09-19T04:18:12.497242+0000","last_active":"2024-09-19T04:18:13.478119+0000","last_peered":"2024-09-19T04:18:13.478119+0000","last_clean":"2024-09-19T04:18:13.478119+0000","last_became_active":"2024-09-19T04:18:12.496732+0000","last_became_peered":"2024-09-19T04:18:12.496732+0000","last_unstale":"2024-09-19T04:18:13.478119+0000","last_undegraded":"2024-09-19T04:18:13.478119+0000","last_fullsized":"2024-09-19T04:18:13.478119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:56.520341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.576520+0000","last_change":"2024-09-19T04:18:10.103905+0000","last_active":"2024-09-19T04:18:24.576520+0000","last_peered":"2024-09-19T04:18:24.576520+0000","last_clean":"2024-09-19T04:18:24.576520+0000","last_became_active":"2024-09-19T04:18:10.103517+0000","last_became_peered":"2024-09-19T04:18:10.103517+0000","last_unstale":"2024-09-19T04:18:24.576520+0000","last_undegraded":"2024-09-19T04:18:24.576520+0000","last_fullsized":"2024-09-19T04:18:24.576520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:48.292878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490128+0000","last_change":"2024-09-19T04:18:12.499380+0000","last_active":"2024-09-19T04:18:13.490128+0000","last_peered":"2024-09-19T04:18:13.490128+0000","last_clean":"2024-09-19T04:18:13.490128+0000","last_became_active":"2024-09-19T04:18:12.498995+0000","last_became_peered":"2024-09-19T04:18:12.498995+0000","last_unstale":"2024-09-19T04:18:13.490128+0000","last_undegraded":"2024-09-19T04:18:13.490128+0000","last_fullsized":"2024-09-19T04:18:13.490128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:20.653923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:36.597466+0000","last_change":"2024-09-19T04:18:10.105619+0000","last_active":"2024-09-19T04:18:36.597466+0000","last_peered":"2024-09-19T04:18:36.597466+0000","last_clean":"2024-09-19T04:18:36.597466+0000","last_became_active":"2024-09-19T04:18:10.105082+0000","last_became_peered":"2024-09-19T04:18:10.105082+0000","last_unstale":"2024-09-19T04:18:36.597466+0000","last_undegraded":"2024-09-19T04:18:36.597466+0000","last_fullsized":"2024-09-19T04:18:36.597466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:56.843068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.489971+0000","last_change":"2024-09-19T04:18:12.498473+0000","last_active":"2024-09-19T04:18:13.489971+0000","last_peered":"2024-09-19T04:18:13.489971+0000","last_clean":"2024-09-19T04:18:13.489971+0000","last_became_active":"2024-09-19T04:18:12.496904+0000","last_became_peered":"2024-09-19T04:18:12.496904+0000","last_unstale":"2024-09-19T04:18:13.489971+0000","last_undegraded":"2024-09-19T04:18:13.489971+0000","last_fullsized":"2024-09-19T04:18:13.489971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:03.155899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.578125+0000","last_change":"2024-09-19T04:18:10.365182+0000","last_active":"2024-09-19T04:18:24.578125+0000","last_peered":"2024-09-19T04:18:24.578125+0000","last_clean":"2024-09-19T04:18:24.578125+0000","last_became_active":"2024-09-19T04:18:10.364717+0000","last_became_peered":"2024-09-19T04:18:10.364717+0000","last_unstale":"2024-09-19T04:18:24.578125+0000","last_undegraded":"2024-09-19T04:18:24.578125+0000","last_fullsized":"2024-09-19T04:18:24.578125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:32.063066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487770+0000","last_change":"2024-09-19T04:18:12.493585+0000","last_active":"2024-09-19T04:18:13.487770+0000","last_peered":"2024-09-19T04:18:13.487770+0000","last_clean":"2024-09-19T04:18:13.487770+0000","last_became_active":"2024-09-19T04:18:12.493255+0000","last_became_peered":"2024-09-19T04:18:12.493255+0000","last_unstale":"2024-09-19T04:18:13.487770+0000","last_undegraded":"2024-09-19T04:18:13.487770+0000","last_fullsized":"2024-09-19T04:18:13.487770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:33.139727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.551615+0000","last_change":"2024-09-19T04:18:10.101537+0000","last_active":"2024-09-19T04:18:24.551615+0000","last_peered":"2024-09-19T04:18:24.551615+0000","last_clean":"2024-09-19T04:18:24.551615+0000","last_became_active":"2024-09-19T04:18:10.100951+0000","last_became_peered":"2024-09-19T04:18:10.100951+0000","last_unstale":"2024-09-19T04:18:24.551615+0000","last_undegraded":"2024-09-19T04:18:24.551615+0000","last_fullsized":"2024-09-19T04:18:24.551615+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:08.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480839+0000","last_change":"2024-09-19T04:18:12.495530+0000","last_active":"2024-09-19T04:18:13.480839+0000","last_peered":"2024-09-19T04:18:13.480839+0000","last_clean":"2024-09-19T04:18:13.480839+0000","last_became_active":"2024-09-19T04:18:12.495176+0000","last_became_peered":"2024-09-19T04:18:12.495176+0000","last_unstale":"2024-09-19T04:18:13.480839+0000","last_undegraded":"2024-09-19T04:18:13.480839+0000","last_fullsized":"2024-09-19T04:18:13.480839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:24.308047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.576172+0000","last_change":"2024-09-19T04:18:10.102539+0000","last_active":"2024-09-19T04:18:24.576172+0000","last_peered":"2024-09-19T04:18:24.576172+0000","last_clean":"2024-09-19T04:18:24.576172+0000","last_became_active":"2024-09-19T04:18:10.102109+0000","last_became_peered":"2024-09-19T04:18:10.102109+0000","last_unstale":"2024-09-19T04:18:24.576172+0000","last_undegraded":"2024-09-19T04:18:24.576172+0000","last_fullsized":"2024-09-19T04:18:24.576172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:58.852767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490945+0000","last_change":"2024-09-19T04:18:12.491769+0000","last_active":"2024-09-19T04:18:13.490945+0000","last_peered":"2024-09-19T04:18:13.490945+0000","last_clean":"2024-09-19T04:18:13.490945+0000","last_became_active":"2024-09-19T04:18:12.491435+0000","last_became_peered":"2024-09-19T04:18:12.491435+0000","last_unstale":"2024-09-19T04:18:13.490945+0000","last_undegraded":"2024-09-19T04:18:13.490945+0000","last_fullsized":"2024-09-19T04:18:13.490945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:54.880533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.576052+0000","last_change":"2024-09-19T04:18:10.103559+0000","last_active":"2024-09-19T04:18:24.576052+0000","last_peered":"2024-09-19T04:18:24.576052+0000","last_clean":"2024-09-19T04:18:24.576052+0000","last_became_active":"2024-09-19T04:18:10.103208+0000","last_became_peered":"2024-09-19T04:18:10.103208+0000","last_unstale":"2024-09-19T04:18:24.576052+0000","last_undegraded":"2024-09-19T04:18:24.576052+0000","last_fullsized":"2024-09-19T04:18:24.576052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:16.564916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.999740+0000","last_change":"2024-09-19T04:18:12.495616+0000","last_active":"2024-09-19T04:18:13.999740+0000","last_peered":"2024-09-19T04:18:13.999740+0000","last_clean":"2024-09-19T04:18:13.999740+0000","last_became_active":"2024-09-19T04:18:12.495243+0000","last_became_peered":"2024-09-19T04:18:12.495243+0000","last_unstale":"2024-09-19T04:18:13.999740+0000","last_undegraded":"2024-09-19T04:18:13.999740+0000","last_fullsized":"2024-09-19T04:18:13.999740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:12.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:36.598514+0000","last_change":"2024-09-19T04:18:10.626874+0000","last_active":"2024-09-19T04:18:36.598514+0000","last_peered":"2024-09-19T04:18:36.598514+0000","last_clean":"2024-09-19T04:18:36.598514+0000","last_became_active":"2024-09-19T04:18:10.626506+0000","last_became_peered":"2024-09-19T04:18:10.626506+0000","last_unstale":"2024-09-19T04:18:36.598514+0000","last_undegraded":"2024-09-19T04:18:36.598514+0000","last_fullsized":"2024-09-19T04:18:36.598514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:48.725143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490975+0000","last_change":"2024-09-19T04:18:12.496350+0000","last_active":"2024-09-19T04:18:13.490975+0000","last_peered":"2024-09-19T04:18:13.490975+0000","last_clean":"2024-09-19T04:18:13.490975+0000","last_became_active":"2024-09-19T04:18:12.495904+0000","last_became_peered":"2024-09-19T04:18:12.495904+0000","last_unstale":"2024-09-19T04:18:13.490975+0000","last_undegraded":"2024-09-19T04:18:13.490975+0000","last_fullsized":"2024-09-19T04:18:13.490975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:08.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":343,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490536+0000","last_change":"2024-09-19T04:16:56.164450+0000","last_active":"2024-09-19T04:18:13.490536+0000","last_peered":"2024-09-19T04:18:13.490536+0000","last_clean":"2024-09-19T04:18:13.490536+0000","last_became_active":"2024-09-19T04:16:56.164008+0000","last_became_peered":"2024-09-19T04:16:56.164008+0000","last_unstale":"2024-09-19T04:18:13.490536+0000","last_undegraded":"2024-09-19T04:18:13.490536+0000","last_fullsized":"2024-09-19T04:18:13.490536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_clean_scrub_stamp":"2024-09-19T04:13:18.308718+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:14.697725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:25.578520+0000","last_change":"2024-09-19T04:18:10.101598+0000","last_active":"2024-09-19T04:18:25.578520+0000","last_peered":"2024-09-19T04:18:25.578520+0000","last_clean":"2024-09-19T04:18:25.578520+0000","last_became_active":"2024-09-19T04:18:10.101210+0000","last_became_peered":"2024-09-19T04:18:10.101210+0000","last_unstale":"2024-09-19T04:18:25.578520+0000","last_undegraded":"2024-09-19T04:18:25.578520+0000","last_fullsized":"2024-09-19T04:18:25.578520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:10.653481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.486046+0000","last_change":"2024-09-19T04:18:12.495034+0000","last_active":"2024-09-19T04:18:13.486046+0000","last_peered":"2024-09-19T04:18:13.486046+0000","last_clean":"2024-09-19T04:18:13.486046+0000","last_became_active":"2024-09-19T04:18:12.494611+0000","last_became_peered":"2024-09-19T04:18:12.494611+0000","last_unstale":"2024-09-19T04:18:13.486046+0000","last_undegraded":"2024-09-19T04:18:13.486046+0000","last_fullsized":"2024-09-19T04:18:13.486046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:28.448927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:22.555493+0000","last_change":"2024-09-19T04:18:10.105212+0000","last_active":"2024-09-19T04:18:22.555493+0000","last_peered":"2024-09-19T04:18:22.555493+0000","last_clean":"2024-09-19T04:18:22.555493+0000","last_became_active":"2024-09-19T04:18:10.104710+0000","last_became_peered":"2024-09-19T04:18:10.104710+0000","last_unstale":"2024-09-19T04:18:22.555493+0000","last_undegraded":"2024-09-19T04:18:22.555493+0000","last_fullsized":"2024-09-19T04:18:22.555493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:58.007773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478020+0000","last_change":"2024-09-19T04:18:12.492773+0000","last_active":"2024-09-19T04:18:13.478020+0000","last_peered":"2024-09-19T04:18:13.478020+0000","last_clean":"2024-09-19T04:18:13.478020+0000","last_became_active":"2024-09-19T04:18:12.492442+0000","last_became_peered":"2024-09-19T04:18:12.492442+0000","last_unstale":"2024-09-19T04:18:13.478020+0000","last_undegraded":"2024-09-19T04:18:13.478020+0000","last_fullsized":"2024-09-19T04:18:13.478020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:31:36.616232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:35.595850+0000","last_change":"2024-09-19T04:18:10.364530+0000","last_active":"2024-09-19T04:18:35.595850+0000","last_peered":"2024-09-19T04:18:35.595850+0000","last_clean":"2024-09-19T04:18:35.595850+0000","last_became_active":"2024-09-19T04:18:10.364204+0000","last_became_peered":"2024-09-19T04:18:10.364204+0000","last_unstale":"2024-09-19T04:18:35.595850+0000","last_undegraded":"2024-09-19T04:18:35.595850+0000","last_fullsized":"2024-09-19T04:18:35.595850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:04.229530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.488063+0000","last_change":"2024-09-19T04:18:12.493730+0000","last_active":"2024-09-19T04:18:13.488063+0000","last_peered":"2024-09-19T04:18:13.488063+0000","last_clean":"2024-09-19T04:18:13.488063+0000","last_became_active":"2024-09-19T04:18:12.493153+0000","last_became_peered":"2024-09-19T04:18:12.493153+0000","last_unstale":"2024-09-19T04:18:13.488063+0000","last_undegraded":"2024-09-19T04:18:13.488063+0000","last_fullsized":"2024-09-19T04:18:13.488063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.758490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:37.598625+0000","last_change":"2024-09-19T04:18:10.102073+0000","last_active":"2024-09-19T04:18:37.598625+0000","last_peered":"2024-09-19T04:18:37.598625+0000","last_clean":"2024-09-19T04:18:37.598625+0000","last_became_active":"2024-09-19T04:18:10.101755+0000","last_became_peered":"2024-09-19T04:18:10.101755+0000","last_unstale":"2024-09-19T04:18:37.598625+0000","last_undegraded":"2024-09-19T04:18:37.598625+0000","last_fullsized":"2024-09-19T04:18:37.598625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:21.489146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478571+0000","last_change":"2024-09-19T04:18:12.494560+0000","last_active":"2024-09-19T04:18:13.478571+0000","last_peered":"2024-09-19T04:18:13.478571+0000","last_clean":"2024-09-19T04:18:13.478571+0000","last_became_active":"2024-09-19T04:18:12.494203+0000","last_became_peered":"2024-09-19T04:18:12.494203+0000","last_unstale":"2024-09-19T04:18:13.478571+0000","last_undegraded":"2024-09-19T04:18:13.478571+0000","last_fullsized":"2024-09-19T04:18:13.478571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:35:17.129649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484849+0000","last_change":"2024-09-19T04:18:10.365781+0000","last_active":"2024-09-19T04:18:13.484849+0000","last_peered":"2024-09-19T04:18:13.484849+0000","last_clean":"2024-09-19T04:18:13.484849+0000","last_became_active":"2024-09-19T04:18:10.365417+0000","last_became_peered":"2024-09-19T04:18:10.365417+0000","last_unstale":"2024-09-19T04:18:13.484849+0000","last_undegraded":"2024-09-19T04:18:13.484849+0000","last_fullsized":"2024-09-19T04:18:13.484849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:20.097572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.479073+0000","last_change":"2024-09-19T04:18:12.495570+0000","last_active":"2024-09-19T04:18:13.479073+0000","last_peered":"2024-09-19T04:18:13.479073+0000","last_clean":"2024-09-19T04:18:13.479073+0000","last_became_active":"2024-09-19T04:18:12.495242+0000","last_became_peered":"2024-09-19T04:18:12.495242+0000","last_unstale":"2024-09-19T04:18:13.479073+0000","last_undegraded":"2024-09-19T04:18:13.479073+0000","last_fullsized":"2024-09-19T04:18:13.479073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:57.448182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.578940+0000","last_change":"2024-09-19T04:18:10.365134+0000","last_active":"2024-09-19T04:18:24.578940+0000","last_peered":"2024-09-19T04:18:24.578940+0000","last_clean":"2024-09-19T04:18:24.578940+0000","last_became_active":"2024-09-19T04:18:10.364726+0000","last_became_peered":"2024-09-19T04:18:10.364726+0000","last_unstale":"2024-09-19T04:18:24.578940+0000","last_undegraded":"2024-09-19T04:18:24.578940+0000","last_fullsized":"2024-09-19T04:18:24.578940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:44.207855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478464+0000","last_change":"2024-09-19T04:18:12.488836+0000","last_active":"2024-09-19T04:18:13.478464+0000","last_peered":"2024-09-19T04:18:13.478464+0000","last_clean":"2024-09-19T04:18:13.478464+0000","last_became_active":"2024-09-19T04:18:12.488507+0000","last_became_peered":"2024-09-19T04:18:12.488507+0000","last_unstale":"2024-09-19T04:18:13.478464+0000","last_undegraded":"2024-09-19T04:18:13.478464+0000","last_fullsized":"2024-09-19T04:18:13.478464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:31.308114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:34.486102+0000","last_change":"2024-09-19T04:18:10.103623+0000","last_active":"2024-09-19T04:18:34.486102+0000","last_peered":"2024-09-19T04:18:34.486102+0000","last_clean":"2024-09-19T04:18:34.486102+0000","last_became_active":"2024-09-19T04:18:10.103340+0000","last_became_peered":"2024-09-19T04:18:10.103340+0000","last_unstale":"2024-09-19T04:18:34.486102+0000","last_undegraded":"2024-09-19T04:18:34.486102+0000","last_fullsized":"2024-09-19T04:18:34.486102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:11.724013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487239+0000","last_change":"2024-09-19T04:18:12.498167+0000","last_active":"2024-09-19T04:18:13.487239+0000","last_peered":"2024-09-19T04:18:13.487239+0000","last_clean":"2024-09-19T04:18:13.487239+0000","last_became_active":"2024-09-19T04:18:12.496924+0000","last_became_peered":"2024-09-19T04:18:12.496924+0000","last_unstale":"2024-09-19T04:18:13.487239+0000","last_undegraded":"2024-09-19T04:18:13.487239+0000","last_fullsized":"2024-09-19T04:18:13.487239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:13.952030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.577159+0000","last_change":"2024-09-19T04:18:10.627106+0000","last_active":"2024-09-19T04:18:24.577159+0000","last_peered":"2024-09-19T04:18:24.577159+0000","last_clean":"2024-09-19T04:18:24.577159+0000","last_became_active":"2024-09-19T04:18:10.626708+0000","last_became_peered":"2024-09-19T04:18:10.626708+0000","last_unstale":"2024-09-19T04:18:24.577159+0000","last_undegraded":"2024-09-19T04:18:24.577159+0000","last_fullsized":"2024-09-19T04:18:24.577159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:46.049699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491685+0000","last_change":"2024-09-19T04:18:12.506995+0000","last_active":"2024-09-19T04:18:13.491685+0000","last_peered":"2024-09-19T04:18:13.491685+0000","last_clean":"2024-09-19T04:18:13.491685+0000","last_became_active":"2024-09-19T04:18:12.500607+0000","last_became_peered":"2024-09-19T04:18:12.500607+0000","last_unstale":"2024-09-19T04:18:13.491685+0000","last_undegraded":"2024-09-19T04:18:13.491685+0000","last_fullsized":"2024-09-19T04:18:13.491685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:37.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:21.542606+0000","last_change":"2024-09-19T04:18:10.365150+0000","last_active":"2024-09-19T04:18:21.542606+0000","last_peered":"2024-09-19T04:18:21.542606+0000","last_clean":"2024-09-19T04:18:21.542606+0000","last_became_active":"2024-09-19T04:18:10.364755+0000","last_became_peered":"2024-09-19T04:18:10.364755+0000","last_unstale":"2024-09-19T04:18:21.542606+0000","last_undegraded":"2024-09-19T04:18:21.542606+0000","last_fullsized":"2024-09-19T04:18:21.542606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:11.639532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484654+0000","last_change":"2024-09-19T04:18:12.484588+0000","last_active":"2024-09-19T04:18:13.484654+0000","last_peered":"2024-09-19T04:18:13.484654+0000","last_clean":"2024-09-19T04:18:13.484654+0000","last_became_active":"2024-09-19T04:18:12.484218+0000","last_became_peered":"2024-09-19T04:18:12.484218+0000","last_unstale":"2024-09-19T04:18:13.484654+0000","last_undegraded":"2024-09-19T04:18:13.484654+0000","last_fullsized":"2024-09-19T04:18:13.484654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:55.634557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.577026+0000","last_change":"2024-09-19T04:18:10.100832+0000","last_active":"2024-09-19T04:18:24.577026+0000","last_peered":"2024-09-19T04:18:24.577026+0000","last_clean":"2024-09-19T04:18:24.577026+0000","last_became_active":"2024-09-19T04:18:10.100518+0000","last_became_peered":"2024-09-19T04:18:10.100518+0000","last_unstale":"2024-09-19T04:18:24.577026+0000","last_undegraded":"2024-09-19T04:18:24.577026+0000","last_fullsized":"2024-09-19T04:18:24.577026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:37.028120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480804+0000","last_change":"2024-09-19T04:18:12.496914+0000","last_active":"2024-09-19T04:18:13.480804+0000","last_peered":"2024-09-19T04:18:13.480804+0000","last_clean":"2024-09-19T04:18:13.480804+0000","last_became_active":"2024-09-19T04:18:12.496556+0000","last_became_peered":"2024-09-19T04:18:12.496556+0000","last_unstale":"2024-09-19T04:18:13.480804+0000","last_undegraded":"2024-09-19T04:18:13.480804+0000","last_fullsized":"2024-09-19T04:18:13.480804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:52.391412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:23.566003+0000","last_change":"2024-09-19T04:18:10.101905+0000","last_active":"2024-09-19T04:18:23.566003+0000","last_peered":"2024-09-19T04:18:23.566003+0000","last_clean":"2024-09-19T04:18:23.566003+0000","last_became_active":"2024-09-19T04:18:10.101472+0000","last_became_peered":"2024-09-19T04:18:10.101472+0000","last_unstale":"2024-09-19T04:18:23.566003+0000","last_undegraded":"2024-09-19T04:18:23.566003+0000","last_fullsized":"2024-09-19T04:18:23.566003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:12:06.667079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.486294+0000","last_change":"2024-09-19T04:18:12.496852+0000","last_active":"2024-09-19T04:18:13.486294+0000","last_peered":"2024-09-19T04:18:13.486294+0000","last_clean":"2024-09-19T04:18:13.486294+0000","last_became_active":"2024-09-19T04:18:12.495365+0000","last_became_peered":"2024-09-19T04:18:12.495365+0000","last_unstale":"2024-09-19T04:18:13.486294+0000","last_undegraded":"2024-09-19T04:18:13.486294+0000","last_fullsized":"2024-09-19T04:18:13.486294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:14:57.012618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:21.542641+0000","last_change":"2024-09-19T04:18:10.104641+0000","last_active":"2024-09-19T04:18:21.542641+0000","last_peered":"2024-09-19T04:18:21.542641+0000","last_clean":"2024-09-19T04:18:21.542641+0000","last_became_active":"2024-09-19T04:18:10.104295+0000","last_became_peered":"2024-09-19T04:18:10.104295+0000","last_unstale":"2024-09-19T04:18:21.542641+0000","last_undegraded":"2024-09-19T04:18:21.542641+0000","last_fullsized":"2024-09-19T04:18:21.542641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:58.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490708+0000","last_change":"2024-09-19T04:18:12.488396+0000","last_active":"2024-09-19T04:18:13.490708+0000","last_peered":"2024-09-19T04:18:13.490708+0000","last_clean":"2024-09-19T04:18:13.490708+0000","last_became_active":"2024-09-19T04:18:12.488048+0000","last_became_peered":"2024-09-19T04:18:12.488048+0000","last_unstale":"2024-09-19T04:18:13.490708+0000","last_undegraded":"2024-09-19T04:18:13.490708+0000","last_fullsized":"2024-09-19T04:18:13.490708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:27.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.488793+0000","last_change":"2024-09-19T04:18:12.493722+0000","last_active":"2024-09-19T04:18:13.488793+0000","last_peered":"2024-09-19T04:18:13.488793+0000","last_clean":"2024-09-19T04:18:13.488793+0000","last_became_active":"2024-09-19T04:18:12.492483+0000","last_became_peered":"2024-09-19T04:18:12.492483+0000","last_unstale":"2024-09-19T04:18:13.488793+0000","last_undegraded":"2024-09-19T04:18:13.488793+0000","last_fullsized":"2024-09-19T04:18:13.488793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:33.492623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.577548+0000","last_change":"2024-09-19T04:18:10.101326+0000","last_active":"2024-09-19T04:18:24.577548+0000","last_peered":"2024-09-19T04:18:24.577548+0000","last_clean":"2024-09-19T04:18:24.577548+0000","last_became_active":"2024-09-19T04:18:10.100955+0000","last_became_peered":"2024-09-19T04:18:10.100955+0000","last_unstale":"2024-09-19T04:18:24.577548+0000","last_undegraded":"2024-09-19T04:18:24.577548+0000","last_fullsized":"2024-09-19T04:18:24.577548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:41.939741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484687+0000","last_change":"2024-09-19T04:18:12.492750+0000","last_active":"2024-09-19T04:18:13.484687+0000","last_peered":"2024-09-19T04:18:13.484687+0000","last_clean":"2024-09-19T04:18:13.484687+0000","last_became_active":"2024-09-19T04:18:12.492417+0000","last_became_peered":"2024-09-19T04:18:12.492417+0000","last_unstale":"2024-09-19T04:18:13.484687+0000","last_undegraded":"2024-09-19T04:18:13.484687+0000","last_fullsized":"2024-09-19T04:18:13.484687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:25.279452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:25.572578+0000","last_change":"2024-09-19T04:18:10.103762+0000","last_active":"2024-09-19T04:18:25.572578+0000","last_peered":"2024-09-19T04:18:25.572578+0000","last_clean":"2024-09-19T04:18:25.572578+0000","last_became_active":"2024-09-19T04:18:10.103441+0000","last_became_peered":"2024-09-19T04:18:10.103441+0000","last_unstale":"2024-09-19T04:18:25.572578+0000","last_undegraded":"2024-09-19T04:18:25.572578+0000","last_fullsized":"2024-09-19T04:18:25.572578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:03.451196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491910+0000","last_change":"2024-09-19T04:18:12.503510+0000","last_active":"2024-09-19T04:18:13.491910+0000","last_peered":"2024-09-19T04:18:13.491910+0000","last_clean":"2024-09-19T04:18:13.491910+0000","last_became_active":"2024-09-19T04:18:12.495165+0000","last_became_peered":"2024-09-19T04:18:12.495165+0000","last_unstale":"2024-09-19T04:18:13.491910+0000","last_undegraded":"2024-09-19T04:18:13.491910+0000","last_fullsized":"2024-09-19T04:18:13.491910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:13.934328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.492423+0000","last_change":"2024-09-19T04:18:10.364783+0000","last_active":"2024-09-19T04:18:13.492423+0000","last_peered":"2024-09-19T04:18:13.492423+0000","last_clean":"2024-09-19T04:18:13.492423+0000","last_became_active":"2024-09-19T04:18:10.364202+0000","last_became_peered":"2024-09-19T04:18:10.364202+0000","last_unstale":"2024-09-19T04:18:13.492423+0000","last_undegraded":"2024-09-19T04:18:13.492423+0000","last_fullsized":"2024-09-19T04:18:13.492423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:18:55.770637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.486574+0000","last_change":"2024-09-19T04:18:12.495825+0000","last_active":"2024-09-19T04:18:13.486574+0000","last_peered":"2024-09-19T04:18:13.486574+0000","last_clean":"2024-09-19T04:18:13.486574+0000","last_became_active":"2024-09-19T04:18:12.494525+0000","last_became_peered":"2024-09-19T04:18:12.494525+0000","last_unstale":"2024-09-19T04:18:13.486574+0000","last_undegraded":"2024-09-19T04:18:13.486574+0000","last_fullsized":"2024-09-19T04:18:13.486574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:57.827135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:35.595797+0000","last_change":"2024-09-19T04:18:10.102145+0000","last_active":"2024-09-19T04:18:35.595797+0000","last_peered":"2024-09-19T04:18:35.595797+0000","last_clean":"2024-09-19T04:18:35.595797+0000","last_became_active":"2024-09-19T04:18:10.101410+0000","last_became_peered":"2024-09-19T04:18:10.101410+0000","last_unstale":"2024-09-19T04:18:35.595797+0000","last_undegraded":"2024-09-19T04:18:35.595797+0000","last_fullsized":"2024-09-19T04:18:35.595797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:20.354296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57320,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":169,"num_read_kb":146,"num_write":234,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":171960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281762,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":424211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":1064959,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972381,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":423748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201314,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":424126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1016646,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593772,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":423465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822707,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":1016324,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":423326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215163,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":423290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":475496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":474490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":425197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":51864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":51848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:18:42.250 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:18:42.619 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:43.069 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:42 smithi107 ceph-mon[30659]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:18:43.069 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:42 smithi107 ceph-mon[30659]: pgmap v302: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-19T04:18:43.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.176+0000 7fbad3200640 1 -- 172.21.15.107:0/502316022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbad41047e0 msgr2=0x7fbad4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:43.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.176+0000 7fbad3200640 1 --2- 172.21.15.107:0/502316022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbad41047e0 0x7fbad4108830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009900 tx=0x7fbabc02a410 comp rx=0 tx=0).stop 2024-09-19T04:18:43.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.176+0000 7fbad3200640 1 -- 172.21.15.107:0/502316022 shutdown_connections 2024-09-19T04:18:43.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.176+0000 7fbad3200640 1 --2- 172.21.15.107:0/502316022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad4108f60 0x7fbad4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:43.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.176+0000 7fbad3200640 1 --2- 172.21.15.107:0/502316022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbad41047e0 0x7fbad4108830 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:43.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.176+0000 7fbad3200640 1 --2- 172.21.15.107:0/502316022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbad4103e30 0x7fbad4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:43.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.176+0000 7fbad3200640 1 -- 172.21.15.107:0/502316022 >> 172.21.15.107:0/502316022 conn(0x7fbad40ff8d0 msgr2=0x7fbad4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:43.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.176+0000 7fbad3200640 1 -- 172.21.15.107:0/502316022 shutdown_connections 2024-09-19T04:18:43.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.176+0000 7fbad3200640 1 -- 172.21.15.107:0/502316022 wait complete. 2024-09-19T04:18:43.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.177+0000 7fbad3200640 1 Processor -- start 2024-09-19T04:18:43.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.177+0000 7fbad3200640 1 -- start start 2024-09-19T04:18:43.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.177+0000 7fbad3200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad4103e30 0x7fbad419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:43.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.177+0000 7fbad3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbad41047e0 0x7fbad419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:43.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.177+0000 7fbad3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbad4108f60 0x7fbad419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.177+0000 7fbad3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4114010 con 0x7fbad41047e0 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.177+0000 7fbad3200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4113e90 con 0x7fbad4103e30 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.177+0000 7fbad3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4114190 con 0x7fbad4108f60 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.177+0000 7fbacbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbad41047e0 0x7fbad419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbacbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbad41047e0 0x7fbad419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45994/0 (socket says 172.21.15.107:45994) 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbacbe00640 1 -- 172.21.15.107:0/4058892486 learned_addr learned my addr 172.21.15.107:0/4058892486 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbad1e00640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad4103e30 0x7fbad419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbad2800640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbad4108f60 0x7fbad419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbacbe00640 1 -- 172.21.15.107:0/4058892486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbad4108f60 msgr2=0x7fbad419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbacbe00640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbad4108f60 0x7fbad419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbacbe00640 1 -- 172.21.15.107:0/4058892486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad4103e30 msgr2=0x7fbad419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbacbe00640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad4103e30 0x7fbad419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:43.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbacbe00640 1 -- 172.21.15.107:0/4058892486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad419f440 con 0x7fbad41047e0 2024-09-19T04:18:43.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbad1e00640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad4103e30 0x7fbad419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:43.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.178+0000 7fbacbe00640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbad41047e0 0x7fbad419a9d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbabc004770 tx=0x7fbabc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:43.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.179+0000 7fbacaa00640 1 -- 172.21.15.107:0/4058892486 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc03f070 con 0x7fbad41047e0 2024-09-19T04:18:43.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.179+0000 7fbacaa00640 1 -- 172.21.15.107:0/4058892486 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbabc031ca0 con 0x7fbad41047e0 2024-09-19T04:18:43.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.179+0000 7fbacaa00640 1 -- 172.21.15.107:0/4058892486 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc035030 con 0x7fbad41047e0 2024-09-19T04:18:43.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.179+0000 7fbad3200640 1 -- 172.21.15.107:0/4058892486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad419f6d0 con 0x7fbad41047e0 2024-09-19T04:18:43.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.179+0000 7fbad3200640 1 -- 172.21.15.107:0/4058892486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad419fb60 con 0x7fbad41047e0 2024-09-19T04:18:43.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.180+0000 7fbad3200640 1 -- 172.21.15.107:0/4058892486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad4104210 con 0x7fbad41047e0 2024-09-19T04:18:43.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.182+0000 7fbacaa00640 1 -- 172.21.15.107:0/4058892486 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbabc042070 con 0x7fbad41047e0 2024-09-19T04:18:43.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.183+0000 7fbacaa00640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbab80a3350 0x7fbab80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:43.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.183+0000 7fbacaa00640 1 -- 172.21.15.107:0/4058892486 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fbabc0eff00 con 0x7fbad41047e0 2024-09-19T04:18:43.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.183+0000 7fbad1e00640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbab80a3350 0x7fbab80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:43.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.184+0000 7fbad1e00640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbab80a3350 0x7fbab80a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009fc0 tx=0x7fbac4005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:43.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.187+0000 7fbacaa00640 1 -- 172.21.15.107:0/4058892486 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbabc0bc5e0 con 0x7fbad41047e0 2024-09-19T04:18:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:42 smithi115 ceph-mon[35477]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:18:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:42 smithi115 ceph-mon[35477]: pgmap v302: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-19T04:18:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:42 smithi145 ceph-mon[35096]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:18:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:42 smithi145 ceph-mon[35096]: pgmap v302: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-19T04:18:43.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.358+0000 7fbad3200640 1 -- 172.21.15.107:0/4058892486 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbad4062630 con 0x7fbab80a3350 2024-09-19T04:18:43.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.365+0000 7fbacaa00640 1 -- 172.21.15.107:0/4058892486 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258910 (secure 0 0 0) 0x7fbad4062630 con 0x7fbab80a3350 2024-09-19T04:18:43.369 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:43.370 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-19T04:18:43.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 -- 172.21.15.107:0/4058892486 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbab80a3350 msgr2=0x7fbab80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:43.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbab80a3350 0x7fbab80a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009fc0 tx=0x7fbac4005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:43.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 -- 172.21.15.107:0/4058892486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbad41047e0 msgr2=0x7fbad419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:43.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbad41047e0 0x7fbad419a9d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbabc004770 tx=0x7fbabc003c10 comp rx=0 tx=0).stop 2024-09-19T04:18:43.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 -- 172.21.15.107:0/4058892486 shutdown_connections 2024-09-19T04:18:43.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbab80a3350 0x7fbab80a5810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:43.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbad4108f60 0x7fbad419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:43.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbad41047e0 0x7fbad419a9d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:43.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 --2- 172.21.15.107:0/4058892486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad4103e30 0x7fbad419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:43.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 -- 172.21.15.107:0/4058892486 >> 172.21.15.107:0/4058892486 conn(0x7fbad40ff8d0 msgr2=0x7fbad4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:43.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 -- 172.21.15.107:0/4058892486 shutdown_connections 2024-09-19T04:18:43.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:43.369+0000 7fbad3200640 1 -- 172.21.15.107:0/4058892486 wait complete. 2024-09-19T04:18:44.221 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-19T04:18:41.814226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647922,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":315,"num_read_kb":275,"num_write":425,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372648,"kb_used_data":10920,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569176,"statfs":{"total":1151940427776,"available":1151558836224,"internally_reserved":0,"allocated":11182080,"data_stored":7015278,"data_compressed":107517,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42132,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":102,"num_write":34,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003074"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480938+0000","last_change":"2024-09-19T04:18:12.499720+0000","last_active":"2024-09-19T04:18:13.480938+0000","last_peered":"2024-09-19T04:18:13.480938+0000","last_clean":"2024-09-19T04:18:13.480938+0000","last_became_active":"2024-09-19T04:18:12.499381+0000","last_became_peered":"2024-09-19T04:18:12.499381+0000","last_unstale":"2024-09-19T04:18:13.480938+0000","last_undegraded":"2024-09-19T04:18:13.480938+0000","last_fullsized":"2024-09-19T04:18:13.480938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:46.708870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491536+0000","last_change":"2024-09-19T04:18:12.498163+0000","last_active":"2024-09-19T04:18:13.491536+0000","last_peered":"2024-09-19T04:18:13.491536+0000","last_clean":"2024-09-19T04:18:13.491536+0000","last_became_active":"2024-09-19T04:18:12.495706+0000","last_became_peered":"2024-09-19T04:18:12.495706+0000","last_unstale":"2024-09-19T04:18:13.491536+0000","last_undegraded":"2024-09-19T04:18:13.491536+0000","last_fullsized":"2024-09-19T04:18:13.491536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:50.604410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491537+0000","last_change":"2024-09-19T04:18:12.493903+0000","last_active":"2024-09-19T04:18:13.491537+0000","last_peered":"2024-09-19T04:18:13.491537+0000","last_clean":"2024-09-19T04:18:13.491537+0000","last_became_active":"2024-09-19T04:18:12.493443+0000","last_became_peered":"2024-09-19T04:18:12.493443+0000","last_unstale":"2024-09-19T04:18:13.491537+0000","last_undegraded":"2024-09-19T04:18:13.491537+0000","last_fullsized":"2024-09-19T04:18:13.491537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:51.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487370+0000","last_change":"2024-09-19T04:18:12.500460+0000","last_active":"2024-09-19T04:18:13.487370+0000","last_peered":"2024-09-19T04:18:13.487370+0000","last_clean":"2024-09-19T04:18:13.487370+0000","last_became_active":"2024-09-19T04:18:12.494675+0000","last_became_peered":"2024-09-19T04:18:12.494675+0000","last_unstale":"2024-09-19T04:18:13.487370+0000","last_undegraded":"2024-09-19T04:18:13.487370+0000","last_fullsized":"2024-09-19T04:18:13.487370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:13:32.863156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484850+0000","last_change":"2024-09-19T04:18:12.494936+0000","last_active":"2024-09-19T04:18:13.484850+0000","last_peered":"2024-09-19T04:18:13.484850+0000","last_clean":"2024-09-19T04:18:13.484850+0000","last_became_active":"2024-09-19T04:18:12.494556+0000","last_became_peered":"2024-09-19T04:18:12.494556+0000","last_unstale":"2024-09-19T04:18:13.484850+0000","last_undegraded":"2024-09-19T04:18:13.484850+0000","last_fullsized":"2024-09-19T04:18:13.484850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:06.001529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478573+0000","last_change":"2024-09-19T04:18:12.495880+0000","last_active":"2024-09-19T04:18:13.478573+0000","last_peered":"2024-09-19T04:18:13.478573+0000","last_clean":"2024-09-19T04:18:13.478573+0000","last_became_active":"2024-09-19T04:18:12.495562+0000","last_became_peered":"2024-09-19T04:18:12.495562+0000","last_unstale":"2024-09-19T04:18:13.478573+0000","last_undegraded":"2024-09-19T04:18:13.478573+0000","last_fullsized":"2024-09-19T04:18:13.478573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:20.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490803+0000","last_change":"2024-09-19T04:18:12.497991+0000","last_active":"2024-09-19T04:18:13.490803+0000","last_peered":"2024-09-19T04:18:13.490803+0000","last_clean":"2024-09-19T04:18:13.490803+0000","last_became_active":"2024-09-19T04:18:12.497644+0000","last_became_peered":"2024-09-19T04:18:12.497644+0000","last_unstale":"2024-09-19T04:18:13.490803+0000","last_undegraded":"2024-09-19T04:18:13.490803+0000","last_fullsized":"2024-09-19T04:18:13.490803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.851930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484998+0000","last_change":"2024-09-19T04:18:12.495789+0000","last_active":"2024-09-19T04:18:13.484998+0000","last_peered":"2024-09-19T04:18:13.484998+0000","last_clean":"2024-09-19T04:18:13.484998+0000","last_became_active":"2024-09-19T04:18:12.494637+0000","last_became_peered":"2024-09-19T04:18:12.494637+0000","last_unstale":"2024-09-19T04:18:13.484998+0000","last_undegraded":"2024-09-19T04:18:13.484998+0000","last_fullsized":"2024-09-19T04:18:13.484998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:14.245180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485190+0000","last_change":"2024-09-19T04:18:12.491433+0000","last_active":"2024-09-19T04:18:13.485190+0000","last_peered":"2024-09-19T04:18:13.485190+0000","last_clean":"2024-09-19T04:18:13.485190+0000","last_became_active":"2024-09-19T04:18:12.490444+0000","last_became_peered":"2024-09-19T04:18:12.490444+0000","last_unstale":"2024-09-19T04:18:13.485190+0000","last_undegraded":"2024-09-19T04:18:13.485190+0000","last_fullsized":"2024-09-19T04:18:13.485190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:43.068355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.479183+0000","last_change":"2024-09-19T04:18:12.493188+0000","last_active":"2024-09-19T04:18:13.479183+0000","last_peered":"2024-09-19T04:18:13.479183+0000","last_clean":"2024-09-19T04:18:13.479183+0000","last_became_active":"2024-09-19T04:18:12.492788+0000","last_became_peered":"2024-09-19T04:18:12.492788+0000","last_unstale":"2024-09-19T04:18:13.479183+0000","last_undegraded":"2024-09-19T04:18:13.479183+0000","last_fullsized":"2024-09-19T04:18:13.479183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:17.912820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490852+0000","last_change":"2024-09-19T04:18:12.490616+0000","last_active":"2024-09-19T04:18:13.490852+0000","last_peered":"2024-09-19T04:18:13.490852+0000","last_clean":"2024-09-19T04:18:13.490852+0000","last_became_active":"2024-09-19T04:18:12.490155+0000","last_became_peered":"2024-09-19T04:18:12.490155+0000","last_unstale":"2024-09-19T04:18:13.490852+0000","last_undegraded":"2024-09-19T04:18:13.490852+0000","last_fullsized":"2024-09-19T04:18:13.490852+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:47.082082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.486611+0000","last_change":"2024-09-19T04:18:12.494191+0000","last_active":"2024-09-19T04:18:13.486611+0000","last_peered":"2024-09-19T04:18:13.486611+0000","last_clean":"2024-09-19T04:18:13.486611+0000","last_became_active":"2024-09-19T04:18:12.493786+0000","last_became_peered":"2024-09-19T04:18:12.493786+0000","last_unstale":"2024-09-19T04:18:13.486611+0000","last_undegraded":"2024-09-19T04:18:13.486611+0000","last_fullsized":"2024-09-19T04:18:13.486611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:29.691556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478270+0000","last_change":"2024-09-19T04:18:12.482167+0000","last_active":"2024-09-19T04:18:13.478270+0000","last_peered":"2024-09-19T04:18:13.478270+0000","last_clean":"2024-09-19T04:18:13.478270+0000","last_became_active":"2024-09-19T04:18:12.481314+0000","last_became_peered":"2024-09-19T04:18:12.481314+0000","last_unstale":"2024-09-19T04:18:13.478270+0000","last_undegraded":"2024-09-19T04:18:13.478270+0000","last_fullsized":"2024-09-19T04:18:13.478270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:28.081641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491377+0000","last_change":"2024-09-19T04:18:12.491711+0000","last_active":"2024-09-19T04:18:13.491377+0000","last_peered":"2024-09-19T04:18:13.491377+0000","last_clean":"2024-09-19T04:18:13.491377+0000","last_became_active":"2024-09-19T04:18:12.490964+0000","last_became_peered":"2024-09-19T04:18:12.490964+0000","last_unstale":"2024-09-19T04:18:13.491377+0000","last_undegraded":"2024-09-19T04:18:13.491377+0000","last_fullsized":"2024-09-19T04:18:13.491377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:16.607679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491940+0000","last_change":"2024-09-19T04:18:12.504737+0000","last_active":"2024-09-19T04:18:13.491940+0000","last_peered":"2024-09-19T04:18:13.491940+0000","last_clean":"2024-09-19T04:18:13.491940+0000","last_became_active":"2024-09-19T04:18:12.497583+0000","last_became_peered":"2024-09-19T04:18:12.497583+0000","last_unstale":"2024-09-19T04:18:13.491940+0000","last_undegraded":"2024-09-19T04:18:13.491940+0000","last_fullsized":"2024-09-19T04:18:13.491940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:48.456738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484709+0000","last_change":"2024-09-19T04:18:12.499045+0000","last_active":"2024-09-19T04:18:13.484709+0000","last_peered":"2024-09-19T04:18:13.484709+0000","last_clean":"2024-09-19T04:18:13.484709+0000","last_became_active":"2024-09-19T04:18:12.498713+0000","last_became_peered":"2024-09-19T04:18:12.498713+0000","last_unstale":"2024-09-19T04:18:13.484709+0000","last_undegraded":"2024-09-19T04:18:13.484709+0000","last_fullsized":"2024-09-19T04:18:13.484709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:54.337769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480975+0000","last_change":"2024-09-19T04:18:12.495989+0000","last_active":"2024-09-19T04:18:13.480975+0000","last_peered":"2024-09-19T04:18:13.480975+0000","last_clean":"2024-09-19T04:18:13.480975+0000","last_became_active":"2024-09-19T04:18:12.495633+0000","last_became_peered":"2024-09-19T04:18:12.495633+0000","last_unstale":"2024-09-19T04:18:13.480975+0000","last_undegraded":"2024-09-19T04:18:13.480975+0000","last_fullsized":"2024-09-19T04:18:13.480975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:17.225861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485052+0000","last_change":"2024-09-19T04:18:12.488532+0000","last_active":"2024-09-19T04:18:13.485052+0000","last_peered":"2024-09-19T04:18:13.485052+0000","last_clean":"2024-09-19T04:18:13.485052+0000","last_became_active":"2024-09-19T04:18:12.488117+0000","last_became_peered":"2024-09-19T04:18:12.488117+0000","last_unstale":"2024-09-19T04:18:13.485052+0000","last_undegraded":"2024-09-19T04:18:13.485052+0000","last_fullsized":"2024-09-19T04:18:13.485052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:45.318852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487085+0000","last_change":"2024-09-19T04:18:12.496784+0000","last_active":"2024-09-19T04:18:13.487085+0000","last_peered":"2024-09-19T04:18:13.487085+0000","last_clean":"2024-09-19T04:18:13.487085+0000","last_became_active":"2024-09-19T04:18:12.496443+0000","last_became_peered":"2024-09-19T04:18:12.496443+0000","last_unstale":"2024-09-19T04:18:13.487085+0000","last_undegraded":"2024-09-19T04:18:13.487085+0000","last_fullsized":"2024-09-19T04:18:13.487085+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:25:46.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480595+0000","last_change":"2024-09-19T04:18:12.496196+0000","last_active":"2024-09-19T04:18:13.480595+0000","last_peered":"2024-09-19T04:18:13.480595+0000","last_clean":"2024-09-19T04:18:13.480595+0000","last_became_active":"2024-09-19T04:18:12.495517+0000","last_became_peered":"2024-09-19T04:18:12.495517+0000","last_unstale":"2024-09-19T04:18:13.480595+0000","last_undegraded":"2024-09-19T04:18:13.480595+0000","last_fullsized":"2024-09-19T04:18:13.480595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:32.281885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478408+0000","last_change":"2024-09-19T04:18:12.493731+0000","last_active":"2024-09-19T04:18:13.478408+0000","last_peered":"2024-09-19T04:18:13.478408+0000","last_clean":"2024-09-19T04:18:13.478408+0000","last_became_active":"2024-09-19T04:18:12.493412+0000","last_became_peered":"2024-09-19T04:18:12.493412+0000","last_unstale":"2024-09-19T04:18:13.478408+0000","last_undegraded":"2024-09-19T04:18:13.478408+0000","last_fullsized":"2024-09-19T04:18:13.478408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:41.089637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485124+0000","last_change":"2024-09-19T04:18:12.497033+0000","last_active":"2024-09-19T04:18:13.485124+0000","last_peered":"2024-09-19T04:18:13.485124+0000","last_clean":"2024-09-19T04:18:13.485124+0000","last_became_active":"2024-09-19T04:18:12.496710+0000","last_became_peered":"2024-09-19T04:18:12.496710+0000","last_unstale":"2024-09-19T04:18:13.485124+0000","last_undegraded":"2024-09-19T04:18:13.485124+0000","last_fullsized":"2024-09-19T04:18:13.485124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:19.193869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485428+0000","last_change":"2024-09-19T04:18:12.499213+0000","last_active":"2024-09-19T04:18:13.485428+0000","last_peered":"2024-09-19T04:18:13.485428+0000","last_clean":"2024-09-19T04:18:13.485428+0000","last_became_active":"2024-09-19T04:18:12.498728+0000","last_became_peered":"2024-09-19T04:18:12.498728+0000","last_unstale":"2024-09-19T04:18:13.485428+0000","last_undegraded":"2024-09-19T04:18:13.485428+0000","last_fullsized":"2024-09-19T04:18:13.485428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:37.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487908+0000","last_change":"2024-09-19T04:18:12.498603+0000","last_active":"2024-09-19T04:18:13.487908+0000","last_peered":"2024-09-19T04:18:13.487908+0000","last_clean":"2024-09-19T04:18:13.487908+0000","last_became_active":"2024-09-19T04:18:12.497958+0000","last_became_peered":"2024-09-19T04:18:12.497958+0000","last_unstale":"2024-09-19T04:18:13.487908+0000","last_undegraded":"2024-09-19T04:18:13.487908+0000","last_fullsized":"2024-09-19T04:18:13.487908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:57.699594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487095+0000","last_change":"2024-09-19T04:18:12.506679+0000","last_active":"2024-09-19T04:18:13.487095+0000","last_peered":"2024-09-19T04:18:13.487095+0000","last_clean":"2024-09-19T04:18:13.487095+0000","last_became_active":"2024-09-19T04:18:12.505953+0000","last_became_peered":"2024-09-19T04:18:12.505953+0000","last_unstale":"2024-09-19T04:18:13.487095+0000","last_undegraded":"2024-09-19T04:18:13.487095+0000","last_fullsized":"2024-09-19T04:18:13.487095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:53.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.489669+0000","last_change":"2024-09-19T04:18:12.490511+0000","last_active":"2024-09-19T04:18:13.489669+0000","last_peered":"2024-09-19T04:18:13.489669+0000","last_clean":"2024-09-19T04:18:13.489669+0000","last_became_active":"2024-09-19T04:18:12.490051+0000","last_became_peered":"2024-09-19T04:18:12.490051+0000","last_unstale":"2024-09-19T04:18:13.489669+0000","last_undegraded":"2024-09-19T04:18:13.489669+0000","last_fullsized":"2024-09-19T04:18:13.489669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:32.283312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480846+0000","last_change":"2024-09-19T04:18:12.491465+0000","last_active":"2024-09-19T04:18:13.480846+0000","last_peered":"2024-09-19T04:18:13.480846+0000","last_clean":"2024-09-19T04:18:13.480846+0000","last_became_active":"2024-09-19T04:18:12.491136+0000","last_became_peered":"2024-09-19T04:18:12.491136+0000","last_unstale":"2024-09-19T04:18:13.480846+0000","last_undegraded":"2024-09-19T04:18:13.480846+0000","last_fullsized":"2024-09-19T04:18:13.480846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:29.498909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478381+0000","last_change":"2024-09-19T04:18:12.494879+0000","last_active":"2024-09-19T04:18:13.478381+0000","last_peered":"2024-09-19T04:18:13.478381+0000","last_clean":"2024-09-19T04:18:13.478381+0000","last_became_active":"2024-09-19T04:18:12.494534+0000","last_became_peered":"2024-09-19T04:18:12.494534+0000","last_unstale":"2024-09-19T04:18:13.478381+0000","last_undegraded":"2024-09-19T04:18:13.478381+0000","last_fullsized":"2024-09-19T04:18:13.478381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.647417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491875+0000","last_change":"2024-09-19T04:18:12.504364+0000","last_active":"2024-09-19T04:18:13.491875+0000","last_peered":"2024-09-19T04:18:13.491875+0000","last_clean":"2024-09-19T04:18:13.491875+0000","last_became_active":"2024-09-19T04:18:12.504006+0000","last_became_peered":"2024-09-19T04:18:12.504006+0000","last_unstale":"2024-09-19T04:18:13.491875+0000","last_undegraded":"2024-09-19T04:18:13.491875+0000","last_fullsized":"2024-09-19T04:18:13.491875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:53.309963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485337+0000","last_change":"2024-09-19T04:18:12.498223+0000","last_active":"2024-09-19T04:18:13.485337+0000","last_peered":"2024-09-19T04:18:13.485337+0000","last_clean":"2024-09-19T04:18:13.485337+0000","last_became_active":"2024-09-19T04:18:12.497862+0000","last_became_peered":"2024-09-19T04:18:12.497862+0000","last_unstale":"2024-09-19T04:18:13.485337+0000","last_undegraded":"2024-09-19T04:18:13.485337+0000","last_fullsized":"2024-09-19T04:18:13.485337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:28.858620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485415+0000","last_change":"2024-09-19T04:18:12.502335+0000","last_active":"2024-09-19T04:18:13.485415+0000","last_peered":"2024-09-19T04:18:13.485415+0000","last_clean":"2024-09-19T04:18:13.485415+0000","last_became_active":"2024-09-19T04:18:12.501686+0000","last_became_peered":"2024-09-19T04:18:12.501686+0000","last_unstale":"2024-09-19T04:18:13.485415+0000","last_undegraded":"2024-09-19T04:18:13.485415+0000","last_fullsized":"2024-09-19T04:18:13.485415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:22.348745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480527+0000","last_change":"2024-09-19T04:18:12.490905+0000","last_active":"2024-09-19T04:18:13.480527+0000","last_peered":"2024-09-19T04:18:13.480527+0000","last_clean":"2024-09-19T04:18:13.480527+0000","last_became_active":"2024-09-19T04:18:12.490489+0000","last_became_peered":"2024-09-19T04:18:12.490489+0000","last_unstale":"2024-09-19T04:18:13.480527+0000","last_undegraded":"2024-09-19T04:18:13.480527+0000","last_fullsized":"2024-09-19T04:18:13.480527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.946474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485197+0000","last_change":"2024-09-19T04:18:12.494607+0000","last_active":"2024-09-19T04:18:13.485197+0000","last_peered":"2024-09-19T04:18:13.485197+0000","last_clean":"2024-09-19T04:18:13.485197+0000","last_became_active":"2024-09-19T04:18:12.494288+0000","last_became_peered":"2024-09-19T04:18:12.494288+0000","last_unstale":"2024-09-19T04:18:13.485197+0000","last_undegraded":"2024-09-19T04:18:13.485197+0000","last_fullsized":"2024-09-19T04:18:13.485197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:42.824951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:37.600084+0000","last_change":"2024-09-19T04:18:10.103108+0000","last_active":"2024-09-19T04:18:37.600084+0000","last_peered":"2024-09-19T04:18:37.600084+0000","last_clean":"2024-09-19T04:18:37.600084+0000","last_became_active":"2024-09-19T04:18:10.102783+0000","last_became_peered":"2024-09-19T04:18:10.102783+0000","last_unstale":"2024-09-19T04:18:37.600084+0000","last_undegraded":"2024-09-19T04:18:37.600084+0000","last_fullsized":"2024-09-19T04:18:37.600084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:25.521953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.488677+0000","last_change":"2024-09-19T04:18:12.495453+0000","last_active":"2024-09-19T04:18:13.488677+0000","last_peered":"2024-09-19T04:18:13.488677+0000","last_clean":"2024-09-19T04:18:13.488677+0000","last_became_active":"2024-09-19T04:18:12.494744+0000","last_became_peered":"2024-09-19T04:18:12.494744+0000","last_unstale":"2024-09-19T04:18:13.488677+0000","last_undegraded":"2024-09-19T04:18:13.488677+0000","last_fullsized":"2024-09-19T04:18:13.488677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:41.071723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:23.567040+0000","last_change":"2024-09-19T04:18:10.102675+0000","last_active":"2024-09-19T04:18:23.567040+0000","last_peered":"2024-09-19T04:18:23.567040+0000","last_clean":"2024-09-19T04:18:23.567040+0000","last_became_active":"2024-09-19T04:18:10.102368+0000","last_became_peered":"2024-09-19T04:18:10.102368+0000","last_unstale":"2024-09-19T04:18:23.567040+0000","last_undegraded":"2024-09-19T04:18:23.567040+0000","last_fullsized":"2024-09-19T04:18:23.567040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:45.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:22.558114+0000","last_change":"2024-09-19T04:18:10.099932+0000","last_active":"2024-09-19T04:18:22.558114+0000","last_peered":"2024-09-19T04:18:22.558114+0000","last_clean":"2024-09-19T04:18:22.558114+0000","last_became_active":"2024-09-19T04:18:10.099637+0000","last_became_peered":"2024-09-19T04:18:10.099637+0000","last_unstale":"2024-09-19T04:18:22.558114+0000","last_undegraded":"2024-09-19T04:18:22.558114+0000","last_fullsized":"2024-09-19T04:18:22.558114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:50.594382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485265+0000","last_change":"2024-09-19T04:18:12.502229+0000","last_active":"2024-09-19T04:18:13.485265+0000","last_peered":"2024-09-19T04:18:13.485265+0000","last_clean":"2024-09-19T04:18:13.485265+0000","last_became_active":"2024-09-19T04:18:12.501921+0000","last_became_peered":"2024-09-19T04:18:12.501921+0000","last_unstale":"2024-09-19T04:18:13.485265+0000","last_undegraded":"2024-09-19T04:18:13.485265+0000","last_fullsized":"2024-09-19T04:18:13.485265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:55.412103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.576553+0000","last_change":"2024-09-19T04:18:10.103961+0000","last_active":"2024-09-19T04:18:24.576553+0000","last_peered":"2024-09-19T04:18:24.576553+0000","last_clean":"2024-09-19T04:18:24.576553+0000","last_became_active":"2024-09-19T04:18:10.102737+0000","last_became_peered":"2024-09-19T04:18:10.102737+0000","last_unstale":"2024-09-19T04:18:24.576553+0000","last_undegraded":"2024-09-19T04:18:24.576553+0000","last_fullsized":"2024-09-19T04:18:24.576553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:31:01.806124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484985+0000","last_change":"2024-09-19T04:18:12.502385+0000","last_active":"2024-09-19T04:18:13.484985+0000","last_peered":"2024-09-19T04:18:13.484985+0000","last_clean":"2024-09-19T04:18:13.484985+0000","last_became_active":"2024-09-19T04:18:12.502025+0000","last_became_peered":"2024-09-19T04:18:12.502025+0000","last_unstale":"2024-09-19T04:18:13.484985+0000","last_undegraded":"2024-09-19T04:18:13.484985+0000","last_fullsized":"2024-09-19T04:18:13.484985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:24.650911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:37.599189+0000","last_change":"2024-09-19T04:18:10.103526+0000","last_active":"2024-09-19T04:18:37.599189+0000","last_peered":"2024-09-19T04:18:37.599189+0000","last_clean":"2024-09-19T04:18:37.599189+0000","last_became_active":"2024-09-19T04:18:10.103191+0000","last_became_peered":"2024-09-19T04:18:10.103191+0000","last_unstale":"2024-09-19T04:18:37.599189+0000","last_undegraded":"2024-09-19T04:18:37.599189+0000","last_fullsized":"2024-09-19T04:18:37.599189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:07.695715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485430+0000","last_change":"2024-09-19T04:18:12.493169+0000","last_active":"2024-09-19T04:18:13.485430+0000","last_peered":"2024-09-19T04:18:13.485430+0000","last_clean":"2024-09-19T04:18:13.485430+0000","last_became_active":"2024-09-19T04:18:12.492669+0000","last_became_peered":"2024-09-19T04:18:12.492669+0000","last_unstale":"2024-09-19T04:18:13.485430+0000","last_undegraded":"2024-09-19T04:18:13.485430+0000","last_fullsized":"2024-09-19T04:18:13.485430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:04.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.574947+0000","last_change":"2024-09-19T04:18:10.102835+0000","last_active":"2024-09-19T04:18:24.574947+0000","last_peered":"2024-09-19T04:18:24.574947+0000","last_clean":"2024-09-19T04:18:24.574947+0000","last_became_active":"2024-09-19T04:18:10.102494+0000","last_became_peered":"2024-09-19T04:18:10.102494+0000","last_unstale":"2024-09-19T04:18:24.574947+0000","last_undegraded":"2024-09-19T04:18:24.574947+0000","last_fullsized":"2024-09-19T04:18:24.574947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:03:17.218245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.492030+0000","last_change":"2024-09-19T04:18:12.504348+0000","last_active":"2024-09-19T04:18:13.492030+0000","last_peered":"2024-09-19T04:18:13.492030+0000","last_clean":"2024-09-19T04:18:13.492030+0000","last_became_active":"2024-09-19T04:18:12.503180+0000","last_became_peered":"2024-09-19T04:18:12.503180+0000","last_unstale":"2024-09-19T04:18:13.492030+0000","last_undegraded":"2024-09-19T04:18:13.492030+0000","last_fullsized":"2024-09-19T04:18:13.492030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:03.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:23.566490+0000","last_change":"2024-09-19T04:18:10.365136+0000","last_active":"2024-09-19T04:18:23.566490+0000","last_peered":"2024-09-19T04:18:23.566490+0000","last_clean":"2024-09-19T04:18:23.566490+0000","last_became_active":"2024-09-19T04:18:10.364594+0000","last_became_peered":"2024-09-19T04:18:10.364594+0000","last_unstale":"2024-09-19T04:18:23.566490+0000","last_undegraded":"2024-09-19T04:18:23.566490+0000","last_fullsized":"2024-09-19T04:18:23.566490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:52.252157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484882+0000","last_change":"2024-09-19T04:18:12.502051+0000","last_active":"2024-09-19T04:18:13.484882+0000","last_peered":"2024-09-19T04:18:13.484882+0000","last_clean":"2024-09-19T04:18:13.484882+0000","last_became_active":"2024-09-19T04:18:12.501578+0000","last_became_peered":"2024-09-19T04:18:12.501578+0000","last_unstale":"2024-09-19T04:18:13.484882+0000","last_undegraded":"2024-09-19T04:18:13.484882+0000","last_fullsized":"2024-09-19T04:18:13.484882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:59.693175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'27","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:37.599800+0000","last_change":"2024-09-19T04:18:10.101507+0000","last_active":"2024-09-19T04:18:37.599800+0000","last_peered":"2024-09-19T04:18:37.599800+0000","last_clean":"2024-09-19T04:18:37.599800+0000","last_became_active":"2024-09-19T04:18:10.101152+0000","last_became_peered":"2024-09-19T04:18:10.101152+0000","last_unstale":"2024-09-19T04:18:37.599800+0000","last_undegraded":"2024-09-19T04:18:37.599800+0000","last_fullsized":"2024-09-19T04:18:37.599800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:01.397605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":33,"num_read_kb":64,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.485358+0000","last_change":"2024-09-19T04:18:12.491611+0000","last_active":"2024-09-19T04:18:13.485358+0000","last_peered":"2024-09-19T04:18:13.485358+0000","last_clean":"2024-09-19T04:18:13.485358+0000","last_became_active":"2024-09-19T04:18:12.491279+0000","last_became_peered":"2024-09-19T04:18:12.491279+0000","last_unstale":"2024-09-19T04:18:13.485358+0000","last_undegraded":"2024-09-19T04:18:13.485358+0000","last_fullsized":"2024-09-19T04:18:13.485358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:01.414713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:25.578520+0000","last_change":"2024-09-19T04:18:10.105574+0000","last_active":"2024-09-19T04:18:25.578520+0000","last_peered":"2024-09-19T04:18:25.578520+0000","last_clean":"2024-09-19T04:18:25.578520+0000","last_became_active":"2024-09-19T04:18:10.105203+0000","last_became_peered":"2024-09-19T04:18:10.105203+0000","last_unstale":"2024-09-19T04:18:25.578520+0000","last_undegraded":"2024-09-19T04:18:25.578520+0000","last_fullsized":"2024-09-19T04:18:25.578520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:00.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478119+0000","last_change":"2024-09-19T04:18:12.497242+0000","last_active":"2024-09-19T04:18:13.478119+0000","last_peered":"2024-09-19T04:18:13.478119+0000","last_clean":"2024-09-19T04:18:13.478119+0000","last_became_active":"2024-09-19T04:18:12.496732+0000","last_became_peered":"2024-09-19T04:18:12.496732+0000","last_unstale":"2024-09-19T04:18:13.478119+0000","last_undegraded":"2024-09-19T04:18:13.478119+0000","last_fullsized":"2024-09-19T04:18:13.478119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:56.520341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.576520+0000","last_change":"2024-09-19T04:18:10.103905+0000","last_active":"2024-09-19T04:18:24.576520+0000","last_peered":"2024-09-19T04:18:24.576520+0000","last_clean":"2024-09-19T04:18:24.576520+0000","last_became_active":"2024-09-19T04:18:10.103517+0000","last_became_peered":"2024-09-19T04:18:10.103517+0000","last_unstale":"2024-09-19T04:18:24.576520+0000","last_undegraded":"2024-09-19T04:18:24.576520+0000","last_fullsized":"2024-09-19T04:18:24.576520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:48.292878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490128+0000","last_change":"2024-09-19T04:18:12.499380+0000","last_active":"2024-09-19T04:18:13.490128+0000","last_peered":"2024-09-19T04:18:13.490128+0000","last_clean":"2024-09-19T04:18:13.490128+0000","last_became_active":"2024-09-19T04:18:12.498995+0000","last_became_peered":"2024-09-19T04:18:12.498995+0000","last_unstale":"2024-09-19T04:18:13.490128+0000","last_undegraded":"2024-09-19T04:18:13.490128+0000","last_fullsized":"2024-09-19T04:18:13.490128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:20.653923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:36.597466+0000","last_change":"2024-09-19T04:18:10.105619+0000","last_active":"2024-09-19T04:18:36.597466+0000","last_peered":"2024-09-19T04:18:36.597466+0000","last_clean":"2024-09-19T04:18:36.597466+0000","last_became_active":"2024-09-19T04:18:10.105082+0000","last_became_peered":"2024-09-19T04:18:10.105082+0000","last_unstale":"2024-09-19T04:18:36.597466+0000","last_undegraded":"2024-09-19T04:18:36.597466+0000","last_fullsized":"2024-09-19T04:18:36.597466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:56.843068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.489971+0000","last_change":"2024-09-19T04:18:12.498473+0000","last_active":"2024-09-19T04:18:13.489971+0000","last_peered":"2024-09-19T04:18:13.489971+0000","last_clean":"2024-09-19T04:18:13.489971+0000","last_became_active":"2024-09-19T04:18:12.496904+0000","last_became_peered":"2024-09-19T04:18:12.496904+0000","last_unstale":"2024-09-19T04:18:13.489971+0000","last_undegraded":"2024-09-19T04:18:13.489971+0000","last_fullsized":"2024-09-19T04:18:13.489971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:03.155899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.578125+0000","last_change":"2024-09-19T04:18:10.365182+0000","last_active":"2024-09-19T04:18:24.578125+0000","last_peered":"2024-09-19T04:18:24.578125+0000","last_clean":"2024-09-19T04:18:24.578125+0000","last_became_active":"2024-09-19T04:18:10.364717+0000","last_became_peered":"2024-09-19T04:18:10.364717+0000","last_unstale":"2024-09-19T04:18:24.578125+0000","last_undegraded":"2024-09-19T04:18:24.578125+0000","last_fullsized":"2024-09-19T04:18:24.578125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:32.063066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487770+0000","last_change":"2024-09-19T04:18:12.493585+0000","last_active":"2024-09-19T04:18:13.487770+0000","last_peered":"2024-09-19T04:18:13.487770+0000","last_clean":"2024-09-19T04:18:13.487770+0000","last_became_active":"2024-09-19T04:18:12.493255+0000","last_became_peered":"2024-09-19T04:18:12.493255+0000","last_unstale":"2024-09-19T04:18:13.487770+0000","last_undegraded":"2024-09-19T04:18:13.487770+0000","last_fullsized":"2024-09-19T04:18:13.487770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:33.139727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.551615+0000","last_change":"2024-09-19T04:18:10.101537+0000","last_active":"2024-09-19T04:18:24.551615+0000","last_peered":"2024-09-19T04:18:24.551615+0000","last_clean":"2024-09-19T04:18:24.551615+0000","last_became_active":"2024-09-19T04:18:10.100951+0000","last_became_peered":"2024-09-19T04:18:10.100951+0000","last_unstale":"2024-09-19T04:18:24.551615+0000","last_undegraded":"2024-09-19T04:18:24.551615+0000","last_fullsized":"2024-09-19T04:18:24.551615+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:08.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480839+0000","last_change":"2024-09-19T04:18:12.495530+0000","last_active":"2024-09-19T04:18:13.480839+0000","last_peered":"2024-09-19T04:18:13.480839+0000","last_clean":"2024-09-19T04:18:13.480839+0000","last_became_active":"2024-09-19T04:18:12.495176+0000","last_became_peered":"2024-09-19T04:18:12.495176+0000","last_unstale":"2024-09-19T04:18:13.480839+0000","last_undegraded":"2024-09-19T04:18:13.480839+0000","last_fullsized":"2024-09-19T04:18:13.480839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:24.308047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.576172+0000","last_change":"2024-09-19T04:18:10.102539+0000","last_active":"2024-09-19T04:18:24.576172+0000","last_peered":"2024-09-19T04:18:24.576172+0000","last_clean":"2024-09-19T04:18:24.576172+0000","last_became_active":"2024-09-19T04:18:10.102109+0000","last_became_peered":"2024-09-19T04:18:10.102109+0000","last_unstale":"2024-09-19T04:18:24.576172+0000","last_undegraded":"2024-09-19T04:18:24.576172+0000","last_fullsized":"2024-09-19T04:18:24.576172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:58.852767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490945+0000","last_change":"2024-09-19T04:18:12.491769+0000","last_active":"2024-09-19T04:18:13.490945+0000","last_peered":"2024-09-19T04:18:13.490945+0000","last_clean":"2024-09-19T04:18:13.490945+0000","last_became_active":"2024-09-19T04:18:12.491435+0000","last_became_peered":"2024-09-19T04:18:12.491435+0000","last_unstale":"2024-09-19T04:18:13.490945+0000","last_undegraded":"2024-09-19T04:18:13.490945+0000","last_fullsized":"2024-09-19T04:18:13.490945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:54.880533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.576052+0000","last_change":"2024-09-19T04:18:10.103559+0000","last_active":"2024-09-19T04:18:24.576052+0000","last_peered":"2024-09-19T04:18:24.576052+0000","last_clean":"2024-09-19T04:18:24.576052+0000","last_became_active":"2024-09-19T04:18:10.103208+0000","last_became_peered":"2024-09-19T04:18:10.103208+0000","last_unstale":"2024-09-19T04:18:24.576052+0000","last_undegraded":"2024-09-19T04:18:24.576052+0000","last_fullsized":"2024-09-19T04:18:24.576052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:16.564916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.999740+0000","last_change":"2024-09-19T04:18:12.495616+0000","last_active":"2024-09-19T04:18:13.999740+0000","last_peered":"2024-09-19T04:18:13.999740+0000","last_clean":"2024-09-19T04:18:13.999740+0000","last_became_active":"2024-09-19T04:18:12.495243+0000","last_became_peered":"2024-09-19T04:18:12.495243+0000","last_unstale":"2024-09-19T04:18:13.999740+0000","last_undegraded":"2024-09-19T04:18:13.999740+0000","last_fullsized":"2024-09-19T04:18:13.999740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:12.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:36.598514+0000","last_change":"2024-09-19T04:18:10.626874+0000","last_active":"2024-09-19T04:18:36.598514+0000","last_peered":"2024-09-19T04:18:36.598514+0000","last_clean":"2024-09-19T04:18:36.598514+0000","last_became_active":"2024-09-19T04:18:10.626506+0000","last_became_peered":"2024-09-19T04:18:10.626506+0000","last_unstale":"2024-09-19T04:18:36.598514+0000","last_undegraded":"2024-09-19T04:18:36.598514+0000","last_fullsized":"2024-09-19T04:18:36.598514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:48.725143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490975+0000","last_change":"2024-09-19T04:18:12.496350+0000","last_active":"2024-09-19T04:18:13.490975+0000","last_peered":"2024-09-19T04:18:13.490975+0000","last_clean":"2024-09-19T04:18:13.490975+0000","last_became_active":"2024-09-19T04:18:12.495904+0000","last_became_peered":"2024-09-19T04:18:12.495904+0000","last_unstale":"2024-09-19T04:18:13.490975+0000","last_undegraded":"2024-09-19T04:18:13.490975+0000","last_fullsized":"2024-09-19T04:18:13.490975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:08.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":343,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490536+0000","last_change":"2024-09-19T04:16:56.164450+0000","last_active":"2024-09-19T04:18:13.490536+0000","last_peered":"2024-09-19T04:18:13.490536+0000","last_clean":"2024-09-19T04:18:13.490536+0000","last_became_active":"2024-09-19T04:16:56.164008+0000","last_became_peered":"2024-09-19T04:16:56.164008+0000","last_unstale":"2024-09-19T04:18:13.490536+0000","last_undegraded":"2024-09-19T04:18:13.490536+0000","last_fullsized":"2024-09-19T04:18:13.490536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_clean_scrub_stamp":"2024-09-19T04:13:18.308718+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:14.697725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:25.578520+0000","last_change":"2024-09-19T04:18:10.101598+0000","last_active":"2024-09-19T04:18:25.578520+0000","last_peered":"2024-09-19T04:18:25.578520+0000","last_clean":"2024-09-19T04:18:25.578520+0000","last_became_active":"2024-09-19T04:18:10.101210+0000","last_became_peered":"2024-09-19T04:18:10.101210+0000","last_unstale":"2024-09-19T04:18:25.578520+0000","last_undegraded":"2024-09-19T04:18:25.578520+0000","last_fullsized":"2024-09-19T04:18:25.578520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:10.653481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.486046+0000","last_change":"2024-09-19T04:18:12.495034+0000","last_active":"2024-09-19T04:18:13.486046+0000","last_peered":"2024-09-19T04:18:13.486046+0000","last_clean":"2024-09-19T04:18:13.486046+0000","last_became_active":"2024-09-19T04:18:12.494611+0000","last_became_peered":"2024-09-19T04:18:12.494611+0000","last_unstale":"2024-09-19T04:18:13.486046+0000","last_undegraded":"2024-09-19T04:18:13.486046+0000","last_fullsized":"2024-09-19T04:18:13.486046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:28.448927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:22.555493+0000","last_change":"2024-09-19T04:18:10.105212+0000","last_active":"2024-09-19T04:18:22.555493+0000","last_peered":"2024-09-19T04:18:22.555493+0000","last_clean":"2024-09-19T04:18:22.555493+0000","last_became_active":"2024-09-19T04:18:10.104710+0000","last_became_peered":"2024-09-19T04:18:10.104710+0000","last_unstale":"2024-09-19T04:18:22.555493+0000","last_undegraded":"2024-09-19T04:18:22.555493+0000","last_fullsized":"2024-09-19T04:18:22.555493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:58.007773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478020+0000","last_change":"2024-09-19T04:18:12.492773+0000","last_active":"2024-09-19T04:18:13.478020+0000","last_peered":"2024-09-19T04:18:13.478020+0000","last_clean":"2024-09-19T04:18:13.478020+0000","last_became_active":"2024-09-19T04:18:12.492442+0000","last_became_peered":"2024-09-19T04:18:12.492442+0000","last_unstale":"2024-09-19T04:18:13.478020+0000","last_undegraded":"2024-09-19T04:18:13.478020+0000","last_fullsized":"2024-09-19T04:18:13.478020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:31:36.616232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:40.602187+0000","last_change":"2024-09-19T04:18:10.364530+0000","last_active":"2024-09-19T04:18:40.602187+0000","last_peered":"2024-09-19T04:18:40.602187+0000","last_clean":"2024-09-19T04:18:40.602187+0000","last_became_active":"2024-09-19T04:18:10.364204+0000","last_became_peered":"2024-09-19T04:18:10.364204+0000","last_unstale":"2024-09-19T04:18:40.602187+0000","last_undegraded":"2024-09-19T04:18:40.602187+0000","last_fullsized":"2024-09-19T04:18:40.602187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:04.229530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.488063+0000","last_change":"2024-09-19T04:18:12.493730+0000","last_active":"2024-09-19T04:18:13.488063+0000","last_peered":"2024-09-19T04:18:13.488063+0000","last_clean":"2024-09-19T04:18:13.488063+0000","last_became_active":"2024-09-19T04:18:12.493153+0000","last_became_peered":"2024-09-19T04:18:12.493153+0000","last_unstale":"2024-09-19T04:18:13.488063+0000","last_undegraded":"2024-09-19T04:18:13.488063+0000","last_fullsized":"2024-09-19T04:18:13.488063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.758490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:37.598625+0000","last_change":"2024-09-19T04:18:10.102073+0000","last_active":"2024-09-19T04:18:37.598625+0000","last_peered":"2024-09-19T04:18:37.598625+0000","last_clean":"2024-09-19T04:18:37.598625+0000","last_became_active":"2024-09-19T04:18:10.101755+0000","last_became_peered":"2024-09-19T04:18:10.101755+0000","last_unstale":"2024-09-19T04:18:37.598625+0000","last_undegraded":"2024-09-19T04:18:37.598625+0000","last_fullsized":"2024-09-19T04:18:37.598625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:21.489146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478571+0000","last_change":"2024-09-19T04:18:12.494560+0000","last_active":"2024-09-19T04:18:13.478571+0000","last_peered":"2024-09-19T04:18:13.478571+0000","last_clean":"2024-09-19T04:18:13.478571+0000","last_became_active":"2024-09-19T04:18:12.494203+0000","last_became_peered":"2024-09-19T04:18:12.494203+0000","last_unstale":"2024-09-19T04:18:13.478571+0000","last_undegraded":"2024-09-19T04:18:13.478571+0000","last_fullsized":"2024-09-19T04:18:13.478571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:35:17.129649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484849+0000","last_change":"2024-09-19T04:18:10.365781+0000","last_active":"2024-09-19T04:18:13.484849+0000","last_peered":"2024-09-19T04:18:13.484849+0000","last_clean":"2024-09-19T04:18:13.484849+0000","last_became_active":"2024-09-19T04:18:10.365417+0000","last_became_peered":"2024-09-19T04:18:10.365417+0000","last_unstale":"2024-09-19T04:18:13.484849+0000","last_undegraded":"2024-09-19T04:18:13.484849+0000","last_fullsized":"2024-09-19T04:18:13.484849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:20.097572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.479073+0000","last_change":"2024-09-19T04:18:12.495570+0000","last_active":"2024-09-19T04:18:13.479073+0000","last_peered":"2024-09-19T04:18:13.479073+0000","last_clean":"2024-09-19T04:18:13.479073+0000","last_became_active":"2024-09-19T04:18:12.495242+0000","last_became_peered":"2024-09-19T04:18:12.495242+0000","last_unstale":"2024-09-19T04:18:13.479073+0000","last_undegraded":"2024-09-19T04:18:13.479073+0000","last_fullsized":"2024-09-19T04:18:13.479073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:57.448182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.578940+0000","last_change":"2024-09-19T04:18:10.365134+0000","last_active":"2024-09-19T04:18:24.578940+0000","last_peered":"2024-09-19T04:18:24.578940+0000","last_clean":"2024-09-19T04:18:24.578940+0000","last_became_active":"2024-09-19T04:18:10.364726+0000","last_became_peered":"2024-09-19T04:18:10.364726+0000","last_unstale":"2024-09-19T04:18:24.578940+0000","last_undegraded":"2024-09-19T04:18:24.578940+0000","last_fullsized":"2024-09-19T04:18:24.578940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:44.207855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.478464+0000","last_change":"2024-09-19T04:18:12.488836+0000","last_active":"2024-09-19T04:18:13.478464+0000","last_peered":"2024-09-19T04:18:13.478464+0000","last_clean":"2024-09-19T04:18:13.478464+0000","last_became_active":"2024-09-19T04:18:12.488507+0000","last_became_peered":"2024-09-19T04:18:12.488507+0000","last_unstale":"2024-09-19T04:18:13.478464+0000","last_undegraded":"2024-09-19T04:18:13.478464+0000","last_fullsized":"2024-09-19T04:18:13.478464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:31.308114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:40.487252+0000","last_change":"2024-09-19T04:18:10.103623+0000","last_active":"2024-09-19T04:18:40.487252+0000","last_peered":"2024-09-19T04:18:40.487252+0000","last_clean":"2024-09-19T04:18:40.487252+0000","last_became_active":"2024-09-19T04:18:10.103340+0000","last_became_peered":"2024-09-19T04:18:10.103340+0000","last_unstale":"2024-09-19T04:18:40.487252+0000","last_undegraded":"2024-09-19T04:18:40.487252+0000","last_fullsized":"2024-09-19T04:18:40.487252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:11.724013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.487239+0000","last_change":"2024-09-19T04:18:12.498167+0000","last_active":"2024-09-19T04:18:13.487239+0000","last_peered":"2024-09-19T04:18:13.487239+0000","last_clean":"2024-09-19T04:18:13.487239+0000","last_became_active":"2024-09-19T04:18:12.496924+0000","last_became_peered":"2024-09-19T04:18:12.496924+0000","last_unstale":"2024-09-19T04:18:13.487239+0000","last_undegraded":"2024-09-19T04:18:13.487239+0000","last_fullsized":"2024-09-19T04:18:13.487239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:13.952030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.577159+0000","last_change":"2024-09-19T04:18:10.627106+0000","last_active":"2024-09-19T04:18:24.577159+0000","last_peered":"2024-09-19T04:18:24.577159+0000","last_clean":"2024-09-19T04:18:24.577159+0000","last_became_active":"2024-09-19T04:18:10.626708+0000","last_became_peered":"2024-09-19T04:18:10.626708+0000","last_unstale":"2024-09-19T04:18:24.577159+0000","last_undegraded":"2024-09-19T04:18:24.577159+0000","last_fullsized":"2024-09-19T04:18:24.577159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:46.049699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491685+0000","last_change":"2024-09-19T04:18:12.506995+0000","last_active":"2024-09-19T04:18:13.491685+0000","last_peered":"2024-09-19T04:18:13.491685+0000","last_clean":"2024-09-19T04:18:13.491685+0000","last_became_active":"2024-09-19T04:18:12.500607+0000","last_became_peered":"2024-09-19T04:18:12.500607+0000","last_unstale":"2024-09-19T04:18:13.491685+0000","last_undegraded":"2024-09-19T04:18:13.491685+0000","last_fullsized":"2024-09-19T04:18:13.491685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:37.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:21.542606+0000","last_change":"2024-09-19T04:18:10.365150+0000","last_active":"2024-09-19T04:18:21.542606+0000","last_peered":"2024-09-19T04:18:21.542606+0000","last_clean":"2024-09-19T04:18:21.542606+0000","last_became_active":"2024-09-19T04:18:10.364755+0000","last_became_peered":"2024-09-19T04:18:10.364755+0000","last_unstale":"2024-09-19T04:18:21.542606+0000","last_undegraded":"2024-09-19T04:18:21.542606+0000","last_fullsized":"2024-09-19T04:18:21.542606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:11.639532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484654+0000","last_change":"2024-09-19T04:18:12.484588+0000","last_active":"2024-09-19T04:18:13.484654+0000","last_peered":"2024-09-19T04:18:13.484654+0000","last_clean":"2024-09-19T04:18:13.484654+0000","last_became_active":"2024-09-19T04:18:12.484218+0000","last_became_peered":"2024-09-19T04:18:12.484218+0000","last_unstale":"2024-09-19T04:18:13.484654+0000","last_undegraded":"2024-09-19T04:18:13.484654+0000","last_fullsized":"2024-09-19T04:18:13.484654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:55.634557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.577026+0000","last_change":"2024-09-19T04:18:10.100832+0000","last_active":"2024-09-19T04:18:24.577026+0000","last_peered":"2024-09-19T04:18:24.577026+0000","last_clean":"2024-09-19T04:18:24.577026+0000","last_became_active":"2024-09-19T04:18:10.100518+0000","last_became_peered":"2024-09-19T04:18:10.100518+0000","last_unstale":"2024-09-19T04:18:24.577026+0000","last_undegraded":"2024-09-19T04:18:24.577026+0000","last_fullsized":"2024-09-19T04:18:24.577026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:37.028120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.480804+0000","last_change":"2024-09-19T04:18:12.496914+0000","last_active":"2024-09-19T04:18:13.480804+0000","last_peered":"2024-09-19T04:18:13.480804+0000","last_clean":"2024-09-19T04:18:13.480804+0000","last_became_active":"2024-09-19T04:18:12.496556+0000","last_became_peered":"2024-09-19T04:18:12.496556+0000","last_unstale":"2024-09-19T04:18:13.480804+0000","last_undegraded":"2024-09-19T04:18:13.480804+0000","last_fullsized":"2024-09-19T04:18:13.480804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:52.391412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:23.566003+0000","last_change":"2024-09-19T04:18:10.101905+0000","last_active":"2024-09-19T04:18:23.566003+0000","last_peered":"2024-09-19T04:18:23.566003+0000","last_clean":"2024-09-19T04:18:23.566003+0000","last_became_active":"2024-09-19T04:18:10.101472+0000","last_became_peered":"2024-09-19T04:18:10.101472+0000","last_unstale":"2024-09-19T04:18:23.566003+0000","last_undegraded":"2024-09-19T04:18:23.566003+0000","last_fullsized":"2024-09-19T04:18:23.566003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:12:06.667079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.486294+0000","last_change":"2024-09-19T04:18:12.496852+0000","last_active":"2024-09-19T04:18:13.486294+0000","last_peered":"2024-09-19T04:18:13.486294+0000","last_clean":"2024-09-19T04:18:13.486294+0000","last_became_active":"2024-09-19T04:18:12.495365+0000","last_became_peered":"2024-09-19T04:18:12.495365+0000","last_unstale":"2024-09-19T04:18:13.486294+0000","last_undegraded":"2024-09-19T04:18:13.486294+0000","last_fullsized":"2024-09-19T04:18:13.486294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:14:57.012618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:21.542641+0000","last_change":"2024-09-19T04:18:10.104641+0000","last_active":"2024-09-19T04:18:21.542641+0000","last_peered":"2024-09-19T04:18:21.542641+0000","last_clean":"2024-09-19T04:18:21.542641+0000","last_became_active":"2024-09-19T04:18:10.104295+0000","last_became_peered":"2024-09-19T04:18:10.104295+0000","last_unstale":"2024-09-19T04:18:21.542641+0000","last_undegraded":"2024-09-19T04:18:21.542641+0000","last_fullsized":"2024-09-19T04:18:21.542641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:58.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.490708+0000","last_change":"2024-09-19T04:18:12.488396+0000","last_active":"2024-09-19T04:18:13.490708+0000","last_peered":"2024-09-19T04:18:13.490708+0000","last_clean":"2024-09-19T04:18:13.490708+0000","last_became_active":"2024-09-19T04:18:12.488048+0000","last_became_peered":"2024-09-19T04:18:12.488048+0000","last_unstale":"2024-09-19T04:18:13.490708+0000","last_undegraded":"2024-09-19T04:18:13.490708+0000","last_fullsized":"2024-09-19T04:18:13.490708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:27.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.488793+0000","last_change":"2024-09-19T04:18:12.493722+0000","last_active":"2024-09-19T04:18:13.488793+0000","last_peered":"2024-09-19T04:18:13.488793+0000","last_clean":"2024-09-19T04:18:13.488793+0000","last_became_active":"2024-09-19T04:18:12.492483+0000","last_became_peered":"2024-09-19T04:18:12.492483+0000","last_unstale":"2024-09-19T04:18:13.488793+0000","last_undegraded":"2024-09-19T04:18:13.488793+0000","last_fullsized":"2024-09-19T04:18:13.488793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:33.492623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:24.577548+0000","last_change":"2024-09-19T04:18:10.101326+0000","last_active":"2024-09-19T04:18:24.577548+0000","last_peered":"2024-09-19T04:18:24.577548+0000","last_clean":"2024-09-19T04:18:24.577548+0000","last_became_active":"2024-09-19T04:18:10.100955+0000","last_became_peered":"2024-09-19T04:18:10.100955+0000","last_unstale":"2024-09-19T04:18:24.577548+0000","last_undegraded":"2024-09-19T04:18:24.577548+0000","last_fullsized":"2024-09-19T04:18:24.577548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:41.939741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.484687+0000","last_change":"2024-09-19T04:18:12.492750+0000","last_active":"2024-09-19T04:18:13.484687+0000","last_peered":"2024-09-19T04:18:13.484687+0000","last_clean":"2024-09-19T04:18:13.484687+0000","last_became_active":"2024-09-19T04:18:12.492417+0000","last_became_peered":"2024-09-19T04:18:12.492417+0000","last_unstale":"2024-09-19T04:18:13.484687+0000","last_undegraded":"2024-09-19T04:18:13.484687+0000","last_fullsized":"2024-09-19T04:18:13.484687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:25.279452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:25.572578+0000","last_change":"2024-09-19T04:18:10.103762+0000","last_active":"2024-09-19T04:18:25.572578+0000","last_peered":"2024-09-19T04:18:25.572578+0000","last_clean":"2024-09-19T04:18:25.572578+0000","last_became_active":"2024-09-19T04:18:10.103441+0000","last_became_peered":"2024-09-19T04:18:10.103441+0000","last_unstale":"2024-09-19T04:18:25.572578+0000","last_undegraded":"2024-09-19T04:18:25.572578+0000","last_fullsized":"2024-09-19T04:18:25.572578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:03.451196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.491910+0000","last_change":"2024-09-19T04:18:12.503510+0000","last_active":"2024-09-19T04:18:13.491910+0000","last_peered":"2024-09-19T04:18:13.491910+0000","last_clean":"2024-09-19T04:18:13.491910+0000","last_became_active":"2024-09-19T04:18:12.495165+0000","last_became_peered":"2024-09-19T04:18:12.495165+0000","last_unstale":"2024-09-19T04:18:13.491910+0000","last_undegraded":"2024-09-19T04:18:13.491910+0000","last_fullsized":"2024-09-19T04:18:13.491910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:13.934328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.492423+0000","last_change":"2024-09-19T04:18:10.364783+0000","last_active":"2024-09-19T04:18:13.492423+0000","last_peered":"2024-09-19T04:18:13.492423+0000","last_clean":"2024-09-19T04:18:13.492423+0000","last_became_active":"2024-09-19T04:18:10.364202+0000","last_became_peered":"2024-09-19T04:18:10.364202+0000","last_unstale":"2024-09-19T04:18:13.492423+0000","last_undegraded":"2024-09-19T04:18:13.492423+0000","last_fullsized":"2024-09-19T04:18:13.492423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:18:55.770637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:13.486574+0000","last_change":"2024-09-19T04:18:12.495825+0000","last_active":"2024-09-19T04:18:13.486574+0000","last_peered":"2024-09-19T04:18:13.486574+0000","last_clean":"2024-09-19T04:18:13.486574+0000","last_became_active":"2024-09-19T04:18:12.494525+0000","last_became_peered":"2024-09-19T04:18:12.494525+0000","last_unstale":"2024-09-19T04:18:13.486574+0000","last_undegraded":"2024-09-19T04:18:13.486574+0000","last_fullsized":"2024-09-19T04:18:13.486574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:57.827135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T04:18:40.602188+0000","last_change":"2024-09-19T04:18:10.102145+0000","last_active":"2024-09-19T04:18:40.602188+0000","last_peered":"2024-09-19T04:18:40.602188+0000","last_clean":"2024-09-19T04:18:40.602188+0000","last_became_active":"2024-09-19T04:18:10.101410+0000","last_became_peered":"2024-09-19T04:18:10.101410+0000","last_unstale":"2024-09-19T04:18:40.602188+0000","last_undegraded":"2024-09-19T04:18:40.602188+0000","last_fullsized":"2024-09-19T04:18:40.602188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:20.354296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57320,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":179,"num_read_kb":156,"num_write":242,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":171960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281762,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":424211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":1064959,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972382,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":423748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201314,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":424126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1016646,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593773,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":423465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822707,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":1016324,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":423326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":423290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":475496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":474490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":425197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":51864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":51848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:18:44.225 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T04:18:44.225 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T04:18:44.226 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T04:18:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:45.071 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:44 smithi107 ceph-mon[30659]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:18:45.071 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:44 smithi107 ceph-mon[30659]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-19T04:18:45.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.189+0000 7fb869c00640 1 -- 172.21.15.107:0/3830523150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8641037c0 msgr2=0x7fb86410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.189+0000 7fb869c00640 1 --2- 172.21.15.107:0/3830523150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8641037c0 0x7fb86410a050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009ca0 tx=0x7fb85c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.189+0000 7fb869c00640 1 -- 172.21.15.107:0/3830523150 shutdown_connections 2024-09-19T04:18:45.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.190+0000 7fb869c00640 1 --2- 172.21.15.107:0/3830523150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8641037c0 0x7fb86410a050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.190+0000 7fb869c00640 1 --2- 172.21.15.107:0/3830523150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864102e00 0x7fb864103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.190+0000 7fb869c00640 1 --2- 172.21.15.107:0/3830523150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb864101c00 0x7fb864102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.190+0000 7fb869c00640 1 -- 172.21.15.107:0/3830523150 >> 172.21.15.107:0/3830523150 conn(0x7fb8640fd390 msgr2=0x7fb8640ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:45.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.190+0000 7fb869c00640 1 -- 172.21.15.107:0/3830523150 shutdown_connections 2024-09-19T04:18:45.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.190+0000 7fb869c00640 1 -- 172.21.15.107:0/3830523150 wait complete. 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.190+0000 7fb869c00640 1 Processor -- start 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb869c00640 1 -- start start 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb869c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864101c00 0x7fb864197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb869c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb864102e00 0x7fb864198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb869c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8641037c0 0x7fb86419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb869c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb86410d860 con 0x7fb8641037c0 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb869c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb86410d6e0 con 0x7fb864101c00 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb869c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb86410d9e0 con 0x7fb864102e00 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb85be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb864102e00 0x7fb864198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb863400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8641037c0 0x7fb86419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb85be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb864102e00 0x7fb864198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41636/0 (socket says 172.21.15.107:41636) 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb85be00640 1 -- 172.21.15.107:0/3429968956 learned_addr learned my addr 172.21.15.107:0/3429968956 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.191+0000 7fb862a00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864101c00 0x7fb864197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb85be00640 1 -- 172.21.15.107:0/3429968956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864101c00 msgr2=0x7fb864197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb85be00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864101c00 0x7fb864197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb85be00640 1 -- 172.21.15.107:0/3429968956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8641037c0 msgr2=0x7fb86419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb85be00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8641037c0 0x7fb86419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb85be00640 1 -- 172.21.15.107:0/3429968956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86419fa80 con 0x7fb864102e00 2024-09-19T04:18:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb863400640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8641037c0 0x7fb86419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb85be00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb864102e00 0x7fb864198390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00d950 tx=0x7fb84c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb85b400640 1 -- 172.21.15.107:0/3429968956 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c017030 con 0x7fb864102e00 2024-09-19T04:18:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb869c00640 1 -- 172.21.15.107:0/3429968956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb864074480 con 0x7fb864102e00 2024-09-19T04:18:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb85b400640 1 -- 172.21.15.107:0/3429968956 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84c004020 con 0x7fb864102e00 2024-09-19T04:18:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb85b400640 1 -- 172.21.15.107:0/3429968956 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0047a0 con 0x7fb864102e00 2024-09-19T04:18:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.192+0000 7fb869c00640 1 -- 172.21.15.107:0/3429968956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb864074a10 con 0x7fb864102e00 2024-09-19T04:18:45.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:44 smithi115 ceph-mon[35477]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:18:45.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:44 smithi115 ceph-mon[35477]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-19T04:18:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.193+0000 7fb869c00640 1 -- 172.21.15.107:0/3429968956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb864108fb0 con 0x7fb864102e00 2024-09-19T04:18:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.195+0000 7fb85b400640 1 -- 172.21.15.107:0/3429968956 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb84c0041d0 con 0x7fb864102e00 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.197+0000 7fb85b400640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb8480a3380 0x7fb8480a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.197+0000 7fb85b400640 1 -- 172.21.15.107:0/3429968956 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb84c0cf900 con 0x7fb864102e00 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.197+0000 7fb862a00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb8480a3380 0x7fb8480a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:45.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.197+0000 7fb862a00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb8480a3380 0x7fb8480a5840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb8540095f0 tx=0x7fb854005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:45.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.199+0000 7fb85b400640 1 -- 172.21.15.107:0/3429968956 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb84c09c5f0 con 0x7fb864102e00 2024-09-19T04:18:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:44 smithi145 ceph-mon[35096]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:18:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:44 smithi145 ceph-mon[35096]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-19T04:18:45.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.434+0000 7fb869c00640 1 -- 172.21.15.107:0/3429968956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb86402d7c0 con 0x7fb864102e00 2024-09-19T04:18:45.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.440+0000 7fb85b400640 1 -- 172.21.15.107:0/3429968956 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb84c09bef0 con 0x7fb864102e00 2024-09-19T04:18:45.441 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-09-19T04:18:45.441 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAlputmsfv/GRAA5N3+gWeLMJlkHnFphgorHQ== 2024-09-19T04:18:45.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 -- 172.21.15.107:0/3429968956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb8480a3380 msgr2=0x7fb8480a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb8480a3380 0x7fb8480a5840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb8540095f0 tx=0x7fb854005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:45.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 -- 172.21.15.107:0/3429968956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb864102e00 msgr2=0x7fb864198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb864102e00 0x7fb864198390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00d950 tx=0x7fb84c00de10 comp rx=0 tx=0).stop 2024-09-19T04:18:45.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 -- 172.21.15.107:0/3429968956 shutdown_connections 2024-09-19T04:18:45.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fb8480a3380 0x7fb8480a5840 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8641037c0 0x7fb86419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb864102e00 0x7fb864198390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 --2- 172.21.15.107:0/3429968956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864101c00 0x7fb864197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 -- 172.21.15.107:0/3429968956 >> 172.21.15.107:0/3429968956 conn(0x7fb8640fd390 msgr2=0x7fb864106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:45.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 -- 172.21.15.107:0/3429968956 shutdown_connections 2024-09-19T04:18:45.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:45.442+0000 7fb869c00640 1 -- 172.21.15.107:0/3429968956 wait complete. 2024-09-19T04:18:46.117 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:45 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3429968956' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:18:46.117 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:45 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:18:46.117 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:45 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:18:46.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:45 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3429968956' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:18:46.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:45 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:18:46.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:45 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:18:46.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:45 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3429968956' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:18:46.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:45 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:18:46.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:45 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:18:46.270 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:18:46.270 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T04:18:46.270 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T04:18:46.354 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T04:18:46.354 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T04:18:46.354 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T04:18:46.781 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:46 smithi115 ceph-mon[35477]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T04:18:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:46 smithi145 ceph-mon[35096]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T04:18:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:46 smithi107 ceph-mon[30659]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T04:18:47.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.396+0000 7f0ceec00640 1 -- 172.21.15.107:0/3595542541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8074590 msgr2=0x7f0ce8074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:47.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.396+0000 7f0ceec00640 1 --2- 172.21.15.107:0/3595542541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8074590 0x7f0ce8074970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009900 tx=0x7f0cd802a480 comp rx=0 tx=0).stop 2024-09-19T04:18:47.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.396+0000 7f0ceec00640 1 -- 172.21.15.107:0/3595542541 shutdown_connections 2024-09-19T04:18:47.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.396+0000 7f0ceec00640 1 --2- 172.21.15.107:0/3595542541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8112b90 0x7f0ce8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:47.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.396+0000 7f0ceec00640 1 --2- 172.21.15.107:0/3595542541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074eb0 0x7f0ce8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:47.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.396+0000 7f0ceec00640 1 --2- 172.21.15.107:0/3595542541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8074590 0x7f0ce8074970 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:47.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.396+0000 7f0ceec00640 1 -- 172.21.15.107:0/3595542541 >> 172.21.15.107:0/3595542541 conn(0x7f0ce80fd8c0 msgr2=0x7f0ce80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:47.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.396+0000 7f0ceec00640 1 -- 172.21.15.107:0/3595542541 shutdown_connections 2024-09-19T04:18:47.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.396+0000 7f0ceec00640 1 -- 172.21.15.107:0/3595542541 wait complete. 2024-09-19T04:18:47.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.397+0000 7f0ceec00640 1 Processor -- start 2024-09-19T04:18:47.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.397+0000 7f0ceec00640 1 -- start start 2024-09-19T04:18:47.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.397+0000 7f0ceec00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074590 0x7f0ce81a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:47.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.397+0000 7f0ceec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074eb0 0x7f0ce81a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:47.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.397+0000 7f0ceec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:47.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.397+0000 7f0ceec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81185c0 con 0x7f0ce8112b90 2024-09-19T04:18:47.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.397+0000 7f0ceec00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8118440 con 0x7f0ce8074590 2024-09-19T04:18:47.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.397+0000 7f0ceec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8118740 con 0x7f0ce8074eb0 2024-09-19T04:18:47.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074590 0x7f0ce81a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:47.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0cecc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:47.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074590 0x7f0ce81a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57102/0 (socket says 172.21.15.107:57102) 2024-09-19T04:18:47.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0ce7e00640 1 -- 172.21.15.107:0/1818048763 learned_addr learned my addr 172.21.15.107:0/1818048763 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0cdf400640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074eb0 0x7f0ce81a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0ce7e00640 1 -- 172.21.15.107:0/1818048763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074eb0 msgr2=0x7f0ce81a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074eb0 0x7f0ce81a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0ce7e00640 1 -- 172.21.15.107:0/1818048763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 msgr2=0x7f0ce819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0ce7e00640 1 -- 172.21.15.107:0/1818048763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce819f050 con 0x7f0ce8074590 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0cecc00640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.398+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074590 0x7f0ce81a4760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8004770 tx=0x7f0cd8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.399+0000 7f0ce5600640 1 -- 172.21.15.107:0/1818048763 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd803f070 con 0x7f0ce8074590 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.399+0000 7f0ceec00640 1 -- 172.21.15.107:0/1818048763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce819f2e0 con 0x7f0ce8074590 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.399+0000 7f0ce5600640 1 -- 172.21.15.107:0/1818048763 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8031ca0 con 0x7f0ce8074590 2024-09-19T04:18:47.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.399+0000 7f0ce5600640 1 -- 172.21.15.107:0/1818048763 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8035030 con 0x7f0ce8074590 2024-09-19T04:18:47.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.399+0000 7f0ceec00640 1 -- 172.21.15.107:0/1818048763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce81ab630 con 0x7f0ce8074590 2024-09-19T04:18:47.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.400+0000 7f0ceec00640 1 -- 172.21.15.107:0/1818048763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce8074970 con 0x7f0ce8074590 2024-09-19T04:18:47.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.402+0000 7f0ce5600640 1 -- 172.21.15.107:0/1818048763 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0cd8042070 con 0x7f0ce8074590 2024-09-19T04:18:47.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.403+0000 7f0ce5600640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0ccc0a32d0 0x7f0ccc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:47.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.403+0000 7f0ce5600640 1 -- 172.21.15.107:0/1818048763 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f0cd80f0570 con 0x7f0ce8074590 2024-09-19T04:18:47.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.403+0000 7f0cdf400640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0ccc0a32d0 0x7f0ccc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:47.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.404+0000 7f0cdf400640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0ccc0a32d0 0x7f0ccc0a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80ffec0 tx=0x7f0cd0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:47.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.405+0000 7f0ce5600640 1 -- 172.21.15.107:0/1818048763 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0cd80bcc50 con 0x7f0ce8074590 2024-09-19T04:18:47.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.631+0000 7f0ceec00640 1 -- 172.21.15.107:0/1818048763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0ce8104fd0 con 0x7f0ce8074590 2024-09-19T04:18:47.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.636+0000 7f0ce5600640 1 -- 172.21.15.107:0/1818048763 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264059 (secure 0 0 0) 0x7f0cd80c0f50 con 0x7f0ce8074590 2024-09-19T04:18:47.642 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:47.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 -- 172.21.15.107:0/1818048763 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0ccc0a32d0 msgr2=0x7f0ccc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0ccc0a32d0 0x7f0ccc0a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80ffec0 tx=0x7f0cd0005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 -- 172.21.15.107:0/1818048763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074590 msgr2=0x7f0ce81a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074590 0x7f0ce81a4760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8004770 tx=0x7f0cd8003c10 comp rx=0 tx=0).stop 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 -- 172.21.15.107:0/1818048763 shutdown_connections 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0ccc0a32d0 0x7f0ccc0a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074eb0 0x7f0ce81a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 --2- 172.21.15.107:0/1818048763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074590 0x7f0ce81a4760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 -- 172.21.15.107:0/1818048763 >> 172.21.15.107:0/1818048763 conn(0x7f0ce80fd8c0 msgr2=0x7f0ce80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 -- 172.21.15.107:0/1818048763 shutdown_connections 2024-09-19T04:18:47.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:47.640+0000 7f0ceec00640 1 -- 172.21.15.107:0/1818048763 wait complete. 2024-09-19T04:18:48.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:47 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1818048763' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:18:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:47 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1818048763' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:18:48.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:47 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1818048763' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:18:48.372 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"flags":0,"active_gid":14146,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3991722475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3991722475}]},"active_addr":"172.21.15.107:6801/3991722475","active_change":"2024-09-19T04:10:59.721367+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3889998535}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3489252098}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3866113757}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1307215535}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3949353317}]}]} 2024-09-19T04:18:48.377 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T04:18:48.377 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T04:18:48.377 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:18:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:49.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:48 smithi107 ceph-mon[30659]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-19T04:18:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:48 smithi115 ceph-mon[35477]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-19T04:18:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:48 smithi145 ceph-mon[35096]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-19T04:18:49.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.296+0000 7f902ee00640 1 -- 172.21.15.107:0/3208674904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9028102790 msgr2=0x7f90281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.296+0000 7f902ee00640 1 --2- 172.21.15.107:0/3208674904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9028102790 0x7f90281067e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9014009900 tx=0x7f901402a410 comp rx=0 tx=0).stop 2024-09-19T04:18:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.296+0000 7f902ee00640 1 -- 172.21.15.107:0/3208674904 shutdown_connections 2024-09-19T04:18:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.296+0000 7f902ee00640 1 --2- 172.21.15.107:0/3208674904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9028106f10 0x7f902810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.296+0000 7f902ee00640 1 --2- 172.21.15.107:0/3208674904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9028102790 0x7f90281067e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.296+0000 7f902ee00640 1 --2- 172.21.15.107:0/3208674904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9028101de0 0x7f90281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.296+0000 7f902ee00640 1 -- 172.21.15.107:0/3208674904 >> 172.21.15.107:0/3208674904 conn(0x7f90280fd8c0 msgr2=0x7f90280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.296+0000 7f902ee00640 1 -- 172.21.15.107:0/3208674904 shutdown_connections 2024-09-19T04:18:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.296+0000 7f902ee00640 1 -- 172.21.15.107:0/3208674904 wait complete. 2024-09-19T04:18:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.297+0000 7f902ee00640 1 Processor -- start 2024-09-19T04:18:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.297+0000 7f902ee00640 1 -- start start 2024-09-19T04:18:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.297+0000 7f902ee00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9028101de0 0x7f90280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.297+0000 7f902ee00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9028102790 0x7f9028074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.297+0000 7f902ee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9028106f10 0x7f9028074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.297+0000 7f902ee00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9028111fc0 con 0x7f9028102790 2024-09-19T04:18:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.297+0000 7f902ee00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9028111e40 con 0x7f9028101de0 2024-09-19T04:18:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.297+0000 7f902ee00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9028112140 con 0x7f9028106f10 2024-09-19T04:18:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.297+0000 7f9027400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9028102790 0x7f9028074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.298+0000 7f9027400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9028102790 0x7f9028074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40992/0 (socket says 172.21.15.107:40992) 2024-09-19T04:18:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.298+0000 7f9027400640 1 -- 172.21.15.107:0/2983330131 learned_addr learned my addr 172.21.15.107:0/2983330131 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.298+0000 7f9027e00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9028101de0 0x7f90280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.298+0000 7f902ce00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9028106f10 0x7f9028074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.298+0000 7f9027400640 1 -- 172.21.15.107:0/2983330131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9028106f10 msgr2=0x7f9028074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.298+0000 7f9027400640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9028106f10 0x7f9028074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.298+0000 7f9027400640 1 -- 172.21.15.107:0/2983330131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9028101de0 msgr2=0x7f90280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.298+0000 7f9027400640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9028101de0 0x7f90280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.298+0000 7f9027400640 1 -- 172.21.15.107:0/2983330131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90281a6920 con 0x7f9028102790 2024-09-19T04:18:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.298+0000 7f9027400640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9028102790 0x7f9028074a30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9014004770 tx=0x7f9014003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.299+0000 7f9024c00640 1 -- 172.21.15.107:0/2983330131 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901403f070 con 0x7f9028102790 2024-09-19T04:18:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.299+0000 7f9024c00640 1 -- 172.21.15.107:0/2983330131 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9014031ca0 con 0x7f9028102790 2024-09-19T04:18:49.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.299+0000 7f9024c00640 1 -- 172.21.15.107:0/2983330131 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014035030 con 0x7f9028102790 2024-09-19T04:18:49.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.299+0000 7f902ee00640 1 -- 172.21.15.107:0/2983330131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90281a6bb0 con 0x7f9028102790 2024-09-19T04:18:49.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.299+0000 7f902ee00640 1 -- 172.21.15.107:0/2983330131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90281a7040 con 0x7f9028102790 2024-09-19T04:18:49.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.300+0000 7f902ee00640 1 -- 172.21.15.107:0/2983330131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90281021c0 con 0x7f9028102790 2024-09-19T04:18:49.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.302+0000 7f9024c00640 1 -- 172.21.15.107:0/2983330131 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9014042070 con 0x7f9028102790 2024-09-19T04:18:49.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.304+0000 7f9024c00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f900c0a3350 0x7f900c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.304+0000 7f9024c00640 1 -- 172.21.15.107:0/2983330131 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f90140effc0 con 0x7f9028102790 2024-09-19T04:18:49.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.304+0000 7f9027e00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f900c0a3350 0x7f900c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.305+0000 7f9027e00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f900c0a3350 0x7f900c0a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9018005fd0 tx=0x7f9018009760 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.308+0000 7f9024c00640 1 -- 172.21.15.107:0/2983330131 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f90140bc5e0 con 0x7f9028102790 2024-09-19T04:18:49.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.485+0000 7f902ee00640 1 -- 172.21.15.107:0/2983330131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9028109b60 con 0x7f9028102790 2024-09-19T04:18:49.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.486+0000 7f9024c00640 1 -- 172.21.15.107:0/2983330131 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22348 (secure 0 0 0) 0x7f90140c08e0 con 0x7f9028102790 2024-09-19T04:18:49.488 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:49.488 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":81,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:18:13.349028+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:49.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 -- 172.21.15.107:0/2983330131 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f900c0a3350 msgr2=0x7f900c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f900c0a3350 0x7f900c0a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9018005fd0 tx=0x7f9018009760 comp rx=0 tx=0).stop 2024-09-19T04:18:49.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 -- 172.21.15.107:0/2983330131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9028102790 msgr2=0x7f9028074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9028102790 0x7f9028074a30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9014004770 tx=0x7f9014003c10 comp rx=0 tx=0).stop 2024-09-19T04:18:49.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 -- 172.21.15.107:0/2983330131 shutdown_connections 2024-09-19T04:18:49.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f900c0a3350 0x7f900c0a5810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9028106f10 0x7f9028074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9028102790 0x7f9028074a30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 --2- 172.21.15.107:0/2983330131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9028101de0 0x7f90280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 -- 172.21.15.107:0/2983330131 >> 172.21.15.107:0/2983330131 conn(0x7f90280fd8c0 msgr2=0x7f90280fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:49.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 -- 172.21.15.107:0/2983330131 shutdown_connections 2024-09-19T04:18:49.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:49.489+0000 7f902ee00640 1 -- 172.21.15.107:0/2983330131 wait complete. 2024-09-19T04:18:50.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:49 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2983330131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:50.169 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T04:18:50.169 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:18:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:49 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2983330131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:50.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:49 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2983330131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:50.541 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:51.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.120+0000 7f5c5e400640 1 -- 172.21.15.107:0/319112118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c581079c0 msgr2=0x7f5c58107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.120+0000 7f5c5e400640 1 --2- 172.21.15.107:0/319112118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c581079c0 0x7f5c58107da0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009930 tx=0x7f5c4c02a420 comp rx=0 tx=0).stop 2024-09-19T04:18:51.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.120+0000 7f5c5e400640 1 -- 172.21.15.107:0/319112118 shutdown_connections 2024-09-19T04:18:51.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.120+0000 7f5c5e400640 1 --2- 172.21.15.107:0/319112118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c58102360 0x7f5c5810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.120+0000 7f5c5e400640 1 --2- 172.21.15.107:0/319112118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c581019c0 0x7f5c58101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.120+0000 7f5c5e400640 1 --2- 172.21.15.107:0/319112118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c581079c0 0x7f5c58107da0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.120+0000 7f5c5e400640 1 -- 172.21.15.107:0/319112118 >> 172.21.15.107:0/319112118 conn(0x7f5c580fd6e0 msgr2=0x7f5c580ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.121+0000 7f5c5e400640 1 -- 172.21.15.107:0/319112118 shutdown_connections 2024-09-19T04:18:51.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.121+0000 7f5c5e400640 1 -- 172.21.15.107:0/319112118 wait complete. 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.121+0000 7f5c5e400640 1 Processor -- start 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5e400640 1 -- start start 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5e400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c581019c0 0x7f5c580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5e400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c58102360 0x7f5c58074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c581079c0 0x7f5c58074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5e400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58111dc0 con 0x7f5c581079c0 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5e400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58111c40 con 0x7f5c58102360 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5e400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58111f40 con 0x7f5c581019c0 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c581079c0 0x7f5c58074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c581079c0 0x7f5c58074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41010/0 (socket says 172.21.15.107:41010) 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5da00640 1 -- 172.21.15.107:0/486445467 learned_addr learned my addr 172.21.15.107:0/486445467 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:51.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c5d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c581019c0 0x7f5c580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.122+0000 7f5c57e00640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c58102360 0x7f5c58074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c5d000640 1 -- 172.21.15.107:0/486445467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c58102360 msgr2=0x7f5c58074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c5d000640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c58102360 0x7f5c58074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c5d000640 1 -- 172.21.15.107:0/486445467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c581079c0 msgr2=0x7f5c58074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c5d000640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c581079c0 0x7f5c58074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c5d000640 1 -- 172.21.15.107:0/486445467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c581a6780 con 0x7f5c581019c0 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c57e00640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c58102360 0x7f5c58074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c5da00640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c581079c0 0x7f5c58074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c5d000640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c581019c0 0x7f5c580744f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c005940 tx=0x7f5c4c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c55600640 1 -- 172.21.15.107:0/486445467 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c03f070 con 0x7f5c581019c0 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c55600640 1 -- 172.21.15.107:0/486445467 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c03b460 con 0x7f5c581019c0 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c5e400640 1 -- 172.21.15.107:0/486445467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c581a6a10 con 0x7f5c581019c0 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.123+0000 7f5c55600640 1 -- 172.21.15.107:0/486445467 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c035070 con 0x7f5c581019c0 2024-09-19T04:18:51.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.124+0000 7f5c5e400640 1 -- 172.21.15.107:0/486445467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c581a6ef0 con 0x7f5c581019c0 2024-09-19T04:18:51.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.125+0000 7f5c5e400640 1 -- 172.21.15.107:0/486445467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c58101e20 con 0x7f5c581019c0 2024-09-19T04:18:51.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.126+0000 7f5c55600640 1 -- 172.21.15.107:0/486445467 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5c4c03b610 con 0x7f5c581019c0 2024-09-19T04:18:51.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.127+0000 7f5c55600640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f5c400a32d0 0x7f5c400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.128+0000 7f5c55600640 1 -- 172.21.15.107:0/486445467 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f5c4c0efe70 con 0x7f5c581019c0 2024-09-19T04:18:51.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.128+0000 7f5c57e00640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f5c400a32d0 0x7f5c400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.128+0000 7f5c57e00640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f5c400a32d0 0x7f5c400a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c58075f80 tx=0x7f5c4400b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.132+0000 7f5c55600640 1 -- 172.21.15.107:0/486445467 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5c4c0bc490 con 0x7f5c581019c0 2024-09-19T04:18:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:50 smithi115 ceph-mon[35477]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-19T04:18:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:50 smithi145 ceph-mon[35096]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-19T04:18:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:50 smithi107 ceph-mon[30659]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-19T04:18:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.309+0000 7f5c5e400640 1 -- 172.21.15.107:0/486445467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c58109f30 con 0x7f5c581019c0 2024-09-19T04:18:51.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.311+0000 7f5c55600640 1 -- 172.21.15.107:0/486445467 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22348 (secure 0 0 0) 0x7f5c4c0c0790 con 0x7f5c581019c0 2024-09-19T04:18:51.313 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:18:51.313 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":81,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:18:13.349028+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.313+0000 7f5c5e400640 1 -- 172.21.15.107:0/486445467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f5c400a32d0 msgr2=0x7f5c400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.313+0000 7f5c5e400640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f5c400a32d0 0x7f5c400a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c58075f80 tx=0x7f5c4400b390 comp rx=0 tx=0).stop 2024-09-19T04:18:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.313+0000 7f5c5e400640 1 -- 172.21.15.107:0/486445467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c581019c0 msgr2=0x7f5c580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.313+0000 7f5c5e400640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c581019c0 0x7f5c580744f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c005940 tx=0x7f5c4c0042f0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.313+0000 7f5c5e400640 1 -- 172.21.15.107:0/486445467 shutdown_connections 2024-09-19T04:18:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.313+0000 7f5c5e400640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f5c400a32d0 0x7f5c400a5790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.313+0000 7f5c5e400640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c581079c0 0x7f5c58074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.313+0000 7f5c5e400640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c58102360 0x7f5c58074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.313+0000 7f5c5e400640 1 --2- 172.21.15.107:0/486445467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c581019c0 0x7f5c580744f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.313+0000 7f5c5e400640 1 -- 172.21.15.107:0/486445467 >> 172.21.15.107:0/486445467 conn(0x7f5c580fd6e0 msgr2=0x7f5c580fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.314+0000 7f5c5e400640 1 -- 172.21.15.107:0/486445467 shutdown_connections 2024-09-19T04:18:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:51.314+0000 7f5c5e400640 1 -- 172.21.15.107:0/486445467 wait complete. 2024-09-19T04:18:52.090 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T04:18:52.091 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T04:18:52.091 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T04:18:52.092 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T04:18:52.092 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T04:18:52.093 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T04:18:52.093 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T04:18:52.094 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T04:18:52.095 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T04:18:52.095 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T04:18:52.095 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T04:18:52.096 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T04:18:52.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:51 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/486445467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:52.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:51 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/486445467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:52.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:51 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/486445467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:18:53.081 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:53.098 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:53.099 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:53.109 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:53.117 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:53.186 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:52 smithi145 ceph-mon[35096]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T04:18:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:52 smithi107 ceph-mon[30659]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T04:18:53.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:52 smithi115 ceph-mon[35477]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T04:18:54.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.989+0000 7f8de1200640 1 -- 172.21.15.107:0/3506548095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ddc074510 msgr2=0x7f8ddc0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.989+0000 7f8de1200640 1 --2- 172.21.15.107:0/3506548095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ddc074510 0x7f8ddc0748f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009900 tx=0x7f8dcc02a480 comp rx=0 tx=0).stop 2024-09-19T04:18:54.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.989+0000 7f8de1200640 1 -- 172.21.15.107:0/3506548095 shutdown_connections 2024-09-19T04:18:54.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.989+0000 7f8de1200640 1 --2- 172.21.15.107:0/3506548095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ddc10e440 0x7f8ddc110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.989+0000 7f8de1200640 1 --2- 172.21.15.107:0/3506548095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ddc074e30 0x7f8ddc10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.989+0000 7f8de1200640 1 --2- 172.21.15.107:0/3506548095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ddc074510 0x7f8ddc0748f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.989+0000 7f8de1200640 1 -- 172.21.15.107:0/3506548095 >> 172.21.15.107:0/3506548095 conn(0x7f8ddc0fd8c0 msgr2=0x7f8ddc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:54.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.989+0000 7f8de1200640 1 -- 172.21.15.107:0/3506548095 shutdown_connections 2024-09-19T04:18:54.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.989+0000 7f8de1200640 1 -- 172.21.15.107:0/3506548095 wait complete. 2024-09-19T04:18:54.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.990+0000 7f8de1200640 1 Processor -- start 2024-09-19T04:18:54.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.990+0000 7f8de1200640 1 -- start start 2024-09-19T04:18:54.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.990+0000 7f8de1200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ddc074510 0x7f8ddc19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.990+0000 7f8de1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ddc074e30 0x7f8ddc19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.990+0000 7f8de1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ddc10e440 0x7f8ddc19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.990+0000 7f8de1200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc1141c0 con 0x7f8ddc074e30 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.990+0000 7f8de1200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc114040 con 0x7f8ddc074510 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.990+0000 7f8de1200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc114340 con 0x7f8ddc10e440 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dd1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ddc074e30 0x7f8ddc19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dd1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ddc074e30 0x7f8ddc19acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41032/0 (socket says 172.21.15.107:41032) 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dd1400640 1 -- 172.21.15.107:0/3920484228 learned_addr learned my addr 172.21.15.107:0/3920484228 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dd9e00640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ddc074510 0x7f8ddc19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dda800640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ddc10e440 0x7f8ddc19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dd1400640 1 -- 172.21.15.107:0/3920484228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ddc10e440 msgr2=0x7f8ddc19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dd1400640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ddc10e440 0x7f8ddc19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dd1400640 1 -- 172.21.15.107:0/3920484228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ddc074510 msgr2=0x7f8ddc19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dd1400640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ddc074510 0x7f8ddc19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dd1400640 1 -- 172.21.15.107:0/3920484228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ddc19e6d0 con 0x7f8ddc074e30 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.991+0000 7f8dd1400640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ddc074e30 0x7f8ddc19acc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400d950 tx=0x7f8dc400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.992+0000 7f8dd3400640 1 -- 172.21.15.107:0/3920484228 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4017030 con 0x7f8ddc074e30 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.992+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ddc19e9c0 con 0x7f8ddc074e30 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.992+0000 7f8dd3400640 1 -- 172.21.15.107:0/3920484228 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dc4004020 con 0x7f8ddc074e30 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.992+0000 7f8dd3400640 1 -- 172.21.15.107:0/3920484228 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40047a0 con 0x7f8ddc074e30 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.992+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ddc1a7330 con 0x7f8ddc074e30 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.992+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ddc062900 con 0x7f8ddc074e30 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.995+0000 7f8dd3400640 1 -- 172.21.15.107:0/3920484228 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8dc40041d0 con 0x7f8ddc074e30 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.996+0000 7f8dd3400640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8dc00a32d0 0x7f8dc00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.996+0000 7f8dd3400640 1 -- 172.21.15.107:0/3920484228 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f8dc40cfa10 con 0x7f8ddc074e30 2024-09-19T04:18:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.996+0000 7f8dd9e00640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8dc00a32d0 0x7f8dc00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.996+0000 7f8dd3400640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] conn(0x7f8dc00acb20 0x7f8dc00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.996+0000 7f8dd3400640 1 -- 172.21.15.107:0/3920484228 --> [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8dc00af5f0 con 0x7f8dc00acb20 2024-09-19T04:18:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.996+0000 7f8dd3400640 1 -- 172.21.15.107:0/3920484228 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f8dc40cfdc0 con 0x7f8ddc074e30 2024-09-19T04:18:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.997+0000 7f8dda800640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] conn(0x7f8dc00acb20 0x7f8dc00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.997+0000 7f8dd9e00640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8dc00a32d0 0x7f8dc00a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc000c00 tx=0x7f8dcc005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.997+0000 7f8dda800640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] conn(0x7f8dc00acb20 0x7f8dc00aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:54.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:53.998+0000 7f8dd3400640 1 -- 172.21.15.107:0/3920484228 <== osd.9 v2:172.21.15.107:6826/3784218099 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8dc00af5f0 con 0x7f8dc00acb20 2024-09-19T04:18:54.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.012+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 --> [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ddc062d00 con 0x7f8dc00acb20 2024-09-19T04:18:54.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.012+0000 7f8dd3400640 1 -- 172.21.15.107:0/3920484228 <== osd.9 v2:172.21.15.107:6826/3784218099 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8ddc062d00 con 0x7f8dc00acb20 2024-09-19T04:18:54.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] conn(0x7f8dc00acb20 msgr2=0x7f8dc00aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8de1200640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] conn(0x7f8dc00acb20 0x7f8dc00aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8dc00a32d0 msgr2=0x7f8dc00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8de1200640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8dc00a32d0 0x7f8dc00a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc000c00 tx=0x7f8dcc005b50 comp rx=0 tx=0).stop 2024-09-19T04:18:54.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ddc074e30 msgr2=0x7f8ddc19acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8de1200640 1 --2- 172.21.15.107:0/3920484228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ddc074e30 0x7f8ddc19acc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400d950 tx=0x7f8dc400de10 comp rx=0 tx=0).stop 2024-09-19T04:18:54.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8dda800640 1 -- 172.21.15.107:0/3920484228 reap_dead start 2024-09-19T04:18:54.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 shutdown_connections 2024-09-19T04:18:54.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 >> 172.21.15.107:0/3920484228 conn(0x7f8ddc0fd8c0 msgr2=0x7f8ddc0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:54.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 shutdown_connections 2024-09-19T04:18:54.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.013+0000 7f8de1200640 1 -- 172.21.15.107:0/3920484228 wait complete. 2024-09-19T04:18:54.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.580+0000 7f73d6800640 1 -- 172.21.15.107:0/735422589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d0104010 msgr2=0x7f73d01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.580+0000 7f73d6800640 1 --2- 172.21.15.107:0/735422589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d0104010 0x7f73d01043f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009900 tx=0x7f73c002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.580+0000 7f73d6800640 1 -- 172.21.15.107:0/735422589 shutdown_connections 2024-09-19T04:18:54.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.580+0000 7f73d6800640 1 --2- 172.21.15.107:0/735422589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d0109140 0x7f73d0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.581+0000 7f73d6800640 1 --2- 172.21.15.107:0/735422589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d01049c0 0x7f73d0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.581+0000 7f73d6800640 1 --2- 172.21.15.107:0/735422589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d0104010 0x7f73d01043f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.581+0000 7f73d6800640 1 -- 172.21.15.107:0/735422589 >> 172.21.15.107:0/735422589 conn(0x7f73d00ffa50 msgr2=0x7f73d0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:54.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.581+0000 7f73d6800640 1 -- 172.21.15.107:0/735422589 shutdown_connections 2024-09-19T04:18:54.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.581+0000 7f73d6800640 1 -- 172.21.15.107:0/735422589 wait complete. 2024-09-19T04:18:54.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73d6800640 1 Processor -- start 2024-09-19T04:18:54.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73d6800640 1 -- start start 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73d6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d0104010 0x7f73d019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73d6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d01049c0 0x7f73d019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73d6800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0109140 0x7f73d019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73d6800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0114140 con 0x7f73d0104010 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73d6800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0113fc0 con 0x7f73d01049c0 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73d6800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01142c0 con 0x7f73d0109140 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73cf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d0104010 0x7f73d019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73cf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d0104010 0x7f73d019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41042/0 (socket says 172.21.15.107:41042) 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73cfe00640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0109140 0x7f73d019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.582+0000 7f73cf400640 1 -- 172.21.15.107:0/4245082683 learned_addr learned my addr 172.21.15.107:0/4245082683 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.583+0000 7f73cea00640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d01049c0 0x7f73d019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.583+0000 7f73cfe00640 1 -- 172.21.15.107:0/4245082683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d01049c0 msgr2=0x7f73d019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.583+0000 7f73cfe00640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d01049c0 0x7f73d019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.583+0000 7f73cfe00640 1 -- 172.21.15.107:0/4245082683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d0104010 msgr2=0x7f73d019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.583+0000 7f73cfe00640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d0104010 0x7f73d019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.583+0000 7f73cfe00640 1 -- 172.21.15.107:0/4245082683 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d019f5f0 con 0x7f73d0109140 2024-09-19T04:18:54.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.583+0000 7f73cfe00640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0109140 0x7f73d019ef10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f73c800da10 tx=0x7f73c800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:54.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.583+0000 7f73c7e00640 1 -- 172.21.15.107:0/4245082683 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c800bd70 con 0x7f73d0109140 2024-09-19T04:18:54.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.584+0000 7f73c7e00640 1 -- 172.21.15.107:0/4245082683 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c8003e50 con 0x7f73d0109140 2024-09-19T04:18:54.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.584+0000 7f73c7e00640 1 -- 172.21.15.107:0/4245082683 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c80106a0 con 0x7f73d0109140 2024-09-19T04:18:54.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.584+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d019f8e0 con 0x7f73d0109140 2024-09-19T04:18:54.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.584+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d01a71c0 con 0x7f73d0109140 2024-09-19T04:18:54.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.585+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73d0062980 con 0x7f73d0109140 2024-09-19T04:18:54.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.588+0000 7f73c7e00640 1 -- 172.21.15.107:0/4245082683 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f73c8004b40 con 0x7f73d0109140 2024-09-19T04:18:54.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.590+0000 7f73c7e00640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f73b40a32d0 0x7f73b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.590+0000 7f73c7e00640 1 -- 172.21.15.107:0/4245082683 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f73c80cfc00 con 0x7f73d0109140 2024-09-19T04:18:54.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.590+0000 7f73c7e00640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] conn(0x7f73b40acb20 0x7f73b40aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.590+0000 7f73c7e00640 1 -- 172.21.15.107:0/4245082683 --> [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73b40af5f0 con 0x7f73b40acb20 2024-09-19T04:18:54.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.590+0000 7f73c7e00640 1 -- 172.21.15.107:0/4245082683 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f73c80d00e0 con 0x7f73d0109140 2024-09-19T04:18:54.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.592+0000 7f73cea00640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] conn(0x7f73b40acb20 0x7f73b40aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.592+0000 7f73cf400640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f73b40a32d0 0x7f73b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.593+0000 7f73cea00640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] conn(0x7f73b40acb20 0x7f73b40aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.593+0000 7f73cf400640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f73b40a32d0 0x7f73b40a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f73c00096c0 tx=0x7f73c0005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.594+0000 7f73c7e00640 1 -- 172.21.15.107:0/4245082683 <== osd.1 v2:172.21.15.115:6800/3371389240 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73b40af5f0 con 0x7f73b40acb20 2024-09-19T04:18:54.598 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:54.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.610+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 --> [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f73d0062e00 con 0x7f73b40acb20 2024-09-19T04:18:54.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.610+0000 7f73c7e00640 1 -- 172.21.15.107:0/4245082683 <== osd.1 v2:172.21.15.115:6800/3371389240 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f73d0062e00 con 0x7f73b40acb20 2024-09-19T04:18:54.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.611+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 >> [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] conn(0x7f73b40acb20 msgr2=0x7f73b40aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.611+0000 7f73d6800640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] conn(0x7f73b40acb20 0x7f73b40aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.611+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f73b40a32d0 msgr2=0x7f73b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.611+0000 7f73d6800640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f73b40a32d0 0x7f73b40a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f73c00096c0 tx=0x7f73c0005a20 comp rx=0 tx=0).stop 2024-09-19T04:18:54.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.611+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0109140 msgr2=0x7f73d019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.611+0000 7f73d6800640 1 --2- 172.21.15.107:0/4245082683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0109140 0x7f73d019ef10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f73c800da10 tx=0x7f73c800ded0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.611+0000 7f73cfe00640 1 -- 172.21.15.107:0/4245082683 reap_dead start 2024-09-19T04:18:54.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.611+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 shutdown_connections 2024-09-19T04:18:54.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.611+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 >> 172.21.15.107:0/4245082683 conn(0x7f73d00ffa50 msgr2=0x7f73d0100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:54.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.612+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 shutdown_connections 2024-09-19T04:18:54.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:54.612+0000 7f73d6800640 1 -- 172.21.15.107:0/4245082683 wait complete. 2024-09-19T04:18:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:54 smithi145 ceph-mon[35096]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T04:18:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:54 smithi107 ceph-mon[30659]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T04:18:55.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:54 smithi115 ceph-mon[35477]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T04:18:55.977 INFO:teuthology.orchestra.run.smithi107.stdout:266287972385 2024-09-19T04:18:55.978 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T04:18:56.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.731+0000 7f693a400640 1 -- 172.21.15.107:0/2867714864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6934103ff0 msgr2=0x7f69341043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.731+0000 7f693a400640 1 --2- 172.21.15.107:0/2867714864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6934103ff0 0x7f69341043d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6924009900 tx=0x7f692402a410 comp rx=0 tx=0).stop 2024-09-19T04:18:56.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.731+0000 7f693a400640 1 -- 172.21.15.107:0/2867714864 shutdown_connections 2024-09-19T04:18:56.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.731+0000 7f693a400640 1 --2- 172.21.15.107:0/2867714864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6934109120 0x7f6934110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.731+0000 7f693a400640 1 --2- 172.21.15.107:0/2867714864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69341049a0 0x7f69341089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.731+0000 7f693a400640 1 --2- 172.21.15.107:0/2867714864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6934103ff0 0x7f69341043d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.731+0000 7f693a400640 1 -- 172.21.15.107:0/2867714864 >> 172.21.15.107:0/2867714864 conn(0x7f69340ffa90 msgr2=0x7f6934101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.731+0000 7f693a400640 1 -- 172.21.15.107:0/2867714864 shutdown_connections 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.731+0000 7f693a400640 1 -- 172.21.15.107:0/2867714864 wait complete. 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f693a400640 1 Processor -- start 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f693a400640 1 -- start start 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f693a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6934103ff0 0x7f693419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f693a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69341049a0 0x7f693419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f693a400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6934109120 0x7f693419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f693a400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6934114150 con 0x7f69341049a0 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f693a400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6934113fd0 con 0x7f6934109120 2024-09-19T04:18:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f693a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f69341142d0 con 0x7f6934103ff0 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f692be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69341049a0 0x7f693419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f692be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69341049a0 0x7f693419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41048/0 (socket says 172.21.15.107:41048) 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.732+0000 7f692be00640 1 -- 172.21.15.107:0/520923618 learned_addr learned my addr 172.21.15.107:0/520923618 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f6933400640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6934103ff0 0x7f693419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f6933e00640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6934109120 0x7f693419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f692be00640 1 -- 172.21.15.107:0/520923618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6934103ff0 msgr2=0x7f693419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f692be00640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6934103ff0 0x7f693419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f692be00640 1 -- 172.21.15.107:0/520923618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6934109120 msgr2=0x7f693419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f692be00640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6934109120 0x7f693419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f692be00640 1 -- 172.21.15.107:0/520923618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693419f5e0 con 0x7f69341049a0 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f692be00640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69341049a0 0x7f693419ab70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f691c00e9f0 tx=0x7f691c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f6930c00640 1 -- 172.21.15.107:0/520923618 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c00cd80 con 0x7f69341049a0 2024-09-19T04:18:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f6930c00640 1 -- 172.21.15.107:0/520923618 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f691c010910 con 0x7f69341049a0 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693419f8d0 con 0x7f69341049a0 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.733+0000 7f6930c00640 1 -- 172.21.15.107:0/520923618 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c018990 con 0x7f69341049a0 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.734+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69341a71b0 con 0x7f69341049a0 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.734+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6934062940 con 0x7f69341049a0 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.736+0000 7f6930c00640 1 -- 172.21.15.107:0/520923618 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f691c010ac0 con 0x7f69341049a0 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.738+0000 7f6930c00640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69180a32d0 0x7f69180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.738+0000 7f6930c00640 1 -- 172.21.15.107:0/520923618 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f691c0d0370 con 0x7f69341049a0 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.738+0000 7f6933400640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69180a32d0 0x7f69180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.738+0000 7f6930c00640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] conn(0x7f69180acb20 0x7f69180aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.738+0000 7f6930c00640 1 -- 172.21.15.107:0/520923618 --> [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f691c022e50 con 0x7f69180acb20 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.738+0000 7f6930c00640 1 -- 172.21.15.107:0/520923618 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f691c0d0720 con 0x7f69341049a0 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.738+0000 7f6933e00640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] conn(0x7f69180acb20 0x7f69180aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.738+0000 7f6933400640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69180a32d0 0x7f69180a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6924009230 tx=0x7f69240058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.739+0000 7f6933e00640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] conn(0x7f69180acb20 0x7f69180aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.740+0000 7f6930c00640 1 -- 172.21.15.107:0/520923618 <== osd.4 v2:172.21.15.115:6808/1123876846 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f691c022e50 con 0x7f69180acb20 2024-09-19T04:18:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.754+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 --> [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6934062dc0 con 0x7f69180acb20 2024-09-19T04:18:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.754+0000 7f6930c00640 1 -- 172.21.15.107:0/520923618 <== osd.4 v2:172.21.15.115:6808/1123876846 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6934062dc0 con 0x7f69180acb20 2024-09-19T04:18:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 >> [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] conn(0x7f69180acb20 msgr2=0x7f69180aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f693a400640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] conn(0x7f69180acb20 0x7f69180aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69180a32d0 msgr2=0x7f69180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f693a400640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f69180a32d0 0x7f69180a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6924009230 tx=0x7f69240058d0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69341049a0 msgr2=0x7f693419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f693a400640 1 --2- 172.21.15.107:0/520923618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69341049a0 0x7f693419ab70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f691c00e9f0 tx=0x7f691c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f6933e00640 1 -- 172.21.15.107:0/520923618 reap_dead start 2024-09-19T04:18:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 shutdown_connections 2024-09-19T04:18:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 >> 172.21.15.107:0/520923618 conn(0x7f69340ffa90 msgr2=0x7f6934100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 shutdown_connections 2024-09-19T04:18:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:55.755+0000 7f693a400640 1 -- 172.21.15.107:0/520923618 wait complete. 2024-09-19T04:18:56.169 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:56.170 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:56.425 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:56.426 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:18:56.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.283+0000 7f9885a00640 1 -- 172.21.15.107:0/2120692526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9880109140 msgr2=0x7f9880110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.283+0000 7f9885a00640 1 --2- 172.21.15.107:0/2120692526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9880109140 0x7f9880110cc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9878009ca0 tx=0x7f987802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.284+0000 7f9885a00640 1 -- 172.21.15.107:0/2120692526 shutdown_connections 2024-09-19T04:18:56.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.284+0000 7f9885a00640 1 --2- 172.21.15.107:0/2120692526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9880109140 0x7f9880110cc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.284+0000 7f9885a00640 1 --2- 172.21.15.107:0/2120692526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98801049c0 0x7f9880108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.284+0000 7f9885a00640 1 --2- 172.21.15.107:0/2120692526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9880104010 0x7f98801043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.284+0000 7f9885a00640 1 -- 172.21.15.107:0/2120692526 >> 172.21.15.107:0/2120692526 conn(0x7f98800ffa50 msgr2=0x7f9880101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:56.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.284+0000 7f9885a00640 1 -- 172.21.15.107:0/2120692526 shutdown_connections 2024-09-19T04:18:56.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.284+0000 7f9885a00640 1 -- 172.21.15.107:0/2120692526 wait complete. 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.285+0000 7f9885a00640 1 Processor -- start 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.285+0000 7f9885a00640 1 -- start start 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.285+0000 7f9885a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9880104010 0x7f988019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.285+0000 7f9885a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98801049c0 0x7f988019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.285+0000 7f9885a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9880109140 0x7f988019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.285+0000 7f9885a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98801141a0 con 0x7f9880109140 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.285+0000 7f9885a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9880114020 con 0x7f98801049c0 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.285+0000 7f9885a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9880114320 con 0x7f9880104010 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.285+0000 7f987f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9880109140 0x7f988019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f987ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9880104010 0x7f988019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:56.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f987e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98801049c0 0x7f988019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f987ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9880104010 0x7f988019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43358/0 (socket says 172.21.15.107:43358) 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f987ea00640 1 -- 172.21.15.107:0/2400914451 learned_addr learned my addr 172.21.15.107:0/2400914451 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f987ea00640 1 -- 172.21.15.107:0/2400914451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98801049c0 msgr2=0x7f988019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f987ea00640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98801049c0 0x7f988019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f987ea00640 1 -- 172.21.15.107:0/2400914451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9880109140 msgr2=0x7f988019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f987ea00640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9880109140 0x7f988019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f987ea00640 1 -- 172.21.15.107:0/2400914451 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988019f670 con 0x7f9880104010 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f987ea00640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9880104010 0x7f988019a6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f987000e9f0 tx=0x7f987000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f9877400640 1 -- 172.21.15.107:0/2400914451 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987000cd80 con 0x7f9880104010 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f9877400640 1 -- 172.21.15.107:0/2400914451 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9870010910 con 0x7f9880104010 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.286+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988019f960 con 0x7f9880104010 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.287+0000 7f9877400640 1 -- 172.21.15.107:0/2400914451 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870018990 con 0x7f9880104010 2024-09-19T04:18:56.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.287+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98801a7240 con 0x7f9880104010 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.287+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9880062900 con 0x7f9880104010 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.289+0000 7f9877400640 1 -- 172.21.15.107:0/2400914451 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9870010ac0 con 0x7f9880104010 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.291+0000 7f9877400640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f98640a32d0 0x7f98640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.291+0000 7f9877400640 1 -- 172.21.15.107:0/2400914451 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f98700d0370 con 0x7f9880104010 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.291+0000 7f987e000640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f98640a32d0 0x7f98640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.291+0000 7f9877400640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] conn(0x7f98640acb20 0x7f98640aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.291+0000 7f9877400640 1 -- 172.21.15.107:0/2400914451 --> [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9870022e50 con 0x7f98640acb20 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.291+0000 7f9877400640 1 -- 172.21.15.107:0/2400914451 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f98700d0720 con 0x7f9880104010 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.291+0000 7f987f400640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] conn(0x7f98640acb20 0x7f98640aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.292+0000 7f987e000640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f98640a32d0 0x7f98640a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f988010aab0 tx=0x7f986c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.292+0000 7f987f400640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] conn(0x7f98640acb20 0x7f98640aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:56.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.293+0000 7f9877400640 1 -- 172.21.15.107:0/2400914451 <== osd.2 v2:172.21.15.145:6800/2439686508 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9870022e50 con 0x7f98640acb20 2024-09-19T04:18:56.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.307+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 --> [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9880062d80 con 0x7f98640acb20 2024-09-19T04:18:56.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.308+0000 7f9877400640 1 -- 172.21.15.107:0/2400914451 <== osd.2 v2:172.21.15.145:6800/2439686508 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9880062d80 con 0x7f98640acb20 2024-09-19T04:18:56.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.308+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 >> [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] conn(0x7f98640acb20 msgr2=0x7f98640aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.308+0000 7f9885a00640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] conn(0x7f98640acb20 0x7f98640aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.308+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f98640a32d0 msgr2=0x7f98640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.308+0000 7f9885a00640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f98640a32d0 0x7f98640a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f988010aab0 tx=0x7f986c005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:56.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.308+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9880104010 msgr2=0x7f988019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:56.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.308+0000 7f9885a00640 1 --2- 172.21.15.107:0/2400914451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9880104010 0x7f988019a6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f987000e9f0 tx=0x7f987000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:18:56.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.308+0000 7f987f400640 1 -- 172.21.15.107:0/2400914451 reap_dead start 2024-09-19T04:18:56.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.309+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 shutdown_connections 2024-09-19T04:18:56.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.309+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 >> 172.21.15.107:0/2400914451 conn(0x7f98800ffa50 msgr2=0x7f9880100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:56.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.309+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 shutdown_connections 2024-09-19T04:18:56.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:56.309+0000 7f9885a00640 1 -- 172.21.15.107:0/2400914451 wait complete. 2024-09-19T04:18:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:56 smithi145 ceph-mon[35096]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T04:18:57.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:56 smithi107 ceph-mon[30659]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T04:18:57.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:56 smithi115 ceph-mon[35477]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T04:18:57.709 INFO:teuthology.orchestra.run.smithi107.stdout:77309411398 2024-09-19T04:18:57.710 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T04:18:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.549+0000 7f0599400640 1 -- 172.21.15.107:0/1403468396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0594074e30 msgr2=0x7f059410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.549+0000 7f0599400640 1 --2- 172.21.15.107:0/1403468396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0594074e30 0x7f059410dd40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0580009900 tx=0x7f058002a480 comp rx=0 tx=0).stop 2024-09-19T04:18:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.549+0000 7f0599400640 1 -- 172.21.15.107:0/1403468396 shutdown_connections 2024-09-19T04:18:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.549+0000 7f0599400640 1 --2- 172.21.15.107:0/1403468396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f059410e280 0x7f0594110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.549+0000 7f0599400640 1 --2- 172.21.15.107:0/1403468396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0594074e30 0x7f059410dd40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.549+0000 7f0599400640 1 --2- 172.21.15.107:0/1403468396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0594074510 0x7f05940748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.549+0000 7f0599400640 1 -- 172.21.15.107:0/1403468396 >> 172.21.15.107:0/1403468396 conn(0x7f05940fd740 msgr2=0x7f05940ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.549+0000 7f0599400640 1 -- 172.21.15.107:0/1403468396 shutdown_connections 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.549+0000 7f0599400640 1 -- 172.21.15.107:0/1403468396 wait complete. 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.550+0000 7f0599400640 1 Processor -- start 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.551+0000 7f0599400640 1 -- start start 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.551+0000 7f0599400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0594074510 0x7f05941019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.551+0000 7f0599400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0594074e30 0x7f0594101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.551+0000 7f0599400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f059410e280 0x7f0594102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.551+0000 7f0599400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0594114000 con 0x7f0594074510 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.551+0000 7f0599400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0594113e80 con 0x7f059410e280 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.551+0000 7f0599400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0594114180 con 0x7f0594074e30 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.551+0000 7f0593e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0594074510 0x7f05941019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.551+0000 7f0593e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0594074510 0x7f05941019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41090/0 (socket says 172.21.15.107:41090) 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.551+0000 7f0593e00640 1 -- 172.21.15.107:0/2755888350 learned_addr learned my addr 172.21.15.107:0/2755888350 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.552+0000 7f0598a00640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f059410e280 0x7f0594102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.552+0000 7f0593e00640 1 -- 172.21.15.107:0/2755888350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0594074e30 msgr2=0x7f0594101ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.552+0000 7f0593e00640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0594074e30 0x7f0594101ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.552+0000 7f0593e00640 1 -- 172.21.15.107:0/2755888350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f059410e280 msgr2=0x7f0594102420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.552+0000 7f0593e00640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f059410e280 0x7f0594102420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.552+0000 7f0593e00640 1 -- 172.21.15.107:0/2755888350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05941aabc0 con 0x7f0594074510 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.552+0000 7f0593e00640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0594074510 0x7f05941019a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f058400ea90 tx=0x7f058400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.552+0000 7f0590c00640 1 -- 172.21.15.107:0/2755888350 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058400ce40 con 0x7f0594074510 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.553+0000 7f0590c00640 1 -- 172.21.15.107:0/2755888350 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05840035f0 con 0x7f0594074510 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.553+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05941aaeb0 con 0x7f0594074510 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.553+0000 7f0590c00640 1 -- 172.21.15.107:0/2755888350 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584010690 con 0x7f0594074510 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.553+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05941ab360 con 0x7f0594074510 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.554+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0594062980 con 0x7f0594074510 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.556+0000 7f0590c00640 1 -- 172.21.15.107:0/2755888350 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f05840037a0 con 0x7f0594074510 2024-09-19T04:18:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.558+0000 7f0590c00640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f05780a32d0 0x7f05780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.558+0000 7f0590c00640 1 -- 172.21.15.107:0/2755888350 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f05840d04a0 con 0x7f0594074510 2024-09-19T04:18:58.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.558+0000 7f0593400640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f05780a32d0 0x7f05780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.558+0000 7f0590c00640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] conn(0x7f05780acb20 0x7f05780aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.558+0000 7f0590c00640 1 -- 172.21.15.107:0/2755888350 --> [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f058401aec0 con 0x7f05780acb20 2024-09-19T04:18:58.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.558+0000 7f0590c00640 1 -- 172.21.15.107:0/2755888350 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f05840d0850 con 0x7f0594074510 2024-09-19T04:18:58.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.558+0000 7f0598a00640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] conn(0x7f05780acb20 0x7f05780aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.558+0000 7f0593400640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f05780a32d0 0x7f05780a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f058002a9a0 tx=0x7f0580005fd0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.559+0000 7f0598a00640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] conn(0x7f05780acb20 0x7f05780aef40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.560+0000 7f0590c00640 1 -- 172.21.15.107:0/2755888350 <== osd.5 v2:172.21.15.145:6808/1693312722 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f058401aec0 con 0x7f05780acb20 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.574+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 --> [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0594062e00 con 0x7f05780acb20 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.575+0000 7f0590c00640 1 -- 172.21.15.107:0/2755888350 <== osd.5 v2:172.21.15.145:6808/1693312722 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0594062e00 con 0x7f05780acb20 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.575+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 >> [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] conn(0x7f05780acb20 msgr2=0x7f05780aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.575+0000 7f0599400640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] conn(0x7f05780acb20 0x7f05780aef40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.575+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f05780a32d0 msgr2=0x7f05780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.575+0000 7f0599400640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f05780a32d0 0x7f05780a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f058002a9a0 tx=0x7f0580005fd0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.575+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0594074510 msgr2=0x7f05941019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.575+0000 7f0599400640 1 --2- 172.21.15.107:0/2755888350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0594074510 0x7f05941019a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f058400ea90 tx=0x7f058400ef50 comp rx=0 tx=0).stop 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.575+0000 7f0598a00640 1 -- 172.21.15.107:0/2755888350 reap_dead start 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.575+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 shutdown_connections 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.575+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 >> 172.21.15.107:0/2755888350 conn(0x7f05940fd740 msgr2=0x7f05940fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.576+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 shutdown_connections 2024-09-19T04:18:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:58.576+0000 7f0599400640 1 -- 172.21.15.107:0/2755888350 wait complete. 2024-09-19T04:18:59.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.050+0000 7f49d5c00640 1 -- 172.21.15.107:0/2920480329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d010c170 msgr2=0x7f49d010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.050+0000 7f49d5c00640 1 --2- 172.21.15.107:0/2920480329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d010c170 0x7f49d010e610 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009ca0 tx=0x7f49c802a950 comp rx=0 tx=0).stop 2024-09-19T04:18:59.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.051+0000 7f49d5c00640 1 -- 172.21.15.107:0/2920480329 shutdown_connections 2024-09-19T04:18:59.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.051+0000 7f49d5c00640 1 --2- 172.21.15.107:0/2920480329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d010c170 0x7f49d010e610 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.051+0000 7f49d5c00640 1 --2- 172.21.15.107:0/2920480329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49d0103660 0x7f49d010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.051+0000 7f49d5c00640 1 --2- 172.21.15.107:0/2920480329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d0102d40 0x7f49d0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.051+0000 7f49d5c00640 1 -- 172.21.15.107:0/2920480329 >> 172.21.15.107:0/2920480329 conn(0x7f49d00fc7a0 msgr2=0x7f49d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:59.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.051+0000 7f49d5c00640 1 -- 172.21.15.107:0/2920480329 shutdown_connections 2024-09-19T04:18:59.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.051+0000 7f49d5c00640 1 -- 172.21.15.107:0/2920480329 wait complete. 2024-09-19T04:18:59.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49d5c00640 1 Processor -- start 2024-09-19T04:18:59.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49d5c00640 1 -- start start 2024-09-19T04:18:59.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49d5c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d0102d40 0x7f49d01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49d5c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49d0103660 0x7f49d0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49d5c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d010c170 0x7f49d0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49d5c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0111f80 con 0x7f49d0102d40 2024-09-19T04:18:59.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49d5c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0111e00 con 0x7f49d010c170 2024-09-19T04:18:59.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49d5c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0112100 con 0x7f49d0103660 2024-09-19T04:18:59.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49cea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d0102d40 0x7f49d01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49cea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d0102d40 0x7f49d01009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53528/0 (socket says 172.21.15.107:53528) 2024-09-19T04:18:59.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.052+0000 7f49cea00640 1 -- 172.21.15.107:0/4012823362 learned_addr learned my addr 172.21.15.107:0/4012823362 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:18:59.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.053+0000 7f49ce000640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49d0103660 0x7f49d0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.053+0000 7f49cf400640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d010c170 0x7f49d0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.053+0000 7f49ce000640 1 -- 172.21.15.107:0/4012823362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d010c170 msgr2=0x7f49d0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.053+0000 7f49ce000640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d010c170 0x7f49d0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.053+0000 7f49ce000640 1 -- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d0102d40 msgr2=0x7f49d01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.053+0000 7f49ce000640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d0102d40 0x7f49d01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.053+0000 7f49ce000640 1 -- 172.21.15.107:0/4012823362 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d01a6860 con 0x7f49d0103660 2024-09-19T04:18:59.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.053+0000 7f49cea00640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d0102d40 0x7f49d01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:59.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.053+0000 7f49ce000640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49d0103660 0x7f49d0100ee0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00e7a0 tx=0x7f49bc00ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:59.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.054+0000 7f49c7400640 1 -- 172.21.15.107:0/4012823362 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc00ccc0 con 0x7f49d0103660 2024-09-19T04:18:59.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.054+0000 7f49c7400640 1 -- 172.21.15.107:0/4012823362 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49bc009530 con 0x7f49d0103660 2024-09-19T04:18:59.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.054+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d01a6b50 con 0x7f49d0103660 2024-09-19T04:18:59.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.054+0000 7f49c7400640 1 -- 172.21.15.107:0/4012823362 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc018880 con 0x7f49d0103660 2024-09-19T04:18:59.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.054+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d01a70e0 con 0x7f49d0103660 2024-09-19T04:18:59.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.055+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49d00628c0 con 0x7f49d0103660 2024-09-19T04:18:59.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.058+0000 7f49c7400640 1 -- 172.21.15.107:0/4012823362 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f49bc018a20 con 0x7f49d0103660 2024-09-19T04:18:59.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.059+0000 7f49c7400640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49b40a3350 0x7f49b40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.059+0000 7f49c7400640 1 -- 172.21.15.107:0/4012823362 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f49bc0cf9a0 con 0x7f49d0103660 2024-09-19T04:18:59.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.060+0000 7f49cea00640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49b40a3350 0x7f49b40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.060+0000 7f49c7400640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] conn(0x7f49b40acba0 0x7f49b40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:59.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.060+0000 7f49c7400640 1 -- 172.21.15.107:0/4012823362 --> [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49bc012ec0 con 0x7f49b40acba0 2024-09-19T04:18:59.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.060+0000 7f49c7400640 1 -- 172.21.15.107:0/4012823362 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f49bc0d42c0 con 0x7f49d0103660 2024-09-19T04:18:59.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.060+0000 7f49cf400640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] conn(0x7f49b40acba0 0x7f49b40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:59.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.060+0000 7f49cea00640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49b40a3350 0x7f49b40a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f49c0011550 tx=0x7f49c0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:59.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.061+0000 7f49cf400640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] conn(0x7f49b40acba0 0x7f49b40aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:59.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.062+0000 7f49c7400640 1 -- 172.21.15.107:0/4012823362 <== osd.0 v2:172.21.15.107:6802/1501799384 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49bc012ec0 con 0x7f49b40acba0 2024-09-19T04:18:59.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.079+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 --> [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49d0062cf0 con 0x7f49b40acba0 2024-09-19T04:18:59.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.080+0000 7f49c7400640 1 -- 172.21.15.107:0/4012823362 <== osd.0 v2:172.21.15.107:6802/1501799384 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f49d0062cf0 con 0x7f49b40acba0 2024-09-19T04:18:59.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.080+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] conn(0x7f49b40acba0 msgr2=0x7f49b40aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.080+0000 7f49d5c00640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] conn(0x7f49b40acba0 0x7f49b40aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:59.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.081+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49b40a3350 msgr2=0x7f49b40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.081+0000 7f49d5c00640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f49b40a3350 0x7f49b40a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f49c0011550 tx=0x7f49c0005f90 comp rx=0 tx=0).stop 2024-09-19T04:18:59.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.081+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49d0103660 msgr2=0x7f49d0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:59.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.081+0000 7f49d5c00640 1 --2- 172.21.15.107:0/4012823362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49d0103660 0x7f49d0100ee0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00e7a0 tx=0x7f49bc00ec60 comp rx=0 tx=0).stop 2024-09-19T04:18:59.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.081+0000 7f49cf400640 1 -- 172.21.15.107:0/4012823362 reap_dead start 2024-09-19T04:18:59.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.081+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 shutdown_connections 2024-09-19T04:18:59.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.081+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 >> 172.21.15.107:0/4012823362 conn(0x7f49d00fc7a0 msgr2=0x7f49d00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:59.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.081+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 shutdown_connections 2024-09-19T04:18:59.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:18:59.081+0000 7f49d5c00640 1 -- 172.21.15.107:0/4012823362 wait complete. 2024-09-19T04:18:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:58 smithi145 ceph-mon[35096]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T04:18:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:58 smithi107 ceph-mon[30659]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T04:18:59.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:58 smithi115 ceph-mon[35477]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T04:19:00.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T04:19:00.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T04:19:00.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-19T04:19:00.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-19T04:19:00.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-19T04:19:00.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-19T04:19:00.210 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:18:59 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T04:19:00.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:59 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T04:19:00.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:59 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T04:19:00.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:59 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-19T04:19:00.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:59 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-19T04:19:00.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:59 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-19T04:19:00.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:59 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-19T04:19:00.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:18:59 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T04:19:00.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.207+0000 7f04dae00640 1 -- 172.21.15.107:0/1089119525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d4102b90 msgr2=0x7f04d4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.207+0000 7f04dae00640 1 --2- 172.21.15.107:0/1089119525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d4102b90 0x7f04d4102f70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009900 tx=0x7f04c802a480 comp rx=0 tx=0).stop 2024-09-19T04:19:00.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.207+0000 7f04dae00640 1 -- 172.21.15.107:0/1089119525 shutdown_connections 2024-09-19T04:19:00.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.207+0000 7f04dae00640 1 --2- 172.21.15.107:0/1089119525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04d410bfc0 0x7f04d410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.207+0000 7f04dae00640 1 --2- 172.21.15.107:0/1089119525 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04d41034b0 0x7f04d410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.207+0000 7f04dae00640 1 --2- 172.21.15.107:0/1089119525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d4102b90 0x7f04d4102f70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.207+0000 7f04dae00640 1 -- 172.21.15.107:0/1089119525 >> 172.21.15.107:0/1089119525 conn(0x7f04d40fc610 msgr2=0x7f04d40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:00.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.207+0000 7f04dae00640 1 -- 172.21.15.107:0/1089119525 shutdown_connections 2024-09-19T04:19:00.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.207+0000 7f04dae00640 1 -- 172.21.15.107:0/1089119525 wait complete. 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.208+0000 7f04dae00640 1 Processor -- start 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.208+0000 7f04dae00640 1 -- start start 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04dae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d4102b90 0x7f04d4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04dae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04d41034b0 0x7f04d4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04dae00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04d410bfc0 0x7f04d419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04dae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4111e10 con 0x7f04d4102b90 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04dae00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4111c90 con 0x7f04d410bfc0 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04dae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4111f90 con 0x7f04d41034b0 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04d9a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d4102b90 0x7f04d4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04d9a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d4102b90 0x7f04d4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53566/0 (socket says 172.21.15.107:53566) 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04d9a00640 1 -- 172.21.15.107:0/86915802 learned_addr learned my addr 172.21.15.107:0/86915802 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04da400640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04d410bfc0 0x7f04d419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:00.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04d9000640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04d41034b0 0x7f04d4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04d9a00640 1 -- 172.21.15.107:0/86915802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04d41034b0 msgr2=0x7f04d4198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04d9a00640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04d41034b0 0x7f04d4198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04d9a00640 1 -- 172.21.15.107:0/86915802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04d410bfc0 msgr2=0x7f04d419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.209+0000 7f04d9a00640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04d410bfc0 0x7f04d419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.210+0000 7f04d9a00640 1 -- 172.21.15.107:0/86915802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d419d2b0 con 0x7f04d4102b90 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.210+0000 7f04da400640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04d410bfc0 0x7f04d419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.210+0000 7f04d9a00640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d4102b90 0x7f04d4198300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f04c8004770 tx=0x7f04c8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.210+0000 7f04d2000640 1 -- 172.21.15.107:0/86915802 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c803f070 con 0x7f04d4102b90 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.210+0000 7f04d2000640 1 -- 172.21.15.107:0/86915802 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c8031ca0 con 0x7f04d4102b90 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.210+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d419d540 con 0x7f04d4102b90 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.210+0000 7f04d2000640 1 -- 172.21.15.107:0/86915802 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8035030 con 0x7f04d4102b90 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.210+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d41a4e80 con 0x7f04d4102b90 2024-09-19T04:19:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.212+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04d4062900 con 0x7f04d4102b90 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.213+0000 7f04d2000640 1 -- 172.21.15.107:0/86915802 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f04c8042070 con 0x7f04d4102b90 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.214+0000 7f04d2000640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f04bc0a3350 0x7f04bc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.215+0000 7f04d2000640 1 -- 172.21.15.107:0/86915802 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f04c80efd80 con 0x7f04d4102b90 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.215+0000 7f04d9000640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f04bc0a3350 0x7f04bc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.215+0000 7f04d2000640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] conn(0x7f04bc0ace80 0x7f04bc0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.215+0000 7f04d2000640 1 -- 172.21.15.107:0/86915802 --> [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04c8004e50 con 0x7f04bc0ace80 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.215+0000 7f04d2000640 1 -- 172.21.15.107:0/86915802 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f04c80f4030 con 0x7f04d4102b90 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.215+0000 7f04d9000640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f04bc0a3350 0x7f04bc0a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f04d410d4b0 tx=0x7f04c000b360 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.215+0000 7f04da400640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] conn(0x7f04bc0ace80 0x7f04bc0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.216+0000 7f04da400640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] conn(0x7f04bc0ace80 0x7f04bc0af320 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.217+0000 7f04d2000640 1 -- 172.21.15.107:0/86915802 <== osd.7 v2:172.21.15.115:6816/278000330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04c8004e50 con 0x7f04bc0ace80 2024-09-19T04:19:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.232+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 --> [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04d4062d30 con 0x7f04bc0ace80 2024-09-19T04:19:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.232+0000 7f04d2000640 1 -- 172.21.15.107:0/86915802 <== osd.7 v2:172.21.15.115:6816/278000330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f04d4062d30 con 0x7f04bc0ace80 2024-09-19T04:19:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 >> [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] conn(0x7f04bc0ace80 msgr2=0x7f04bc0af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04dae00640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] conn(0x7f04bc0ace80 0x7f04bc0af320 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f04bc0a3350 msgr2=0x7f04bc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04dae00640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f04bc0a3350 0x7f04bc0a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f04d410d4b0 tx=0x7f04c000b360 comp rx=0 tx=0).stop 2024-09-19T04:19:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d4102b90 msgr2=0x7f04d4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04dae00640 1 --2- 172.21.15.107:0/86915802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d4102b90 0x7f04d4198300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f04c8004770 tx=0x7f04c8003c10 comp rx=0 tx=0).stop 2024-09-19T04:19:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04da400640 1 -- 172.21.15.107:0/86915802 reap_dead start 2024-09-19T04:19:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 shutdown_connections 2024-09-19T04:19:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 >> 172.21.15.107:0/86915802 conn(0x7f04d40fc610 msgr2=0x7f04d40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 shutdown_connections 2024-09-19T04:19:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.233+0000 7f04dae00640 1 -- 172.21.15.107:0/86915802 wait complete. 2024-09-19T04:19:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:59 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T04:19:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:59 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T04:19:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:59 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-19T04:19:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:59 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-19T04:19:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:59 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-19T04:19:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:59 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-19T04:19:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:18:59 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T04:19:00.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.767+0000 7fd25ae00640 1 -- 172.21.15.107:0/1470262401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd254101de0 msgr2=0x7fd2541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.767+0000 7fd25ae00640 1 --2- 172.21.15.107:0/1470262401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd254101de0 0x7fd2541021c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd244009900 tx=0x7fd24402a410 comp rx=0 tx=0).stop 2024-09-19T04:19:00.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.768+0000 7fd25ae00640 1 -- 172.21.15.107:0/1470262401 shutdown_connections 2024-09-19T04:19:00.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.768+0000 7fd25ae00640 1 --2- 172.21.15.107:0/1470262401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254106f10 0x7fd25410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.768+0000 7fd25ae00640 1 --2- 172.21.15.107:0/1470262401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd254102790 0x7fd2541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.768+0000 7fd25ae00640 1 --2- 172.21.15.107:0/1470262401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd254101de0 0x7fd2541021c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.768+0000 7fd25ae00640 1 -- 172.21.15.107:0/1470262401 >> 172.21.15.107:0/1470262401 conn(0x7fd2540fd8c0 msgr2=0x7fd2540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:00.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.768+0000 7fd25ae00640 1 -- 172.21.15.107:0/1470262401 shutdown_connections 2024-09-19T04:19:00.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.768+0000 7fd25ae00640 1 -- 172.21.15.107:0/1470262401 wait complete. 2024-09-19T04:19:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.769+0000 7fd25ae00640 1 Processor -- start 2024-09-19T04:19:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.769+0000 7fd25ae00640 1 -- start start 2024-09-19T04:19:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.769+0000 7fd25ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd254101de0 0x7fd25419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.769+0000 7fd25ae00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254102790 0x7fd25419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.769+0000 7fd25ae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd254106f10 0x7fd25419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.769+0000 7fd25ae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd254111f70 con 0x7fd254106f10 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.769+0000 7fd25ae00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd254111df0 con 0x7fd254102790 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.769+0000 7fd25ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541120f0 con 0x7fd254101de0 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.769+0000 7fd258e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd254106f10 0x7fd25419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd258e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd254106f10 0x7fd25419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53594/0 (socket says 172.21.15.107:53594) 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd258e00640 1 -- 172.21.15.107:0/1519639970 learned_addr learned my addr 172.21.15.107:0/1519639970 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd24be00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254102790 0x7fd25419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd253e00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd254101de0 0x7fd25419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd24be00640 1 -- 172.21.15.107:0/1519639970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd254101de0 msgr2=0x7fd25419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd24be00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd254101de0 0x7fd25419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd24be00640 1 -- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd254106f10 msgr2=0x7fd25419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd24be00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd254106f10 0x7fd25419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd24be00640 1 -- 172.21.15.107:0/1519639970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25419f5e0 con 0x7fd254102790 2024-09-19T04:19:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd258e00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd254106f10 0x7fd25419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.770+0000 7fd24be00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254102790 0x7fd25419ab70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e9f0 tx=0x7fd23c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.771+0000 7fd251600640 1 -- 172.21.15.107:0/1519639970 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00ccc0 con 0x7fd254102790 2024-09-19T04:19:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.771+0000 7fd251600640 1 -- 172.21.15.107:0/1519639970 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd23c005020 con 0x7fd254102790 2024-09-19T04:19:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.771+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25419f8d0 con 0x7fd254102790 2024-09-19T04:19:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.771+0000 7fd251600640 1 -- 172.21.15.107:0/1519639970 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0049e0 con 0x7fd254102790 2024-09-19T04:19:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.771+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2541a71b0 con 0x7fd254102790 2024-09-19T04:19:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.772+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd25410e150 con 0x7fd254102790 2024-09-19T04:19:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.774+0000 7fd251600640 1 -- 172.21.15.107:0/1519639970 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd23c005560 con 0x7fd254102790 2024-09-19T04:19:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.775+0000 7fd251600640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd2380a32d0 0x7fd2380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.775+0000 7fd251600640 1 -- 172.21.15.107:0/1519639970 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd23c0cfcb0 con 0x7fd254102790 2024-09-19T04:19:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.775+0000 7fd253e00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd2380a32d0 0x7fd2380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.775+0000 7fd251600640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] conn(0x7fd2380ace00 0x7fd2380af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.775+0000 7fd251600640 1 -- 172.21.15.107:0/1519639970 --> [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2380af970 con 0x7fd2380ace00 2024-09-19T04:19:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.775+0000 7fd251600640 1 -- 172.21.15.107:0/1519639970 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fd23c0d0060 con 0x7fd254102790 2024-09-19T04:19:00.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.776+0000 7fd258e00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] conn(0x7fd2380ace00 0x7fd2380af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:00.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.776+0000 7fd253e00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd2380a32d0 0x7fd2380a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd244009230 tx=0x7fd2440058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:00.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.776+0000 7fd258e00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] conn(0x7fd2380ace00 0x7fd2380af2a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:00.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.777+0000 7fd251600640 1 -- 172.21.15.107:0/1519639970 <== osd.6 v2:172.21.15.107:6818/2695066565 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2380af970 con 0x7fd2380ace00 2024-09-19T04:19:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.791+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 --> [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd2541032c0 con 0x7fd2380ace00 2024-09-19T04:19:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd251600640 1 -- 172.21.15.107:0/1519639970 <== osd.6 v2:172.21.15.107:6818/2695066565 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd2541032c0 con 0x7fd2380ace00 2024-09-19T04:19:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] conn(0x7fd2380ace00 msgr2=0x7fd2380af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd25ae00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] conn(0x7fd2380ace00 0x7fd2380af2a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd2380a32d0 msgr2=0x7fd2380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd25ae00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd2380a32d0 0x7fd2380a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd244009230 tx=0x7fd2440058d0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254102790 msgr2=0x7fd25419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd25ae00640 1 --2- 172.21.15.107:0/1519639970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254102790 0x7fd25419ab70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e9f0 tx=0x7fd23c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:19:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd258e00640 1 -- 172.21.15.107:0/1519639970 reap_dead start 2024-09-19T04:19:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 shutdown_connections 2024-09-19T04:19:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 >> 172.21.15.107:0/1519639970 conn(0x7fd2540fd8c0 msgr2=0x7fd2540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.792+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 shutdown_connections 2024-09-19T04:19:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.793+0000 7fd25ae00640 1 -- 172.21.15.107:0/1519639970 wait complete. 2024-09-19T04:19:01.059 INFO:teuthology.orchestra.run.smithi107.stdout:133143986234 2024-09-19T04:19:01.059 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T04:19:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:00 smithi107 ceph-mon[30659]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T04:19:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T04:19:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T04:19:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-19T04:19:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-19T04:19:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-19T04:19:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-19T04:19:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:00 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T04:19:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:00 smithi107 ceph-mon[30659]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:19:01.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.101+0000 7f23f6200640 1 -- 172.21.15.107:0/1643706812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23f01049c0 msgr2=0x7f23f0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.101+0000 7f23f6200640 1 --2- 172.21.15.107:0/1643706812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23f01049c0 0x7f23f0108a10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009900 tx=0x7f23dc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.101+0000 7f23f6200640 1 -- 172.21.15.107:0/1643706812 shutdown_connections 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.101+0000 7f23f6200640 1 --2- 172.21.15.107:0/1643706812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23f0109140 0x7f23f0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.101+0000 7f23f6200640 1 --2- 172.21.15.107:0/1643706812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23f01049c0 0x7f23f0108a10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.101+0000 7f23f6200640 1 --2- 172.21.15.107:0/1643706812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23f0104010 0x7f23f01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.101+0000 7f23f6200640 1 -- 172.21.15.107:0/1643706812 >> 172.21.15.107:0/1643706812 conn(0x7f23f00ffa50 msgr2=0x7f23f0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.101+0000 7f23f6200640 1 -- 172.21.15.107:0/1643706812 shutdown_connections 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.101+0000 7f23f6200640 1 -- 172.21.15.107:0/1643706812 wait complete. 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.102+0000 7f23f6200640 1 Processor -- start 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.102+0000 7f23f6200640 1 -- start start 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.102+0000 7f23f6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23f0104010 0x7f23f019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.102+0000 7f23f6200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23f01049c0 0x7f23f019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.102+0000 7f23f6200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23f0109140 0x7f23f019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.102+0000 7f23f6200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0114190 con 0x7f23f0104010 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.102+0000 7f23f6200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0114010 con 0x7f23f0109140 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.102+0000 7f23f6200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0114310 con 0x7f23f01049c0 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23ef400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23f0104010 0x7f23f019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23ef400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23f0104010 0x7f23f019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53556/0 (socket says 172.21.15.107:53556) 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23ef400640 1 -- 172.21.15.107:0/2184273109 learned_addr learned my addr 172.21.15.107:0/2184273109 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23eea00640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23f01049c0 0x7f23f019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23efe00640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23f0109140 0x7f23f019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23ef400640 1 -- 172.21.15.107:0/2184273109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23f01049c0 msgr2=0x7f23f019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23ef400640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23f01049c0 0x7f23f019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23ef400640 1 -- 172.21.15.107:0/2184273109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23f0109140 msgr2=0x7f23f019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23ef400640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23f0109140 0x7f23f019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23ef400640 1 -- 172.21.15.107:0/2184273109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f019f5c0 con 0x7f23f0104010 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.103+0000 7f23ef400640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23f0104010 0x7f23f019a610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f23e000d950 tx=0x7f23e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.104+0000 7f23e7e00640 1 -- 172.21.15.107:0/2184273109 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0017030 con 0x7f23f0104010 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.104+0000 7f23e7e00640 1 -- 172.21.15.107:0/2184273109 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e0004020 con 0x7f23f0104010 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.104+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f019f8b0 con 0x7f23f0104010 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.104+0000 7f23e7e00640 1 -- 172.21.15.107:0/2184273109 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e00047a0 con 0x7f23f0104010 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.104+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f01a7190 con 0x7f23f0104010 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.105+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23f0062980 con 0x7f23f0104010 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.107+0000 7f23e7e00640 1 -- 172.21.15.107:0/2184273109 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f23e00041d0 con 0x7f23f0104010 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.108+0000 7f23e7e00640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f23d40a3350 0x7f23d40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.108+0000 7f23e7e00640 1 -- 172.21.15.107:0/2184273109 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f23e00cfa80 con 0x7f23f0104010 2024-09-19T04:19:01.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.108+0000 7f23eea00640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f23d40a3350 0x7f23d40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.109+0000 7f23e7e00640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] conn(0x7f23d40ace80 0x7f23d40af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:01.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.109+0000 7f23e7e00640 1 -- 172.21.15.107:0/2184273109 --> [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23d40af9f0 con 0x7f23d40ace80 2024-09-19T04:19:01.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.109+0000 7f23e7e00640 1 -- 172.21.15.107:0/2184273109 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f23e00cfe30 con 0x7f23f0104010 2024-09-19T04:19:01.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.109+0000 7f23efe00640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] conn(0x7f23d40ace80 0x7f23d40af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:01.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.109+0000 7f23eea00640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f23d40a3350 0x7f23d40a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f23dc000c00 tx=0x7f23dc005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:01.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.109+0000 7f23efe00640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] conn(0x7f23d40ace80 0x7f23d40af320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:01.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.110+0000 7f23e7e00640 1 -- 172.21.15.107:0/2184273109 <== osd.3 v2:172.21.15.107:6810/3212660448 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23d40af9f0 con 0x7f23d40ace80 2024-09-19T04:19:01.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.125+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 --> [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f23f0062e00 con 0x7f23d40ace80 2024-09-19T04:19:01.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.125+0000 7f23e7e00640 1 -- 172.21.15.107:0/2184273109 <== osd.3 v2:172.21.15.107:6810/3212660448 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f23f0062e00 con 0x7f23d40ace80 2024-09-19T04:19:01.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.125+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] conn(0x7f23d40ace80 msgr2=0x7f23d40af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.125+0000 7f23f6200640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] conn(0x7f23d40ace80 0x7f23d40af320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:01.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.126+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f23d40a3350 msgr2=0x7f23d40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.126+0000 7f23f6200640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f23d40a3350 0x7f23d40a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f23dc000c00 tx=0x7f23dc005b50 comp rx=0 tx=0).stop 2024-09-19T04:19:01.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.126+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23f0104010 msgr2=0x7f23f019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:01.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.126+0000 7f23f6200640 1 --2- 172.21.15.107:0/2184273109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23f0104010 0x7f23f019a610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f23e000d950 tx=0x7f23e000de10 comp rx=0 tx=0).stop 2024-09-19T04:19:01.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.126+0000 7f23efe00640 1 -- 172.21.15.107:0/2184273109 reap_dead start 2024-09-19T04:19:01.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.126+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 shutdown_connections 2024-09-19T04:19:01.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.126+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 >> 172.21.15.107:0/2184273109 conn(0x7f23f00ffa50 msgr2=0x7f23f0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:01.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.126+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 shutdown_connections 2024-09-19T04:19:01.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:00.126+0000 7f23f6200640 1 -- 172.21.15.107:0/2184273109 wait complete. 2024-09-19T04:19:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:00 smithi115 ceph-mon[35477]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T04:19:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T04:19:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T04:19:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-19T04:19:01.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-19T04:19:01.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-19T04:19:01.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-19T04:19:01.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:00 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T04:19:01.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:00 smithi115 ceph-mon[35477]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:19:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:00 smithi145 ceph-mon[35096]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T04:19:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T04:19:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T04:19:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-19T04:19:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-19T04:19:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-19T04:19:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-19T04:19:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:00 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T04:19:01.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:00 smithi145 ceph-mon[35096]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:19:02.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.031+0000 7f07f5200640 1 -- 172.21.15.107:0/3919096449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103ff0 msgr2=0x7f07f01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:02.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.031+0000 7f07f5200640 1 --2- 172.21.15.107:0/3919096449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103ff0 0x7f07f01043d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009ca0 tx=0x7f07e002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:19:02.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.032+0000 7f07f5200640 1 -- 172.21.15.107:0/3919096449 shutdown_connections 2024-09-19T04:19:02.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.032+0000 7f07f5200640 1 --2- 172.21.15.107:0/3919096449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f0109120 0x7f07f0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:02.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.032+0000 7f07f5200640 1 --2- 172.21.15.107:0/3919096449 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07f01049a0 0x7f07f01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:02.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.032+0000 7f07f5200640 1 --2- 172.21.15.107:0/3919096449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103ff0 0x7f07f01043d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:02.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.032+0000 7f07f5200640 1 -- 172.21.15.107:0/3919096449 >> 172.21.15.107:0/3919096449 conn(0x7f07f00ffa90 msgr2=0x7f07f0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:02.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.032+0000 7f07f5200640 1 -- 172.21.15.107:0/3919096449 shutdown_connections 2024-09-19T04:19:02.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.032+0000 7f07f5200640 1 -- 172.21.15.107:0/3919096449 wait complete. 2024-09-19T04:19:02.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.032+0000 7f07f5200640 1 Processor -- start 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07f5200640 1 -- start start 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07f5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103ff0 0x7f07f0076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07f5200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07f01049a0 0x7f07f0076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07f5200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f0109120 0x7f07f0077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07f5200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07f0114120 con 0x7f07f0103ff0 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07f5200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f07f0113fa0 con 0x7f07f0109120 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07f5200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f07f01142a0 con 0x7f07f01049a0 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07ede00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103ff0 0x7f07f0076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07ede00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103ff0 0x7f07f0076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53652/0 (socket says 172.21.15.107:53652) 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07ede00640 1 -- 172.21.15.107:0/3754654106 learned_addr learned my addr 172.21.15.107:0/3754654106 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07ed400640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07f01049a0 0x7f07f0076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:02.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.033+0000 7f07ee800640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f0109120 0x7f07f0077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.034+0000 7f07ede00640 1 -- 172.21.15.107:0/3754654106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07f01049a0 msgr2=0x7f07f0076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.034+0000 7f07ede00640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07f01049a0 0x7f07f0076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.034+0000 7f07ede00640 1 -- 172.21.15.107:0/3754654106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f0109120 msgr2=0x7f07f0077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.034+0000 7f07ede00640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f0109120 0x7f07f0077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.034+0000 7f07ede00640 1 -- 172.21.15.107:0/3754654106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f01aacc0 con 0x7f07f0103ff0 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.034+0000 7f07ed400640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07f01049a0 0x7f07f0076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.034+0000 7f07ede00640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103ff0 0x7f07f0076700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f07e00097b0 tx=0x7f07e00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.034+0000 7f07e6a00640 1 -- 172.21.15.107:0/3754654106 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e002fa20 con 0x7f07f0103ff0 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.035+0000 7f07e6a00640 1 -- 172.21.15.107:0/3754654106 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e002fbc0 con 0x7f07f0103ff0 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.035+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f01aaf50 con 0x7f07f0103ff0 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.035+0000 7f07e6a00640 1 -- 172.21.15.107:0/3754654106 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e00466e0 con 0x7f07f0103ff0 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.035+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f01ab450 con 0x7f07f0103ff0 2024-09-19T04:19:02.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.035+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07f0062980 con 0x7f07f0103ff0 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.038+0000 7f07e6a00640 1 -- 172.21.15.107:0/3754654106 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f07e00305c0 con 0x7f07f0103ff0 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.039+0000 7f07e6a00640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f07d40a3350 0x7f07d40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.039+0000 7f07e6a00640 1 -- 172.21.15.107:0/3754654106 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f07e00f0f50 con 0x7f07f0103ff0 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.039+0000 7f07ed400640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f07d40a3350 0x7f07d40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.039+0000 7f07e6a00640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] conn(0x7f07d40ace80 0x7f07d40af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.039+0000 7f07e6a00640 1 -- 172.21.15.107:0/3754654106 --> [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07e00020b0 con 0x7f07d40ace80 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.039+0000 7f07e6a00640 1 -- 172.21.15.107:0/3754654106 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f07e00f13a0 con 0x7f07f0103ff0 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.040+0000 7f07ee800640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] conn(0x7f07d40ace80 0x7f07d40af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.040+0000 7f07ed400640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f07d40a3350 0x7f07d40a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f07f0077ec0 tx=0x7f07dc00b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.040+0000 7f07ee800640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] conn(0x7f07d40ace80 0x7f07d40af320 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:02.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.041+0000 7f07e6a00640 1 -- 172.21.15.107:0/3754654106 <== osd.11 v2:172.21.15.145:6824/399346726 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07e00020b0 con 0x7f07d40ace80 2024-09-19T04:19:02.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.055+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 --> [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07f0062e00 con 0x7f07d40ace80 2024-09-19T04:19:02.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.056+0000 7f07e6a00640 1 -- 172.21.15.107:0/3754654106 <== osd.11 v2:172.21.15.145:6824/399346726 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f07f0062e00 con 0x7f07d40ace80 2024-09-19T04:19:02.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.056+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 >> [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] conn(0x7f07d40ace80 msgr2=0x7f07d40af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:02.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.056+0000 7f07f5200640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] conn(0x7f07d40ace80 0x7f07d40af320 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:02.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.056+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f07d40a3350 msgr2=0x7f07d40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:02.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.056+0000 7f07f5200640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f07d40a3350 0x7f07d40a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f07f0077ec0 tx=0x7f07dc00b390 comp rx=0 tx=0).stop 2024-09-19T04:19:02.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.056+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103ff0 msgr2=0x7f07f0076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:02.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.056+0000 7f07f5200640 1 --2- 172.21.15.107:0/3754654106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103ff0 0x7f07f0076700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f07e00097b0 tx=0x7f07e00319f0 comp rx=0 tx=0).stop 2024-09-19T04:19:02.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.056+0000 7f07ee800640 1 -- 172.21.15.107:0/3754654106 reap_dead start 2024-09-19T04:19:02.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.057+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 shutdown_connections 2024-09-19T04:19:02.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.057+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 >> 172.21.15.107:0/3754654106 conn(0x7f07f00ffa90 msgr2=0x7f07f0100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:02.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.057+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 shutdown_connections 2024-09-19T04:19:02.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:02.057+0000 7f07f5200640 1 -- 172.21.15.107:0/3754654106 wait complete. 2024-09-19T04:19:02.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:01 smithi107 ceph-mon[30659]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:19:02.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:01 smithi107 ceph-mon[30659]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-19T04:19:02.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:01 smithi115 ceph-mon[35477]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:19:02.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:01 smithi115 ceph-mon[35477]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-19T04:19:02.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:01 smithi145 ceph-mon[35096]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:19:02.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:01 smithi145 ceph-mon[35096]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-19T04:19:02.710 INFO:teuthology.orchestra.run.smithi107.stdout:34359738449 2024-09-19T04:19:02.710 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T04:19:03.046 INFO:teuthology.orchestra.run.smithi107.stdout:55834574923 2024-09-19T04:19:03.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T04:19:03.391 INFO:teuthology.orchestra.run.smithi107.stdout:154618822711 2024-09-19T04:19:03.391 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T04:19:03.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.743+0000 7fbf4ac00640 1 -- 172.21.15.107:0/308104072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf44077040 msgr2=0x7fbf441124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.743+0000 7fbf4ac00640 1 --2- 172.21.15.107:0/308104072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf44077040 0x7fbf441124f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009900 tx=0x7fbf2c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.744+0000 7fbf4ac00640 1 -- 172.21.15.107:0/308104072 shutdown_connections 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.744+0000 7fbf4ac00640 1 --2- 172.21.15.107:0/308104072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf44112a30 0x7fbf44114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.744+0000 7fbf4ac00640 1 --2- 172.21.15.107:0/308104072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf44077040 0x7fbf441124f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.744+0000 7fbf4ac00640 1 --2- 172.21.15.107:0/308104072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf44076720 0x7fbf44076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.744+0000 7fbf4ac00640 1 -- 172.21.15.107:0/308104072 >> 172.21.15.107:0/308104072 conn(0x7fbf440ffa90 msgr2=0x7fbf44101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.744+0000 7fbf4ac00640 1 -- 172.21.15.107:0/308104072 shutdown_connections 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.744+0000 7fbf4ac00640 1 -- 172.21.15.107:0/308104072 wait complete. 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf4ac00640 1 Processor -- start 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf4ac00640 1 -- start start 2024-09-19T04:19:03.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf4ac00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf44076720 0x7fbf441a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf4ac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf44077040 0x7fbf441a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf4ac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf44112a30 0x7fbf4419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf4ac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441185c0 con 0x7fbf44112a30 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf4ac00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44118440 con 0x7fbf44076720 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf4ac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44118740 con 0x7fbf44077040 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf48c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf44112a30 0x7fbf4419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf48c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf44112a30 0x7fbf4419e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53632/0 (socket says 172.21.15.107:53632) 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.745+0000 7fbf48c00640 1 -- 172.21.15.107:0/1702339965 learned_addr learned my addr 172.21.15.107:0/1702339965 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf43e00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf44076720 0x7fbf441a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf3be00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf44077040 0x7fbf441a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf48c00640 1 -- 172.21.15.107:0/1702339965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf44077040 msgr2=0x7fbf441a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf48c00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf44077040 0x7fbf441a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf48c00640 1 -- 172.21.15.107:0/1702339965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf44076720 msgr2=0x7fbf441a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf48c00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf44076720 0x7fbf441a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf48c00640 1 -- 172.21.15.107:0/1702339965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4419f000 con 0x7fbf44112a30 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf48c00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf44112a30 0x7fbf4419e8d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009fd0 tx=0x7fbf3c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf41600640 1 -- 172.21.15.107:0/1702339965 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c004d20 con 0x7fbf44112a30 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf41600640 1 -- 172.21.15.107:0/1702339965 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf3c005020 con 0x7fbf44112a30 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.746+0000 7fbf41600640 1 -- 172.21.15.107:0/1702339965 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c0116d0 con 0x7fbf44112a30 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.747+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4419f2f0 con 0x7fbf44112a30 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.747+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf441ab5e0 con 0x7fbf44112a30 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.748+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf44114a30 con 0x7fbf44112a30 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.750+0000 7fbf41600640 1 -- 172.21.15.107:0/1702339965 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbf3c0051d0 con 0x7fbf44112a30 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.751+0000 7fbf41600640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf280a32d0 0x7fbf280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.751+0000 7fbf41600640 1 -- 172.21.15.107:0/1702339965 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fbf3c0d0140 con 0x7fbf44112a30 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.751+0000 7fbf43e00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf280a32d0 0x7fbf280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:03.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.751+0000 7fbf41600640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] conn(0x7fbf280ace00 0x7fbf280af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.751+0000 7fbf41600640 1 -- 172.21.15.107:0/1702339965 --> [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf3c014ec0 con 0x7fbf280ace00 2024-09-19T04:19:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.751+0000 7fbf41600640 1 -- 172.21.15.107:0/1702339965 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fbf3c0d04f0 con 0x7fbf44112a30 2024-09-19T04:19:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.752+0000 7fbf3be00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] conn(0x7fbf280ace00 0x7fbf280af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.752+0000 7fbf43e00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf280a32d0 0x7fbf280a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbf4419fa10 tx=0x7fbf34005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.752+0000 7fbf3be00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] conn(0x7fbf280ace00 0x7fbf280af2a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.753+0000 7fbf41600640 1 -- 172.21.15.107:0/1702339965 <== osd.10 v2:172.21.15.115:6824/442507474 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf3c014ec0 con 0x7fbf280ace00 2024-09-19T04:19:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.767+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 --> [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf44077e60 con 0x7fbf280ace00 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.768+0000 7fbf41600640 1 -- 172.21.15.107:0/1702339965 <== osd.10 v2:172.21.15.115:6824/442507474 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbf44077e60 con 0x7fbf280ace00 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.768+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 >> [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] conn(0x7fbf280ace00 msgr2=0x7fbf280af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.768+0000 7fbf4ac00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] conn(0x7fbf280ace00 0x7fbf280af2a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.768+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf280a32d0 msgr2=0x7fbf280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.768+0000 7fbf4ac00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf280a32d0 0x7fbf280a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbf4419fa10 tx=0x7fbf34005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.768+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf44112a30 msgr2=0x7fbf4419e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.768+0000 7fbf4ac00640 1 --2- 172.21.15.107:0/1702339965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf44112a30 0x7fbf4419e8d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009fd0 tx=0x7fbf3c00ee60 comp rx=0 tx=0).stop 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.768+0000 7fbf48c00640 1 -- 172.21.15.107:0/1702339965 reap_dead start 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.768+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 shutdown_connections 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.768+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 >> 172.21.15.107:0/1702339965 conn(0x7fbf440ffa90 msgr2=0x7fbf44101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.769+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 shutdown_connections 2024-09-19T04:19:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.769+0000 7fbf4ac00640 1 -- 172.21.15.107:0/1702339965 wait complete. 2024-09-19T04:19:04.368 INFO:teuthology.orchestra.run.smithi107.stdout:292057776156 2024-09-19T04:19:04.369 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T04:19:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.017+0000 7f92bbe00640 1 -- 172.21.15.107:0/2334006454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92bc074ee0 msgr2=0x7f92bc10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.017+0000 7f92bbe00640 1 --2- 172.21.15.107:0/2334006454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92bc074ee0 0x7f92bc10dd20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009900 tx=0x7f92a802a480 comp rx=0 tx=0).stop 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.017+0000 7f92bbe00640 1 -- 172.21.15.107:0/2334006454 shutdown_connections 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.017+0000 7f92bbe00640 1 --2- 172.21.15.107:0/2334006454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92bc10e260 0x7f92bc110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.017+0000 7f92bbe00640 1 --2- 172.21.15.107:0/2334006454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92bc074ee0 0x7f92bc10dd20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.017+0000 7f92bbe00640 1 --2- 172.21.15.107:0/2334006454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92bc0745c0 0x7f92bc0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.017+0000 7f92bbe00640 1 -- 172.21.15.107:0/2334006454 >> 172.21.15.107:0/2334006454 conn(0x7f92bc0fd740 msgr2=0x7f92bc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.017+0000 7f92bbe00640 1 -- 172.21.15.107:0/2334006454 shutdown_connections 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.017+0000 7f92bbe00640 1 -- 172.21.15.107:0/2334006454 wait complete. 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.018+0000 7f92bbe00640 1 Processor -- start 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.018+0000 7f92bbe00640 1 -- start start 2024-09-19T04:19:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92bbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92bc0745c0 0x7f92bc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92bc074ee0 0x7f92bc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92bbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92bc10e260 0x7f92bc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92bbe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc113fb0 con 0x7f92bc074ee0 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92bbe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc113e30 con 0x7f92bc10e260 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92bbe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc114130 con 0x7f92bc0745c0 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92ba000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92bc074ee0 0x7f92bc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92baa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92bc0745c0 0x7f92bc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92ba000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92bc074ee0 0x7f92bc19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53602/0 (socket says 172.21.15.107:53602) 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92ba000640 1 -- 172.21.15.107:0/2461714339 learned_addr learned my addr 172.21.15.107:0/2461714339 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.019+0000 7f92bb400640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92bc10e260 0x7f92bc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92baa00640 1 -- 172.21.15.107:0/2461714339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92bc10e260 msgr2=0x7f92bc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92baa00640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92bc10e260 0x7f92bc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92baa00640 1 -- 172.21.15.107:0/2461714339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92bc074ee0 msgr2=0x7f92bc19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92baa00640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92bc074ee0 0x7f92bc19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92baa00640 1 -- 172.21.15.107:0/2461714339 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92bc19f460 con 0x7f92bc0745c0 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92baa00640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92bc0745c0 0x7f92bc19a4b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00e9f0 tx=0x7f92ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92b3400640 1 -- 172.21.15.107:0/2461714339 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00ccc0 con 0x7f92bc0745c0 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92bc19f750 con 0x7f92bc0745c0 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92b3400640 1 -- 172.21.15.107:0/2461714339 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92ac005020 con 0x7f92bc0745c0 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92b3400640 1 -- 172.21.15.107:0/2461714339 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0049e0 con 0x7f92bc0745c0 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.020+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92bc1a7030 con 0x7f92bc0745c0 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.021+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f92bc062980 con 0x7f92bc0745c0 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.024+0000 7f92b3400640 1 -- 172.21.15.107:0/2461714339 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f92ac005560 con 0x7f92bc0745c0 2024-09-19T04:19:04.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.026+0000 7f92b3400640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92a00a32d0 0x7f92a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.026+0000 7f92b3400640 1 -- 172.21.15.107:0/2461714339 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f92ac0cfea0 con 0x7f92bc0745c0 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.026+0000 7f92ba000640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92a00a32d0 0x7f92a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.026+0000 7f92b3400640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] conn(0x7f92a00ace00 0x7f92a00af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.026+0000 7f92b3400640 1 -- 172.21.15.107:0/2461714339 --> [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92ac00e1a0 con 0x7f92a00ace00 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.026+0000 7f92b3400640 1 -- 172.21.15.107:0/2461714339 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f92ac0d0250 con 0x7f92bc0745c0 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.027+0000 7f92bb400640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] conn(0x7f92a00ace00 0x7f92a00af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.027+0000 7f92ba000640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92a00a32d0 0x7f92a00a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92a8031550 tx=0x7f92a8005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.027+0000 7f92bb400640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] conn(0x7f92a00ace00 0x7f92a00af2a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.028+0000 7f92b3400640 1 -- 172.21.15.107:0/2461714339 <== osd.8 v2:172.21.15.145:6816/2759275478 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92ac00e1a0 con 0x7f92a00ace00 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.043+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 --> [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92bc062e00 con 0x7f92a00ace00 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.043+0000 7f92b3400640 1 -- 172.21.15.107:0/2461714339 <== osd.8 v2:172.21.15.145:6816/2759275478 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f92bc062e00 con 0x7f92a00ace00 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.044+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 >> [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] conn(0x7f92a00ace00 msgr2=0x7f92a00af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.044+0000 7f92bbe00640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] conn(0x7f92a00ace00 0x7f92a00af2a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.044+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92a00a32d0 msgr2=0x7f92a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.044+0000 7f92bbe00640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f92a00a32d0 0x7f92a00a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92a8031550 tx=0x7f92a8005a20 comp rx=0 tx=0).stop 2024-09-19T04:19:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.044+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92bc0745c0 msgr2=0x7f92bc19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.044+0000 7f92bbe00640 1 --2- 172.21.15.107:0/2461714339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92bc0745c0 0x7f92bc19a4b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00e9f0 tx=0x7f92ac00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:19:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.044+0000 7f92bb400640 1 -- 172.21.15.107:0/2461714339 reap_dead start 2024-09-19T04:19:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.044+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 shutdown_connections 2024-09-19T04:19:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.044+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 >> 172.21.15.107:0/2461714339 conn(0x7f92bc0fd740 msgr2=0x7f92bc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.044+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 shutdown_connections 2024-09-19T04:19:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:01.045+0000 7f92bbe00640 1 -- 172.21.15.107:0/2461714339 wait complete. 2024-09-19T04:19:05.154 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:04 smithi107 ceph-mon[30659]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:04 smithi115 ceph-mon[35477]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:04 smithi145 ceph-mon[35096]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:05.650 INFO:teuthology.orchestra.run.smithi107.stdout:322122547223 2024-09-19T04:19:05.650 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T04:19:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:236223201319 2024-09-19T04:19:06.237 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T04:19:06.702 INFO:teuthology.orchestra.run.smithi107.stdout:206158430251 2024-09-19T04:19:06.702 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T04:19:06.737 INFO:teuthology.orchestra.run.smithi107.stdout:103079215168 2024-09-19T04:19:06.737 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T04:19:06.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:06 smithi107 ceph-mon[30659]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:07.110 INFO:teuthology.orchestra.run.smithi107.stdout:184683593777 2024-09-19T04:19:07.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T04:19:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:06 smithi115 ceph-mon[35477]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:06 smithi145 ceph-mon[35096]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:07.244 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:07.276 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:07.276 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:07.299 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:07.312 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:08.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.073+0000 7fd1fd000640 1 -- 172.21.15.107:0/2617456968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1f8074eb0 msgr2=0x7fd1f8112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:08.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.073+0000 7fd1fd000640 1 --2- 172.21.15.107:0/2617456968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1f8074eb0 0x7fd1f8112650 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009900 tx=0x7fd1e402a480 comp rx=0 tx=0).stop 2024-09-19T04:19:08.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.073+0000 7fd1fd000640 1 -- 172.21.15.107:0/2617456968 shutdown_connections 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.073+0000 7fd1fd000640 1 --2- 172.21.15.107:0/2617456968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1f8112b90 0x7fd1f8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.073+0000 7fd1fd000640 1 --2- 172.21.15.107:0/2617456968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1f8074eb0 0x7fd1f8112650 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.073+0000 7fd1fd000640 1 --2- 172.21.15.107:0/2617456968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1f8074590 0x7fd1f8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.073+0000 7fd1fd000640 1 -- 172.21.15.107:0/2617456968 >> 172.21.15.107:0/2617456968 conn(0x7fd1f80fd8c0 msgr2=0x7fd1f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.073+0000 7fd1fd000640 1 -- 172.21.15.107:0/2617456968 shutdown_connections 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.073+0000 7fd1fd000640 1 -- 172.21.15.107:0/2617456968 wait complete. 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.074+0000 7fd1fd000640 1 Processor -- start 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.074+0000 7fd1fd000640 1 -- start start 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1fd000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1f8074590 0x7fd1f81a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1fd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1f8074eb0 0x7fd1f81a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1fd000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1f8112b90 0x7fd1f819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:08.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1fd000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f81185b0 con 0x7fd1f8074eb0 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1fd000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8118430 con 0x7fd1f8074590 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1fd000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8118730 con 0x7fd1f8112b90 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1f5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1f8074eb0 0x7fd1f81a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1f5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1f8074eb0 0x7fd1f81a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53668/0 (socket says 172.21.15.107:53668) 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1f5400640 1 -- 172.21.15.107:0/2370054959 learned_addr learned my addr 172.21.15.107:0/2370054959 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1f6800640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1f8112b90 0x7fd1f819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1f5e00640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1f8074590 0x7fd1f81a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1f5400640 1 -- 172.21.15.107:0/2370054959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1f8112b90 msgr2=0x7fd1f819e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1f5400640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1f8112b90 0x7fd1f819e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1f5400640 1 -- 172.21.15.107:0/2370054959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1f8074590 msgr2=0x7fd1f81a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1f5400640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1f8074590 0x7fd1f81a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.075+0000 7fd1f5400640 1 -- 172.21.15.107:0/2370054959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f819f030 con 0x7fd1f8074eb0 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.076+0000 7fd1f5400640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1f8074eb0 0x7fd1f81a4c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4004770 tx=0x7fd1e4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:08.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.076+0000 7fd1eea00640 1 -- 172.21.15.107:0/2370054959 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e403f070 con 0x7fd1f8074eb0 2024-09-19T04:19:08.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.076+0000 7fd1eea00640 1 -- 172.21.15.107:0/2370054959 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e4031ca0 con 0x7fd1f8074eb0 2024-09-19T04:19:08.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.076+0000 7fd1fd000640 1 -- 172.21.15.107:0/2370054959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f819f2c0 con 0x7fd1f8074eb0 2024-09-19T04:19:08.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.076+0000 7fd1eea00640 1 -- 172.21.15.107:0/2370054959 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4035030 con 0x7fd1f8074eb0 2024-09-19T04:19:08.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.076+0000 7fd1fd000640 1 -- 172.21.15.107:0/2370054959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f81ab610 con 0x7fd1f8074eb0 2024-09-19T04:19:08.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.077+0000 7fd1fd000640 1 -- 172.21.15.107:0/2370054959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f8074970 con 0x7fd1f8074eb0 2024-09-19T04:19:08.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.079+0000 7fd1eea00640 1 -- 172.21.15.107:0/2370054959 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd1e4042070 con 0x7fd1f8074eb0 2024-09-19T04:19:08.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.080+0000 7fd1eea00640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd1dc0a32d0 0x7fd1dc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:08.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.081+0000 7fd1eea00640 1 -- 172.21.15.107:0/2370054959 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fd1e40f0170 con 0x7fd1f8074eb0 2024-09-19T04:19:08.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.081+0000 7fd1f5e00640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd1dc0a32d0 0x7fd1dc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:08.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.081+0000 7fd1f5e00640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd1dc0a32d0 0x7fd1dc0a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd1f819fa40 tx=0x7fd1e800b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:08.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.084+0000 7fd1eea00640 1 -- 172.21.15.107:0/2370054959 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd1e40bc6b0 con 0x7fd1f8074eb0 2024-09-19T04:19:08.093 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:08.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.271+0000 7fd1fd000640 1 -- 172.21.15.107:0/2370054959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd1f8105000 con 0x7fd1f8074eb0 2024-09-19T04:19:08.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.272+0000 7fd1eea00640 1 -- 172.21.15.107:0/2370054959 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd1e40c09b0 con 0x7fd1f8074eb0 2024-09-19T04:19:08.273 INFO:teuthology.orchestra.run.smithi107.stdout:292057776157 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 -- 172.21.15.107:0/2370054959 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd1dc0a32d0 msgr2=0x7fd1dc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd1dc0a32d0 0x7fd1dc0a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd1f819fa40 tx=0x7fd1e800b410 comp rx=0 tx=0).stop 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 -- 172.21.15.107:0/2370054959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1f8074eb0 msgr2=0x7fd1f81a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1f8074eb0 0x7fd1f81a4c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4004770 tx=0x7fd1e4003c10 comp rx=0 tx=0).stop 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 -- 172.21.15.107:0/2370054959 shutdown_connections 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fd1dc0a32d0 0x7fd1dc0a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1f8112b90 0x7fd1f819e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1f8074eb0 0x7fd1f81a4c80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 --2- 172.21.15.107:0/2370054959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1f8074590 0x7fd1f81a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 -- 172.21.15.107:0/2370054959 >> 172.21.15.107:0/2370054959 conn(0x7fd1f80fd8c0 msgr2=0x7fd1f80ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 -- 172.21.15.107:0/2370054959 shutdown_connections 2024-09-19T04:19:08.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.274+0000 7fd1fd000640 1 -- 172.21.15.107:0/2370054959 wait complete. 2024-09-19T04:19:08.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.301+0000 7f40f7e00640 1 -- 172.21.15.107:0/1797729412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8074eb0 msgr2=0x7f40f81124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:08.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.301+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1797729412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8074eb0 0x7f40f81124a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009900 tx=0x7f40e402a480 comp rx=0 tx=0).stop 2024-09-19T04:19:08.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.301+0000 7f40f7e00640 1 -- 172.21.15.107:0/1797729412 shutdown_connections 2024-09-19T04:19:08.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.301+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1797729412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f81129e0 0x7f40f8114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.301+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1797729412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8074eb0 0x7f40f81124a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.301+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1797729412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8074590 0x7f40f8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.301+0000 7f40f7e00640 1 -- 172.21.15.107:0/1797729412 >> 172.21.15.107:0/1797729412 conn(0x7f40f80fd710 msgr2=0x7f40f80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:08.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.301+0000 7f40f7e00640 1 -- 172.21.15.107:0/1797729412 shutdown_connections 2024-09-19T04:19:08.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.301+0000 7f40f7e00640 1 -- 172.21.15.107:0/1797729412 wait complete. 2024-09-19T04:19:08.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.302+0000 7f40f7e00640 1 Processor -- start 2024-09-19T04:19:08.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.302+0000 7f40f7e00640 1 -- start start 2024-09-19T04:19:08.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.302+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8074590 0x7f40f81a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:08.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.302+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8074eb0 0x7f40f81a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:08.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.302+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f81129e0 0x7f40f819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:08.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.302+0000 7f40f7e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81183b0 con 0x7f40f8074eb0 2024-09-19T04:19:08.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.302+0000 7f40f7e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8118230 con 0x7f40f81129e0 2024-09-19T04:19:08.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.302+0000 7f40f7e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8118530 con 0x7f40f8074590 2024-09-19T04:19:08.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8074590 0x7f40f81a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:08.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8074eb0 0x7f40f81a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:08.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8074590 0x7f40f81a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33062/0 (socket says 172.21.15.107:33062) 2024-09-19T04:19:08.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f6a00640 1 -- 172.21.15.107:0/1663414315 learned_addr learned my addr 172.21.15.107:0/1663414315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:08.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f7400640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f81129e0 0x7f40f819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:08.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f6a00640 1 -- 172.21.15.107:0/1663414315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f81129e0 msgr2=0x7f40f819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f6a00640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f81129e0 0x7f40f819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f6a00640 1 -- 172.21.15.107:0/1663414315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8074eb0 msgr2=0x7f40f81a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f6a00640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8074eb0 0x7f40f81a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f6a00640 1 -- 172.21.15.107:0/1663414315 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f819ef50 con 0x7f40f8074590 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.303+0000 7f40f6a00640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8074590 0x7f40f81a4630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e9f0 tx=0x7f40e800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.304+0000 7f40fca00640 1 -- 172.21.15.107:0/1663414315 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e800cd80 con 0x7f40f8074590 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.304+0000 7f40fca00640 1 -- 172.21.15.107:0/1663414315 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8005500 con 0x7f40f8074590 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.304+0000 7f40f7e00640 1 -- 172.21.15.107:0/1663414315 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f819f240 con 0x7f40f8074590 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.304+0000 7f40fca00640 1 -- 172.21.15.107:0/1663414315 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8010690 con 0x7f40f8074590 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.304+0000 7f40f7e00640 1 -- 172.21.15.107:0/1663414315 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f81ab490 con 0x7f40f8074590 2024-09-19T04:19:08.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.305+0000 7f40f7e00640 1 -- 172.21.15.107:0/1663414315 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f8075cd0 con 0x7f40f8074590 2024-09-19T04:19:08.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.307+0000 7f40fca00640 1 -- 172.21.15.107:0/1663414315 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f40e80046c0 con 0x7f40f8074590 2024-09-19T04:19:08.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.308+0000 7f40fca00640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f40dc0a3350 0x7f40dc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:08.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.308+0000 7f40fca00640 1 -- 172.21.15.107:0/1663414315 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f40e80cfe50 con 0x7f40f8074590 2024-09-19T04:19:08.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.308+0000 7f40f6000640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f40dc0a3350 0x7f40dc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:08.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.309+0000 7f40f6000640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f40dc0a3350 0x7f40dc0a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009230 tx=0x7f40e40058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:08.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.312+0000 7f40fca00640 1 -- 172.21.15.107:0/1663414315 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f40e809c9e0 con 0x7f40f8074590 2024-09-19T04:19:08.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.499+0000 7f40f7e00640 1 -- 172.21.15.107:0/1663414315 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f40f8074970 con 0x7f40f8074590 2024-09-19T04:19:08.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.499+0000 7f40fca00640 1 -- 172.21.15.107:0/1663414315 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f40e809c2e0 con 0x7f40f8074590 2024-09-19T04:19:08.501 INFO:teuthology.orchestra.run.smithi107.stdout:133143986236 2024-09-19T04:19:08.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 -- 172.21.15.107:0/1663414315 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f40dc0a3350 msgr2=0x7f40dc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:08.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f40dc0a3350 0x7f40dc0a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009230 tx=0x7f40e40058d0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 -- 172.21.15.107:0/1663414315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8074590 msgr2=0x7f40f81a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:08.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8074590 0x7f40f81a4630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e9f0 tx=0x7f40e800edf0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 -- 172.21.15.107:0/1663414315 shutdown_connections 2024-09-19T04:19:08.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f40dc0a3350 0x7f40dc0a5810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f81129e0 0x7f40f819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8074eb0 0x7f40f81a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1663414315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8074590 0x7f40f81a4630 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:08.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 -- 172.21.15.107:0/1663414315 >> 172.21.15.107:0/1663414315 conn(0x7f40f80fd710 msgr2=0x7f40f80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:08.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 -- 172.21.15.107:0/1663414315 shutdown_connections 2024-09-19T04:19:08.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:08.502+0000 7f40f7e00640 1 -- 172.21.15.107:0/1663414315 wait complete. 2024-09-19T04:19:09.158 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:08 smithi107 ceph-mon[30659]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:09.158 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:08 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2370054959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:19:09.158 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:08 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1663414315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:19:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:08 smithi115 ceph-mon[35477]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:08 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2370054959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:19:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:08 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1663414315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:19:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:08 smithi145 ceph-mon[35096]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:08 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2370054959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:19:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:08 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1663414315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:19:09.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.704+0000 7fee34e00640 1 -- 172.21.15.107:0/1631445561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee30108f60 msgr2=0x7fee30110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:09.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.704+0000 7fee34e00640 1 --2- 172.21.15.107:0/1631445561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee30108f60 0x7fee30110ae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee28009ca0 tx=0x7fee2802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:19:09.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.704+0000 7fee34e00640 1 -- 172.21.15.107:0/1631445561 shutdown_connections 2024-09-19T04:19:09.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.704+0000 7fee34e00640 1 --2- 172.21.15.107:0/1631445561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee30108f60 0x7fee30110ae0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.704+0000 7fee34e00640 1 --2- 172.21.15.107:0/1631445561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee301047e0 0x7fee30108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.704+0000 7fee34e00640 1 --2- 172.21.15.107:0/1631445561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee30103e30 0x7fee30104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.704+0000 7fee34e00640 1 -- 172.21.15.107:0/1631445561 >> 172.21.15.107:0/1631445561 conn(0x7fee300ff8d0 msgr2=0x7fee30101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.704+0000 7fee34e00640 1 -- 172.21.15.107:0/1631445561 shutdown_connections 2024-09-19T04:19:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.704+0000 7fee34e00640 1 -- 172.21.15.107:0/1631445561 wait complete. 2024-09-19T04:19:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.705+0000 7fee34e00640 1 Processor -- start 2024-09-19T04:19:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.705+0000 7fee34e00640 1 -- start start 2024-09-19T04:19:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee34e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee30103e30 0x7fee3019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee34e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee301047e0 0x7fee3019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee34e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee30108f60 0x7fee3019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee34e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee30113fc0 con 0x7fee30103e30 2024-09-19T04:19:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee34e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fee30113e40 con 0x7fee301047e0 2024-09-19T04:19:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee34e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fee30114140 con 0x7fee30108f60 2024-09-19T04:19:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee2f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee30103e30 0x7fee3019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee2f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee30103e30 0x7fee3019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36852/0 (socket says 172.21.15.107:36852) 2024-09-19T04:19:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee2f400640 1 -- 172.21.15.107:0/2592031051 learned_addr learned my addr 172.21.15.107:0/2592031051 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee2ea00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee301047e0 0x7fee3019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.706+0000 7fee2fe00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee30108f60 0x7fee3019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee2f400640 1 -- 172.21.15.107:0/2592031051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee30108f60 msgr2=0x7fee3019ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee2f400640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee30108f60 0x7fee3019ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee2f400640 1 -- 172.21.15.107:0/2592031051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee301047e0 msgr2=0x7fee3019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee2f400640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee301047e0 0x7fee3019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee2f400640 1 -- 172.21.15.107:0/2592031051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3019f390 con 0x7fee30103e30 2024-09-19T04:19:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee2f400640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee30103e30 0x7fee3019a3e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fee2000ea60 tx=0x7fee2000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee27e00640 1 -- 172.21.15.107:0/2592031051 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2000cde0 con 0x7fee30103e30 2024-09-19T04:19:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee27e00640 1 -- 172.21.15.107:0/2592031051 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee200035f0 con 0x7fee30103e30 2024-09-19T04:19:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee34e00640 1 -- 172.21.15.107:0/2592031051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3019f680 con 0x7fee30103e30 2024-09-19T04:19:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee27e00640 1 -- 172.21.15.107:0/2592031051 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20010670 con 0x7fee30103e30 2024-09-19T04:19:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.707+0000 7fee34e00640 1 -- 172.21.15.107:0/2592031051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee301a6f60 con 0x7fee30103e30 2024-09-19T04:19:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.709+0000 7fee34e00640 1 -- 172.21.15.107:0/2592031051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee30104210 con 0x7fee30103e30 2024-09-19T04:19:09.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.710+0000 7fee27e00640 1 -- 172.21.15.107:0/2592031051 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fee200037a0 con 0x7fee30103e30 2024-09-19T04:19:09.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.712+0000 7fee27e00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fee140a3350 0x7fee140a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:09.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.712+0000 7fee27e00640 1 -- 172.21.15.107:0/2592031051 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fee200cfd10 con 0x7fee30103e30 2024-09-19T04:19:09.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.712+0000 7fee2ea00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fee140a3350 0x7fee140a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:09.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.713+0000 7fee2ea00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fee140a3350 0x7fee140a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0095f0 tx=0x7fee1c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:09.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.716+0000 7fee27e00640 1 -- 172.21.15.107:0/2592031051 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fee200d5020 con 0x7fee30103e30 2024-09-19T04:19:09.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.902+0000 7fee34e00640 1 -- 172.21.15.107:0/2592031051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fee3010bbb0 con 0x7fee30103e30 2024-09-19T04:19:09.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.903+0000 7fee27e00640 1 -- 172.21.15.107:0/2592031051 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fee2009c8a0 con 0x7fee30103e30 2024-09-19T04:19:09.905 INFO:teuthology.orchestra.run.smithi107.stdout:34359738451 2024-09-19T04:19:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 -- 172.21.15.107:0/2592031051 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fee140a3350 msgr2=0x7fee140a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fee140a3350 0x7fee140a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0095f0 tx=0x7fee1c005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 -- 172.21.15.107:0/2592031051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee30103e30 msgr2=0x7fee3019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee30103e30 0x7fee3019a3e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fee2000ea60 tx=0x7fee2000ef20 comp rx=0 tx=0).stop 2024-09-19T04:19:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 -- 172.21.15.107:0/2592031051 shutdown_connections 2024-09-19T04:19:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fee140a3350 0x7fee140a5810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee30108f60 0x7fee3019ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee301047e0 0x7fee3019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 --2- 172.21.15.107:0/2592031051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee30103e30 0x7fee3019a3e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 -- 172.21.15.107:0/2592031051 >> 172.21.15.107:0/2592031051 conn(0x7fee300ff8d0 msgr2=0x7fee30100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:09.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 -- 172.21.15.107:0/2592031051 shutdown_connections 2024-09-19T04:19:09.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:09.905+0000 7fee34e00640 1 -- 172.21.15.107:0/2592031051 wait complete. 2024-09-19T04:19:09.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:09 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:19:10.100 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:09 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.139+0000 7f8762400640 1 -- 172.21.15.107:0/4137640412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f875c1049c0 msgr2=0x7f875c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.139+0000 7f8762400640 1 --2- 172.21.15.107:0/4137640412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f875c1049c0 0x7f875c108a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8748009900 tx=0x7f874802a410 comp rx=0 tx=0).stop 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.139+0000 7f8762400640 1 -- 172.21.15.107:0/4137640412 shutdown_connections 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.139+0000 7f8762400640 1 --2- 172.21.15.107:0/4137640412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875c109140 0x7f875c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.139+0000 7f8762400640 1 --2- 172.21.15.107:0/4137640412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f875c1049c0 0x7f875c108a10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.139+0000 7f8762400640 1 --2- 172.21.15.107:0/4137640412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f875c104010 0x7f875c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.139+0000 7f8762400640 1 -- 172.21.15.107:0/4137640412 >> 172.21.15.107:0/4137640412 conn(0x7f875c0ffa50 msgr2=0x7f875c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.139+0000 7f8762400640 1 -- 172.21.15.107:0/4137640412 shutdown_connections 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.139+0000 7f8762400640 1 -- 172.21.15.107:0/4137640412 wait complete. 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.140+0000 7f8762400640 1 Processor -- start 2024-09-19T04:19:10.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.140+0000 7f8762400640 1 -- start start 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f8762400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f875c104010 0x7f875c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f8762400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f875c1049c0 0x7f875c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f8762400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875c109140 0x7f875c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f8762400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f875c114140 con 0x7f875c109140 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f8762400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f875c113fc0 con 0x7f875c1049c0 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f8762400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f875c1142c0 con 0x7f875c104010 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f875b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f875c104010 0x7f875c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f875be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875c109140 0x7f875c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f875b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f875c104010 0x7f875c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38964/0 (socket says 172.21.15.107:38964) 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f875b400640 1 -- 172.21.15.107:0/2893117407 learned_addr learned my addr 172.21.15.107:0/2893117407 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f875aa00640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f875c1049c0 0x7f875c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f875b400640 1 -- 172.21.15.107:0/2893117407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f875c1049c0 msgr2=0x7f875c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:10.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f875b400640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f875c1049c0 0x7f875c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f875b400640 1 -- 172.21.15.107:0/2893117407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875c109140 msgr2=0x7f875c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f875b400640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875c109140 0x7f875c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.141+0000 7f875b400640 1 -- 172.21.15.107:0/2893117407 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c19f5f0 con 0x7f875c104010 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.142+0000 7f875b400640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f875c104010 0x7f875c19a640 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f874c00ea60 tx=0x7f874c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.142+0000 7f8753e00640 1 -- 172.21.15.107:0/2893117407 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c00cde0 con 0x7f875c104010 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.142+0000 7f8762400640 1 -- 172.21.15.107:0/2893117407 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875c19f8e0 con 0x7f875c104010 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.142+0000 7f8753e00640 1 -- 172.21.15.107:0/2893117407 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f874c0035f0 con 0x7f875c104010 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.142+0000 7f8753e00640 1 -- 172.21.15.107:0/2893117407 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c010670 con 0x7f875c104010 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.142+0000 7f8762400640 1 -- 172.21.15.107:0/2893117407 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875c1a71c0 con 0x7f875c104010 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.143+0000 7f8762400640 1 -- 172.21.15.107:0/2893117407 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f875c1043f0 con 0x7f875c104010 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.145+0000 7f8753e00640 1 -- 172.21.15.107:0/2893117407 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f874c0037a0 con 0x7f875c104010 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.146+0000 7f8753e00640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f87400a32d0 0x7f87400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.146+0000 7f8753e00640 1 -- 172.21.15.107:0/2893117407 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f874c0cfd40 con 0x7f875c104010 2024-09-19T04:19:10.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.146+0000 7f875aa00640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f87400a32d0 0x7f87400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:10.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.147+0000 7f875aa00640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f87400a32d0 0x7f87400a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8748009230 tx=0x7f87480058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:10.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.150+0000 7f8753e00640 1 -- 172.21.15.107:0/2893117407 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f874c09c8d0 con 0x7f875c104010 2024-09-19T04:19:10.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:09 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:19:10.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.340+0000 7f8762400640 1 -- 172.21.15.107:0/2893117407 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f875c19b9a0 con 0x7f875c104010 2024-09-19T04:19:10.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.340+0000 7f8753e00640 1 -- 172.21.15.107:0/2893117407 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f874c09c1d0 con 0x7f875c104010 2024-09-19T04:19:10.342 INFO:teuthology.orchestra.run.smithi107.stdout:236223201321 2024-09-19T04:19:10.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.342+0000 7f8762400640 1 -- 172.21.15.107:0/2893117407 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f87400a32d0 msgr2=0x7f87400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:10.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.342+0000 7f8762400640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f87400a32d0 0x7f87400a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8748009230 tx=0x7f87480058d0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.342+0000 7f8762400640 1 -- 172.21.15.107:0/2893117407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f875c104010 msgr2=0x7f875c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:10.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.342+0000 7f8762400640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f875c104010 0x7f875c19a640 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f874c00ea60 tx=0x7f874c00ef20 comp rx=0 tx=0).stop 2024-09-19T04:19:10.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.342+0000 7f8762400640 1 -- 172.21.15.107:0/2893117407 shutdown_connections 2024-09-19T04:19:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.342+0000 7f8762400640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f87400a32d0 0x7f87400a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.342+0000 7f8762400640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875c109140 0x7f875c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.342+0000 7f8762400640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f875c1049c0 0x7f875c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.342+0000 7f8762400640 1 --2- 172.21.15.107:0/2893117407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f875c104010 0x7f875c19a640 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.342+0000 7f8762400640 1 -- 172.21.15.107:0/2893117407 >> 172.21.15.107:0/2893117407 conn(0x7f875c0ffa50 msgr2=0x7f875c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.343+0000 7f8762400640 1 -- 172.21.15.107:0/2893117407 shutdown_connections 2024-09-19T04:19:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.343+0000 7f8762400640 1 -- 172.21.15.107:0/2893117407 wait complete. 2024-09-19T04:19:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.718+0000 7f9369800640 1 -- 172.21.15.107:0/2327482257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9364101dd0 msgr2=0x7f93641021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.718+0000 7f9369800640 1 --2- 172.21.15.107:0/2327482257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9364101dd0 0x7f93641021b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9354009900 tx=0x7f935402a410 comp rx=0 tx=0).stop 2024-09-19T04:19:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.718+0000 7f9369800640 1 -- 172.21.15.107:0/2327482257 shutdown_connections 2024-09-19T04:19:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.718+0000 7f9369800640 1 --2- 172.21.15.107:0/2327482257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9364106f00 0x7f936410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.718+0000 7f9369800640 1 --2- 172.21.15.107:0/2327482257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9364102780 0x7f93641067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.718+0000 7f9369800640 1 --2- 172.21.15.107:0/2327482257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9364101dd0 0x7f93641021b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.718+0000 7f9369800640 1 -- 172.21.15.107:0/2327482257 >> 172.21.15.107:0/2327482257 conn(0x7f93640fd8b0 msgr2=0x7f93640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.718+0000 7f9369800640 1 -- 172.21.15.107:0/2327482257 shutdown_connections 2024-09-19T04:19:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.719+0000 7f9369800640 1 -- 172.21.15.107:0/2327482257 wait complete. 2024-09-19T04:19:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.720+0000 7f9369800640 1 Processor -- start 2024-09-19T04:19:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.720+0000 7f9369800640 1 -- start start 2024-09-19T04:19:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.720+0000 7f9369800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9364101dd0 0x7f9364108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.720+0000 7f9369800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9364102780 0x7f9364108fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.720+0000 7f9369800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9364106f00 0x7f9364109520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.720+0000 7f9369800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9364111fb0 con 0x7f9364101dd0 2024-09-19T04:19:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.720+0000 7f9369800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9364111e30 con 0x7f9364106f00 2024-09-19T04:19:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.720+0000 7f9369800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9364112130 con 0x7f9364102780 2024-09-19T04:19:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f935a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9364102780 0x7f9364108fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f935a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9364102780 0x7f9364108fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38982/0 (socket says 172.21.15.107:38982) 2024-09-19T04:19:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f935a000640 1 -- 172.21.15.107:0/2941254382 learned_addr learned my addr 172.21.15.107:0/2941254382 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f935a000640 1 -- 172.21.15.107:0/2941254382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9364106f00 msgr2=0x7f9364109520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f935a000640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9364106f00 0x7f9364109520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f9362a00640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9364101dd0 0x7f9364108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f935a000640 1 -- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9364101dd0 msgr2=0x7f9364108aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f935a000640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9364101dd0 0x7f9364108aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f935a000640 1 -- 172.21.15.107:0/2941254382 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93641aad00 con 0x7f9364102780 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f9362a00640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9364101dd0 0x7f9364108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.721+0000 7f935a000640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9364102780 0x7f9364108fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f934c00ea20 tx=0x7f934c00eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.722+0000 7f935be00640 1 -- 172.21.15.107:0/2941254382 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c00cd80 con 0x7f9364102780 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.722+0000 7f935be00640 1 -- 172.21.15.107:0/2941254382 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934c010910 con 0x7f9364102780 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.722+0000 7f935be00640 1 -- 172.21.15.107:0/2941254382 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c018920 con 0x7f9364102780 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.722+0000 7f9369800640 1 -- 172.21.15.107:0/2941254382 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93641aaff0 con 0x7f9364102780 2024-09-19T04:19:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.722+0000 7f9369800640 1 -- 172.21.15.107:0/2941254382 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93641ab4f0 con 0x7f9364102780 2024-09-19T04:19:10.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.723+0000 7f9369800640 1 -- 172.21.15.107:0/2941254382 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93641032b0 con 0x7f9364102780 2024-09-19T04:19:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.726+0000 7f935be00640 1 -- 172.21.15.107:0/2941254382 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f934c010ac0 con 0x7f9364102780 2024-09-19T04:19:10.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.728+0000 7f935be00640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f93480a3350 0x7f93480a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:10.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.728+0000 7f935be00640 1 -- 172.21.15.107:0/2941254382 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f934c0cff40 con 0x7f9364102780 2024-09-19T04:19:10.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.728+0000 7f9362a00640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f93480a3350 0x7f93480a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:10.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.729+0000 7f9362a00640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f93480a3350 0x7f93480a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9354005ab0 tx=0x7f9354000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:10.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.731+0000 7f935be00640 1 -- 172.21.15.107:0/2941254382 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f934c09cad0 con 0x7f9364102780 2024-09-19T04:19:10.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.916+0000 7f9369800640 1 -- 172.21.15.107:0/2941254382 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f93641ab8c0 con 0x7f9364102780 2024-09-19T04:19:10.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.916+0000 7f935be00640 1 -- 172.21.15.107:0/2941254382 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f934c09c3d0 con 0x7f9364102780 2024-09-19T04:19:10.921 INFO:teuthology.orchestra.run.smithi107.stdout:55834574926 2024-09-19T04:19:10.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 -- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f93480a3350 msgr2=0x7f93480a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f93480a3350 0x7f93480a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9354005ab0 tx=0x7f9354000bc0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 -- 172.21.15.107:0/2941254382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9364102780 msgr2=0x7f9364108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9364102780 0x7f9364108fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f934c00ea20 tx=0x7f934c00eee0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 -- 172.21.15.107:0/2941254382 shutdown_connections 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f93480a3350 0x7f93480a5810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9364106f00 0x7f9364109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9364102780 0x7f9364108fe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 --2- 172.21.15.107:0/2941254382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9364101dd0 0x7f9364108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 -- 172.21.15.107:0/2941254382 >> 172.21.15.107:0/2941254382 conn(0x7f93640fd8b0 msgr2=0x7f93640feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 -- 172.21.15.107:0/2941254382 shutdown_connections 2024-09-19T04:19:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.919+0000 7f9369800640 1 -- 172.21.15.107:0/2941254382 wait complete. 2024-09-19T04:19:11.121 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986234 got 133143986236 for osd.4 2024-09-19T04:19:11.122 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:10 smithi115 ceph-mon[35477]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:10 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2592031051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:19:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:10 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2529740566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:19:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:10 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2893117407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:19:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:10 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:10 smithi145 ceph-mon[35096]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:10 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2592031051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:19:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:10 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2529740566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:19:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:10 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2893117407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:19:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:10 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:10 smithi107 ceph-mon[30659]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T04:19:11.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:10 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2592031051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:19:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:10 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2529740566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:19:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:10 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2893117407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:19:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:10 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:11.709 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201319 got 236223201321 for osd.8 2024-09-19T04:19:11.709 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:11.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:11 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2941254382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:19:12.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.021+0000 7ff23f000640 1 -- 172.21.15.107:0/389285785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2381049a0 msgr2=0x7ff2381089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.021+0000 7ff23f000640 1 --2- 172.21.15.107:0/389285785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2381049a0 0x7ff2381089f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff220009900 tx=0x7ff22002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.021+0000 7ff23f000640 1 -- 172.21.15.107:0/389285785 shutdown_connections 2024-09-19T04:19:12.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.021+0000 7ff23f000640 1 --2- 172.21.15.107:0/389285785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238109120 0x7ff238110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.021+0000 7ff23f000640 1 --2- 172.21.15.107:0/389285785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2381049a0 0x7ff2381089f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.021+0000 7ff23f000640 1 --2- 172.21.15.107:0/389285785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238103ff0 0x7ff2381043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.021+0000 7ff23f000640 1 -- 172.21.15.107:0/389285785 >> 172.21.15.107:0/389285785 conn(0x7ff2380ffa90 msgr2=0x7ff238101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.021+0000 7ff23f000640 1 -- 172.21.15.107:0/389285785 shutdown_connections 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.021+0000 7ff23f000640 1 -- 172.21.15.107:0/389285785 wait complete. 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.022+0000 7ff23f000640 1 Processor -- start 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.022+0000 7ff23f000640 1 -- start start 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.022+0000 7ff23f000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238103ff0 0x7ff23819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.022+0000 7ff23f000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2381049a0 0x7ff23819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.022+0000 7ff23f000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238109120 0x7ff23819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.022+0000 7ff23f000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff238114170 con 0x7ff2381049a0 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.022+0000 7ff23f000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff238113ff0 con 0x7ff238109120 2024-09-19T04:19:12.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.022+0000 7ff23f000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381142f0 con 0x7ff238103ff0 2024-09-19T04:19:12.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.022+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2381049a0 0x7ff23819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff237e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238103ff0 0x7ff23819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2381049a0 0x7ff23819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36868/0 (socket says 172.21.15.107:36868) 2024-09-19T04:19:12.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff22fe00640 1 -- 172.21.15.107:0/2529740566 learned_addr learned my addr 172.21.15.107:0/2529740566 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:12.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff23d000640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238109120 0x7ff23819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff237e00640 1 -- 172.21.15.107:0/2529740566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238109120 msgr2=0x7ff23819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff237e00640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238109120 0x7ff23819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff237e00640 1 -- 172.21.15.107:0/2529740566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2381049a0 msgr2=0x7ff23819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff237e00640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2381049a0 0x7ff23819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff237e00640 1 -- 172.21.15.107:0/2529740566 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23819f5e0 con 0x7ff238103ff0 2024-09-19T04:19:12.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff22fe00640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2381049a0 0x7ff23819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:12.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.023+0000 7ff237e00640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238103ff0 0x7ff23819a630 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff22800e9f0 tx=0x7ff22800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.024+0000 7ff235600640 1 -- 172.21.15.107:0/2529740566 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22800cd80 con 0x7ff238103ff0 2024-09-19T04:19:12.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.024+0000 7ff23f000640 1 -- 172.21.15.107:0/2529740566 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23819f8d0 con 0x7ff238103ff0 2024-09-19T04:19:12.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.024+0000 7ff23f000640 1 -- 172.21.15.107:0/2529740566 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2381a71b0 con 0x7ff238103ff0 2024-09-19T04:19:12.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.024+0000 7ff235600640 1 -- 172.21.15.107:0/2529740566 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2280035f0 con 0x7ff238103ff0 2024-09-19T04:19:12.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.024+0000 7ff235600640 1 -- 172.21.15.107:0/2529740566 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228010670 con 0x7ff238103ff0 2024-09-19T04:19:12.053 INFO:teuthology.orchestra.run.smithi107.stdout:266287972388 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.025+0000 7ff23f000640 1 -- 172.21.15.107:0/2529740566 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff214005290 con 0x7ff238103ff0 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.027+0000 7ff235600640 1 -- 172.21.15.107:0/2529740566 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff228004020 con 0x7ff238103ff0 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.028+0000 7ff235600640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff21c0a3350 0x7ff21c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.028+0000 7ff235600640 1 -- 172.21.15.107:0/2529740566 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7ff2280cfc40 con 0x7ff238103ff0 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.029+0000 7ff22fe00640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff21c0a3350 0x7ff21c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.029+0000 7ff22fe00640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff21c0a3350 0x7ff21c0a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff2200096c0 tx=0x7ff2200046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.032+0000 7ff235600640 1 -- 172.21.15.107:0/2529740566 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff22809c7d0 con 0x7ff238103ff0 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.218+0000 7ff23f000640 1 -- 172.21.15.107:0/2529740566 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff214005850 con 0x7ff238103ff0 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.218+0000 7ff235600640 1 -- 172.21.15.107:0/2529740566 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff22809c0d0 con 0x7ff238103ff0 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.220+0000 7ff23f000640 1 -- 172.21.15.107:0/2529740566 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff21c0a3350 msgr2=0x7ff21c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.220+0000 7ff23f000640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff21c0a3350 0x7ff21c0a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff2200096c0 tx=0x7ff2200046c0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.220+0000 7ff23f000640 1 -- 172.21.15.107:0/2529740566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238103ff0 msgr2=0x7ff23819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.220+0000 7ff23f000640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238103ff0 0x7ff23819a630 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff22800e9f0 tx=0x7ff22800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.221+0000 7ff23f000640 1 -- 172.21.15.107:0/2529740566 shutdown_connections 2024-09-19T04:19:12.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.221+0000 7ff23f000640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff21c0a3350 0x7ff21c0a5810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.221+0000 7ff23f000640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238109120 0x7ff23819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.221+0000 7ff23f000640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2381049a0 0x7ff23819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.221+0000 7ff23f000640 1 --2- 172.21.15.107:0/2529740566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238103ff0 0x7ff23819a630 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.221+0000 7ff23f000640 1 -- 172.21.15.107:0/2529740566 >> 172.21.15.107:0/2529740566 conn(0x7ff2380ffa90 msgr2=0x7ff238100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.221+0000 7ff23f000640 1 -- 172.21.15.107:0/2529740566 shutdown_connections 2024-09-19T04:19:12.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:10.221+0000 7ff23f000640 1 -- 172.21.15.107:0/2529740566 wait complete. 2024-09-19T04:19:12.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:11 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2941254382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:19:12.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:11 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2941254382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:19:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:12 smithi115 ceph-mon[35477]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 1 objects/s recovering 2024-09-19T04:19:13.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:12 smithi107 ceph-mon[30659]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 1 objects/s recovering 2024-09-19T04:19:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:12 smithi145 ceph-mon[35096]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 1 objects/s recovering 2024-09-19T04:19:13.254 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738451 for osd.0 2024-09-19T04:19:13.254 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:13.582 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972385 got 266287972388 for osd.9 2024-09-19T04:19:13.582 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.802+0000 7fcb36200640 1 -- 172.21.15.107:0/3268149214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb30102360 msgr2=0x7fcb3010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.802+0000 7fcb36200640 1 --2- 172.21.15.107:0/3268149214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb30102360 0x7fcb3010e890 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009cd0 tx=0x7fcb2802a8e0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.803+0000 7fcb36200640 1 -- 172.21.15.107:0/3268149214 shutdown_connections 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.803+0000 7fcb36200640 1 --2- 172.21.15.107:0/3268149214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb30102360 0x7fcb3010e890 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.803+0000 7fcb36200640 1 --2- 172.21.15.107:0/3268149214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb301019c0 0x7fcb30101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.803+0000 7fcb36200640 1 --2- 172.21.15.107:0/3268149214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb301079c0 0x7fcb30107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.803+0000 7fcb36200640 1 -- 172.21.15.107:0/3268149214 >> 172.21.15.107:0/3268149214 conn(0x7fcb300fd6e0 msgr2=0x7fcb300ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.803+0000 7fcb36200640 1 -- 172.21.15.107:0/3268149214 shutdown_connections 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.803+0000 7fcb36200640 1 -- 172.21.15.107:0/3268149214 wait complete. 2024-09-19T04:19:13.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.804+0000 7fcb36200640 1 Processor -- start 2024-09-19T04:19:13.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.804+0000 7fcb36200640 1 -- start start 2024-09-19T04:19:13.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb36200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb301019c0 0x7fcb300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb36200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb30102360 0x7fcb30074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb36200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb301079c0 0x7fcb30074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb36200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30111e10 con 0x7fcb30102360 2024-09-19T04:19:13.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb36200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30111c90 con 0x7fcb301079c0 2024-09-19T04:19:13.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb36200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30111f90 con 0x7fcb301019c0 2024-09-19T04:19:13.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb34e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb301019c0 0x7fcb300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb27e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb30102360 0x7fcb30074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb35800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb301079c0 0x7fcb30074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb34e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb301019c0 0x7fcb300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38990/0 (socket says 172.21.15.107:38990) 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.805+0000 7fcb34e00640 1 -- 172.21.15.107:0/1104356467 learned_addr learned my addr 172.21.15.107:0/1104356467 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb35800640 1 -- 172.21.15.107:0/1104356467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb301019c0 msgr2=0x7fcb300744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb35800640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb301019c0 0x7fcb300744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb35800640 1 -- 172.21.15.107:0/1104356467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb30102360 msgr2=0x7fcb30074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb35800640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb30102360 0x7fcb30074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb35800640 1 -- 172.21.15.107:0/1104356467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb301a66d0 con 0x7fcb301079c0 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb34e00640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb301019c0 0x7fcb300744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb27e00640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb30102360 0x7fcb30074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:13.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb35800640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb301079c0 0x7fcb30074f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009ca0 tx=0x7fcb28031e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb2e000640 1 -- 172.21.15.107:0/1104356467 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28004040 con 0x7fcb301079c0 2024-09-19T04:19:13.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb2e000640 1 -- 172.21.15.107:0/1104356467 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb280046c0 con 0x7fcb301079c0 2024-09-19T04:19:13.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.806+0000 7fcb36200640 1 -- 172.21.15.107:0/1104356467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb301a6960 con 0x7fcb301079c0 2024-09-19T04:19:13.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.807+0000 7fcb2e000640 1 -- 172.21.15.107:0/1104356467 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28030e40 con 0x7fcb301079c0 2024-09-19T04:19:13.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.807+0000 7fcb36200640 1 -- 172.21.15.107:0/1104356467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb301a6e70 con 0x7fcb301079c0 2024-09-19T04:19:13.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.808+0000 7fcb36200640 1 -- 172.21.15.107:0/1104356467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb30101e20 con 0x7fcb301079c0 2024-09-19T04:19:13.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.809+0000 7fcb2e000640 1 -- 172.21.15.107:0/1104356467 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcb280305a0 con 0x7fcb301079c0 2024-09-19T04:19:13.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.811+0000 7fcb2e000640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fcb140a32d0 0x7fcb140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.811+0000 7fcb2e000640 1 -- 172.21.15.107:0/1104356467 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fcb28074080 con 0x7fcb301079c0 2024-09-19T04:19:13.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.811+0000 7fcb34e00640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fcb140a32d0 0x7fcb140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.811+0000 7fcb34e00640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fcb140a32d0 0x7fcb140a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb20011440 tx=0x7fcb20005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.813+0000 7fcb2e000640 1 -- 172.21.15.107:0/1104356467 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcb2804b060 con 0x7fcb301079c0 2024-09-19T04:19:13.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.998+0000 7fcb36200640 1 -- 172.21.15.107:0/1104356467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fcb301a7100 con 0x7fcb301079c0 2024-09-19T04:19:14.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:13.999+0000 7fcb2e000640 1 -- 172.21.15.107:0/1104356467 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcb280bde90 con 0x7fcb301079c0 2024-09-19T04:19:14.000 INFO:teuthology.orchestra.run.smithi107.stdout:322122547226 2024-09-19T04:19:14.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 -- 172.21.15.107:0/1104356467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fcb140a32d0 msgr2=0x7fcb140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fcb140a32d0 0x7fcb140a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb20011440 tx=0x7fcb20005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:14.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 -- 172.21.15.107:0/1104356467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb301079c0 msgr2=0x7fcb30074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb301079c0 0x7fcb30074f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009ca0 tx=0x7fcb28031e60 comp rx=0 tx=0).stop 2024-09-19T04:19:14.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 -- 172.21.15.107:0/1104356467 shutdown_connections 2024-09-19T04:19:14.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fcb140a32d0 0x7fcb140a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb301079c0 0x7fcb30074f70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb30102360 0x7fcb30074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 --2- 172.21.15.107:0/1104356467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb301019c0 0x7fcb300744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 -- 172.21.15.107:0/1104356467 >> 172.21.15.107:0/1104356467 conn(0x7fcb300fd6e0 msgr2=0x7fcb300fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.001+0000 7fcb36200640 1 -- 172.21.15.107:0/1104356467 shutdown_connections 2024-09-19T04:19:14.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.002+0000 7fcb36200640 1 -- 172.21.15.107:0/1104356467 wait complete. 2024-09-19T04:19:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.027+0000 7f6557200640 1 -- 172.21.15.107:0/2075191457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6558102b90 msgr2=0x7f6558102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.027+0000 7f6557200640 1 --2- 172.21.15.107:0/2075191457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6558102b90 0x7f6558102f70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6548009900 tx=0x7f654802a480 comp rx=0 tx=0).stop 2024-09-19T04:19:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.027+0000 7f6557200640 1 -- 172.21.15.107:0/2075191457 shutdown_connections 2024-09-19T04:19:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.027+0000 7f6557200640 1 --2- 172.21.15.107:0/2075191457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f655810bfc0 0x7f655810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.027+0000 7f6557200640 1 --2- 172.21.15.107:0/2075191457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65581034b0 0x7f655810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.027+0000 7f6557200640 1 --2- 172.21.15.107:0/2075191457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6558102b90 0x7f6558102f70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.027+0000 7f6557200640 1 -- 172.21.15.107:0/2075191457 >> 172.21.15.107:0/2075191457 conn(0x7f65580fc610 msgr2=0x7f65580fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.027+0000 7f6557200640 1 -- 172.21.15.107:0/2075191457 shutdown_connections 2024-09-19T04:19:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.027+0000 7f6557200640 1 -- 172.21.15.107:0/2075191457 wait complete. 2024-09-19T04:19:14.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.028+0000 7f6557200640 1 Processor -- start 2024-09-19T04:19:14.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.028+0000 7f6557200640 1 -- start start 2024-09-19T04:19:14.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6557200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6558102b90 0x7f6558196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6557200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65581034b0 0x7f6558196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6557200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f655810bfc0 0x7f655819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6557200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f655810fbb0 con 0x7f6558102b90 2024-09-19T04:19:14.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6557200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f655810fa30 con 0x7f655810bfc0 2024-09-19T04:19:14.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6557200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f655810fd30 con 0x7f65581034b0 2024-09-19T04:19:14.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6555e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6558102b90 0x7f6558196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6556800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f655810bfc0 0x7f655819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6556800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f655810bfc0 0x7f655819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:32852/0 (socket says 172.21.15.107:32852) 2024-09-19T04:19:14.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6556800640 1 -- 172.21.15.107:0/4137797746 learned_addr learned my addr 172.21.15.107:0/4137797746 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:14.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6555400640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65581034b0 0x7f6558196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.029+0000 7f6556800640 1 -- 172.21.15.107:0/4137797746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65581034b0 msgr2=0x7f6558196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.030+0000 7f6556800640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65581034b0 0x7f6558196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.030+0000 7f6556800640 1 -- 172.21.15.107:0/4137797746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6558102b90 msgr2=0x7f6558196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.030+0000 7f6556800640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6558102b90 0x7f6558196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.030+0000 7f6556800640 1 -- 172.21.15.107:0/4137797746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655819b0b0 con 0x7f655810bfc0 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.030+0000 7f6556800640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f655810bfc0 0x7f655819a9d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6550009fd0 tx=0x7f655000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.030+0000 7f654ea00640 1 -- 172.21.15.107:0/4137797746 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550017070 con 0x7f655810bfc0 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.030+0000 7f654ea00640 1 -- 172.21.15.107:0/4137797746 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6550003e50 con 0x7f655810bfc0 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.030+0000 7f6557200640 1 -- 172.21.15.107:0/4137797746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655819b3a0 con 0x7f655810bfc0 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.030+0000 7f654ea00640 1 -- 172.21.15.107:0/4137797746 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65500106c0 con 0x7f655810bfc0 2024-09-19T04:19:14.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.030+0000 7f6557200640 1 -- 172.21.15.107:0/4137797746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65581a2c80 con 0x7f655810bfc0 2024-09-19T04:19:14.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.031+0000 7f6557200640 1 -- 172.21.15.107:0/4137797746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6558102f70 con 0x7f655810bfc0 2024-09-19T04:19:14.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.033+0000 7f654ea00640 1 -- 172.21.15.107:0/4137797746 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6550003970 con 0x7f655810bfc0 2024-09-19T04:19:14.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.034+0000 7f654ea00640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f653c0a32d0 0x7f653c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.035+0000 7f654ea00640 1 -- 172.21.15.107:0/4137797746 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f65500cf790 con 0x7f655810bfc0 2024-09-19T04:19:14.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.035+0000 7f6555e00640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f653c0a32d0 0x7f653c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.036+0000 7f6555e00640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f653c0a32d0 0x7f653c0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6548009230 tx=0x7f65480058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.039+0000 7f654ea00640 1 -- 172.21.15.107:0/4137797746 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f655009bcd0 con 0x7f655810bfc0 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.218+0000 7f6557200640 1 -- 172.21.15.107:0/4137797746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6558107090 con 0x7f655810bfc0 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.219+0000 7f654ea00640 1 -- 172.21.15.107:0/4137797746 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f655009ffd0 con 0x7f655810bfc0 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi107.stdout:77309411402 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 -- 172.21.15.107:0/4137797746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f653c0a32d0 msgr2=0x7f653c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f653c0a32d0 0x7f653c0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6548009230 tx=0x7f65480058d0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 -- 172.21.15.107:0/4137797746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f655810bfc0 msgr2=0x7f655819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f655810bfc0 0x7f655819a9d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6550009fd0 tx=0x7f655000b260 comp rx=0 tx=0).stop 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 -- 172.21.15.107:0/4137797746 shutdown_connections 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f653c0a32d0 0x7f653c0a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f655810bfc0 0x7f655819a9d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65581034b0 0x7f6558196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 --2- 172.21.15.107:0/4137797746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6558102b90 0x7f6558196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 -- 172.21.15.107:0/4137797746 >> 172.21.15.107:0/4137797746 conn(0x7f65580fc610 msgr2=0x7f65580fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 -- 172.21.15.107:0/4137797746 shutdown_connections 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:14.221+0000 7f6557200640 1 -- 172.21.15.107:0/4137797746 wait complete. 2024-09-19T04:19:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.153+0000 7f734a000640 1 -- 172.21.15.107:0/2208118988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73440745c0 msgr2=0x7f73440749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.153+0000 7f734a000640 1 --2- 172.21.15.107:0/2208118988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73440745c0 0x7f73440749a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7338009900 tx=0x7f733802a480 comp rx=0 tx=0).stop 2024-09-19T04:19:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.154+0000 7f734a000640 1 -- 172.21.15.107:0/2208118988 shutdown_connections 2024-09-19T04:19:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.154+0000 7f734a000640 1 --2- 172.21.15.107:0/2208118988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f734410e260 0x7f7344110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.154+0000 7f734a000640 1 --2- 172.21.15.107:0/2208118988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7344074ee0 0x7f734410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.154+0000 7f734a000640 1 --2- 172.21.15.107:0/2208118988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73440745c0 0x7f73440749a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.154+0000 7f734a000640 1 -- 172.21.15.107:0/2208118988 >> 172.21.15.107:0/2208118988 conn(0x7f73440fd740 msgr2=0x7f73440ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.154+0000 7f734a000640 1 -- 172.21.15.107:0/2208118988 shutdown_connections 2024-09-19T04:19:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.154+0000 7f734a000640 1 -- 172.21.15.107:0/2208118988 wait complete. 2024-09-19T04:19:15.157 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:14 smithi107 ceph-mon[30659]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:19:15.157 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:14 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1104356467' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:19:15.157 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:14 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/4137797746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.155+0000 7f734a000640 1 Processor -- start 2024-09-19T04:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.155+0000 7f734a000640 1 -- start start 2024-09-19T04:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.155+0000 7f734a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73440745c0 0x7f734419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.155+0000 7f734a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7344074ee0 0x7f734419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.155+0000 7f734a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f734410e260 0x7f734419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.155+0000 7f734a000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7344113fb0 con 0x7f7344074ee0 2024-09-19T04:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.155+0000 7f734a000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7344113e30 con 0x7f73440745c0 2024-09-19T04:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.155+0000 7f734a000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7344114130 con 0x7f734410e260 2024-09-19T04:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7343e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7344074ee0 0x7f734419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7343e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7344074ee0 0x7f734419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36968/0 (socket says 172.21.15.107:36968) 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7343e00640 1 -- 172.21.15.107:0/227407232 learned_addr learned my addr 172.21.15.107:0/227407232 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7348c00640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73440745c0 0x7f734419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7349600640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f734410e260 0x7f734419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7348c00640 1 -- 172.21.15.107:0/227407232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f734410e260 msgr2=0x7f734419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7348c00640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f734410e260 0x7f734419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7348c00640 1 -- 172.21.15.107:0/227407232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7344074ee0 msgr2=0x7f734419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7348c00640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7344074ee0 0x7f734419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7348c00640 1 -- 172.21.15.107:0/227407232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734419f440 con 0x7f73440745c0 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.156+0000 7f7343e00640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7344074ee0 0x7f734419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.157+0000 7f7348c00640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73440745c0 0x7f734419a490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7338005940 tx=0x7f7338004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.157+0000 7f7341600640 1 -- 172.21.15.107:0/227407232 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733803f030 con 0x7f73440745c0 2024-09-19T04:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.157+0000 7f7341600640 1 -- 172.21.15.107:0/227407232 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7338003ad0 con 0x7f73440745c0 2024-09-19T04:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.157+0000 7f734a000640 1 -- 172.21.15.107:0/227407232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734419f6d0 con 0x7f73440745c0 2024-09-19T04:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.157+0000 7f7341600640 1 -- 172.21.15.107:0/227407232 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338035070 con 0x7f73440745c0 2024-09-19T04:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.157+0000 7f734a000640 1 -- 172.21.15.107:0/227407232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73441a7010 con 0x7f73440745c0 2024-09-19T04:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.158+0000 7f734a000640 1 -- 172.21.15.107:0/227407232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7344075c70 con 0x7f73440745c0 2024-09-19T04:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.160+0000 7f7341600640 1 -- 172.21.15.107:0/227407232 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7338042030 con 0x7f73440745c0 2024-09-19T04:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.161+0000 7f7341600640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f732c0a32d0 0x7f732c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.161+0000 7f7341600640 1 -- 172.21.15.107:0/227407232 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f73380efce0 con 0x7f73440745c0 2024-09-19T04:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.161+0000 7f7343e00640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f732c0a32d0 0x7f732c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.162+0000 7f7343e00640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f732c0a32d0 0x7f732c0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f734410f5f0 tx=0x7f7330005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.165+0000 7f7341600640 1 -- 172.21.15.107:0/227407232 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f73380bc220 con 0x7f73440745c0 2024-09-19T04:19:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:14 smithi115 ceph-mon[35477]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:19:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:14 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1104356467' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:19:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:14 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/4137797746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:19:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:14 smithi145 ceph-mon[35096]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T04:19:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:14 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1104356467' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:19:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:14 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/4137797746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:19:15.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.343+0000 7f734a000640 1 -- 172.21.15.107:0/227407232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7344109410 con 0x7f73440745c0 2024-09-19T04:19:15.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.344+0000 7f7341600640 1 -- 172.21.15.107:0/227407232 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f73380c0520 con 0x7f73440745c0 2024-09-19T04:19:15.346 INFO:teuthology.orchestra.run.smithi107.stdout:103079215171 2024-09-19T04:19:15.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 -- 172.21.15.107:0/227407232 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f732c0a32d0 msgr2=0x7f732c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f732c0a32d0 0x7f732c0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f734410f5f0 tx=0x7f7330005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:15.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 -- 172.21.15.107:0/227407232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73440745c0 msgr2=0x7f734419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73440745c0 0x7f734419a490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7338005940 tx=0x7f7338004270 comp rx=0 tx=0).stop 2024-09-19T04:19:15.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 -- 172.21.15.107:0/227407232 shutdown_connections 2024-09-19T04:19:15.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f732c0a32d0 0x7f732c0a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f734410e260 0x7f734419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7344074ee0 0x7f734419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 --2- 172.21.15.107:0/227407232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73440745c0 0x7f734419a490 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 -- 172.21.15.107:0/227407232 >> 172.21.15.107:0/227407232 conn(0x7f73440fd740 msgr2=0x7f73440fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:15.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 -- 172.21.15.107:0/227407232 shutdown_connections 2024-09-19T04:19:15.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:15.346+0000 7f734a000640 1 -- 172.21.15.107:0/227407232 wait complete. 2024-09-19T04:19:15.472 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574923 got 55834574926 for osd.1 2024-09-19T04:19:15.472 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:16.082 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:15 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/227407232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:19:16.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:15 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/227407232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:19:16.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:15 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/227407232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:19:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.288+0000 7f5088400640 1 -- 172.21.15.107:0/2203589421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5080102790 msgr2=0x7f50801067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.288+0000 7f5088400640 1 --2- 172.21.15.107:0/2203589421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5080102790 0x7f50801067e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f506c009900 tx=0x7f506c02a410 comp rx=0 tx=0).stop 2024-09-19T04:19:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.288+0000 7f5088400640 1 -- 172.21.15.107:0/2203589421 shutdown_connections 2024-09-19T04:19:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.288+0000 7f5088400640 1 --2- 172.21.15.107:0/2203589421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5080106f10 0x7f508010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.288+0000 7f5088400640 1 --2- 172.21.15.107:0/2203589421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5080102790 0x7f50801067e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.288+0000 7f5088400640 1 --2- 172.21.15.107:0/2203589421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5080101de0 0x7f50801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.288+0000 7f5088400640 1 -- 172.21.15.107:0/2203589421 >> 172.21.15.107:0/2203589421 conn(0x7f50800fd8c0 msgr2=0x7f50800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.288+0000 7f5088400640 1 -- 172.21.15.107:0/2203589421 shutdown_connections 2024-09-19T04:19:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.288+0000 7f5088400640 1 -- 172.21.15.107:0/2203589421 wait complete. 2024-09-19T04:19:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.289+0000 7f5088400640 1 Processor -- start 2024-09-19T04:19:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.289+0000 7f5088400640 1 -- start start 2024-09-19T04:19:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.289+0000 7f5088400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5080101de0 0x7f508019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.289+0000 7f5088400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5080102790 0x7f508019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.289+0000 7f5088400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5080106f10 0x7f508019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.289+0000 7f5088400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5080111f30 con 0x7f5080102790 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.289+0000 7f5088400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5080111db0 con 0x7f5080106f10 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.289+0000 7f5088400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f50801120b0 con 0x7f5080101de0 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5085000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5080102790 0x7f508019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5086400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5080106f10 0x7f508019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5085a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5080101de0 0x7f508019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5086400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5080106f10 0x7f508019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:32898/0 (socket says 172.21.15.107:32898) 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5086400640 1 -- 172.21.15.107:0/2519977052 learned_addr learned my addr 172.21.15.107:0/2519977052 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5086400640 1 -- 172.21.15.107:0/2519977052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5080101de0 msgr2=0x7f508019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5086400640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5080101de0 0x7f508019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5086400640 1 -- 172.21.15.107:0/2519977052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5080102790 msgr2=0x7f508019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5086400640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5080102790 0x7f508019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5086400640 1 -- 172.21.15.107:0/2519977052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508019f640 con 0x7f5080106f10 2024-09-19T04:19:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.290+0000 7f5086400640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5080106f10 0x7f508019ef60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f507800eaa0 tx=0x7f507800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.291+0000 7f507e000640 1 -- 172.21.15.107:0/2519977052 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507800cdb0 con 0x7f5080106f10 2024-09-19T04:19:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.291+0000 7f507e000640 1 -- 172.21.15.107:0/2519977052 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5078005020 con 0x7f5080106f10 2024-09-19T04:19:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.291+0000 7f5088400640 1 -- 172.21.15.107:0/2519977052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508019f930 con 0x7f5080106f10 2024-09-19T04:19:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.291+0000 7f507e000640 1 -- 172.21.15.107:0/2519977052 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50780116b0 con 0x7f5080106f10 2024-09-19T04:19:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.291+0000 7f5088400640 1 -- 172.21.15.107:0/2519977052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50801a7210 con 0x7f5080106f10 2024-09-19T04:19:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.292+0000 7f5088400640 1 -- 172.21.15.107:0/2519977052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50801021c0 con 0x7f5080106f10 2024-09-19T04:19:16.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.294+0000 7f507e000640 1 -- 172.21.15.107:0/2519977052 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5078005560 con 0x7f5080106f10 2024-09-19T04:19:16.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.295+0000 7f507e000640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f50680a3350 0x7f50680a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.295+0000 7f507e000640 1 -- 172.21.15.107:0/2519977052 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f50780cfba0 con 0x7f5080106f10 2024-09-19T04:19:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.295+0000 7f5085a00640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f50680a3350 0x7f50680a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.296+0000 7f5085a00640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f50680a3350 0x7f50680a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5074009bd0 tx=0x7f5074005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:16.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.299+0000 7f507e000640 1 -- 172.21.15.107:0/2519977052 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5078016030 con 0x7f5080106f10 2024-09-19T04:19:16.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.479+0000 7f5088400640 1 -- 172.21.15.107:0/2519977052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f50801a74b0 con 0x7f5080106f10 2024-09-19T04:19:16.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.479+0000 7f507e000640 1 -- 172.21.15.107:0/2519977052 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f507809c0e0 con 0x7f5080106f10 2024-09-19T04:19:16.481 INFO:teuthology.orchestra.run.smithi107.stdout:154618822715 2024-09-19T04:19:16.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 -- 172.21.15.107:0/2519977052 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f50680a3350 msgr2=0x7f50680a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f50680a3350 0x7f50680a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5074009bd0 tx=0x7f5074005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:16.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 -- 172.21.15.107:0/2519977052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5080106f10 msgr2=0x7f508019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5080106f10 0x7f508019ef60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f507800eaa0 tx=0x7f507800ef60 comp rx=0 tx=0).stop 2024-09-19T04:19:16.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 -- 172.21.15.107:0/2519977052 shutdown_connections 2024-09-19T04:19:16.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f50680a3350 0x7f50680a5810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5080106f10 0x7f508019ef60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5080102790 0x7f508019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 --2- 172.21.15.107:0/2519977052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5080101de0 0x7f508019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 -- 172.21.15.107:0/2519977052 >> 172.21.15.107:0/2519977052 conn(0x7f50800fd8c0 msgr2=0x7f50800fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:16.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 -- 172.21.15.107:0/2519977052 shutdown_connections 2024-09-19T04:19:16.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:16.482+0000 7f5088400640 1 -- 172.21.15.107:0/2519977052 wait complete. 2024-09-19T04:19:16.662 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776156 got 292057776157 for osd.10 2024-09-19T04:19:16.663 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:17.182 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:16 smithi107 ceph-mon[30659]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T04:19:17.182 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:16 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2519977052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:19:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:16 smithi115 ceph-mon[35477]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T04:19:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:16 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2519977052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:19:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:16 smithi145 ceph-mon[35096]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T04:19:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:16 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2519977052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:19:17.281 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822711 got 154618822715 for osd.5 2024-09-19T04:19:17.281 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:17.836 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215168 got 103079215171 for osd.3 2024-09-19T04:19:17.837 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:18.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.306+0000 7f0671400640 1 -- 172.21.15.107:0/1963577092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c074f40 msgr2=0x7f066c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.306+0000 7f0671400640 1 --2- 172.21.15.107:0/1963577092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c074f40 0x7f066c10de80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0658009900 tx=0x7f065802a480 comp rx=0 tx=0).stop 2024-09-19T04:19:18.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.306+0000 7f0671400640 1 -- 172.21.15.107:0/1963577092 shutdown_connections 2024-09-19T04:19:18.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.306+0000 7f0671400640 1 --2- 172.21.15.107:0/1963577092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066c10e3c0 0x7f066c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.306+0000 7f0671400640 1 --2- 172.21.15.107:0/1963577092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c074f40 0x7f066c10de80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.306+0000 7f0671400640 1 --2- 172.21.15.107:0/1963577092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f066c074620 0x7f066c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.306+0000 7f0671400640 1 -- 172.21.15.107:0/1963577092 >> 172.21.15.107:0/1963577092 conn(0x7f066c0fd8c0 msgr2=0x7f066c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:18.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.306+0000 7f0671400640 1 -- 172.21.15.107:0/1963577092 shutdown_connections 2024-09-19T04:19:18.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.306+0000 7f0671400640 1 -- 172.21.15.107:0/1963577092 wait complete. 2024-09-19T04:19:18.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.307+0000 7f0671400640 1 Processor -- start 2024-09-19T04:19:18.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.307+0000 7f0671400640 1 -- start start 2024-09-19T04:19:18.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.307+0000 7f0671400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f066c074620 0x7f066c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.307+0000 7f0671400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c074f40 0x7f066c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.307+0000 7f0671400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066c10e3c0 0x7f066c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.307+0000 7f0671400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1141b0 con 0x7f066c074f40 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.307+0000 7f0671400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f066c114030 con 0x7f066c10e3c0 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.307+0000 7f0671400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f066c114330 con 0x7f066c074620 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f0669e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c074f40 0x7f066c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f066a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f066c074620 0x7f066c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f0669e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c074f40 0x7f066c19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37000/0 (socket says 172.21.15.107:37000) 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f0669e00640 1 -- 172.21.15.107:0/3697568417 learned_addr learned my addr 172.21.15.107:0/3697568417 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f066b200640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066c10e3c0 0x7f066c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f0669e00640 1 -- 172.21.15.107:0/3697568417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f066c074620 msgr2=0x7f066c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f0669e00640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f066c074620 0x7f066c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f0669e00640 1 -- 172.21.15.107:0/3697568417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066c10e3c0 msgr2=0x7f066c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f0669e00640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066c10e3c0 0x7f066c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f0669e00640 1 -- 172.21.15.107:0/3697568417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066c19f640 con 0x7f066c074f40 2024-09-19T04:19:18.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.308+0000 7f0669e00640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c074f40 0x7f066c19abd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0658004770 tx=0x7f0658003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:18.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.309+0000 7f0663400640 1 -- 172.21.15.107:0/3697568417 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065803f070 con 0x7f066c074f40 2024-09-19T04:19:18.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.309+0000 7f0663400640 1 -- 172.21.15.107:0/3697568417 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0658031ca0 con 0x7f066c074f40 2024-09-19T04:19:18.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.309+0000 7f0671400640 1 -- 172.21.15.107:0/3697568417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066c19f8d0 con 0x7f066c074f40 2024-09-19T04:19:18.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.309+0000 7f0663400640 1 -- 172.21.15.107:0/3697568417 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658035030 con 0x7f066c074f40 2024-09-19T04:19:18.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.309+0000 7f0671400640 1 -- 172.21.15.107:0/3697568417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066c1a7210 con 0x7f066c074f40 2024-09-19T04:19:18.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.310+0000 7f0671400640 1 -- 172.21.15.107:0/3697568417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066c075cd0 con 0x7f066c074f40 2024-09-19T04:19:18.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.312+0000 7f0663400640 1 -- 172.21.15.107:0/3697568417 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0658042070 con 0x7f066c074f40 2024-09-19T04:19:18.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.313+0000 7f0663400640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f06500a32d0 0x7f06500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.313+0000 7f0663400640 1 -- 172.21.15.107:0/3697568417 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f06580f0050 con 0x7f066c074f40 2024-09-19T04:19:18.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.313+0000 7f066a800640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f06500a32d0 0x7f06500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.314+0000 7f066a800640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f06500a32d0 0x7f06500a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f065c009bd0 tx=0x7f065c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.317+0000 7f0663400640 1 -- 172.21.15.107:0/3697568417 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f06580bc6b0 con 0x7f066c074f40 2024-09-19T04:19:18.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.499+0000 7f0671400640 1 -- 172.21.15.107:0/3697568417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f066c109570 con 0x7f066c074f40 2024-09-19T04:19:18.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.499+0000 7f0663400640 1 -- 172.21.15.107:0/3697568417 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f06580c09b0 con 0x7f066c074f40 2024-09-19T04:19:18.501 INFO:teuthology.orchestra.run.smithi107.stdout:206158430254 2024-09-19T04:19:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.501+0000 7f0671400640 1 -- 172.21.15.107:0/3697568417 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f06500a32d0 msgr2=0x7f06500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.501+0000 7f0671400640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f06500a32d0 0x7f06500a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f065c009bd0 tx=0x7f065c005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.501+0000 7f0671400640 1 -- 172.21.15.107:0/3697568417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c074f40 msgr2=0x7f066c19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.501+0000 7f0671400640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c074f40 0x7f066c19abd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0658004770 tx=0x7f0658003c10 comp rx=0 tx=0).stop 2024-09-19T04:19:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.502+0000 7f0671400640 1 -- 172.21.15.107:0/3697568417 shutdown_connections 2024-09-19T04:19:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.502+0000 7f0671400640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f06500a32d0 0x7f06500a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.502+0000 7f0671400640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066c10e3c0 0x7f066c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.502+0000 7f0671400640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c074f40 0x7f066c19abd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.502+0000 7f0671400640 1 --2- 172.21.15.107:0/3697568417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f066c074620 0x7f066c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.502+0000 7f0671400640 1 -- 172.21.15.107:0/3697568417 >> 172.21.15.107:0/3697568417 conn(0x7f066c0fd8c0 msgr2=0x7f066c0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.502+0000 7f0671400640 1 -- 172.21.15.107:0/3697568417 shutdown_connections 2024-09-19T04:19:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:18.502+0000 7f0671400640 1 -- 172.21.15.107:0/3697568417 wait complete. 2024-09-19T04:19:18.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411402 for osd.2 2024-09-19T04:19:18.895 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:18 smithi115 ceph-mon[35477]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:18 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3697568417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:19:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:18 smithi145 ceph-mon[35096]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:18 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3697568417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:19:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:18 smithi107 ceph-mon[30659]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:18 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3697568417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:19:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:19.292 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547223 got 322122547226 for osd.11 2024-09-19T04:19:19.293 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:19.908 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430251 got 206158430254 for osd.7 2024-09-19T04:19:19.908 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:21.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.001+0000 7f6684200640 1 -- 172.21.15.107:0/4117433803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f667c106f10 msgr2=0x7f667c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.001+0000 7f6684200640 1 --2- 172.21.15.107:0/4117433803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f667c106f10 0x7f667c10ea90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6674009ca0 tx=0x7f667402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.001+0000 7f6684200640 1 -- 172.21.15.107:0/4117433803 shutdown_connections 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.001+0000 7f6684200640 1 --2- 172.21.15.107:0/4117433803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f667c106f10 0x7f667c10ea90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.001+0000 7f6684200640 1 --2- 172.21.15.107:0/4117433803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c102790 0x7f667c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.001+0000 7f6684200640 1 --2- 172.21.15.107:0/4117433803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f667c101de0 0x7f667c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.001+0000 7f6684200640 1 -- 172.21.15.107:0/4117433803 >> 172.21.15.107:0/4117433803 conn(0x7f667c0fd8c0 msgr2=0x7f667c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.001+0000 7f6684200640 1 -- 172.21.15.107:0/4117433803 shutdown_connections 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.001+0000 7f6684200640 1 -- 172.21.15.107:0/4117433803 wait complete. 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.002+0000 7f6684200640 1 Processor -- start 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.002+0000 7f6684200640 1 -- start start 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.002+0000 7f6684200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c101de0 0x7f667c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.002+0000 7f6684200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f667c102790 0x7f667c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.002+0000 7f6684200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f667c106f10 0x7f667c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.002+0000 7f6684200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f667c111fc0 con 0x7f667c106f10 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.002+0000 7f6684200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f667c111e40 con 0x7f667c101de0 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.002+0000 7f6684200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f667c112140 con 0x7f667c102790 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6682200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f667c106f10 0x7f667c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6680e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f667c102790 0x7f667c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6682200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f667c106f10 0x7f667c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37898/0 (socket says 172.21.15.107:37898) 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6682200640 1 -- 172.21.15.107:0/2813449856 learned_addr learned my addr 172.21.15.107:0/2813449856 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6681800640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c101de0 0x7f667c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6680e00640 1 -- 172.21.15.107:0/2813449856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c101de0 msgr2=0x7f667c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6680e00640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c101de0 0x7f667c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6680e00640 1 -- 172.21.15.107:0/2813449856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f667c106f10 msgr2=0x7f667c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6680e00640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f667c106f10 0x7f667c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6680e00640 1 -- 172.21.15.107:0/2813449856 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c19f590 con 0x7f667c102790 2024-09-19T04:19:21.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6682200640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f667c106f10 0x7f667c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:21.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.003+0000 7f6680e00640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f667c102790 0x7f667c19ab20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f666800e9f0 tx=0x7f666800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.004+0000 7f667a000640 1 -- 172.21.15.107:0/2813449856 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666800ccc0 con 0x7f667c102790 2024-09-19T04:19:21.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.004+0000 7f6684200640 1 -- 172.21.15.107:0/2813449856 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c19f880 con 0x7f667c102790 2024-09-19T04:19:21.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.004+0000 7f667a000640 1 -- 172.21.15.107:0/2813449856 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6668010910 con 0x7f667c102790 2024-09-19T04:19:21.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.004+0000 7f667a000640 1 -- 172.21.15.107:0/2813449856 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668018960 con 0x7f667c102790 2024-09-19T04:19:21.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.004+0000 7f6684200640 1 -- 172.21.15.107:0/2813449856 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c1a7160 con 0x7f667c102790 2024-09-19T04:19:21.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.005+0000 7f6684200640 1 -- 172.21.15.107:0/2813449856 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667c1021c0 con 0x7f667c102790 2024-09-19T04:19:21.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.007+0000 7f667a000640 1 -- 172.21.15.107:0/2813449856 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6668010ac0 con 0x7f667c102790 2024-09-19T04:19:21.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.008+0000 7f667a000640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f66640a32d0 0x7f66640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.008+0000 7f667a000640 1 -- 172.21.15.107:0/2813449856 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f66680cfdf0 con 0x7f667c102790 2024-09-19T04:19:21.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.008+0000 7f6681800640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f66640a32d0 0x7f66640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.009+0000 7f6681800640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f66640a32d0 0x7f66640a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6670005d00 tx=0x7f667000b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.012+0000 7f667a000640 1 -- 172.21.15.107:0/2813449856 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f666809c330 con 0x7f667c102790 2024-09-19T04:19:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:20 smithi115 ceph-mon[35477]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:21.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.208+0000 7f6684200640 1 -- 172.21.15.107:0/2813449856 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f667c1a7400 con 0x7f667c102790 2024-09-19T04:19:21.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:20 smithi107 ceph-mon[30659]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:21.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.208+0000 7f667a000640 1 -- 172.21.15.107:0/2813449856 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f66680a0630 con 0x7f667c102790 2024-09-19T04:19:21.210 INFO:teuthology.orchestra.run.smithi107.stdout:184683593781 2024-09-19T04:19:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.211+0000 7f6684200640 1 -- 172.21.15.107:0/2813449856 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f66640a32d0 msgr2=0x7f66640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.211+0000 7f6684200640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f66640a32d0 0x7f66640a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6670005d00 tx=0x7f667000b410 comp rx=0 tx=0).stop 2024-09-19T04:19:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.211+0000 7f6684200640 1 -- 172.21.15.107:0/2813449856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f667c102790 msgr2=0x7f667c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.211+0000 7f6684200640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f667c102790 0x7f667c19ab20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f666800e9f0 tx=0x7f666800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.212+0000 7f6684200640 1 -- 172.21.15.107:0/2813449856 shutdown_connections 2024-09-19T04:19:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.212+0000 7f6684200640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f66640a32d0 0x7f66640a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.212+0000 7f6684200640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f667c106f10 0x7f667c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.212+0000 7f6684200640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f667c102790 0x7f667c19ab20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.212+0000 7f6684200640 1 --2- 172.21.15.107:0/2813449856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c101de0 0x7f667c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.212+0000 7f6684200640 1 -- 172.21.15.107:0/2813449856 >> 172.21.15.107:0/2813449856 conn(0x7f667c0fd8c0 msgr2=0x7f667c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.212+0000 7f6684200640 1 -- 172.21.15.107:0/2813449856 shutdown_connections 2024-09-19T04:19:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:21.212+0000 7f6684200640 1 -- 172.21.15.107:0/2813449856 wait complete. 2024-09-19T04:19:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:20 smithi145 ceph-mon[35096]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:21.906 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593777 got 184683593781 for osd.6 2024-09-19T04:19:21.906 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:21.906 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T04:19:21.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:19:22.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:21 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2813449856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:19:22.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:21 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2813449856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:19:22.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:21 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2813449856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:19:22.322 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:22.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.924+0000 7f9c33800640 1 -- 172.21.15.107:0/3605973434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c2c101dd0 msgr2=0x7f9c2c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:22.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.924+0000 7f9c33800640 1 --2- 172.21.15.107:0/3605973434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c2c101dd0 0x7f9c2c1021b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009900 tx=0x7f9c2002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:22.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.924+0000 7f9c33800640 1 -- 172.21.15.107:0/3605973434 shutdown_connections 2024-09-19T04:19:22.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.924+0000 7f9c33800640 1 --2- 172.21.15.107:0/3605973434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c2c106f00 0x7f9c2c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:22.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.924+0000 7f9c33800640 1 --2- 172.21.15.107:0/3605973434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c2c102780 0x7f9c2c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:22.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.924+0000 7f9c33800640 1 --2- 172.21.15.107:0/3605973434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c2c101dd0 0x7f9c2c1021b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:22.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.924+0000 7f9c33800640 1 -- 172.21.15.107:0/3605973434 >> 172.21.15.107:0/3605973434 conn(0x7f9c2c0fd8b0 msgr2=0x7f9c2c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:22.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.924+0000 7f9c33800640 1 -- 172.21.15.107:0/3605973434 shutdown_connections 2024-09-19T04:19:22.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.924+0000 7f9c33800640 1 -- 172.21.15.107:0/3605973434 wait complete. 2024-09-19T04:19:22.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.925+0000 7f9c33800640 1 Processor -- start 2024-09-19T04:19:22.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c33800640 1 -- start start 2024-09-19T04:19:22.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c33800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c2c101dd0 0x7f9c2c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:22.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c33800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c2c102780 0x7f9c2c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:22.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c33800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c2c106f00 0x7f9c2c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:22.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c33800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c111f70 con 0x7f9c2c106f00 2024-09-19T04:19:22.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c33800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c111df0 con 0x7f9c2c101dd0 2024-09-19T04:19:22.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c33800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c1120f0 con 0x7f9c2c102780 2024-09-19T04:19:22.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c2be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c2c102780 0x7f9c2c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:22.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c31800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c2c106f00 0x7f9c2c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:22.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c2be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c2c102780 0x7f9c2c19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:40130/0 (socket says 172.21.15.107:40130) 2024-09-19T04:19:22.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c31800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c2c106f00 0x7f9c2c19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37926/0 (socket says 172.21.15.107:37926) 2024-09-19T04:19:22.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c2be00640 1 -- 172.21.15.107:0/2180880039 learned_addr learned my addr 172.21.15.107:0/2180880039 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:22.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.926+0000 7f9c30e00640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c2c101dd0 0x7f9c2c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:22.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.927+0000 7f9c2be00640 1 -- 172.21.15.107:0/2180880039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c2c101dd0 msgr2=0x7f9c2c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:22.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.927+0000 7f9c2be00640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c2c101dd0 0x7f9c2c19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:22.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.927+0000 7f9c2be00640 1 -- 172.21.15.107:0/2180880039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c2c106f00 msgr2=0x7f9c2c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:22.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.927+0000 7f9c2be00640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c2c106f00 0x7f9c2c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:22.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.927+0000 7f9c2be00640 1 -- 172.21.15.107:0/2180880039 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2c19f700 con 0x7f9c2c102780 2024-09-19T04:19:22.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.927+0000 7f9c2be00640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c2c102780 0x7f9c2c19ac90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e9f0 tx=0x7f9c1800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:22.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.927+0000 7f9c29600640 1 -- 172.21.15.107:0/2180880039 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1800cd80 con 0x7f9c2c102780 2024-09-19T04:19:22.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.928+0000 7f9c29600640 1 -- 172.21.15.107:0/2180880039 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c18010910 con 0x7f9c2c102780 2024-09-19T04:19:22.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.928+0000 7f9c33800640 1 -- 172.21.15.107:0/2180880039 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2c19f9f0 con 0x7f9c2c102780 2024-09-19T04:19:22.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.928+0000 7f9c29600640 1 -- 172.21.15.107:0/2180880039 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18018990 con 0x7f9c2c102780 2024-09-19T04:19:22.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.928+0000 7f9c33800640 1 -- 172.21.15.107:0/2180880039 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c2c1a72d0 con 0x7f9c2c102780 2024-09-19T04:19:22.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.929+0000 7f9c33800640 1 -- 172.21.15.107:0/2180880039 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c005290 con 0x7f9c2c102780 2024-09-19T04:19:22.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.930+0000 7f9c29600640 1 -- 172.21.15.107:0/2180880039 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9c18018b30 con 0x7f9c2c102780 2024-09-19T04:19:22.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.932+0000 7f9c29600640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9c140a3350 0x7f9c140a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:22.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.933+0000 7f9c29600640 1 -- 172.21.15.107:0/2180880039 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f9c180cfc60 con 0x7f9c2c102780 2024-09-19T04:19:22.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.933+0000 7f9c30e00640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9c140a3350 0x7f9c140a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:22.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.933+0000 7f9c30e00640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9c140a3350 0x7f9c140a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9c20000c00 tx=0x7f9c20005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:22.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:22.936+0000 7f9c29600640 1 -- 172.21.15.107:0/2180880039 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c1809c1a0 con 0x7f9c2c102780 2024-09-19T04:19:23.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.116+0000 7f9c33800640 1 -- 172.21.15.107:0/2180880039 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c0c000bc0 con 0x7f9c140a3350 2024-09-19T04:19:23.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:22 smithi107 ceph-mon[30659]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:23.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:23.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:22 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:23.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.123+0000 7f9c29600640 1 -- 172.21.15.107:0/2180880039 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258923 (secure 0 0 0) 0x7f9c0c000bc0 con 0x7f9c140a3350 2024-09-19T04:19:23.127 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:23.128 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-19T04:19:23.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.127+0000 7f9c33800640 1 -- 172.21.15.107:0/2180880039 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9c140a3350 msgr2=0x7f9c140a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.127+0000 7f9c33800640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9c140a3350 0x7f9c140a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9c20000c00 tx=0x7f9c20005b50 comp rx=0 tx=0).stop 2024-09-19T04:19:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.127+0000 7f9c33800640 1 -- 172.21.15.107:0/2180880039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c2c102780 msgr2=0x7f9c2c19ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.127+0000 7f9c33800640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c2c102780 0x7f9c2c19ac90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e9f0 tx=0x7f9c1800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.127+0000 7f9c33800640 1 -- 172.21.15.107:0/2180880039 shutdown_connections 2024-09-19T04:19:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.127+0000 7f9c33800640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9c140a3350 0x7f9c140a5810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.127+0000 7f9c33800640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c2c106f00 0x7f9c2c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.127+0000 7f9c33800640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c2c102780 0x7f9c2c19ac90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.127+0000 7f9c33800640 1 --2- 172.21.15.107:0/2180880039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c2c101dd0 0x7f9c2c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.127+0000 7f9c33800640 1 -- 172.21.15.107:0/2180880039 >> 172.21.15.107:0/2180880039 conn(0x7f9c2c0fd8b0 msgr2=0x7f9c2c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.128+0000 7f9c33800640 1 -- 172.21.15.107:0/2180880039 shutdown_connections 2024-09-19T04:19:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:23.128+0000 7f9c33800640 1 -- 172.21.15.107:0/2180880039 wait complete. 2024-09-19T04:19:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:22 smithi115 ceph-mon[35477]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:22 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:22 smithi145 ceph-mon[35096]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:22 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:23.886 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-19T04:19:21.824885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657574,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":714,"num_read_kb":483,"num_write":428,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373024,"kb_used_data":11232,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568800,"statfs":{"total":1151940427776,"available":1151558451200,"internally_reserved":0,"allocated":11501568,"data_stored":7248222,"data_compressed":107517,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003131"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975267+0000","last_change":"2024-09-19T04:18:12.499720+0000","last_active":"2024-09-19T04:19:00.975267+0000","last_peered":"2024-09-19T04:19:00.975267+0000","last_clean":"2024-09-19T04:19:00.975267+0000","last_became_active":"2024-09-19T04:18:12.499381+0000","last_became_peered":"2024-09-19T04:18:12.499381+0000","last_unstale":"2024-09-19T04:19:00.975267+0000","last_undegraded":"2024-09-19T04:19:00.975267+0000","last_fullsized":"2024-09-19T04:19:00.975267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:46.708870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598633+0000","last_change":"2024-09-19T04:18:12.498163+0000","last_active":"2024-09-19T04:19:01.598633+0000","last_peered":"2024-09-19T04:19:01.598633+0000","last_clean":"2024-09-19T04:19:01.598633+0000","last_became_active":"2024-09-19T04:18:12.495706+0000","last_became_peered":"2024-09-19T04:18:12.495706+0000","last_unstale":"2024-09-19T04:19:01.598633+0000","last_undegraded":"2024-09-19T04:19:01.598633+0000","last_fullsized":"2024-09-19T04:19:01.598633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:50.604410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598798+0000","last_change":"2024-09-19T04:18:12.493903+0000","last_active":"2024-09-19T04:19:01.598798+0000","last_peered":"2024-09-19T04:19:01.598798+0000","last_clean":"2024-09-19T04:19:01.598798+0000","last_became_active":"2024-09-19T04:18:12.493443+0000","last_became_peered":"2024-09-19T04:18:12.493443+0000","last_unstale":"2024-09-19T04:19:01.598798+0000","last_undegraded":"2024-09-19T04:19:01.598798+0000","last_fullsized":"2024-09-19T04:19:01.598798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:51.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002757+0000","last_change":"2024-09-19T04:18:12.500460+0000","last_active":"2024-09-19T04:19:01.002757+0000","last_peered":"2024-09-19T04:19:01.002757+0000","last_clean":"2024-09-19T04:19:01.002757+0000","last_became_active":"2024-09-19T04:18:12.494675+0000","last_became_peered":"2024-09-19T04:18:12.494675+0000","last_unstale":"2024-09-19T04:19:01.002757+0000","last_undegraded":"2024-09-19T04:19:01.002757+0000","last_fullsized":"2024-09-19T04:19:01.002757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:13:32.863156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978519+0000","last_change":"2024-09-19T04:18:12.494936+0000","last_active":"2024-09-19T04:19:00.978519+0000","last_peered":"2024-09-19T04:19:00.978519+0000","last_clean":"2024-09-19T04:19:00.978519+0000","last_became_active":"2024-09-19T04:18:12.494556+0000","last_became_peered":"2024-09-19T04:18:12.494556+0000","last_unstale":"2024-09-19T04:19:00.978519+0000","last_undegraded":"2024-09-19T04:19:00.978519+0000","last_fullsized":"2024-09-19T04:19:00.978519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:06.001529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002015+0000","last_change":"2024-09-19T04:18:12.495880+0000","last_active":"2024-09-19T04:19:01.002015+0000","last_peered":"2024-09-19T04:19:01.002015+0000","last_clean":"2024-09-19T04:19:01.002015+0000","last_became_active":"2024-09-19T04:18:12.495562+0000","last_became_peered":"2024-09-19T04:18:12.495562+0000","last_unstale":"2024-09-19T04:19:01.002015+0000","last_undegraded":"2024-09-19T04:19:01.002015+0000","last_fullsized":"2024-09-19T04:19:01.002015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:20.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975163+0000","last_change":"2024-09-19T04:18:12.497991+0000","last_active":"2024-09-19T04:19:00.975163+0000","last_peered":"2024-09-19T04:19:00.975163+0000","last_clean":"2024-09-19T04:19:00.975163+0000","last_became_active":"2024-09-19T04:18:12.497644+0000","last_became_peered":"2024-09-19T04:18:12.497644+0000","last_unstale":"2024-09-19T04:19:00.975163+0000","last_undegraded":"2024-09-19T04:19:00.975163+0000","last_fullsized":"2024-09-19T04:19:00.975163+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.851930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977676+0000","last_change":"2024-09-19T04:18:12.495789+0000","last_active":"2024-09-19T04:19:00.977676+0000","last_peered":"2024-09-19T04:19:00.977676+0000","last_clean":"2024-09-19T04:19:00.977676+0000","last_became_active":"2024-09-19T04:18:12.494637+0000","last_became_peered":"2024-09-19T04:18:12.494637+0000","last_unstale":"2024-09-19T04:19:00.977676+0000","last_undegraded":"2024-09-19T04:19:00.977676+0000","last_fullsized":"2024-09-19T04:19:00.977676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:14.245180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.974103+0000","last_change":"2024-09-19T04:18:12.491433+0000","last_active":"2024-09-19T04:19:00.974103+0000","last_peered":"2024-09-19T04:19:00.974103+0000","last_clean":"2024-09-19T04:19:00.974103+0000","last_became_active":"2024-09-19T04:18:12.490444+0000","last_became_peered":"2024-09-19T04:18:12.490444+0000","last_unstale":"2024-09-19T04:19:00.974103+0000","last_undegraded":"2024-09-19T04:19:00.974103+0000","last_fullsized":"2024-09-19T04:19:00.974103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:43.068355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002508+0000","last_change":"2024-09-19T04:18:12.493188+0000","last_active":"2024-09-19T04:19:01.002508+0000","last_peered":"2024-09-19T04:19:01.002508+0000","last_clean":"2024-09-19T04:19:01.002508+0000","last_became_active":"2024-09-19T04:18:12.492788+0000","last_became_peered":"2024-09-19T04:18:12.492788+0000","last_unstale":"2024-09-19T04:19:01.002508+0000","last_undegraded":"2024-09-19T04:19:01.002508+0000","last_fullsized":"2024-09-19T04:19:01.002508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:17.912820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975419+0000","last_change":"2024-09-19T04:18:12.490616+0000","last_active":"2024-09-19T04:19:00.975419+0000","last_peered":"2024-09-19T04:19:00.975419+0000","last_clean":"2024-09-19T04:19:00.975419+0000","last_became_active":"2024-09-19T04:18:12.490155+0000","last_became_peered":"2024-09-19T04:18:12.490155+0000","last_unstale":"2024-09-19T04:19:00.975419+0000","last_undegraded":"2024-09-19T04:19:00.975419+0000","last_fullsized":"2024-09-19T04:19:00.975419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:47.082082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002827+0000","last_change":"2024-09-19T04:18:12.494191+0000","last_active":"2024-09-19T04:19:01.002827+0000","last_peered":"2024-09-19T04:19:01.002827+0000","last_clean":"2024-09-19T04:19:01.002827+0000","last_became_active":"2024-09-19T04:18:12.493786+0000","last_became_peered":"2024-09-19T04:18:12.493786+0000","last_unstale":"2024-09-19T04:19:01.002827+0000","last_undegraded":"2024-09-19T04:19:01.002827+0000","last_fullsized":"2024-09-19T04:19:01.002827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:29.691556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975367+0000","last_change":"2024-09-19T04:18:12.482167+0000","last_active":"2024-09-19T04:19:00.975367+0000","last_peered":"2024-09-19T04:19:00.975367+0000","last_clean":"2024-09-19T04:19:00.975367+0000","last_became_active":"2024-09-19T04:18:12.481314+0000","last_became_peered":"2024-09-19T04:18:12.481314+0000","last_unstale":"2024-09-19T04:19:00.975367+0000","last_undegraded":"2024-09-19T04:19:00.975367+0000","last_fullsized":"2024-09-19T04:19:00.975367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:28.081641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598564+0000","last_change":"2024-09-19T04:18:12.491711+0000","last_active":"2024-09-19T04:19:01.598564+0000","last_peered":"2024-09-19T04:19:01.598564+0000","last_clean":"2024-09-19T04:19:01.598564+0000","last_became_active":"2024-09-19T04:18:12.490964+0000","last_became_peered":"2024-09-19T04:18:12.490964+0000","last_unstale":"2024-09-19T04:19:01.598564+0000","last_undegraded":"2024-09-19T04:19:01.598564+0000","last_fullsized":"2024-09-19T04:19:01.598564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:16.607679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598664+0000","last_change":"2024-09-19T04:18:12.504737+0000","last_active":"2024-09-19T04:19:01.598664+0000","last_peered":"2024-09-19T04:19:01.598664+0000","last_clean":"2024-09-19T04:19:01.598664+0000","last_became_active":"2024-09-19T04:18:12.497583+0000","last_became_peered":"2024-09-19T04:18:12.497583+0000","last_unstale":"2024-09-19T04:19:01.598664+0000","last_undegraded":"2024-09-19T04:19:01.598664+0000","last_fullsized":"2024-09-19T04:19:01.598664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:48.456738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973467+0000","last_change":"2024-09-19T04:18:12.499045+0000","last_active":"2024-09-19T04:19:00.973467+0000","last_peered":"2024-09-19T04:19:00.973467+0000","last_clean":"2024-09-19T04:19:00.973467+0000","last_became_active":"2024-09-19T04:18:12.498713+0000","last_became_peered":"2024-09-19T04:18:12.498713+0000","last_unstale":"2024-09-19T04:19:00.973467+0000","last_undegraded":"2024-09-19T04:19:00.973467+0000","last_fullsized":"2024-09-19T04:19:00.973467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:54.337769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975318+0000","last_change":"2024-09-19T04:18:12.495989+0000","last_active":"2024-09-19T04:19:00.975318+0000","last_peered":"2024-09-19T04:19:00.975318+0000","last_clean":"2024-09-19T04:19:00.975318+0000","last_became_active":"2024-09-19T04:18:12.495633+0000","last_became_peered":"2024-09-19T04:18:12.495633+0000","last_unstale":"2024-09-19T04:19:00.975318+0000","last_undegraded":"2024-09-19T04:19:00.975318+0000","last_fullsized":"2024-09-19T04:19:00.975318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:17.225861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975106+0000","last_change":"2024-09-19T04:18:12.488532+0000","last_active":"2024-09-19T04:19:00.975106+0000","last_peered":"2024-09-19T04:19:00.975106+0000","last_clean":"2024-09-19T04:19:00.975106+0000","last_became_active":"2024-09-19T04:18:12.488117+0000","last_became_peered":"2024-09-19T04:18:12.488117+0000","last_unstale":"2024-09-19T04:19:00.975106+0000","last_undegraded":"2024-09-19T04:19:00.975106+0000","last_fullsized":"2024-09-19T04:19:00.975106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:45.318852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973745+0000","last_change":"2024-09-19T04:18:12.496784+0000","last_active":"2024-09-19T04:19:00.973745+0000","last_peered":"2024-09-19T04:19:00.973745+0000","last_clean":"2024-09-19T04:19:00.973745+0000","last_became_active":"2024-09-19T04:18:12.496443+0000","last_became_peered":"2024-09-19T04:18:12.496443+0000","last_unstale":"2024-09-19T04:19:00.973745+0000","last_undegraded":"2024-09-19T04:19:00.973745+0000","last_fullsized":"2024-09-19T04:19:00.973745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:25:46.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.980557+0000","last_change":"2024-09-19T04:19:00.980557+0000","last_active":"2024-09-19T04:19:00.980557+0000","last_peered":"2024-09-19T04:19:00.980557+0000","last_clean":"2024-09-19T04:19:00.980557+0000","last_became_active":"2024-09-19T04:19:00.980149+0000","last_became_peered":"2024-09-19T04:19:00.980149+0000","last_unstale":"2024-09-19T04:19:00.980557+0000","last_undegraded":"2024-09-19T04:19:00.980557+0000","last_fullsized":"2024-09-19T04:19:00.980557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:06.140085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975449+0000","last_change":"2024-09-19T04:18:12.493731+0000","last_active":"2024-09-19T04:19:00.975449+0000","last_peered":"2024-09-19T04:19:00.975449+0000","last_clean":"2024-09-19T04:19:00.975449+0000","last_became_active":"2024-09-19T04:18:12.493412+0000","last_became_peered":"2024-09-19T04:18:12.493412+0000","last_unstale":"2024-09-19T04:19:00.975449+0000","last_undegraded":"2024-09-19T04:19:00.975449+0000","last_fullsized":"2024-09-19T04:19:00.975449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:41.089637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973552+0000","last_change":"2024-09-19T04:18:12.497033+0000","last_active":"2024-09-19T04:19:00.973552+0000","last_peered":"2024-09-19T04:19:00.973552+0000","last_clean":"2024-09-19T04:19:00.973552+0000","last_became_active":"2024-09-19T04:18:12.496710+0000","last_became_peered":"2024-09-19T04:18:12.496710+0000","last_unstale":"2024-09-19T04:19:00.973552+0000","last_undegraded":"2024-09-19T04:19:00.973552+0000","last_fullsized":"2024-09-19T04:19:00.973552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:19.193869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977658+0000","last_change":"2024-09-19T04:18:12.499213+0000","last_active":"2024-09-19T04:19:00.977658+0000","last_peered":"2024-09-19T04:19:00.977658+0000","last_clean":"2024-09-19T04:19:00.977658+0000","last_became_active":"2024-09-19T04:18:12.498728+0000","last_became_peered":"2024-09-19T04:18:12.498728+0000","last_unstale":"2024-09-19T04:19:00.977658+0000","last_undegraded":"2024-09-19T04:19:00.977658+0000","last_fullsized":"2024-09-19T04:19:00.977658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:37.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.983070+0000","last_change":"2024-09-19T04:19:00.982647+0000","last_active":"2024-09-19T04:19:00.983070+0000","last_peered":"2024-09-19T04:19:00.983070+0000","last_clean":"2024-09-19T04:19:00.983070+0000","last_became_active":"2024-09-19T04:19:00.982246+0000","last_became_peered":"2024-09-19T04:19:00.982246+0000","last_unstale":"2024-09-19T04:19:00.983070+0000","last_undegraded":"2024-09-19T04:19:00.983070+0000","last_fullsized":"2024-09-19T04:19:00.983070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:48.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977926+0000","last_change":"2024-09-19T04:18:12.506679+0000","last_active":"2024-09-19T04:19:00.977926+0000","last_peered":"2024-09-19T04:19:00.977926+0000","last_clean":"2024-09-19T04:19:00.977926+0000","last_became_active":"2024-09-19T04:18:12.505953+0000","last_became_peered":"2024-09-19T04:18:12.505953+0000","last_unstale":"2024-09-19T04:19:00.977926+0000","last_undegraded":"2024-09-19T04:19:00.977926+0000","last_fullsized":"2024-09-19T04:19:00.977926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:53.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973348+0000","last_change":"2024-09-19T04:18:12.490511+0000","last_active":"2024-09-19T04:19:00.973348+0000","last_peered":"2024-09-19T04:19:00.973348+0000","last_clean":"2024-09-19T04:19:00.973348+0000","last_became_active":"2024-09-19T04:18:12.490051+0000","last_became_peered":"2024-09-19T04:18:12.490051+0000","last_unstale":"2024-09-19T04:19:00.973348+0000","last_undegraded":"2024-09-19T04:19:00.973348+0000","last_fullsized":"2024-09-19T04:19:00.973348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:32.283312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975844+0000","last_change":"2024-09-19T04:18:12.491465+0000","last_active":"2024-09-19T04:19:00.975844+0000","last_peered":"2024-09-19T04:19:00.975844+0000","last_clean":"2024-09-19T04:19:00.975844+0000","last_became_active":"2024-09-19T04:18:12.491136+0000","last_became_peered":"2024-09-19T04:18:12.491136+0000","last_unstale":"2024-09-19T04:19:00.975844+0000","last_undegraded":"2024-09-19T04:19:00.975844+0000","last_fullsized":"2024-09-19T04:19:00.975844+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:29.498909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.599944+0000","last_change":"2024-09-19T04:19:01.599944+0000","last_active":"2024-09-19T04:19:01.599944+0000","last_peered":"2024-09-19T04:19:01.599944+0000","last_clean":"2024-09-19T04:19:01.599944+0000","last_became_active":"2024-09-19T04:19:01.599571+0000","last_became_peered":"2024-09-19T04:19:01.599571+0000","last_unstale":"2024-09-19T04:19:01.599944+0000","last_undegraded":"2024-09-19T04:19:01.599944+0000","last_fullsized":"2024-09-19T04:19:01.599944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:17.929785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598676+0000","last_change":"2024-09-19T04:18:12.504364+0000","last_active":"2024-09-19T04:19:01.598676+0000","last_peered":"2024-09-19T04:19:01.598676+0000","last_clean":"2024-09-19T04:19:01.598676+0000","last_became_active":"2024-09-19T04:18:12.504006+0000","last_became_peered":"2024-09-19T04:18:12.504006+0000","last_unstale":"2024-09-19T04:19:01.598676+0000","last_undegraded":"2024-09-19T04:19:01.598676+0000","last_fullsized":"2024-09-19T04:19:01.598676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:53.309963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973049+0000","last_change":"2024-09-19T04:18:12.498223+0000","last_active":"2024-09-19T04:19:00.973049+0000","last_peered":"2024-09-19T04:19:00.973049+0000","last_clean":"2024-09-19T04:19:00.973049+0000","last_became_active":"2024-09-19T04:18:12.497862+0000","last_became_peered":"2024-09-19T04:18:12.497862+0000","last_unstale":"2024-09-19T04:19:00.973049+0000","last_undegraded":"2024-09-19T04:19:00.973049+0000","last_fullsized":"2024-09-19T04:19:00.973049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:28.858620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973075+0000","last_change":"2024-09-19T04:18:12.502335+0000","last_active":"2024-09-19T04:19:00.973075+0000","last_peered":"2024-09-19T04:19:00.973075+0000","last_clean":"2024-09-19T04:19:00.973075+0000","last_became_active":"2024-09-19T04:18:12.501686+0000","last_became_peered":"2024-09-19T04:18:12.501686+0000","last_unstale":"2024-09-19T04:19:00.973075+0000","last_undegraded":"2024-09-19T04:19:00.973075+0000","last_fullsized":"2024-09-19T04:19:00.973075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:22.348745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.974873+0000","last_change":"2024-09-19T04:18:12.490905+0000","last_active":"2024-09-19T04:19:00.974873+0000","last_peered":"2024-09-19T04:19:00.974873+0000","last_clean":"2024-09-19T04:19:00.974873+0000","last_became_active":"2024-09-19T04:18:12.490489+0000","last_became_peered":"2024-09-19T04:18:12.490489+0000","last_unstale":"2024-09-19T04:19:00.974873+0000","last_undegraded":"2024-09-19T04:19:00.974873+0000","last_fullsized":"2024-09-19T04:19:00.974873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.946474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978569+0000","last_change":"2024-09-19T04:18:12.494607+0000","last_active":"2024-09-19T04:19:00.978569+0000","last_peered":"2024-09-19T04:19:00.978569+0000","last_clean":"2024-09-19T04:19:00.978569+0000","last_became_active":"2024-09-19T04:18:12.494288+0000","last_became_peered":"2024-09-19T04:18:12.494288+0000","last_unstale":"2024-09-19T04:19:00.978569+0000","last_undegraded":"2024-09-19T04:19:00.978569+0000","last_fullsized":"2024-09-19T04:19:00.978569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:42.824951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":85,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:17.653814+0000","last_change":"2024-09-19T04:18:10.103108+0000","last_active":"2024-09-19T04:19:17.653814+0000","last_peered":"2024-09-19T04:19:17.653814+0000","last_clean":"2024-09-19T04:19:17.653814+0000","last_became_active":"2024-09-19T04:18:10.102783+0000","last_became_peered":"2024-09-19T04:18:10.102783+0000","last_unstale":"2024-09-19T04:19:17.653814+0000","last_undegraded":"2024-09-19T04:19:17.653814+0000","last_fullsized":"2024-09-19T04:19:17.653814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:25.521953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002255+0000","last_change":"2024-09-19T04:18:12.495453+0000","last_active":"2024-09-19T04:19:01.002255+0000","last_peered":"2024-09-19T04:19:01.002255+0000","last_clean":"2024-09-19T04:19:01.002255+0000","last_became_active":"2024-09-19T04:18:12.494744+0000","last_became_peered":"2024-09-19T04:18:12.494744+0000","last_unstale":"2024-09-19T04:19:01.002255+0000","last_undegraded":"2024-09-19T04:19:01.002255+0000","last_fullsized":"2024-09-19T04:19:01.002255+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:41.071723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002121+0000","last_change":"2024-09-19T04:18:10.102675+0000","last_active":"2024-09-19T04:19:01.002121+0000","last_peered":"2024-09-19T04:19:01.002121+0000","last_clean":"2024-09-19T04:19:01.002121+0000","last_became_active":"2024-09-19T04:18:10.102368+0000","last_became_peered":"2024-09-19T04:18:10.102368+0000","last_unstale":"2024-09-19T04:19:01.002121+0000","last_undegraded":"2024-09-19T04:19:01.002121+0000","last_fullsized":"2024-09-19T04:19:01.002121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:45.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975790+0000","last_change":"2024-09-19T04:18:10.099932+0000","last_active":"2024-09-19T04:19:00.975790+0000","last_peered":"2024-09-19T04:19:00.975790+0000","last_clean":"2024-09-19T04:19:00.975790+0000","last_became_active":"2024-09-19T04:18:10.099637+0000","last_became_peered":"2024-09-19T04:18:10.099637+0000","last_unstale":"2024-09-19T04:19:00.975790+0000","last_undegraded":"2024-09-19T04:19:00.975790+0000","last_fullsized":"2024-09-19T04:19:00.975790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:50.594382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977722+0000","last_change":"2024-09-19T04:18:12.502229+0000","last_active":"2024-09-19T04:19:00.977722+0000","last_peered":"2024-09-19T04:19:00.977722+0000","last_clean":"2024-09-19T04:19:00.977722+0000","last_became_active":"2024-09-19T04:18:12.501921+0000","last_became_peered":"2024-09-19T04:18:12.501921+0000","last_unstale":"2024-09-19T04:19:00.977722+0000","last_undegraded":"2024-09-19T04:19:00.977722+0000","last_fullsized":"2024-09-19T04:19:00.977722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:55.412103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002119+0000","last_change":"2024-09-19T04:18:10.103961+0000","last_active":"2024-09-19T04:19:01.002119+0000","last_peered":"2024-09-19T04:19:01.002119+0000","last_clean":"2024-09-19T04:19:01.002119+0000","last_became_active":"2024-09-19T04:18:10.102737+0000","last_became_peered":"2024-09-19T04:18:10.102737+0000","last_unstale":"2024-09-19T04:19:01.002119+0000","last_undegraded":"2024-09-19T04:19:01.002119+0000","last_fullsized":"2024-09-19T04:19:01.002119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:31:01.806124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.974828+0000","last_change":"2024-09-19T04:18:12.502385+0000","last_active":"2024-09-19T04:19:00.974828+0000","last_peered":"2024-09-19T04:19:00.974828+0000","last_clean":"2024-09-19T04:19:00.974828+0000","last_became_active":"2024-09-19T04:18:12.502025+0000","last_became_peered":"2024-09-19T04:18:12.502025+0000","last_unstale":"2024-09-19T04:19:00.974828+0000","last_undegraded":"2024-09-19T04:19:00.974828+0000","last_fullsized":"2024-09-19T04:19:00.974828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:24.650911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:17.653089+0000","last_change":"2024-09-19T04:18:10.103526+0000","last_active":"2024-09-19T04:19:17.653089+0000","last_peered":"2024-09-19T04:19:17.653089+0000","last_clean":"2024-09-19T04:19:17.653089+0000","last_became_active":"2024-09-19T04:18:10.103191+0000","last_became_peered":"2024-09-19T04:18:10.103191+0000","last_unstale":"2024-09-19T04:19:17.653089+0000","last_undegraded":"2024-09-19T04:19:17.653089+0000","last_fullsized":"2024-09-19T04:19:17.653089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:07.695715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973858+0000","last_change":"2024-09-19T04:18:12.493169+0000","last_active":"2024-09-19T04:19:00.973858+0000","last_peered":"2024-09-19T04:19:00.973858+0000","last_clean":"2024-09-19T04:19:00.973858+0000","last_became_active":"2024-09-19T04:18:12.492669+0000","last_became_peered":"2024-09-19T04:18:12.492669+0000","last_unstale":"2024-09-19T04:19:00.973858+0000","last_undegraded":"2024-09-19T04:19:00.973858+0000","last_fullsized":"2024-09-19T04:19:00.973858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:04.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.007275+0000","last_change":"2024-09-19T04:19:01.007275+0000","last_active":"2024-09-19T04:19:01.007275+0000","last_peered":"2024-09-19T04:19:01.007275+0000","last_clean":"2024-09-19T04:19:01.007275+0000","last_became_active":"2024-09-19T04:19:01.003361+0000","last_became_peered":"2024-09-19T04:19:01.003361+0000","last_unstale":"2024-09-19T04:19:01.007275+0000","last_undegraded":"2024-09-19T04:19:01.007275+0000","last_fullsized":"2024-09-19T04:19:01.007275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:07:11.907462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598830+0000","last_change":"2024-09-19T04:18:12.504348+0000","last_active":"2024-09-19T04:19:01.598830+0000","last_peered":"2024-09-19T04:19:01.598830+0000","last_clean":"2024-09-19T04:19:01.598830+0000","last_became_active":"2024-09-19T04:18:12.503180+0000","last_became_peered":"2024-09-19T04:18:12.503180+0000","last_unstale":"2024-09-19T04:19:01.598830+0000","last_undegraded":"2024-09-19T04:19:01.598830+0000","last_fullsized":"2024-09-19T04:19:01.598830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:03.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002213+0000","last_change":"2024-09-19T04:18:10.365136+0000","last_active":"2024-09-19T04:19:01.002213+0000","last_peered":"2024-09-19T04:19:01.002213+0000","last_clean":"2024-09-19T04:19:01.002213+0000","last_became_active":"2024-09-19T04:18:10.364594+0000","last_became_peered":"2024-09-19T04:18:10.364594+0000","last_unstale":"2024-09-19T04:19:01.002213+0000","last_undegraded":"2024-09-19T04:19:01.002213+0000","last_fullsized":"2024-09-19T04:19:01.002213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:52.252157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973750+0000","last_change":"2024-09-19T04:18:12.502051+0000","last_active":"2024-09-19T04:19:00.973750+0000","last_peered":"2024-09-19T04:19:00.973750+0000","last_clean":"2024-09-19T04:19:00.973750+0000","last_became_active":"2024-09-19T04:18:12.501578+0000","last_became_peered":"2024-09-19T04:18:12.501578+0000","last_unstale":"2024-09-19T04:19:00.973750+0000","last_undegraded":"2024-09-19T04:19:00.973750+0000","last_fullsized":"2024-09-19T04:19:00.973750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:59.693175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":167,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:17.653033+0000","last_change":"2024-09-19T04:18:10.101507+0000","last_active":"2024-09-19T04:19:17.653033+0000","last_peered":"2024-09-19T04:19:17.653033+0000","last_clean":"2024-09-19T04:19:17.653033+0000","last_became_active":"2024-09-19T04:18:10.101152+0000","last_became_peered":"2024-09-19T04:18:10.101152+0000","last_unstale":"2024-09-19T04:19:17.653033+0000","last_undegraded":"2024-09-19T04:19:17.653033+0000","last_fullsized":"2024-09-19T04:19:17.653033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:01.397605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978835+0000","last_change":"2024-09-19T04:18:12.491611+0000","last_active":"2024-09-19T04:19:00.978835+0000","last_peered":"2024-09-19T04:19:00.978835+0000","last_clean":"2024-09-19T04:19:00.978835+0000","last_became_active":"2024-09-19T04:18:12.491279+0000","last_became_peered":"2024-09-19T04:18:12.491279+0000","last_unstale":"2024-09-19T04:19:00.978835+0000","last_undegraded":"2024-09-19T04:19:00.978835+0000","last_fullsized":"2024-09-19T04:19:00.978835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:01.414713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973716+0000","last_change":"2024-09-19T04:18:10.105574+0000","last_active":"2024-09-19T04:19:00.973716+0000","last_peered":"2024-09-19T04:19:00.973716+0000","last_clean":"2024-09-19T04:19:00.973716+0000","last_became_active":"2024-09-19T04:18:10.105203+0000","last_became_peered":"2024-09-19T04:18:10.105203+0000","last_unstale":"2024-09-19T04:19:00.973716+0000","last_undegraded":"2024-09-19T04:19:00.973716+0000","last_fullsized":"2024-09-19T04:19:00.973716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:00.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.974853+0000","last_change":"2024-09-19T04:18:12.497242+0000","last_active":"2024-09-19T04:19:00.974853+0000","last_peered":"2024-09-19T04:19:00.974853+0000","last_clean":"2024-09-19T04:19:00.974853+0000","last_became_active":"2024-09-19T04:18:12.496732+0000","last_became_peered":"2024-09-19T04:18:12.496732+0000","last_unstale":"2024-09-19T04:19:00.974853+0000","last_undegraded":"2024-09-19T04:19:00.974853+0000","last_fullsized":"2024-09-19T04:19:00.974853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:56.520341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002017+0000","last_change":"2024-09-19T04:18:10.103905+0000","last_active":"2024-09-19T04:19:01.002017+0000","last_peered":"2024-09-19T04:19:01.002017+0000","last_clean":"2024-09-19T04:19:01.002017+0000","last_became_active":"2024-09-19T04:18:10.103517+0000","last_became_peered":"2024-09-19T04:18:10.103517+0000","last_unstale":"2024-09-19T04:19:01.002017+0000","last_undegraded":"2024-09-19T04:19:01.002017+0000","last_fullsized":"2024-09-19T04:19:01.002017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:48.292878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975328+0000","last_change":"2024-09-19T04:18:12.499380+0000","last_active":"2024-09-19T04:19:00.975328+0000","last_peered":"2024-09-19T04:19:00.975328+0000","last_clean":"2024-09-19T04:19:00.975328+0000","last_became_active":"2024-09-19T04:18:12.498995+0000","last_became_peered":"2024-09-19T04:18:12.498995+0000","last_unstale":"2024-09-19T04:19:00.975328+0000","last_undegraded":"2024-09-19T04:19:00.975328+0000","last_fullsized":"2024-09-19T04:19:00.975328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:20.653923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:16.652044+0000","last_change":"2024-09-19T04:18:10.105619+0000","last_active":"2024-09-19T04:19:16.652044+0000","last_peered":"2024-09-19T04:19:16.652044+0000","last_clean":"2024-09-19T04:19:16.652044+0000","last_became_active":"2024-09-19T04:18:10.105082+0000","last_became_peered":"2024-09-19T04:18:10.105082+0000","last_unstale":"2024-09-19T04:19:16.652044+0000","last_undegraded":"2024-09-19T04:19:16.652044+0000","last_fullsized":"2024-09-19T04:19:16.652044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:56.843068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975098+0000","last_change":"2024-09-19T04:18:12.498473+0000","last_active":"2024-09-19T04:19:00.975098+0000","last_peered":"2024-09-19T04:19:00.975098+0000","last_clean":"2024-09-19T04:19:00.975098+0000","last_became_active":"2024-09-19T04:18:12.496904+0000","last_became_peered":"2024-09-19T04:18:12.496904+0000","last_unstale":"2024-09-19T04:19:00.975098+0000","last_undegraded":"2024-09-19T04:19:00.975098+0000","last_fullsized":"2024-09-19T04:19:00.975098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:03.155899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978505+0000","last_change":"2024-09-19T04:18:10.365182+0000","last_active":"2024-09-19T04:19:00.978505+0000","last_peered":"2024-09-19T04:19:00.978505+0000","last_clean":"2024-09-19T04:19:00.978505+0000","last_became_active":"2024-09-19T04:18:10.364717+0000","last_became_peered":"2024-09-19T04:18:10.364717+0000","last_unstale":"2024-09-19T04:19:00.978505+0000","last_undegraded":"2024-09-19T04:19:00.978505+0000","last_fullsized":"2024-09-19T04:19:00.978505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:32.063066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.001950+0000","last_change":"2024-09-19T04:18:12.493585+0000","last_active":"2024-09-19T04:19:01.001950+0000","last_peered":"2024-09-19T04:19:01.001950+0000","last_clean":"2024-09-19T04:19:01.001950+0000","last_became_active":"2024-09-19T04:18:12.493255+0000","last_became_peered":"2024-09-19T04:18:12.493255+0000","last_unstale":"2024-09-19T04:19:01.001950+0000","last_undegraded":"2024-09-19T04:19:01.001950+0000","last_fullsized":"2024-09-19T04:19:01.001950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:33.139727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.979069+0000","last_change":"2024-09-19T04:18:10.101537+0000","last_active":"2024-09-19T04:19:00.979069+0000","last_peered":"2024-09-19T04:19:00.979069+0000","last_clean":"2024-09-19T04:19:00.979069+0000","last_became_active":"2024-09-19T04:18:10.100951+0000","last_became_peered":"2024-09-19T04:18:10.100951+0000","last_unstale":"2024-09-19T04:19:00.979069+0000","last_undegraded":"2024-09-19T04:19:00.979069+0000","last_fullsized":"2024-09-19T04:19:00.979069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:08.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975115+0000","last_change":"2024-09-19T04:18:12.495530+0000","last_active":"2024-09-19T04:19:00.975115+0000","last_peered":"2024-09-19T04:19:00.975115+0000","last_clean":"2024-09-19T04:19:00.975115+0000","last_became_active":"2024-09-19T04:18:12.495176+0000","last_became_peered":"2024-09-19T04:18:12.495176+0000","last_unstale":"2024-09-19T04:19:00.975115+0000","last_undegraded":"2024-09-19T04:19:00.975115+0000","last_fullsized":"2024-09-19T04:19:00.975115+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:24.308047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.974791+0000","last_change":"2024-09-19T04:18:10.102539+0000","last_active":"2024-09-19T04:19:00.974791+0000","last_peered":"2024-09-19T04:19:00.974791+0000","last_clean":"2024-09-19T04:19:00.974791+0000","last_became_active":"2024-09-19T04:18:10.102109+0000","last_became_peered":"2024-09-19T04:18:10.102109+0000","last_unstale":"2024-09-19T04:19:00.974791+0000","last_undegraded":"2024-09-19T04:19:00.974791+0000","last_fullsized":"2024-09-19T04:19:00.974791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:58.852767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975776+0000","last_change":"2024-09-19T04:18:12.491769+0000","last_active":"2024-09-19T04:19:00.975776+0000","last_peered":"2024-09-19T04:19:00.975776+0000","last_clean":"2024-09-19T04:19:00.975776+0000","last_became_active":"2024-09-19T04:18:12.491435+0000","last_became_peered":"2024-09-19T04:18:12.491435+0000","last_unstale":"2024-09-19T04:19:00.975776+0000","last_undegraded":"2024-09-19T04:19:00.975776+0000","last_fullsized":"2024-09-19T04:19:00.975776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:54.880533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002301+0000","last_change":"2024-09-19T04:18:10.103559+0000","last_active":"2024-09-19T04:19:01.002301+0000","last_peered":"2024-09-19T04:19:01.002301+0000","last_clean":"2024-09-19T04:19:01.002301+0000","last_became_active":"2024-09-19T04:18:10.103208+0000","last_became_peered":"2024-09-19T04:18:10.103208+0000","last_unstale":"2024-09-19T04:19:01.002301+0000","last_undegraded":"2024-09-19T04:19:01.002301+0000","last_fullsized":"2024-09-19T04:19:01.002301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:16.564916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975331+0000","last_change":"2024-09-19T04:18:12.495616+0000","last_active":"2024-09-19T04:19:00.975331+0000","last_peered":"2024-09-19T04:19:00.975331+0000","last_clean":"2024-09-19T04:19:00.975331+0000","last_became_active":"2024-09-19T04:18:12.495243+0000","last_became_peered":"2024-09-19T04:18:12.495243+0000","last_unstale":"2024-09-19T04:19:00.975331+0000","last_undegraded":"2024-09-19T04:19:00.975331+0000","last_fullsized":"2024-09-19T04:19:00.975331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:12.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":84,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:16.652485+0000","last_change":"2024-09-19T04:18:10.626874+0000","last_active":"2024-09-19T04:19:16.652485+0000","last_peered":"2024-09-19T04:19:16.652485+0000","last_clean":"2024-09-19T04:19:16.652485+0000","last_became_active":"2024-09-19T04:18:10.626506+0000","last_became_peered":"2024-09-19T04:18:10.626506+0000","last_unstale":"2024-09-19T04:19:16.652485+0000","last_undegraded":"2024-09-19T04:19:16.652485+0000","last_fullsized":"2024-09-19T04:19:16.652485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:48.725143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975644+0000","last_change":"2024-09-19T04:18:12.496350+0000","last_active":"2024-09-19T04:19:00.975644+0000","last_peered":"2024-09-19T04:19:00.975644+0000","last_clean":"2024-09-19T04:19:00.975644+0000","last_became_active":"2024-09-19T04:18:12.495904+0000","last_became_peered":"2024-09-19T04:18:12.495904+0000","last_unstale":"2024-09-19T04:19:00.975644+0000","last_undegraded":"2024-09-19T04:19:00.975644+0000","last_fullsized":"2024-09-19T04:19:00.975644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:08.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":347,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975303+0000","last_change":"2024-09-19T04:16:56.164450+0000","last_active":"2024-09-19T04:19:00.975303+0000","last_peered":"2024-09-19T04:19:00.975303+0000","last_clean":"2024-09-19T04:19:00.975303+0000","last_became_active":"2024-09-19T04:16:56.164008+0000","last_became_peered":"2024-09-19T04:16:56.164008+0000","last_unstale":"2024-09-19T04:19:00.975303+0000","last_undegraded":"2024-09-19T04:19:00.975303+0000","last_fullsized":"2024-09-19T04:19:00.975303+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_clean_scrub_stamp":"2024-09-19T04:13:18.308718+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:14.697725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002444+0000","last_change":"2024-09-19T04:18:10.101598+0000","last_active":"2024-09-19T04:19:01.002444+0000","last_peered":"2024-09-19T04:19:01.002444+0000","last_clean":"2024-09-19T04:19:01.002444+0000","last_became_active":"2024-09-19T04:18:10.101210+0000","last_became_peered":"2024-09-19T04:18:10.101210+0000","last_unstale":"2024-09-19T04:19:01.002444+0000","last_undegraded":"2024-09-19T04:19:01.002444+0000","last_fullsized":"2024-09-19T04:19:01.002444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:10.653481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977862+0000","last_change":"2024-09-19T04:18:12.495034+0000","last_active":"2024-09-19T04:19:00.977862+0000","last_peered":"2024-09-19T04:19:00.977862+0000","last_clean":"2024-09-19T04:19:00.977862+0000","last_became_active":"2024-09-19T04:18:12.494611+0000","last_became_peered":"2024-09-19T04:18:12.494611+0000","last_unstale":"2024-09-19T04:19:00.977862+0000","last_undegraded":"2024-09-19T04:19:00.977862+0000","last_fullsized":"2024-09-19T04:19:00.977862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:28.448927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975345+0000","last_change":"2024-09-19T04:18:10.105212+0000","last_active":"2024-09-19T04:19:00.975345+0000","last_peered":"2024-09-19T04:19:00.975345+0000","last_clean":"2024-09-19T04:19:00.975345+0000","last_became_active":"2024-09-19T04:18:10.104710+0000","last_became_peered":"2024-09-19T04:18:10.104710+0000","last_unstale":"2024-09-19T04:19:00.975345+0000","last_undegraded":"2024-09-19T04:19:00.975345+0000","last_fullsized":"2024-09-19T04:19:00.975345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:58.007773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.980419+0000","last_change":"2024-09-19T04:19:00.980419+0000","last_active":"2024-09-19T04:19:00.980419+0000","last_peered":"2024-09-19T04:19:00.980419+0000","last_clean":"2024-09-19T04:19:00.980419+0000","last_became_active":"2024-09-19T04:19:00.980008+0000","last_became_peered":"2024-09-19T04:19:00.980008+0000","last_unstale":"2024-09-19T04:19:00.980419+0000","last_undegraded":"2024-09-19T04:19:00.980419+0000","last_fullsized":"2024-09-19T04:19:00.980419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:18.396611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":84,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:19.655325+0000","last_change":"2024-09-19T04:18:10.364530+0000","last_active":"2024-09-19T04:19:19.655325+0000","last_peered":"2024-09-19T04:19:19.655325+0000","last_clean":"2024-09-19T04:19:19.655325+0000","last_became_active":"2024-09-19T04:18:10.364204+0000","last_became_peered":"2024-09-19T04:18:10.364204+0000","last_unstale":"2024-09-19T04:19:19.655325+0000","last_undegraded":"2024-09-19T04:19:19.655325+0000","last_fullsized":"2024-09-19T04:19:19.655325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:04.229530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002088+0000","last_change":"2024-09-19T04:18:12.493730+0000","last_active":"2024-09-19T04:19:01.002088+0000","last_peered":"2024-09-19T04:19:01.002088+0000","last_clean":"2024-09-19T04:19:01.002088+0000","last_became_active":"2024-09-19T04:18:12.493153+0000","last_became_peered":"2024-09-19T04:18:12.493153+0000","last_unstale":"2024-09-19T04:19:01.002088+0000","last_undegraded":"2024-09-19T04:19:01.002088+0000","last_fullsized":"2024-09-19T04:19:01.002088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.758490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:17.653056+0000","last_change":"2024-09-19T04:18:10.102073+0000","last_active":"2024-09-19T04:19:17.653056+0000","last_peered":"2024-09-19T04:19:17.653056+0000","last_clean":"2024-09-19T04:19:17.653056+0000","last_became_active":"2024-09-19T04:18:10.101755+0000","last_became_peered":"2024-09-19T04:18:10.101755+0000","last_unstale":"2024-09-19T04:19:17.653056+0000","last_undegraded":"2024-09-19T04:19:17.653056+0000","last_fullsized":"2024-09-19T04:19:17.653056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:21.489146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.001939+0000","last_change":"2024-09-19T04:18:12.494560+0000","last_active":"2024-09-19T04:19:01.001939+0000","last_peered":"2024-09-19T04:19:01.001939+0000","last_clean":"2024-09-19T04:19:01.001939+0000","last_became_active":"2024-09-19T04:18:12.494203+0000","last_became_peered":"2024-09-19T04:18:12.494203+0000","last_unstale":"2024-09-19T04:19:01.001939+0000","last_undegraded":"2024-09-19T04:19:01.001939+0000","last_fullsized":"2024-09-19T04:19:01.001939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:35:17.129649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975411+0000","last_change":"2024-09-19T04:18:10.365781+0000","last_active":"2024-09-19T04:19:00.975411+0000","last_peered":"2024-09-19T04:19:00.975411+0000","last_clean":"2024-09-19T04:19:00.975411+0000","last_became_active":"2024-09-19T04:18:10.365417+0000","last_became_peered":"2024-09-19T04:18:10.365417+0000","last_unstale":"2024-09-19T04:19:00.975411+0000","last_undegraded":"2024-09-19T04:19:00.975411+0000","last_fullsized":"2024-09-19T04:19:00.975411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:20.097572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.001998+0000","last_change":"2024-09-19T04:18:12.495570+0000","last_active":"2024-09-19T04:19:01.001998+0000","last_peered":"2024-09-19T04:19:01.001998+0000","last_clean":"2024-09-19T04:19:01.001998+0000","last_became_active":"2024-09-19T04:18:12.495242+0000","last_became_peered":"2024-09-19T04:18:12.495242+0000","last_unstale":"2024-09-19T04:19:01.001998+0000","last_undegraded":"2024-09-19T04:19:01.001998+0000","last_fullsized":"2024-09-19T04:19:01.001998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:57.448182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978902+0000","last_change":"2024-09-19T04:18:10.365134+0000","last_active":"2024-09-19T04:19:00.978902+0000","last_peered":"2024-09-19T04:19:00.978902+0000","last_clean":"2024-09-19T04:19:00.978902+0000","last_became_active":"2024-09-19T04:18:10.364726+0000","last_became_peered":"2024-09-19T04:18:10.364726+0000","last_unstale":"2024-09-19T04:19:00.978902+0000","last_undegraded":"2024-09-19T04:19:00.978902+0000","last_fullsized":"2024-09-19T04:19:00.978902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:44.207855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975566+0000","last_change":"2024-09-19T04:18:12.488836+0000","last_active":"2024-09-19T04:19:00.975566+0000","last_peered":"2024-09-19T04:19:00.975566+0000","last_clean":"2024-09-19T04:19:00.975566+0000","last_became_active":"2024-09-19T04:18:12.488507+0000","last_became_peered":"2024-09-19T04:18:12.488507+0000","last_unstale":"2024-09-19T04:19:00.975566+0000","last_undegraded":"2024-09-19T04:19:00.975566+0000","last_fullsized":"2024-09-19T04:19:00.975566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:31.308114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975204+0000","last_change":"2024-09-19T04:18:10.103623+0000","last_active":"2024-09-19T04:19:00.975204+0000","last_peered":"2024-09-19T04:19:00.975204+0000","last_clean":"2024-09-19T04:19:00.975204+0000","last_became_active":"2024-09-19T04:18:10.103340+0000","last_became_peered":"2024-09-19T04:18:10.103340+0000","last_unstale":"2024-09-19T04:19:00.975204+0000","last_undegraded":"2024-09-19T04:19:00.975204+0000","last_fullsized":"2024-09-19T04:19:00.975204+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:11.724013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978023+0000","last_change":"2024-09-19T04:18:12.498167+0000","last_active":"2024-09-19T04:19:00.978023+0000","last_peered":"2024-09-19T04:19:00.978023+0000","last_clean":"2024-09-19T04:19:00.978023+0000","last_became_active":"2024-09-19T04:18:12.496924+0000","last_became_peered":"2024-09-19T04:18:12.496924+0000","last_unstale":"2024-09-19T04:19:00.978023+0000","last_undegraded":"2024-09-19T04:19:00.978023+0000","last_fullsized":"2024-09-19T04:19:00.978023+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:13.952030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002785+0000","last_change":"2024-09-19T04:18:10.627106+0000","last_active":"2024-09-19T04:19:01.002785+0000","last_peered":"2024-09-19T04:19:01.002785+0000","last_clean":"2024-09-19T04:19:01.002785+0000","last_became_active":"2024-09-19T04:18:10.626708+0000","last_became_peered":"2024-09-19T04:18:10.626708+0000","last_unstale":"2024-09-19T04:19:01.002785+0000","last_undegraded":"2024-09-19T04:19:01.002785+0000","last_fullsized":"2024-09-19T04:19:01.002785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:46.049699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598893+0000","last_change":"2024-09-19T04:18:12.506995+0000","last_active":"2024-09-19T04:19:01.598893+0000","last_peered":"2024-09-19T04:19:01.598893+0000","last_clean":"2024-09-19T04:19:01.598893+0000","last_became_active":"2024-09-19T04:18:12.500607+0000","last_became_peered":"2024-09-19T04:18:12.500607+0000","last_unstale":"2024-09-19T04:19:01.598893+0000","last_undegraded":"2024-09-19T04:19:01.598893+0000","last_fullsized":"2024-09-19T04:19:01.598893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:37.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973607+0000","last_change":"2024-09-19T04:18:10.365150+0000","last_active":"2024-09-19T04:19:00.973607+0000","last_peered":"2024-09-19T04:19:00.973607+0000","last_clean":"2024-09-19T04:19:00.973607+0000","last_became_active":"2024-09-19T04:18:10.364755+0000","last_became_peered":"2024-09-19T04:18:10.364755+0000","last_unstale":"2024-09-19T04:19:00.973607+0000","last_undegraded":"2024-09-19T04:19:00.973607+0000","last_fullsized":"2024-09-19T04:19:00.973607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:11.639532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.979071+0000","last_change":"2024-09-19T04:18:12.484588+0000","last_active":"2024-09-19T04:19:00.979071+0000","last_peered":"2024-09-19T04:19:00.979071+0000","last_clean":"2024-09-19T04:19:00.979071+0000","last_became_active":"2024-09-19T04:18:12.484218+0000","last_became_peered":"2024-09-19T04:18:12.484218+0000","last_unstale":"2024-09-19T04:19:00.979071+0000","last_undegraded":"2024-09-19T04:19:00.979071+0000","last_fullsized":"2024-09-19T04:19:00.979071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:55.634557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977839+0000","last_change":"2024-09-19T04:18:10.100832+0000","last_active":"2024-09-19T04:19:00.977839+0000","last_peered":"2024-09-19T04:19:00.977839+0000","last_clean":"2024-09-19T04:19:00.977839+0000","last_became_active":"2024-09-19T04:18:10.100518+0000","last_became_peered":"2024-09-19T04:18:10.100518+0000","last_unstale":"2024-09-19T04:19:00.977839+0000","last_undegraded":"2024-09-19T04:19:00.977839+0000","last_fullsized":"2024-09-19T04:19:00.977839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:37.028120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975234+0000","last_change":"2024-09-19T04:18:12.496914+0000","last_active":"2024-09-19T04:19:00.975234+0000","last_peered":"2024-09-19T04:19:00.975234+0000","last_clean":"2024-09-19T04:19:00.975234+0000","last_became_active":"2024-09-19T04:18:12.496556+0000","last_became_peered":"2024-09-19T04:18:12.496556+0000","last_unstale":"2024-09-19T04:19:00.975234+0000","last_undegraded":"2024-09-19T04:19:00.975234+0000","last_fullsized":"2024-09-19T04:19:00.975234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:52.391412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.993937+0000","last_change":"2024-09-19T04:19:00.993461+0000","last_active":"2024-09-19T04:19:00.993937+0000","last_peered":"2024-09-19T04:19:00.993937+0000","last_clean":"2024-09-19T04:19:00.993937+0000","last_became_active":"2024-09-19T04:19:00.980704+0000","last_became_peered":"2024-09-19T04:19:00.980704+0000","last_unstale":"2024-09-19T04:19:00.993937+0000","last_undegraded":"2024-09-19T04:19:00.993937+0000","last_fullsized":"2024-09-19T04:19:00.993937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:56.090092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978004+0000","last_change":"2024-09-19T04:18:12.496852+0000","last_active":"2024-09-19T04:19:00.978004+0000","last_peered":"2024-09-19T04:19:00.978004+0000","last_clean":"2024-09-19T04:19:00.978004+0000","last_became_active":"2024-09-19T04:18:12.495365+0000","last_became_peered":"2024-09-19T04:18:12.495365+0000","last_unstale":"2024-09-19T04:19:00.978004+0000","last_undegraded":"2024-09-19T04:19:00.978004+0000","last_fullsized":"2024-09-19T04:19:00.978004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:14:57.012618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978566+0000","last_change":"2024-09-19T04:18:10.104641+0000","last_active":"2024-09-19T04:19:00.978566+0000","last_peered":"2024-09-19T04:19:00.978566+0000","last_clean":"2024-09-19T04:19:00.978566+0000","last_became_active":"2024-09-19T04:18:10.104295+0000","last_became_peered":"2024-09-19T04:18:10.104295+0000","last_unstale":"2024-09-19T04:19:00.978566+0000","last_undegraded":"2024-09-19T04:19:00.978566+0000","last_fullsized":"2024-09-19T04:19:00.978566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:58.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975362+0000","last_change":"2024-09-19T04:18:12.488396+0000","last_active":"2024-09-19T04:19:00.975362+0000","last_peered":"2024-09-19T04:19:00.975362+0000","last_clean":"2024-09-19T04:19:00.975362+0000","last_became_active":"2024-09-19T04:18:12.488048+0000","last_became_peered":"2024-09-19T04:18:12.488048+0000","last_unstale":"2024-09-19T04:19:00.975362+0000","last_undegraded":"2024-09-19T04:19:00.975362+0000","last_fullsized":"2024-09-19T04:19:00.975362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:27.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002204+0000","last_change":"2024-09-19T04:18:12.493722+0000","last_active":"2024-09-19T04:19:01.002204+0000","last_peered":"2024-09-19T04:19:01.002204+0000","last_clean":"2024-09-19T04:19:01.002204+0000","last_became_active":"2024-09-19T04:18:12.492483+0000","last_became_peered":"2024-09-19T04:18:12.492483+0000","last_unstale":"2024-09-19T04:19:01.002204+0000","last_undegraded":"2024-09-19T04:19:01.002204+0000","last_fullsized":"2024-09-19T04:19:01.002204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:33.492623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.979376+0000","last_change":"2024-09-19T04:18:10.101326+0000","last_active":"2024-09-19T04:19:00.979376+0000","last_peered":"2024-09-19T04:19:00.979376+0000","last_clean":"2024-09-19T04:19:00.979376+0000","last_became_active":"2024-09-19T04:18:10.100955+0000","last_became_peered":"2024-09-19T04:18:10.100955+0000","last_unstale":"2024-09-19T04:19:00.979376+0000","last_undegraded":"2024-09-19T04:19:00.979376+0000","last_fullsized":"2024-09-19T04:19:00.979376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:41.939741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973618+0000","last_change":"2024-09-19T04:18:12.492750+0000","last_active":"2024-09-19T04:19:00.973618+0000","last_peered":"2024-09-19T04:19:00.973618+0000","last_clean":"2024-09-19T04:19:00.973618+0000","last_became_active":"2024-09-19T04:18:12.492417+0000","last_became_peered":"2024-09-19T04:18:12.492417+0000","last_unstale":"2024-09-19T04:19:00.973618+0000","last_undegraded":"2024-09-19T04:19:00.973618+0000","last_fullsized":"2024-09-19T04:19:00.973618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:25.279452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.990252+0000","last_change":"2024-09-19T04:19:00.989741+0000","last_active":"2024-09-19T04:19:00.990252+0000","last_peered":"2024-09-19T04:19:00.990252+0000","last_clean":"2024-09-19T04:19:00.990252+0000","last_became_active":"2024-09-19T04:19:00.977530+0000","last_became_peered":"2024-09-19T04:19:00.977530+0000","last_unstale":"2024-09-19T04:19:00.990252+0000","last_undegraded":"2024-09-19T04:19:00.990252+0000","last_fullsized":"2024-09-19T04:19:00.990252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:35.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598809+0000","last_change":"2024-09-19T04:18:12.503510+0000","last_active":"2024-09-19T04:19:01.598809+0000","last_peered":"2024-09-19T04:19:01.598809+0000","last_clean":"2024-09-19T04:19:01.598809+0000","last_became_active":"2024-09-19T04:18:12.495165+0000","last_became_peered":"2024-09-19T04:18:12.495165+0000","last_unstale":"2024-09-19T04:19:01.598809+0000","last_undegraded":"2024-09-19T04:19:01.598809+0000","last_fullsized":"2024-09-19T04:19:01.598809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:13.934328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978447+0000","last_change":"2024-09-19T04:18:10.364783+0000","last_active":"2024-09-19T04:19:00.978447+0000","last_peered":"2024-09-19T04:19:00.978447+0000","last_clean":"2024-09-19T04:19:00.978447+0000","last_became_active":"2024-09-19T04:18:10.364202+0000","last_became_peered":"2024-09-19T04:18:10.364202+0000","last_unstale":"2024-09-19T04:19:00.978447+0000","last_undegraded":"2024-09-19T04:19:00.978447+0000","last_fullsized":"2024-09-19T04:19:00.978447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:18:55.770637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978229+0000","last_change":"2024-09-19T04:18:12.495825+0000","last_active":"2024-09-19T04:19:00.978229+0000","last_peered":"2024-09-19T04:19:00.978229+0000","last_clean":"2024-09-19T04:19:00.978229+0000","last_became_active":"2024-09-19T04:18:12.494525+0000","last_became_peered":"2024-09-19T04:18:12.494525+0000","last_unstale":"2024-09-19T04:19:00.978229+0000","last_undegraded":"2024-09-19T04:19:00.978229+0000","last_fullsized":"2024-09-19T04:19:00.978229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:57.827135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:20.656911+0000","last_change":"2024-09-19T04:18:10.102145+0000","last_active":"2024-09-19T04:19:20.656911+0000","last_peered":"2024-09-19T04:19:20.656911+0000","last_clean":"2024-09-19T04:19:20.656911+0000","last_became_active":"2024-09-19T04:18:10.101410+0000","last_became_peered":"2024-09-19T04:18:10.101410+0000","last_unstale":"2024-09-19T04:19:20.656911+0000","last_undegraded":"2024-09-19T04:19:20.656911+0000","last_fullsized":"2024-09-19T04:19:20.656911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:20.354296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66972,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":578,"num_read_kb":364,"num_write":245,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281762,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547227,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":441210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1064960,"data_stored":1091610,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972391,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":440747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201323,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":441125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430255,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":1033645,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593782,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":440464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822716,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":1033323,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":440325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215173,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":440289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":999424,"data_stored":502147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":501141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":937984,"data_stored":442196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:19:23.890 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:19:24.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:19:24.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:19:24.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:23 smithi115 ceph-mon[35477]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:24.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:19:24.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:19:24.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:23 smithi145 ceph-mon[35096]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:24.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:19:24.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:19:24.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:23 smithi107 ceph-mon[30659]: from='mgr.14146 172.21.15.107:0/1512822829' entity='mgr.x' 2024-09-19T04:19:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:24.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.790+0000 7fbf8f600640 1 -- 172.21.15.107:0/4213263245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf88102790 msgr2=0x7fbf881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.790+0000 7fbf8f600640 1 --2- 172.21.15.107:0/4213263245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf88102790 0x7fbf881067e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009900 tx=0x7fbf7402a410 comp rx=0 tx=0).stop 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.790+0000 7fbf8f600640 1 -- 172.21.15.107:0/4213263245 shutdown_connections 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.790+0000 7fbf8f600640 1 --2- 172.21.15.107:0/4213263245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf88106f10 0x7fbf8810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.790+0000 7fbf8f600640 1 --2- 172.21.15.107:0/4213263245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf88102790 0x7fbf881067e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.790+0000 7fbf8f600640 1 --2- 172.21.15.107:0/4213263245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf88101de0 0x7fbf881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.790+0000 7fbf8f600640 1 -- 172.21.15.107:0/4213263245 >> 172.21.15.107:0/4213263245 conn(0x7fbf880fd8c0 msgr2=0x7fbf880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.791+0000 7fbf8f600640 1 -- 172.21.15.107:0/4213263245 shutdown_connections 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.791+0000 7fbf8f600640 1 -- 172.21.15.107:0/4213263245 wait complete. 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.791+0000 7fbf8f600640 1 Processor -- start 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8f600640 1 -- start start 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8f600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf88101de0 0x7fbf8819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8f600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf88102790 0x7fbf8819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8f600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf88106f10 0x7fbf8819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8f600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88111f70 con 0x7fbf88106f10 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8f600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88111df0 con 0x7fbf88101de0 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8f600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881120f0 con 0x7fbf88102790 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf88106f10 0x7fbf8819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf88106f10 0x7fbf8819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37940/0 (socket says 172.21.15.107:37940) 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8cc00640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf88101de0 0x7fbf8819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf8d600640 1 -- 172.21.15.107:0/1376603731 learned_addr learned my addr 172.21.15.107:0/1376603731 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.792+0000 7fbf87e00640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf88102790 0x7fbf8819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf8d600640 1 -- 172.21.15.107:0/1376603731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf88102790 msgr2=0x7fbf8819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf8d600640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf88102790 0x7fbf8819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf8d600640 1 -- 172.21.15.107:0/1376603731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf88101de0 msgr2=0x7fbf8819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf8d600640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf88101de0 0x7fbf8819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf8d600640 1 -- 172.21.15.107:0/1376603731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8819f5e0 con 0x7fbf88106f10 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf8d600640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf88106f10 0x7fbf8819ef00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000eb10 tx=0x7fbf8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf85600640 1 -- 172.21.15.107:0/1376603731 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80004b20 con 0x7fbf88106f10 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf85600640 1 -- 172.21.15.107:0/1376603731 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf80004cc0 con 0x7fbf88106f10 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf8f600640 1 -- 172.21.15.107:0/1376603731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8819f8d0 con 0x7fbf88106f10 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf85600640 1 -- 172.21.15.107:0/1376603731 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80010760 con 0x7fbf88106f10 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.793+0000 7fbf8f600640 1 -- 172.21.15.107:0/1376603731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf881a71b0 con 0x7fbf88106f10 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.795+0000 7fbf8f600640 1 -- 172.21.15.107:0/1376603731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf881021c0 con 0x7fbf88106f10 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.796+0000 7fbf85600640 1 -- 172.21.15.107:0/1376603731 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbf80005020 con 0x7fbf88106f10 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.797+0000 7fbf85600640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf6c0a32d0 0x7fbf6c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.798+0000 7fbf85600640 1 -- 172.21.15.107:0/1376603731 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fbf800d0110 con 0x7fbf88106f10 2024-09-19T04:19:24.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.798+0000 7fbf8cc00640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf6c0a32d0 0x7fbf6c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:24.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.799+0000 7fbf8cc00640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf6c0a32d0 0x7fbf6c0a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009bd0 tx=0x7fbf78005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:24.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.802+0000 7fbf85600640 1 -- 172.21.15.107:0/1376603731 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbf8009cca0 con 0x7fbf88106f10 2024-09-19T04:19:24.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.975+0000 7fbf8f600640 1 -- 172.21.15.107:0/1376603731 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf880fec40 con 0x7fbf6c0a32d0 2024-09-19T04:19:24.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.982+0000 7fbf85600640 1 -- 172.21.15.107:0/1376603731 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258923 (secure 0 0 0) 0x7fbf880fec40 con 0x7fbf6c0a32d0 2024-09-19T04:19:24.986 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:24.987 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-19T04:19:24.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 -- 172.21.15.107:0/1376603731 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf6c0a32d0 msgr2=0x7fbf6c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:24.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf6c0a32d0 0x7fbf6c0a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009bd0 tx=0x7fbf78005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:24.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 -- 172.21.15.107:0/1376603731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf88106f10 msgr2=0x7fbf8819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf88106f10 0x7fbf8819ef00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000eb10 tx=0x7fbf8000c2d0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 -- 172.21.15.107:0/1376603731 shutdown_connections 2024-09-19T04:19:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbf6c0a32d0 0x7fbf6c0a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf88106f10 0x7fbf8819ef00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf88102790 0x7fbf8819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 --2- 172.21.15.107:0/1376603731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf88101de0 0x7fbf8819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 -- 172.21.15.107:0/1376603731 >> 172.21.15.107:0/1376603731 conn(0x7fbf880fd8c0 msgr2=0x7fbf880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 -- 172.21.15.107:0/1376603731 shutdown_connections 2024-09-19T04:19:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:24.986+0000 7fbf8f600640 1 -- 172.21.15.107:0/1376603731 wait complete. 2024-09-19T04:19:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:24 smithi145 ceph-mon[35096]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:19:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:24 smithi145 ceph-mon[35096]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:24 smithi107 ceph-mon[30659]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:19:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:24 smithi107 ceph-mon[30659]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:25.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:24 smithi115 ceph-mon[35477]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:19:25.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:24 smithi115 ceph-mon[35477]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:25.699 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-19T04:19:23.825418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657574,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":744,"num_read_kb":498,"num_write":428,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373024,"kb_used_data":11232,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568800,"statfs":{"total":1151940427776,"available":1151558451200,"internally_reserved":0,"allocated":11501568,"data_stored":7248222,"data_compressed":107517,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003080"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975267+0000","last_change":"2024-09-19T04:18:12.499720+0000","last_active":"2024-09-19T04:19:00.975267+0000","last_peered":"2024-09-19T04:19:00.975267+0000","last_clean":"2024-09-19T04:19:00.975267+0000","last_became_active":"2024-09-19T04:18:12.499381+0000","last_became_peered":"2024-09-19T04:18:12.499381+0000","last_unstale":"2024-09-19T04:19:00.975267+0000","last_undegraded":"2024-09-19T04:19:00.975267+0000","last_fullsized":"2024-09-19T04:19:00.975267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:46.708870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598633+0000","last_change":"2024-09-19T04:18:12.498163+0000","last_active":"2024-09-19T04:19:01.598633+0000","last_peered":"2024-09-19T04:19:01.598633+0000","last_clean":"2024-09-19T04:19:01.598633+0000","last_became_active":"2024-09-19T04:18:12.495706+0000","last_became_peered":"2024-09-19T04:18:12.495706+0000","last_unstale":"2024-09-19T04:19:01.598633+0000","last_undegraded":"2024-09-19T04:19:01.598633+0000","last_fullsized":"2024-09-19T04:19:01.598633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:50.604410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598798+0000","last_change":"2024-09-19T04:18:12.493903+0000","last_active":"2024-09-19T04:19:01.598798+0000","last_peered":"2024-09-19T04:19:01.598798+0000","last_clean":"2024-09-19T04:19:01.598798+0000","last_became_active":"2024-09-19T04:18:12.493443+0000","last_became_peered":"2024-09-19T04:18:12.493443+0000","last_unstale":"2024-09-19T04:19:01.598798+0000","last_undegraded":"2024-09-19T04:19:01.598798+0000","last_fullsized":"2024-09-19T04:19:01.598798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:51.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002757+0000","last_change":"2024-09-19T04:18:12.500460+0000","last_active":"2024-09-19T04:19:01.002757+0000","last_peered":"2024-09-19T04:19:01.002757+0000","last_clean":"2024-09-19T04:19:01.002757+0000","last_became_active":"2024-09-19T04:18:12.494675+0000","last_became_peered":"2024-09-19T04:18:12.494675+0000","last_unstale":"2024-09-19T04:19:01.002757+0000","last_undegraded":"2024-09-19T04:19:01.002757+0000","last_fullsized":"2024-09-19T04:19:01.002757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:13:32.863156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978519+0000","last_change":"2024-09-19T04:18:12.494936+0000","last_active":"2024-09-19T04:19:00.978519+0000","last_peered":"2024-09-19T04:19:00.978519+0000","last_clean":"2024-09-19T04:19:00.978519+0000","last_became_active":"2024-09-19T04:18:12.494556+0000","last_became_peered":"2024-09-19T04:18:12.494556+0000","last_unstale":"2024-09-19T04:19:00.978519+0000","last_undegraded":"2024-09-19T04:19:00.978519+0000","last_fullsized":"2024-09-19T04:19:00.978519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:06.001529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002015+0000","last_change":"2024-09-19T04:18:12.495880+0000","last_active":"2024-09-19T04:19:01.002015+0000","last_peered":"2024-09-19T04:19:01.002015+0000","last_clean":"2024-09-19T04:19:01.002015+0000","last_became_active":"2024-09-19T04:18:12.495562+0000","last_became_peered":"2024-09-19T04:18:12.495562+0000","last_unstale":"2024-09-19T04:19:01.002015+0000","last_undegraded":"2024-09-19T04:19:01.002015+0000","last_fullsized":"2024-09-19T04:19:01.002015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:20.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975163+0000","last_change":"2024-09-19T04:18:12.497991+0000","last_active":"2024-09-19T04:19:00.975163+0000","last_peered":"2024-09-19T04:19:00.975163+0000","last_clean":"2024-09-19T04:19:00.975163+0000","last_became_active":"2024-09-19T04:18:12.497644+0000","last_became_peered":"2024-09-19T04:18:12.497644+0000","last_unstale":"2024-09-19T04:19:00.975163+0000","last_undegraded":"2024-09-19T04:19:00.975163+0000","last_fullsized":"2024-09-19T04:19:00.975163+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.851930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977676+0000","last_change":"2024-09-19T04:18:12.495789+0000","last_active":"2024-09-19T04:19:00.977676+0000","last_peered":"2024-09-19T04:19:00.977676+0000","last_clean":"2024-09-19T04:19:00.977676+0000","last_became_active":"2024-09-19T04:18:12.494637+0000","last_became_peered":"2024-09-19T04:18:12.494637+0000","last_unstale":"2024-09-19T04:19:00.977676+0000","last_undegraded":"2024-09-19T04:19:00.977676+0000","last_fullsized":"2024-09-19T04:19:00.977676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:14.245180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.974103+0000","last_change":"2024-09-19T04:18:12.491433+0000","last_active":"2024-09-19T04:19:00.974103+0000","last_peered":"2024-09-19T04:19:00.974103+0000","last_clean":"2024-09-19T04:19:00.974103+0000","last_became_active":"2024-09-19T04:18:12.490444+0000","last_became_peered":"2024-09-19T04:18:12.490444+0000","last_unstale":"2024-09-19T04:19:00.974103+0000","last_undegraded":"2024-09-19T04:19:00.974103+0000","last_fullsized":"2024-09-19T04:19:00.974103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:43.068355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002508+0000","last_change":"2024-09-19T04:18:12.493188+0000","last_active":"2024-09-19T04:19:01.002508+0000","last_peered":"2024-09-19T04:19:01.002508+0000","last_clean":"2024-09-19T04:19:01.002508+0000","last_became_active":"2024-09-19T04:18:12.492788+0000","last_became_peered":"2024-09-19T04:18:12.492788+0000","last_unstale":"2024-09-19T04:19:01.002508+0000","last_undegraded":"2024-09-19T04:19:01.002508+0000","last_fullsized":"2024-09-19T04:19:01.002508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:17.912820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975419+0000","last_change":"2024-09-19T04:18:12.490616+0000","last_active":"2024-09-19T04:19:00.975419+0000","last_peered":"2024-09-19T04:19:00.975419+0000","last_clean":"2024-09-19T04:19:00.975419+0000","last_became_active":"2024-09-19T04:18:12.490155+0000","last_became_peered":"2024-09-19T04:18:12.490155+0000","last_unstale":"2024-09-19T04:19:00.975419+0000","last_undegraded":"2024-09-19T04:19:00.975419+0000","last_fullsized":"2024-09-19T04:19:00.975419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:47.082082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002827+0000","last_change":"2024-09-19T04:18:12.494191+0000","last_active":"2024-09-19T04:19:01.002827+0000","last_peered":"2024-09-19T04:19:01.002827+0000","last_clean":"2024-09-19T04:19:01.002827+0000","last_became_active":"2024-09-19T04:18:12.493786+0000","last_became_peered":"2024-09-19T04:18:12.493786+0000","last_unstale":"2024-09-19T04:19:01.002827+0000","last_undegraded":"2024-09-19T04:19:01.002827+0000","last_fullsized":"2024-09-19T04:19:01.002827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:29.691556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975367+0000","last_change":"2024-09-19T04:18:12.482167+0000","last_active":"2024-09-19T04:19:00.975367+0000","last_peered":"2024-09-19T04:19:00.975367+0000","last_clean":"2024-09-19T04:19:00.975367+0000","last_became_active":"2024-09-19T04:18:12.481314+0000","last_became_peered":"2024-09-19T04:18:12.481314+0000","last_unstale":"2024-09-19T04:19:00.975367+0000","last_undegraded":"2024-09-19T04:19:00.975367+0000","last_fullsized":"2024-09-19T04:19:00.975367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:28.081641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598564+0000","last_change":"2024-09-19T04:18:12.491711+0000","last_active":"2024-09-19T04:19:01.598564+0000","last_peered":"2024-09-19T04:19:01.598564+0000","last_clean":"2024-09-19T04:19:01.598564+0000","last_became_active":"2024-09-19T04:18:12.490964+0000","last_became_peered":"2024-09-19T04:18:12.490964+0000","last_unstale":"2024-09-19T04:19:01.598564+0000","last_undegraded":"2024-09-19T04:19:01.598564+0000","last_fullsized":"2024-09-19T04:19:01.598564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:16.607679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598664+0000","last_change":"2024-09-19T04:18:12.504737+0000","last_active":"2024-09-19T04:19:01.598664+0000","last_peered":"2024-09-19T04:19:01.598664+0000","last_clean":"2024-09-19T04:19:01.598664+0000","last_became_active":"2024-09-19T04:18:12.497583+0000","last_became_peered":"2024-09-19T04:18:12.497583+0000","last_unstale":"2024-09-19T04:19:01.598664+0000","last_undegraded":"2024-09-19T04:19:01.598664+0000","last_fullsized":"2024-09-19T04:19:01.598664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:48.456738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973467+0000","last_change":"2024-09-19T04:18:12.499045+0000","last_active":"2024-09-19T04:19:00.973467+0000","last_peered":"2024-09-19T04:19:00.973467+0000","last_clean":"2024-09-19T04:19:00.973467+0000","last_became_active":"2024-09-19T04:18:12.498713+0000","last_became_peered":"2024-09-19T04:18:12.498713+0000","last_unstale":"2024-09-19T04:19:00.973467+0000","last_undegraded":"2024-09-19T04:19:00.973467+0000","last_fullsized":"2024-09-19T04:19:00.973467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:54.337769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975318+0000","last_change":"2024-09-19T04:18:12.495989+0000","last_active":"2024-09-19T04:19:00.975318+0000","last_peered":"2024-09-19T04:19:00.975318+0000","last_clean":"2024-09-19T04:19:00.975318+0000","last_became_active":"2024-09-19T04:18:12.495633+0000","last_became_peered":"2024-09-19T04:18:12.495633+0000","last_unstale":"2024-09-19T04:19:00.975318+0000","last_undegraded":"2024-09-19T04:19:00.975318+0000","last_fullsized":"2024-09-19T04:19:00.975318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:17.225861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975106+0000","last_change":"2024-09-19T04:18:12.488532+0000","last_active":"2024-09-19T04:19:00.975106+0000","last_peered":"2024-09-19T04:19:00.975106+0000","last_clean":"2024-09-19T04:19:00.975106+0000","last_became_active":"2024-09-19T04:18:12.488117+0000","last_became_peered":"2024-09-19T04:18:12.488117+0000","last_unstale":"2024-09-19T04:19:00.975106+0000","last_undegraded":"2024-09-19T04:19:00.975106+0000","last_fullsized":"2024-09-19T04:19:00.975106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:45.318852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973745+0000","last_change":"2024-09-19T04:18:12.496784+0000","last_active":"2024-09-19T04:19:00.973745+0000","last_peered":"2024-09-19T04:19:00.973745+0000","last_clean":"2024-09-19T04:19:00.973745+0000","last_became_active":"2024-09-19T04:18:12.496443+0000","last_became_peered":"2024-09-19T04:18:12.496443+0000","last_unstale":"2024-09-19T04:19:00.973745+0000","last_undegraded":"2024-09-19T04:19:00.973745+0000","last_fullsized":"2024-09-19T04:19:00.973745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:25:46.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.980557+0000","last_change":"2024-09-19T04:19:00.980557+0000","last_active":"2024-09-19T04:19:00.980557+0000","last_peered":"2024-09-19T04:19:00.980557+0000","last_clean":"2024-09-19T04:19:00.980557+0000","last_became_active":"2024-09-19T04:19:00.980149+0000","last_became_peered":"2024-09-19T04:19:00.980149+0000","last_unstale":"2024-09-19T04:19:00.980557+0000","last_undegraded":"2024-09-19T04:19:00.980557+0000","last_fullsized":"2024-09-19T04:19:00.980557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:06.140085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975449+0000","last_change":"2024-09-19T04:18:12.493731+0000","last_active":"2024-09-19T04:19:00.975449+0000","last_peered":"2024-09-19T04:19:00.975449+0000","last_clean":"2024-09-19T04:19:00.975449+0000","last_became_active":"2024-09-19T04:18:12.493412+0000","last_became_peered":"2024-09-19T04:18:12.493412+0000","last_unstale":"2024-09-19T04:19:00.975449+0000","last_undegraded":"2024-09-19T04:19:00.975449+0000","last_fullsized":"2024-09-19T04:19:00.975449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:41.089637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973552+0000","last_change":"2024-09-19T04:18:12.497033+0000","last_active":"2024-09-19T04:19:00.973552+0000","last_peered":"2024-09-19T04:19:00.973552+0000","last_clean":"2024-09-19T04:19:00.973552+0000","last_became_active":"2024-09-19T04:18:12.496710+0000","last_became_peered":"2024-09-19T04:18:12.496710+0000","last_unstale":"2024-09-19T04:19:00.973552+0000","last_undegraded":"2024-09-19T04:19:00.973552+0000","last_fullsized":"2024-09-19T04:19:00.973552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:19.193869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977658+0000","last_change":"2024-09-19T04:18:12.499213+0000","last_active":"2024-09-19T04:19:00.977658+0000","last_peered":"2024-09-19T04:19:00.977658+0000","last_clean":"2024-09-19T04:19:00.977658+0000","last_became_active":"2024-09-19T04:18:12.498728+0000","last_became_peered":"2024-09-19T04:18:12.498728+0000","last_unstale":"2024-09-19T04:19:00.977658+0000","last_undegraded":"2024-09-19T04:19:00.977658+0000","last_fullsized":"2024-09-19T04:19:00.977658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:37.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.983070+0000","last_change":"2024-09-19T04:19:00.982647+0000","last_active":"2024-09-19T04:19:00.983070+0000","last_peered":"2024-09-19T04:19:00.983070+0000","last_clean":"2024-09-19T04:19:00.983070+0000","last_became_active":"2024-09-19T04:19:00.982246+0000","last_became_peered":"2024-09-19T04:19:00.982246+0000","last_unstale":"2024-09-19T04:19:00.983070+0000","last_undegraded":"2024-09-19T04:19:00.983070+0000","last_fullsized":"2024-09-19T04:19:00.983070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:48.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977926+0000","last_change":"2024-09-19T04:18:12.506679+0000","last_active":"2024-09-19T04:19:00.977926+0000","last_peered":"2024-09-19T04:19:00.977926+0000","last_clean":"2024-09-19T04:19:00.977926+0000","last_became_active":"2024-09-19T04:18:12.505953+0000","last_became_peered":"2024-09-19T04:18:12.505953+0000","last_unstale":"2024-09-19T04:19:00.977926+0000","last_undegraded":"2024-09-19T04:19:00.977926+0000","last_fullsized":"2024-09-19T04:19:00.977926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:53.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973348+0000","last_change":"2024-09-19T04:18:12.490511+0000","last_active":"2024-09-19T04:19:00.973348+0000","last_peered":"2024-09-19T04:19:00.973348+0000","last_clean":"2024-09-19T04:19:00.973348+0000","last_became_active":"2024-09-19T04:18:12.490051+0000","last_became_peered":"2024-09-19T04:18:12.490051+0000","last_unstale":"2024-09-19T04:19:00.973348+0000","last_undegraded":"2024-09-19T04:19:00.973348+0000","last_fullsized":"2024-09-19T04:19:00.973348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:32.283312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975844+0000","last_change":"2024-09-19T04:18:12.491465+0000","last_active":"2024-09-19T04:19:00.975844+0000","last_peered":"2024-09-19T04:19:00.975844+0000","last_clean":"2024-09-19T04:19:00.975844+0000","last_became_active":"2024-09-19T04:18:12.491136+0000","last_became_peered":"2024-09-19T04:18:12.491136+0000","last_unstale":"2024-09-19T04:19:00.975844+0000","last_undegraded":"2024-09-19T04:19:00.975844+0000","last_fullsized":"2024-09-19T04:19:00.975844+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:29.498909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.599944+0000","last_change":"2024-09-19T04:19:01.599944+0000","last_active":"2024-09-19T04:19:01.599944+0000","last_peered":"2024-09-19T04:19:01.599944+0000","last_clean":"2024-09-19T04:19:01.599944+0000","last_became_active":"2024-09-19T04:19:01.599571+0000","last_became_peered":"2024-09-19T04:19:01.599571+0000","last_unstale":"2024-09-19T04:19:01.599944+0000","last_undegraded":"2024-09-19T04:19:01.599944+0000","last_fullsized":"2024-09-19T04:19:01.599944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:17.929785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598676+0000","last_change":"2024-09-19T04:18:12.504364+0000","last_active":"2024-09-19T04:19:01.598676+0000","last_peered":"2024-09-19T04:19:01.598676+0000","last_clean":"2024-09-19T04:19:01.598676+0000","last_became_active":"2024-09-19T04:18:12.504006+0000","last_became_peered":"2024-09-19T04:18:12.504006+0000","last_unstale":"2024-09-19T04:19:01.598676+0000","last_undegraded":"2024-09-19T04:19:01.598676+0000","last_fullsized":"2024-09-19T04:19:01.598676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:53.309963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973049+0000","last_change":"2024-09-19T04:18:12.498223+0000","last_active":"2024-09-19T04:19:00.973049+0000","last_peered":"2024-09-19T04:19:00.973049+0000","last_clean":"2024-09-19T04:19:00.973049+0000","last_became_active":"2024-09-19T04:18:12.497862+0000","last_became_peered":"2024-09-19T04:18:12.497862+0000","last_unstale":"2024-09-19T04:19:00.973049+0000","last_undegraded":"2024-09-19T04:19:00.973049+0000","last_fullsized":"2024-09-19T04:19:00.973049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:28.858620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973075+0000","last_change":"2024-09-19T04:18:12.502335+0000","last_active":"2024-09-19T04:19:00.973075+0000","last_peered":"2024-09-19T04:19:00.973075+0000","last_clean":"2024-09-19T04:19:00.973075+0000","last_became_active":"2024-09-19T04:18:12.501686+0000","last_became_peered":"2024-09-19T04:18:12.501686+0000","last_unstale":"2024-09-19T04:19:00.973075+0000","last_undegraded":"2024-09-19T04:19:00.973075+0000","last_fullsized":"2024-09-19T04:19:00.973075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:22.348745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.974873+0000","last_change":"2024-09-19T04:18:12.490905+0000","last_active":"2024-09-19T04:19:00.974873+0000","last_peered":"2024-09-19T04:19:00.974873+0000","last_clean":"2024-09-19T04:19:00.974873+0000","last_became_active":"2024-09-19T04:18:12.490489+0000","last_became_peered":"2024-09-19T04:18:12.490489+0000","last_unstale":"2024-09-19T04:19:00.974873+0000","last_undegraded":"2024-09-19T04:19:00.974873+0000","last_fullsized":"2024-09-19T04:19:00.974873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.946474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978569+0000","last_change":"2024-09-19T04:18:12.494607+0000","last_active":"2024-09-19T04:19:00.978569+0000","last_peered":"2024-09-19T04:19:00.978569+0000","last_clean":"2024-09-19T04:19:00.978569+0000","last_became_active":"2024-09-19T04:18:12.494288+0000","last_became_peered":"2024-09-19T04:18:12.494288+0000","last_unstale":"2024-09-19T04:19:00.978569+0000","last_undegraded":"2024-09-19T04:19:00.978569+0000","last_fullsized":"2024-09-19T04:19:00.978569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:42.824951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":85,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:17.653814+0000","last_change":"2024-09-19T04:18:10.103108+0000","last_active":"2024-09-19T04:19:17.653814+0000","last_peered":"2024-09-19T04:19:17.653814+0000","last_clean":"2024-09-19T04:19:17.653814+0000","last_became_active":"2024-09-19T04:18:10.102783+0000","last_became_peered":"2024-09-19T04:18:10.102783+0000","last_unstale":"2024-09-19T04:19:17.653814+0000","last_undegraded":"2024-09-19T04:19:17.653814+0000","last_fullsized":"2024-09-19T04:19:17.653814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:25.521953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002255+0000","last_change":"2024-09-19T04:18:12.495453+0000","last_active":"2024-09-19T04:19:01.002255+0000","last_peered":"2024-09-19T04:19:01.002255+0000","last_clean":"2024-09-19T04:19:01.002255+0000","last_became_active":"2024-09-19T04:18:12.494744+0000","last_became_peered":"2024-09-19T04:18:12.494744+0000","last_unstale":"2024-09-19T04:19:01.002255+0000","last_undegraded":"2024-09-19T04:19:01.002255+0000","last_fullsized":"2024-09-19T04:19:01.002255+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:41.071723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002121+0000","last_change":"2024-09-19T04:18:10.102675+0000","last_active":"2024-09-19T04:19:01.002121+0000","last_peered":"2024-09-19T04:19:01.002121+0000","last_clean":"2024-09-19T04:19:01.002121+0000","last_became_active":"2024-09-19T04:18:10.102368+0000","last_became_peered":"2024-09-19T04:18:10.102368+0000","last_unstale":"2024-09-19T04:19:01.002121+0000","last_undegraded":"2024-09-19T04:19:01.002121+0000","last_fullsized":"2024-09-19T04:19:01.002121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:45.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975790+0000","last_change":"2024-09-19T04:18:10.099932+0000","last_active":"2024-09-19T04:19:00.975790+0000","last_peered":"2024-09-19T04:19:00.975790+0000","last_clean":"2024-09-19T04:19:00.975790+0000","last_became_active":"2024-09-19T04:18:10.099637+0000","last_became_peered":"2024-09-19T04:18:10.099637+0000","last_unstale":"2024-09-19T04:19:00.975790+0000","last_undegraded":"2024-09-19T04:19:00.975790+0000","last_fullsized":"2024-09-19T04:19:00.975790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:50.594382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977722+0000","last_change":"2024-09-19T04:18:12.502229+0000","last_active":"2024-09-19T04:19:00.977722+0000","last_peered":"2024-09-19T04:19:00.977722+0000","last_clean":"2024-09-19T04:19:00.977722+0000","last_became_active":"2024-09-19T04:18:12.501921+0000","last_became_peered":"2024-09-19T04:18:12.501921+0000","last_unstale":"2024-09-19T04:19:00.977722+0000","last_undegraded":"2024-09-19T04:19:00.977722+0000","last_fullsized":"2024-09-19T04:19:00.977722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:55.412103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002119+0000","last_change":"2024-09-19T04:18:10.103961+0000","last_active":"2024-09-19T04:19:01.002119+0000","last_peered":"2024-09-19T04:19:01.002119+0000","last_clean":"2024-09-19T04:19:01.002119+0000","last_became_active":"2024-09-19T04:18:10.102737+0000","last_became_peered":"2024-09-19T04:18:10.102737+0000","last_unstale":"2024-09-19T04:19:01.002119+0000","last_undegraded":"2024-09-19T04:19:01.002119+0000","last_fullsized":"2024-09-19T04:19:01.002119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:31:01.806124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.974828+0000","last_change":"2024-09-19T04:18:12.502385+0000","last_active":"2024-09-19T04:19:00.974828+0000","last_peered":"2024-09-19T04:19:00.974828+0000","last_clean":"2024-09-19T04:19:00.974828+0000","last_became_active":"2024-09-19T04:18:12.502025+0000","last_became_peered":"2024-09-19T04:18:12.502025+0000","last_unstale":"2024-09-19T04:19:00.974828+0000","last_undegraded":"2024-09-19T04:19:00.974828+0000","last_fullsized":"2024-09-19T04:19:00.974828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:24.650911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:22.659699+0000","last_change":"2024-09-19T04:18:10.103526+0000","last_active":"2024-09-19T04:19:22.659699+0000","last_peered":"2024-09-19T04:19:22.659699+0000","last_clean":"2024-09-19T04:19:22.659699+0000","last_became_active":"2024-09-19T04:18:10.103191+0000","last_became_peered":"2024-09-19T04:18:10.103191+0000","last_unstale":"2024-09-19T04:19:22.659699+0000","last_undegraded":"2024-09-19T04:19:22.659699+0000","last_fullsized":"2024-09-19T04:19:22.659699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:07.695715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973858+0000","last_change":"2024-09-19T04:18:12.493169+0000","last_active":"2024-09-19T04:19:00.973858+0000","last_peered":"2024-09-19T04:19:00.973858+0000","last_clean":"2024-09-19T04:19:00.973858+0000","last_became_active":"2024-09-19T04:18:12.492669+0000","last_became_peered":"2024-09-19T04:18:12.492669+0000","last_unstale":"2024-09-19T04:19:00.973858+0000","last_undegraded":"2024-09-19T04:19:00.973858+0000","last_fullsized":"2024-09-19T04:19:00.973858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:04.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.007275+0000","last_change":"2024-09-19T04:19:01.007275+0000","last_active":"2024-09-19T04:19:01.007275+0000","last_peered":"2024-09-19T04:19:01.007275+0000","last_clean":"2024-09-19T04:19:01.007275+0000","last_became_active":"2024-09-19T04:19:01.003361+0000","last_became_peered":"2024-09-19T04:19:01.003361+0000","last_unstale":"2024-09-19T04:19:01.007275+0000","last_undegraded":"2024-09-19T04:19:01.007275+0000","last_fullsized":"2024-09-19T04:19:01.007275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:07:11.907462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598830+0000","last_change":"2024-09-19T04:18:12.504348+0000","last_active":"2024-09-19T04:19:01.598830+0000","last_peered":"2024-09-19T04:19:01.598830+0000","last_clean":"2024-09-19T04:19:01.598830+0000","last_became_active":"2024-09-19T04:18:12.503180+0000","last_became_peered":"2024-09-19T04:18:12.503180+0000","last_unstale":"2024-09-19T04:19:01.598830+0000","last_undegraded":"2024-09-19T04:19:01.598830+0000","last_fullsized":"2024-09-19T04:19:01.598830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:03.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002213+0000","last_change":"2024-09-19T04:18:10.365136+0000","last_active":"2024-09-19T04:19:01.002213+0000","last_peered":"2024-09-19T04:19:01.002213+0000","last_clean":"2024-09-19T04:19:01.002213+0000","last_became_active":"2024-09-19T04:18:10.364594+0000","last_became_peered":"2024-09-19T04:18:10.364594+0000","last_unstale":"2024-09-19T04:19:01.002213+0000","last_undegraded":"2024-09-19T04:19:01.002213+0000","last_fullsized":"2024-09-19T04:19:01.002213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:52.252157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973750+0000","last_change":"2024-09-19T04:18:12.502051+0000","last_active":"2024-09-19T04:19:00.973750+0000","last_peered":"2024-09-19T04:19:00.973750+0000","last_clean":"2024-09-19T04:19:00.973750+0000","last_became_active":"2024-09-19T04:18:12.501578+0000","last_became_peered":"2024-09-19T04:18:12.501578+0000","last_unstale":"2024-09-19T04:19:00.973750+0000","last_undegraded":"2024-09-19T04:19:00.973750+0000","last_fullsized":"2024-09-19T04:19:00.973750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:59.693175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":177,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:22.659489+0000","last_change":"2024-09-19T04:18:10.101507+0000","last_active":"2024-09-19T04:19:22.659489+0000","last_peered":"2024-09-19T04:19:22.659489+0000","last_clean":"2024-09-19T04:19:22.659489+0000","last_became_active":"2024-09-19T04:18:10.101152+0000","last_became_peered":"2024-09-19T04:18:10.101152+0000","last_unstale":"2024-09-19T04:19:22.659489+0000","last_undegraded":"2024-09-19T04:19:22.659489+0000","last_fullsized":"2024-09-19T04:19:22.659489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:01.397605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":125,"num_read_kb":120,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978835+0000","last_change":"2024-09-19T04:18:12.491611+0000","last_active":"2024-09-19T04:19:00.978835+0000","last_peered":"2024-09-19T04:19:00.978835+0000","last_clean":"2024-09-19T04:19:00.978835+0000","last_became_active":"2024-09-19T04:18:12.491279+0000","last_became_peered":"2024-09-19T04:18:12.491279+0000","last_unstale":"2024-09-19T04:19:00.978835+0000","last_undegraded":"2024-09-19T04:19:00.978835+0000","last_fullsized":"2024-09-19T04:19:00.978835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:01.414713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973716+0000","last_change":"2024-09-19T04:18:10.105574+0000","last_active":"2024-09-19T04:19:00.973716+0000","last_peered":"2024-09-19T04:19:00.973716+0000","last_clean":"2024-09-19T04:19:00.973716+0000","last_became_active":"2024-09-19T04:18:10.105203+0000","last_became_peered":"2024-09-19T04:18:10.105203+0000","last_unstale":"2024-09-19T04:19:00.973716+0000","last_undegraded":"2024-09-19T04:19:00.973716+0000","last_fullsized":"2024-09-19T04:19:00.973716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:00.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.974853+0000","last_change":"2024-09-19T04:18:12.497242+0000","last_active":"2024-09-19T04:19:00.974853+0000","last_peered":"2024-09-19T04:19:00.974853+0000","last_clean":"2024-09-19T04:19:00.974853+0000","last_became_active":"2024-09-19T04:18:12.496732+0000","last_became_peered":"2024-09-19T04:18:12.496732+0000","last_unstale":"2024-09-19T04:19:00.974853+0000","last_undegraded":"2024-09-19T04:19:00.974853+0000","last_fullsized":"2024-09-19T04:19:00.974853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:56.520341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002017+0000","last_change":"2024-09-19T04:18:10.103905+0000","last_active":"2024-09-19T04:19:01.002017+0000","last_peered":"2024-09-19T04:19:01.002017+0000","last_clean":"2024-09-19T04:19:01.002017+0000","last_became_active":"2024-09-19T04:18:10.103517+0000","last_became_peered":"2024-09-19T04:18:10.103517+0000","last_unstale":"2024-09-19T04:19:01.002017+0000","last_undegraded":"2024-09-19T04:19:01.002017+0000","last_fullsized":"2024-09-19T04:19:01.002017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:48.292878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975328+0000","last_change":"2024-09-19T04:18:12.499380+0000","last_active":"2024-09-19T04:19:00.975328+0000","last_peered":"2024-09-19T04:19:00.975328+0000","last_clean":"2024-09-19T04:19:00.975328+0000","last_became_active":"2024-09-19T04:18:12.498995+0000","last_became_peered":"2024-09-19T04:18:12.498995+0000","last_unstale":"2024-09-19T04:19:00.975328+0000","last_undegraded":"2024-09-19T04:19:00.975328+0000","last_fullsized":"2024-09-19T04:19:00.975328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:20.653923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":87,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:21.659012+0000","last_change":"2024-09-19T04:18:10.105619+0000","last_active":"2024-09-19T04:19:21.659012+0000","last_peered":"2024-09-19T04:19:21.659012+0000","last_clean":"2024-09-19T04:19:21.659012+0000","last_became_active":"2024-09-19T04:18:10.105082+0000","last_became_peered":"2024-09-19T04:18:10.105082+0000","last_unstale":"2024-09-19T04:19:21.659012+0000","last_undegraded":"2024-09-19T04:19:21.659012+0000","last_fullsized":"2024-09-19T04:19:21.659012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:56.843068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975098+0000","last_change":"2024-09-19T04:18:12.498473+0000","last_active":"2024-09-19T04:19:00.975098+0000","last_peered":"2024-09-19T04:19:00.975098+0000","last_clean":"2024-09-19T04:19:00.975098+0000","last_became_active":"2024-09-19T04:18:12.496904+0000","last_became_peered":"2024-09-19T04:18:12.496904+0000","last_unstale":"2024-09-19T04:19:00.975098+0000","last_undegraded":"2024-09-19T04:19:00.975098+0000","last_fullsized":"2024-09-19T04:19:00.975098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:03.155899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978505+0000","last_change":"2024-09-19T04:18:10.365182+0000","last_active":"2024-09-19T04:19:00.978505+0000","last_peered":"2024-09-19T04:19:00.978505+0000","last_clean":"2024-09-19T04:19:00.978505+0000","last_became_active":"2024-09-19T04:18:10.364717+0000","last_became_peered":"2024-09-19T04:18:10.364717+0000","last_unstale":"2024-09-19T04:19:00.978505+0000","last_undegraded":"2024-09-19T04:19:00.978505+0000","last_fullsized":"2024-09-19T04:19:00.978505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:32.063066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.001950+0000","last_change":"2024-09-19T04:18:12.493585+0000","last_active":"2024-09-19T04:19:01.001950+0000","last_peered":"2024-09-19T04:19:01.001950+0000","last_clean":"2024-09-19T04:19:01.001950+0000","last_became_active":"2024-09-19T04:18:12.493255+0000","last_became_peered":"2024-09-19T04:18:12.493255+0000","last_unstale":"2024-09-19T04:19:01.001950+0000","last_undegraded":"2024-09-19T04:19:01.001950+0000","last_fullsized":"2024-09-19T04:19:01.001950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:33.139727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.979069+0000","last_change":"2024-09-19T04:18:10.101537+0000","last_active":"2024-09-19T04:19:00.979069+0000","last_peered":"2024-09-19T04:19:00.979069+0000","last_clean":"2024-09-19T04:19:00.979069+0000","last_became_active":"2024-09-19T04:18:10.100951+0000","last_became_peered":"2024-09-19T04:18:10.100951+0000","last_unstale":"2024-09-19T04:19:00.979069+0000","last_undegraded":"2024-09-19T04:19:00.979069+0000","last_fullsized":"2024-09-19T04:19:00.979069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:08.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975115+0000","last_change":"2024-09-19T04:18:12.495530+0000","last_active":"2024-09-19T04:19:00.975115+0000","last_peered":"2024-09-19T04:19:00.975115+0000","last_clean":"2024-09-19T04:19:00.975115+0000","last_became_active":"2024-09-19T04:18:12.495176+0000","last_became_peered":"2024-09-19T04:18:12.495176+0000","last_unstale":"2024-09-19T04:19:00.975115+0000","last_undegraded":"2024-09-19T04:19:00.975115+0000","last_fullsized":"2024-09-19T04:19:00.975115+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:24.308047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.974791+0000","last_change":"2024-09-19T04:18:10.102539+0000","last_active":"2024-09-19T04:19:00.974791+0000","last_peered":"2024-09-19T04:19:00.974791+0000","last_clean":"2024-09-19T04:19:00.974791+0000","last_became_active":"2024-09-19T04:18:10.102109+0000","last_became_peered":"2024-09-19T04:18:10.102109+0000","last_unstale":"2024-09-19T04:19:00.974791+0000","last_undegraded":"2024-09-19T04:19:00.974791+0000","last_fullsized":"2024-09-19T04:19:00.974791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:58.852767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975776+0000","last_change":"2024-09-19T04:18:12.491769+0000","last_active":"2024-09-19T04:19:00.975776+0000","last_peered":"2024-09-19T04:19:00.975776+0000","last_clean":"2024-09-19T04:19:00.975776+0000","last_became_active":"2024-09-19T04:18:12.491435+0000","last_became_peered":"2024-09-19T04:18:12.491435+0000","last_unstale":"2024-09-19T04:19:00.975776+0000","last_undegraded":"2024-09-19T04:19:00.975776+0000","last_fullsized":"2024-09-19T04:19:00.975776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:54.880533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002301+0000","last_change":"2024-09-19T04:18:10.103559+0000","last_active":"2024-09-19T04:19:01.002301+0000","last_peered":"2024-09-19T04:19:01.002301+0000","last_clean":"2024-09-19T04:19:01.002301+0000","last_became_active":"2024-09-19T04:18:10.103208+0000","last_became_peered":"2024-09-19T04:18:10.103208+0000","last_unstale":"2024-09-19T04:19:01.002301+0000","last_undegraded":"2024-09-19T04:19:01.002301+0000","last_fullsized":"2024-09-19T04:19:01.002301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:16.564916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975331+0000","last_change":"2024-09-19T04:18:12.495616+0000","last_active":"2024-09-19T04:19:00.975331+0000","last_peered":"2024-09-19T04:19:00.975331+0000","last_clean":"2024-09-19T04:19:00.975331+0000","last_became_active":"2024-09-19T04:18:12.495243+0000","last_became_peered":"2024-09-19T04:18:12.495243+0000","last_unstale":"2024-09-19T04:19:00.975331+0000","last_undegraded":"2024-09-19T04:19:00.975331+0000","last_fullsized":"2024-09-19T04:19:00.975331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:12.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":89,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:21.658961+0000","last_change":"2024-09-19T04:18:10.626874+0000","last_active":"2024-09-19T04:19:21.658961+0000","last_peered":"2024-09-19T04:19:21.658961+0000","last_clean":"2024-09-19T04:19:21.658961+0000","last_became_active":"2024-09-19T04:18:10.626506+0000","last_became_peered":"2024-09-19T04:18:10.626506+0000","last_unstale":"2024-09-19T04:19:21.658961+0000","last_undegraded":"2024-09-19T04:19:21.658961+0000","last_fullsized":"2024-09-19T04:19:21.658961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:48.725143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975644+0000","last_change":"2024-09-19T04:18:12.496350+0000","last_active":"2024-09-19T04:19:00.975644+0000","last_peered":"2024-09-19T04:19:00.975644+0000","last_clean":"2024-09-19T04:19:00.975644+0000","last_became_active":"2024-09-19T04:18:12.495904+0000","last_became_peered":"2024-09-19T04:18:12.495904+0000","last_unstale":"2024-09-19T04:19:00.975644+0000","last_undegraded":"2024-09-19T04:19:00.975644+0000","last_fullsized":"2024-09-19T04:19:00.975644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:08.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":347,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975303+0000","last_change":"2024-09-19T04:16:56.164450+0000","last_active":"2024-09-19T04:19:00.975303+0000","last_peered":"2024-09-19T04:19:00.975303+0000","last_clean":"2024-09-19T04:19:00.975303+0000","last_became_active":"2024-09-19T04:16:56.164008+0000","last_became_peered":"2024-09-19T04:16:56.164008+0000","last_unstale":"2024-09-19T04:19:00.975303+0000","last_undegraded":"2024-09-19T04:19:00.975303+0000","last_fullsized":"2024-09-19T04:19:00.975303+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_clean_scrub_stamp":"2024-09-19T04:13:18.308718+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:14.697725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002444+0000","last_change":"2024-09-19T04:18:10.101598+0000","last_active":"2024-09-19T04:19:01.002444+0000","last_peered":"2024-09-19T04:19:01.002444+0000","last_clean":"2024-09-19T04:19:01.002444+0000","last_became_active":"2024-09-19T04:18:10.101210+0000","last_became_peered":"2024-09-19T04:18:10.101210+0000","last_unstale":"2024-09-19T04:19:01.002444+0000","last_undegraded":"2024-09-19T04:19:01.002444+0000","last_fullsized":"2024-09-19T04:19:01.002444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:10.653481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977862+0000","last_change":"2024-09-19T04:18:12.495034+0000","last_active":"2024-09-19T04:19:00.977862+0000","last_peered":"2024-09-19T04:19:00.977862+0000","last_clean":"2024-09-19T04:19:00.977862+0000","last_became_active":"2024-09-19T04:18:12.494611+0000","last_became_peered":"2024-09-19T04:18:12.494611+0000","last_unstale":"2024-09-19T04:19:00.977862+0000","last_undegraded":"2024-09-19T04:19:00.977862+0000","last_fullsized":"2024-09-19T04:19:00.977862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:28.448927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975345+0000","last_change":"2024-09-19T04:18:10.105212+0000","last_active":"2024-09-19T04:19:00.975345+0000","last_peered":"2024-09-19T04:19:00.975345+0000","last_clean":"2024-09-19T04:19:00.975345+0000","last_became_active":"2024-09-19T04:18:10.104710+0000","last_became_peered":"2024-09-19T04:18:10.104710+0000","last_unstale":"2024-09-19T04:19:00.975345+0000","last_undegraded":"2024-09-19T04:19:00.975345+0000","last_fullsized":"2024-09-19T04:19:00.975345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:58.007773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.980419+0000","last_change":"2024-09-19T04:19:00.980419+0000","last_active":"2024-09-19T04:19:00.980419+0000","last_peered":"2024-09-19T04:19:00.980419+0000","last_clean":"2024-09-19T04:19:00.980419+0000","last_became_active":"2024-09-19T04:19:00.980008+0000","last_became_peered":"2024-09-19T04:19:00.980008+0000","last_unstale":"2024-09-19T04:19:00.980419+0000","last_undegraded":"2024-09-19T04:19:00.980419+0000","last_fullsized":"2024-09-19T04:19:00.980419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:18.396611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":84,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:19.655325+0000","last_change":"2024-09-19T04:18:10.364530+0000","last_active":"2024-09-19T04:19:19.655325+0000","last_peered":"2024-09-19T04:19:19.655325+0000","last_clean":"2024-09-19T04:19:19.655325+0000","last_became_active":"2024-09-19T04:18:10.364204+0000","last_became_peered":"2024-09-19T04:18:10.364204+0000","last_unstale":"2024-09-19T04:19:19.655325+0000","last_undegraded":"2024-09-19T04:19:19.655325+0000","last_fullsized":"2024-09-19T04:19:19.655325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:04.229530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002088+0000","last_change":"2024-09-19T04:18:12.493730+0000","last_active":"2024-09-19T04:19:01.002088+0000","last_peered":"2024-09-19T04:19:01.002088+0000","last_clean":"2024-09-19T04:19:01.002088+0000","last_became_active":"2024-09-19T04:18:12.493153+0000","last_became_peered":"2024-09-19T04:18:12.493153+0000","last_unstale":"2024-09-19T04:19:01.002088+0000","last_undegraded":"2024-09-19T04:19:01.002088+0000","last_fullsized":"2024-09-19T04:19:01.002088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.758490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:17.653056+0000","last_change":"2024-09-19T04:18:10.102073+0000","last_active":"2024-09-19T04:19:17.653056+0000","last_peered":"2024-09-19T04:19:17.653056+0000","last_clean":"2024-09-19T04:19:17.653056+0000","last_became_active":"2024-09-19T04:18:10.101755+0000","last_became_peered":"2024-09-19T04:18:10.101755+0000","last_unstale":"2024-09-19T04:19:17.653056+0000","last_undegraded":"2024-09-19T04:19:17.653056+0000","last_fullsized":"2024-09-19T04:19:17.653056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:21.489146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.001939+0000","last_change":"2024-09-19T04:18:12.494560+0000","last_active":"2024-09-19T04:19:01.001939+0000","last_peered":"2024-09-19T04:19:01.001939+0000","last_clean":"2024-09-19T04:19:01.001939+0000","last_became_active":"2024-09-19T04:18:12.494203+0000","last_became_peered":"2024-09-19T04:18:12.494203+0000","last_unstale":"2024-09-19T04:19:01.001939+0000","last_undegraded":"2024-09-19T04:19:01.001939+0000","last_fullsized":"2024-09-19T04:19:01.001939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:35:17.129649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975411+0000","last_change":"2024-09-19T04:18:10.365781+0000","last_active":"2024-09-19T04:19:00.975411+0000","last_peered":"2024-09-19T04:19:00.975411+0000","last_clean":"2024-09-19T04:19:00.975411+0000","last_became_active":"2024-09-19T04:18:10.365417+0000","last_became_peered":"2024-09-19T04:18:10.365417+0000","last_unstale":"2024-09-19T04:19:00.975411+0000","last_undegraded":"2024-09-19T04:19:00.975411+0000","last_fullsized":"2024-09-19T04:19:00.975411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:20.097572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.001998+0000","last_change":"2024-09-19T04:18:12.495570+0000","last_active":"2024-09-19T04:19:01.001998+0000","last_peered":"2024-09-19T04:19:01.001998+0000","last_clean":"2024-09-19T04:19:01.001998+0000","last_became_active":"2024-09-19T04:18:12.495242+0000","last_became_peered":"2024-09-19T04:18:12.495242+0000","last_unstale":"2024-09-19T04:19:01.001998+0000","last_undegraded":"2024-09-19T04:19:01.001998+0000","last_fullsized":"2024-09-19T04:19:01.001998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:57.448182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978902+0000","last_change":"2024-09-19T04:18:10.365134+0000","last_active":"2024-09-19T04:19:00.978902+0000","last_peered":"2024-09-19T04:19:00.978902+0000","last_clean":"2024-09-19T04:19:00.978902+0000","last_became_active":"2024-09-19T04:18:10.364726+0000","last_became_peered":"2024-09-19T04:18:10.364726+0000","last_unstale":"2024-09-19T04:19:00.978902+0000","last_undegraded":"2024-09-19T04:19:00.978902+0000","last_fullsized":"2024-09-19T04:19:00.978902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:44.207855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975566+0000","last_change":"2024-09-19T04:18:12.488836+0000","last_active":"2024-09-19T04:19:00.975566+0000","last_peered":"2024-09-19T04:19:00.975566+0000","last_clean":"2024-09-19T04:19:00.975566+0000","last_became_active":"2024-09-19T04:18:12.488507+0000","last_became_peered":"2024-09-19T04:18:12.488507+0000","last_unstale":"2024-09-19T04:19:00.975566+0000","last_undegraded":"2024-09-19T04:19:00.975566+0000","last_fullsized":"2024-09-19T04:19:00.975566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:31.308114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975204+0000","last_change":"2024-09-19T04:18:10.103623+0000","last_active":"2024-09-19T04:19:00.975204+0000","last_peered":"2024-09-19T04:19:00.975204+0000","last_clean":"2024-09-19T04:19:00.975204+0000","last_became_active":"2024-09-19T04:18:10.103340+0000","last_became_peered":"2024-09-19T04:18:10.103340+0000","last_unstale":"2024-09-19T04:19:00.975204+0000","last_undegraded":"2024-09-19T04:19:00.975204+0000","last_fullsized":"2024-09-19T04:19:00.975204+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:11.724013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978023+0000","last_change":"2024-09-19T04:18:12.498167+0000","last_active":"2024-09-19T04:19:00.978023+0000","last_peered":"2024-09-19T04:19:00.978023+0000","last_clean":"2024-09-19T04:19:00.978023+0000","last_became_active":"2024-09-19T04:18:12.496924+0000","last_became_peered":"2024-09-19T04:18:12.496924+0000","last_unstale":"2024-09-19T04:19:00.978023+0000","last_undegraded":"2024-09-19T04:19:00.978023+0000","last_fullsized":"2024-09-19T04:19:00.978023+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:13.952030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002785+0000","last_change":"2024-09-19T04:18:10.627106+0000","last_active":"2024-09-19T04:19:01.002785+0000","last_peered":"2024-09-19T04:19:01.002785+0000","last_clean":"2024-09-19T04:19:01.002785+0000","last_became_active":"2024-09-19T04:18:10.626708+0000","last_became_peered":"2024-09-19T04:18:10.626708+0000","last_unstale":"2024-09-19T04:19:01.002785+0000","last_undegraded":"2024-09-19T04:19:01.002785+0000","last_fullsized":"2024-09-19T04:19:01.002785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:46.049699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598893+0000","last_change":"2024-09-19T04:18:12.506995+0000","last_active":"2024-09-19T04:19:01.598893+0000","last_peered":"2024-09-19T04:19:01.598893+0000","last_clean":"2024-09-19T04:19:01.598893+0000","last_became_active":"2024-09-19T04:18:12.500607+0000","last_became_peered":"2024-09-19T04:18:12.500607+0000","last_unstale":"2024-09-19T04:19:01.598893+0000","last_undegraded":"2024-09-19T04:19:01.598893+0000","last_fullsized":"2024-09-19T04:19:01.598893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:37.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973607+0000","last_change":"2024-09-19T04:18:10.365150+0000","last_active":"2024-09-19T04:19:00.973607+0000","last_peered":"2024-09-19T04:19:00.973607+0000","last_clean":"2024-09-19T04:19:00.973607+0000","last_became_active":"2024-09-19T04:18:10.364755+0000","last_became_peered":"2024-09-19T04:18:10.364755+0000","last_unstale":"2024-09-19T04:19:00.973607+0000","last_undegraded":"2024-09-19T04:19:00.973607+0000","last_fullsized":"2024-09-19T04:19:00.973607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:11.639532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.979071+0000","last_change":"2024-09-19T04:18:12.484588+0000","last_active":"2024-09-19T04:19:00.979071+0000","last_peered":"2024-09-19T04:19:00.979071+0000","last_clean":"2024-09-19T04:19:00.979071+0000","last_became_active":"2024-09-19T04:18:12.484218+0000","last_became_peered":"2024-09-19T04:18:12.484218+0000","last_unstale":"2024-09-19T04:19:00.979071+0000","last_undegraded":"2024-09-19T04:19:00.979071+0000","last_fullsized":"2024-09-19T04:19:00.979071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:55.634557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.977839+0000","last_change":"2024-09-19T04:18:10.100832+0000","last_active":"2024-09-19T04:19:00.977839+0000","last_peered":"2024-09-19T04:19:00.977839+0000","last_clean":"2024-09-19T04:19:00.977839+0000","last_became_active":"2024-09-19T04:18:10.100518+0000","last_became_peered":"2024-09-19T04:18:10.100518+0000","last_unstale":"2024-09-19T04:19:00.977839+0000","last_undegraded":"2024-09-19T04:19:00.977839+0000","last_fullsized":"2024-09-19T04:19:00.977839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:37.028120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975234+0000","last_change":"2024-09-19T04:18:12.496914+0000","last_active":"2024-09-19T04:19:00.975234+0000","last_peered":"2024-09-19T04:19:00.975234+0000","last_clean":"2024-09-19T04:19:00.975234+0000","last_became_active":"2024-09-19T04:18:12.496556+0000","last_became_peered":"2024-09-19T04:18:12.496556+0000","last_unstale":"2024-09-19T04:19:00.975234+0000","last_undegraded":"2024-09-19T04:19:00.975234+0000","last_fullsized":"2024-09-19T04:19:00.975234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:52.391412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.993937+0000","last_change":"2024-09-19T04:19:00.993461+0000","last_active":"2024-09-19T04:19:00.993937+0000","last_peered":"2024-09-19T04:19:00.993937+0000","last_clean":"2024-09-19T04:19:00.993937+0000","last_became_active":"2024-09-19T04:19:00.980704+0000","last_became_peered":"2024-09-19T04:19:00.980704+0000","last_unstale":"2024-09-19T04:19:00.993937+0000","last_undegraded":"2024-09-19T04:19:00.993937+0000","last_fullsized":"2024-09-19T04:19:00.993937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:56.090092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978004+0000","last_change":"2024-09-19T04:18:12.496852+0000","last_active":"2024-09-19T04:19:00.978004+0000","last_peered":"2024-09-19T04:19:00.978004+0000","last_clean":"2024-09-19T04:19:00.978004+0000","last_became_active":"2024-09-19T04:18:12.495365+0000","last_became_peered":"2024-09-19T04:18:12.495365+0000","last_unstale":"2024-09-19T04:19:00.978004+0000","last_undegraded":"2024-09-19T04:19:00.978004+0000","last_fullsized":"2024-09-19T04:19:00.978004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:14:57.012618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978566+0000","last_change":"2024-09-19T04:18:10.104641+0000","last_active":"2024-09-19T04:19:00.978566+0000","last_peered":"2024-09-19T04:19:00.978566+0000","last_clean":"2024-09-19T04:19:00.978566+0000","last_became_active":"2024-09-19T04:18:10.104295+0000","last_became_peered":"2024-09-19T04:18:10.104295+0000","last_unstale":"2024-09-19T04:19:00.978566+0000","last_undegraded":"2024-09-19T04:19:00.978566+0000","last_fullsized":"2024-09-19T04:19:00.978566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:58.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.975362+0000","last_change":"2024-09-19T04:18:12.488396+0000","last_active":"2024-09-19T04:19:00.975362+0000","last_peered":"2024-09-19T04:19:00.975362+0000","last_clean":"2024-09-19T04:19:00.975362+0000","last_became_active":"2024-09-19T04:18:12.488048+0000","last_became_peered":"2024-09-19T04:18:12.488048+0000","last_unstale":"2024-09-19T04:19:00.975362+0000","last_undegraded":"2024-09-19T04:19:00.975362+0000","last_fullsized":"2024-09-19T04:19:00.975362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:27.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.002204+0000","last_change":"2024-09-19T04:18:12.493722+0000","last_active":"2024-09-19T04:19:01.002204+0000","last_peered":"2024-09-19T04:19:01.002204+0000","last_clean":"2024-09-19T04:19:01.002204+0000","last_became_active":"2024-09-19T04:18:12.492483+0000","last_became_peered":"2024-09-19T04:18:12.492483+0000","last_unstale":"2024-09-19T04:19:01.002204+0000","last_undegraded":"2024-09-19T04:19:01.002204+0000","last_fullsized":"2024-09-19T04:19:01.002204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:33.492623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.979376+0000","last_change":"2024-09-19T04:18:10.101326+0000","last_active":"2024-09-19T04:19:00.979376+0000","last_peered":"2024-09-19T04:19:00.979376+0000","last_clean":"2024-09-19T04:19:00.979376+0000","last_became_active":"2024-09-19T04:18:10.100955+0000","last_became_peered":"2024-09-19T04:18:10.100955+0000","last_unstale":"2024-09-19T04:19:00.979376+0000","last_undegraded":"2024-09-19T04:19:00.979376+0000","last_fullsized":"2024-09-19T04:19:00.979376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:41.939741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.973618+0000","last_change":"2024-09-19T04:18:12.492750+0000","last_active":"2024-09-19T04:19:00.973618+0000","last_peered":"2024-09-19T04:19:00.973618+0000","last_clean":"2024-09-19T04:19:00.973618+0000","last_became_active":"2024-09-19T04:18:12.492417+0000","last_became_peered":"2024-09-19T04:18:12.492417+0000","last_unstale":"2024-09-19T04:19:00.973618+0000","last_undegraded":"2024-09-19T04:19:00.973618+0000","last_fullsized":"2024-09-19T04:19:00.973618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:25.279452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.990252+0000","last_change":"2024-09-19T04:19:00.989741+0000","last_active":"2024-09-19T04:19:00.990252+0000","last_peered":"2024-09-19T04:19:00.990252+0000","last_clean":"2024-09-19T04:19:00.990252+0000","last_became_active":"2024-09-19T04:19:00.977530+0000","last_became_peered":"2024-09-19T04:19:00.977530+0000","last_unstale":"2024-09-19T04:19:00.990252+0000","last_undegraded":"2024-09-19T04:19:00.990252+0000","last_fullsized":"2024-09-19T04:19:00.990252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:35.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:01.598809+0000","last_change":"2024-09-19T04:18:12.503510+0000","last_active":"2024-09-19T04:19:01.598809+0000","last_peered":"2024-09-19T04:19:01.598809+0000","last_clean":"2024-09-19T04:19:01.598809+0000","last_became_active":"2024-09-19T04:18:12.495165+0000","last_became_peered":"2024-09-19T04:18:12.495165+0000","last_unstale":"2024-09-19T04:19:01.598809+0000","last_undegraded":"2024-09-19T04:19:01.598809+0000","last_fullsized":"2024-09-19T04:19:01.598809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:13.934328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978447+0000","last_change":"2024-09-19T04:18:10.364783+0000","last_active":"2024-09-19T04:19:00.978447+0000","last_peered":"2024-09-19T04:19:00.978447+0000","last_clean":"2024-09-19T04:19:00.978447+0000","last_became_active":"2024-09-19T04:18:10.364202+0000","last_became_peered":"2024-09-19T04:18:10.364202+0000","last_unstale":"2024-09-19T04:19:00.978447+0000","last_undegraded":"2024-09-19T04:19:00.978447+0000","last_fullsized":"2024-09-19T04:19:00.978447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:18:55.770637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:00.978229+0000","last_change":"2024-09-19T04:18:12.495825+0000","last_active":"2024-09-19T04:19:00.978229+0000","last_peered":"2024-09-19T04:19:00.978229+0000","last_clean":"2024-09-19T04:19:00.978229+0000","last_became_active":"2024-09-19T04:18:12.494525+0000","last_became_peered":"2024-09-19T04:18:12.494525+0000","last_unstale":"2024-09-19T04:19:00.978229+0000","last_undegraded":"2024-09-19T04:19:00.978229+0000","last_fullsized":"2024-09-19T04:19:00.978229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:57.827135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T04:19:20.656911+0000","last_change":"2024-09-19T04:18:10.102145+0000","last_active":"2024-09-19T04:19:20.656911+0000","last_peered":"2024-09-19T04:19:20.656911+0000","last_clean":"2024-09-19T04:19:20.656911+0000","last_became_active":"2024-09-19T04:18:10.101410+0000","last_became_peered":"2024-09-19T04:18:10.101410+0000","last_unstale":"2024-09-19T04:19:20.656911+0000","last_undegraded":"2024-09-19T04:19:20.656911+0000","last_fullsized":"2024-09-19T04:19:20.656911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:20.354296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66972,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":608,"num_read_kb":379,"num_write":245,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281762,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547228,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":441210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1064960,"data_stored":1091610,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972391,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":440747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201324,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":441125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430255,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":1033645,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593782,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":440464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822716,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":1033323,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986240,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":440325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215173,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":440289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":999424,"data_stored":502147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":501141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":937984,"data_stored":442196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:19:25.703 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T04:19:25.703 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T04:19:25.704 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T04:19:25.704 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T04:19:26.073 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:26.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.592+0000 7f3a7aa00640 1 -- 172.21.15.107:0/3566475199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74104010 msgr2=0x7f3a741043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.592+0000 7f3a7aa00640 1 --2- 172.21.15.107:0/3566475199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74104010 0x7f3a741043f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009900 tx=0x7f3a6402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.592+0000 7f3a7aa00640 1 -- 172.21.15.107:0/3566475199 shutdown_connections 2024-09-19T04:19:26.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.592+0000 7f3a7aa00640 1 --2- 172.21.15.107:0/3566475199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a74109140 0x7f3a74110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.592+0000 7f3a7aa00640 1 --2- 172.21.15.107:0/3566475199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a741049c0 0x7f3a74108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.592+0000 7f3a7aa00640 1 --2- 172.21.15.107:0/3566475199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74104010 0x7f3a741043f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.592+0000 7f3a7aa00640 1 -- 172.21.15.107:0/3566475199 >> 172.21.15.107:0/3566475199 conn(0x7f3a740ffa50 msgr2=0x7f3a74101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:26.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.592+0000 7f3a7aa00640 1 -- 172.21.15.107:0/3566475199 shutdown_connections 2024-09-19T04:19:26.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.592+0000 7f3a7aa00640 1 -- 172.21.15.107:0/3566475199 wait complete. 2024-09-19T04:19:26.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.593+0000 7f3a7aa00640 1 Processor -- start 2024-09-19T04:19:26.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.593+0000 7f3a7aa00640 1 -- start start 2024-09-19T04:19:26.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.593+0000 7f3a7aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74104010 0x7f3a7419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:26.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.593+0000 7f3a7aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a741049c0 0x7f3a7419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:26.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.593+0000 7f3a7aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a74109140 0x7f3a7419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:26.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.593+0000 7f3a7aa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74114170 con 0x7f3a74104010 2024-09-19T04:19:26.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.593+0000 7f3a7aa00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74113ff0 con 0x7f3a74109140 2024-09-19T04:19:26.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.593+0000 7f3a7aa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3a741142f0 con 0x7f3a741049c0 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74104010 0x7f3a7419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74104010 0x7f3a7419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37950/0 (socket says 172.21.15.107:37950) 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a73e00640 1 -- 172.21.15.107:0/2855679960 learned_addr learned my addr 172.21.15.107:0/2855679960 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a78a00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a74109140 0x7f3a7419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a6be00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a741049c0 0x7f3a7419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a73e00640 1 -- 172.21.15.107:0/2855679960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a741049c0 msgr2=0x7f3a7419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a73e00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a741049c0 0x7f3a7419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a73e00640 1 -- 172.21.15.107:0/2855679960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a74109140 msgr2=0x7f3a7419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a73e00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a74109140 0x7f3a7419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a73e00640 1 -- 172.21.15.107:0/2855679960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7419f600 con 0x7f3a74104010 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.594+0000 7f3a78a00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a74109140 0x7f3a7419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:26.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.595+0000 7f3a73e00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74104010 0x7f3a7419a650 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3a64006010 tx=0x7f3a640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:26.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.595+0000 7f3a71600640 1 -- 172.21.15.107:0/2855679960 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6403f030 con 0x7f3a74104010 2024-09-19T04:19:26.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.595+0000 7f3a71600640 1 -- 172.21.15.107:0/2855679960 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a64003ce0 con 0x7f3a74104010 2024-09-19T04:19:26.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.595+0000 7f3a71600640 1 -- 172.21.15.107:0/2855679960 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64035070 con 0x7f3a74104010 2024-09-19T04:19:26.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.595+0000 7f3a7aa00640 1 -- 172.21.15.107:0/2855679960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7419f890 con 0x7f3a74104010 2024-09-19T04:19:26.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.595+0000 7f3a7aa00640 1 -- 172.21.15.107:0/2855679960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7419fd20 con 0x7f3a74104010 2024-09-19T04:19:26.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.596+0000 7f3a7aa00640 1 -- 172.21.15.107:0/2855679960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a741043f0 con 0x7f3a74104010 2024-09-19T04:19:26.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.598+0000 7f3a71600640 1 -- 172.21.15.107:0/2855679960 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3a64042030 con 0x7f3a74104010 2024-09-19T04:19:26.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.600+0000 7f3a71600640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3a580a3350 0x7f3a580a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:26.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.600+0000 7f3a71600640 1 -- 172.21.15.107:0/2855679960 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f3a640eff60 con 0x7f3a74104010 2024-09-19T04:19:26.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.600+0000 7f3a6be00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3a580a3350 0x7f3a580a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:26.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.601+0000 7f3a6be00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3a580a3350 0x7f3a580a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3a740ffe70 tx=0x7f3a5c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:26.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.604+0000 7f3a71600640 1 -- 172.21.15.107:0/2855679960 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a640bc560 con 0x7f3a74104010 2024-09-19T04:19:26.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.819+0000 7f3a7aa00640 1 -- 172.21.15.107:0/2855679960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3a7402d7c0 con 0x7f3a74104010 2024-09-19T04:19:26.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.820+0000 7f3a71600640 1 -- 172.21.15.107:0/2855679960 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3a640c0860 con 0x7f3a74104010 2024-09-19T04:19:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:26.822 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T04:19:26.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 -- 172.21.15.107:0/2855679960 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3a580a3350 msgr2=0x7f3a580a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3a580a3350 0x7f3a580a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3a740ffe70 tx=0x7f3a5c005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:26.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 -- 172.21.15.107:0/2855679960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74104010 msgr2=0x7f3a7419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74104010 0x7f3a7419a650 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3a64006010 tx=0x7f3a640042f0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 -- 172.21.15.107:0/2855679960 shutdown_connections 2024-09-19T04:19:26.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3a580a3350 0x7f3a580a5810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a74109140 0x7f3a7419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a741049c0 0x7f3a7419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 --2- 172.21.15.107:0/2855679960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74104010 0x7f3a7419a650 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 -- 172.21.15.107:0/2855679960 >> 172.21.15.107:0/2855679960 conn(0x7f3a740ffa50 msgr2=0x7f3a74100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:26.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 -- 172.21.15.107:0/2855679960 shutdown_connections 2024-09-19T04:19:26.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:26.822+0000 7f3a7aa00640 1 -- 172.21.15.107:0/2855679960 wait complete. 2024-09-19T04:19:27.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:26 smithi107 ceph-mon[30659]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:19:27.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:26 smithi107 ceph-mon[30659]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:27.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:26 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2855679960' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:19:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:26 smithi115 ceph-mon[35477]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:19:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:26 smithi115 ceph-mon[35477]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:26 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2855679960' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:19:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:26 smithi145 ceph-mon[35096]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:19:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:26 smithi145 ceph-mon[35096]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:26 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2855679960' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:19:27.547 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T04:19:27.547 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:19:27.930 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:28.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.516+0000 7f6813a00640 1 -- 172.21.15.107:0/1650373561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f680c074620 msgr2=0x7f680c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.516+0000 7f6813a00640 1 --2- 172.21.15.107:0/1650373561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f680c074620 0x7f680c074a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6800009900 tx=0x7f680002a480 comp rx=0 tx=0).stop 2024-09-19T04:19:28.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.516+0000 7f6813a00640 1 -- 172.21.15.107:0/1650373561 shutdown_connections 2024-09-19T04:19:28.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.516+0000 7f6813a00640 1 --2- 172.21.15.107:0/1650373561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c10e3c0 0x7f680c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.516+0000 7f6813a00640 1 --2- 172.21.15.107:0/1650373561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680c074f40 0x7f680c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.516+0000 7f6813a00640 1 --2- 172.21.15.107:0/1650373561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f680c074620 0x7f680c074a00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.516+0000 7f6813a00640 1 -- 172.21.15.107:0/1650373561 >> 172.21.15.107:0/1650373561 conn(0x7f680c0fd8c0 msgr2=0x7f680c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:28.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.517+0000 7f6813a00640 1 -- 172.21.15.107:0/1650373561 shutdown_connections 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.517+0000 7f6813a00640 1 -- 172.21.15.107:0/1650373561 wait complete. 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.517+0000 7f6813a00640 1 Processor -- start 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6813a00640 1 -- start start 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6813a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680c074620 0x7f680c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6813a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c074f40 0x7f680c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6813a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f680c10e3c0 0x7f680c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6813a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1141c0 con 0x7f680c10e3c0 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6813a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f680c114040 con 0x7f680c074f40 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6813a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f680c114340 con 0x7f680c074620 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6811a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f680c10e3c0 0x7f680c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6811a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f680c10e3c0 0x7f680c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37968/0 (socket says 172.21.15.107:37968) 2024-09-19T04:19:28.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6811a00640 1 -- 172.21.15.107:0/4225151519 learned_addr learned my addr 172.21.15.107:0/4225151519 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f680be00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c074f40 0x7f680c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.518+0000 7f6811000640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680c074620 0x7f680c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6811a00640 1 -- 172.21.15.107:0/4225151519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680c074620 msgr2=0x7f680c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6811a00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680c074620 0x7f680c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6811a00640 1 -- 172.21.15.107:0/4225151519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c074f40 msgr2=0x7f680c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6811a00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c074f40 0x7f680c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6811a00640 1 -- 172.21.15.107:0/4225151519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c19f650 con 0x7f680c10e3c0 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f680be00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c074f40 0x7f680c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6811a00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f680c10e3c0 0x7f680c19ef70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f680400eb10 tx=0x7f680400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6809600640 1 -- 172.21.15.107:0/4225151519 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804004b20 con 0x7f680c10e3c0 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6809600640 1 -- 172.21.15.107:0/4225151519 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6804004cc0 con 0x7f680c10e3c0 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6813a00640 1 -- 172.21.15.107:0/4225151519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680c19f940 con 0x7f680c10e3c0 2024-09-19T04:19:28.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6809600640 1 -- 172.21.15.107:0/4225151519 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804010760 con 0x7f680c10e3c0 2024-09-19T04:19:28.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.519+0000 7f6813a00640 1 -- 172.21.15.107:0/4225151519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680c1a7220 con 0x7f680c10e3c0 2024-09-19T04:19:28.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.520+0000 7f6813a00640 1 -- 172.21.15.107:0/4225151519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f680c074a00 con 0x7f680c10e3c0 2024-09-19T04:19:28.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.522+0000 7f6809600640 1 -- 172.21.15.107:0/4225151519 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6804005020 con 0x7f680c10e3c0 2024-09-19T04:19:28.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.523+0000 7f6809600640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f67f40a3350 0x7f67f40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.524+0000 7f6809600640 1 -- 172.21.15.107:0/4225151519 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f68040cff60 con 0x7f680c10e3c0 2024-09-19T04:19:28.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.524+0000 7f6811000640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f67f40a3350 0x7f67f40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.524+0000 7f6811000640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f67f40a3350 0x7f67f40a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6800000c00 tx=0x7f6800005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:28.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.527+0000 7f6809600640 1 -- 172.21.15.107:0/4225151519 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f680409caf0 con 0x7f680c10e3c0 2024-09-19T04:19:28.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.737+0000 7f6813a00640 1 -- 172.21.15.107:0/4225151519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f680c02d7c0 con 0x7f680c10e3c0 2024-09-19T04:19:28.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.738+0000 7f6809600640 1 -- 172.21.15.107:0/4225151519 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11992 (secure 0 0 0) 0x7f680409c3f0 con 0x7f680c10e3c0 2024-09-19T04:19:28.740 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:28.740 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"btime":"2024-09-19T04:18:25:571089+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:18:25.571069+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:19:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-09-19T04:19:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 -- 172.21.15.107:0/4225151519 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f67f40a3350 msgr2=0x7f67f40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f67f40a3350 0x7f67f40a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6800000c00 tx=0x7f6800005b50 comp rx=0 tx=0).stop 2024-09-19T04:19:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 -- 172.21.15.107:0/4225151519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f680c10e3c0 msgr2=0x7f680c19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f680c10e3c0 0x7f680c19ef70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f680400eb10 tx=0x7f680400c2d0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 -- 172.21.15.107:0/4225151519 shutdown_connections 2024-09-19T04:19:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f67f40a3350 0x7f67f40a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f680c10e3c0 0x7f680c19ef70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c074f40 0x7f680c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 --2- 172.21.15.107:0/4225151519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680c074620 0x7f680c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 -- 172.21.15.107:0/4225151519 >> 172.21.15.107:0/4225151519 conn(0x7f680c0fd8c0 msgr2=0x7f680c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 -- 172.21.15.107:0/4225151519 shutdown_connections 2024-09-19T04:19:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:28.740+0000 7f6813a00640 1 -- 172.21.15.107:0/4225151519 wait complete. 2024-09-19T04:19:29.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:28 smithi107 ceph-mon[30659]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:29.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:28 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/4225151519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:28 smithi145 ceph-mon[35096]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:28 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/4225151519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:29.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:28 smithi115 ceph-mon[35477]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:29.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:28 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/4225151519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:29.453 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:19:29.826 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:30.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.467+0000 7fe189a00640 1 -- 172.21.15.107:0/913184206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe184101de0 msgr2=0x7fe1841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.467+0000 7fe189a00640 1 --2- 172.21.15.107:0/913184206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe184101de0 0x7fe1841021c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe174009900 tx=0x7fe17402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.467+0000 7fe189a00640 1 -- 172.21.15.107:0/913184206 shutdown_connections 2024-09-19T04:19:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.467+0000 7fe189a00640 1 --2- 172.21.15.107:0/913184206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe184106f10 0x7fe18410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.467+0000 7fe189a00640 1 --2- 172.21.15.107:0/913184206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe184102790 0x7fe1841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.467+0000 7fe189a00640 1 --2- 172.21.15.107:0/913184206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe184101de0 0x7fe1841021c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.468+0000 7fe189a00640 1 -- 172.21.15.107:0/913184206 >> 172.21.15.107:0/913184206 conn(0x7fe1840fd8c0 msgr2=0x7fe1840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.468+0000 7fe189a00640 1 -- 172.21.15.107:0/913184206 shutdown_connections 2024-09-19T04:19:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.468+0000 7fe189a00640 1 -- 172.21.15.107:0/913184206 wait complete. 2024-09-19T04:19:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.469+0000 7fe189a00640 1 Processor -- start 2024-09-19T04:19:30.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.469+0000 7fe189a00640 1 -- start start 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.469+0000 7fe189a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe184101de0 0x7fe184198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.469+0000 7fe189a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe184102790 0x7fe1841989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.469+0000 7fe189a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe184106f10 0x7fe18419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.469+0000 7fe189a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe184111f70 con 0x7fe184106f10 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.469+0000 7fe189a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe184111df0 con 0x7fe184101de0 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.469+0000 7fe189a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1841120f0 con 0x7fe184102790 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe183400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe184106f10 0x7fe18419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe182a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe184101de0 0x7fe184198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe182000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe184102790 0x7fe1841989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe183400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe184106f10 0x7fe18419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56264/0 (socket says 172.21.15.107:56264) 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe183400640 1 -- 172.21.15.107:0/3574301135 learned_addr learned my addr 172.21.15.107:0/3574301135 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:30.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe182a00640 1 -- 172.21.15.107:0/3574301135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe184102790 msgr2=0x7fe1841989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe182a00640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe184102790 0x7fe1841989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe182a00640 1 -- 172.21.15.107:0/3574301135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe184106f10 msgr2=0x7fe18419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe182a00640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe184106f10 0x7fe18419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe182a00640 1 -- 172.21.15.107:0/3574301135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18419d420 con 0x7fe184101de0 2024-09-19T04:19:30.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.470+0000 7fe183400640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe184106f10 0x7fe18419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.471+0000 7fe182a00640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe184101de0 0x7fe184198470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe174005940 tx=0x7fe1740040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.471+0000 7fe17b400640 1 -- 172.21.15.107:0/3574301135 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17403f030 con 0x7fe184101de0 2024-09-19T04:19:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.471+0000 7fe189a00640 1 -- 172.21.15.107:0/3574301135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18419d6b0 con 0x7fe184101de0 2024-09-19T04:19:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.471+0000 7fe17b400640 1 -- 172.21.15.107:0/3574301135 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1740044c0 con 0x7fe184101de0 2024-09-19T04:19:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.471+0000 7fe17b400640 1 -- 172.21.15.107:0/3574301135 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174035070 con 0x7fe184101de0 2024-09-19T04:19:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.471+0000 7fe189a00640 1 -- 172.21.15.107:0/3574301135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1841a4ff0 con 0x7fe184101de0 2024-09-19T04:19:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.472+0000 7fe189a00640 1 -- 172.21.15.107:0/3574301135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1841021c0 con 0x7fe184101de0 2024-09-19T04:19:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.474+0000 7fe17b400640 1 -- 172.21.15.107:0/3574301135 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe174042030 con 0x7fe184101de0 2024-09-19T04:19:30.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.475+0000 7fe17b400640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1680a32d0 0x7fe1680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.476+0000 7fe17b400640 1 -- 172.21.15.107:0/3574301135 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fe1740effe0 con 0x7fe184101de0 2024-09-19T04:19:30.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.476+0000 7fe182000640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1680a32d0 0x7fe1680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.476+0000 7fe182000640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1680a32d0 0x7fe1680a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1840fddc0 tx=0x7fe170005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.478+0000 7fe17b400640 1 -- 172.21.15.107:0/3574301135 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe1740bc520 con 0x7fe184101de0 2024-09-19T04:19:30.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.689+0000 7fe189a00640 1 -- 172.21.15.107:0/3574301135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1841997d0 con 0x7fe184101de0 2024-09-19T04:19:30.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.690+0000 7fe17b400640 1 -- 172.21.15.107:0/3574301135 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11992 (secure 0 0 0) 0x7fe1740c0820 con 0x7fe184101de0 2024-09-19T04:19:30.692 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:30.692 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"btime":"2024-09-19T04:18:25:571089+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:18:25.571069+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:19:30.692 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-09-19T04:19:30.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.692+0000 7fe189a00640 1 -- 172.21.15.107:0/3574301135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1680a32d0 msgr2=0x7fe1680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.692+0000 7fe189a00640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1680a32d0 0x7fe1680a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1840fddc0 tx=0x7fe170005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:30.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.692+0000 7fe189a00640 1 -- 172.21.15.107:0/3574301135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe184101de0 msgr2=0x7fe184198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.692+0000 7fe189a00640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe184101de0 0x7fe184198470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe174005940 tx=0x7fe1740040c0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.692+0000 7fe189a00640 1 -- 172.21.15.107:0/3574301135 shutdown_connections 2024-09-19T04:19:30.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.692+0000 7fe189a00640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fe1680a32d0 0x7fe1680a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.692+0000 7fe189a00640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe184106f10 0x7fe18419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.692+0000 7fe189a00640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe184102790 0x7fe1841989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.692+0000 7fe189a00640 1 --2- 172.21.15.107:0/3574301135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe184101de0 0x7fe184198470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.692+0000 7fe189a00640 1 -- 172.21.15.107:0/3574301135 >> 172.21.15.107:0/3574301135 conn(0x7fe1840fd8c0 msgr2=0x7fe1840fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:30.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.693+0000 7fe189a00640 1 -- 172.21.15.107:0/3574301135 shutdown_connections 2024-09-19T04:19:30.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:30.693+0000 7fe189a00640 1 -- 172.21.15.107:0/3574301135 wait complete. 2024-09-19T04:19:31.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:30 smithi107 ceph-mon[30659]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:31.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:30 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3574301135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:30 smithi145 ceph-mon[35096]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:30 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3574301135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:31.434 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:19:31.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:30 smithi115 ceph-mon[35477]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:31.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:30 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3574301135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:31.801 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:32.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.379+0000 7f957ec00640 1 -- 172.21.15.107:0/2471853893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95780745c0 msgr2=0x7f95780749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.379+0000 7f957ec00640 1 --2- 172.21.15.107:0/2471853893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95780745c0 0x7f95780749a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9568009ca0 tx=0x7f956802a950 comp rx=0 tx=0).stop 2024-09-19T04:19:32.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.379+0000 7f957ec00640 1 -- 172.21.15.107:0/2471853893 shutdown_connections 2024-09-19T04:19:32.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.379+0000 7f957ec00640 1 --2- 172.21.15.107:0/2471853893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f957810e430 0x7f9578110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.379+0000 7f957ec00640 1 --2- 172.21.15.107:0/2471853893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9578074ee0 0x7f957810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.379+0000 7f957ec00640 1 --2- 172.21.15.107:0/2471853893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95780745c0 0x7f95780749a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.379+0000 7f957ec00640 1 -- 172.21.15.107:0/2471853893 >> 172.21.15.107:0/2471853893 conn(0x7f95780fd8b0 msgr2=0x7f95780ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:32.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.379+0000 7f957ec00640 1 -- 172.21.15.107:0/2471853893 shutdown_connections 2024-09-19T04:19:32.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.379+0000 7f957ec00640 1 -- 172.21.15.107:0/2471853893 wait complete. 2024-09-19T04:19:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.380+0000 7f957ec00640 1 Processor -- start 2024-09-19T04:19:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.380+0000 7f957ec00640 1 -- start start 2024-09-19T04:19:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.380+0000 7f957ec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95780745c0 0x7f957819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.380+0000 7f957ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9578074ee0 0x7f957819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.380+0000 7f957ec00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f957810e430 0x7f957819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.380+0000 7f957ec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9578114190 con 0x7f95780745c0 2024-09-19T04:19:32.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.380+0000 7f957ec00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9578114010 con 0x7f957810e430 2024-09-19T04:19:32.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.380+0000 7f957ec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9578114310 con 0x7f9578074ee0 2024-09-19T04:19:32.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f9577e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95780745c0 0x7f957819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f9577e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95780745c0 0x7f957819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56284/0 (socket says 172.21.15.107:56284) 2024-09-19T04:19:32.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f9577e00640 1 -- 172.21.15.107:0/4091455338 learned_addr learned my addr 172.21.15.107:0/4091455338 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:32.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f957cc00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f957810e430 0x7f957819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f956f400640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9578074ee0 0x7f957819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f9577e00640 1 -- 172.21.15.107:0/4091455338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9578074ee0 msgr2=0x7f957819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f9577e00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9578074ee0 0x7f957819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f9577e00640 1 -- 172.21.15.107:0/4091455338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f957810e430 msgr2=0x7f957819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f9577e00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f957810e430 0x7f957819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f9577e00640 1 -- 172.21.15.107:0/4091455338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957819f600 con 0x7f95780745c0 2024-09-19T04:19:32.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.381+0000 7f957cc00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f957810e430 0x7f957819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:32.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.382+0000 7f9577e00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95780745c0 0x7f957819a650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9568005cc0 tx=0x7f956802f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.382+0000 7f9575600640 1 -- 172.21.15.107:0/4091455338 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568034040 con 0x7f95780745c0 2024-09-19T04:19:32.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.382+0000 7f9575600640 1 -- 172.21.15.107:0/4091455338 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9568030920 con 0x7f95780745c0 2024-09-19T04:19:32.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.382+0000 7f9575600640 1 -- 172.21.15.107:0/4091455338 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95680046e0 con 0x7f95780745c0 2024-09-19T04:19:32.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.382+0000 7f957ec00640 1 -- 172.21.15.107:0/4091455338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957819f890 con 0x7f95780745c0 2024-09-19T04:19:32.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.382+0000 7f957ec00640 1 -- 172.21.15.107:0/4091455338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95781a71d0 con 0x7f95780745c0 2024-09-19T04:19:32.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.383+0000 7f957ec00640 1 -- 172.21.15.107:0/4091455338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95780749a0 con 0x7f95780745c0 2024-09-19T04:19:32.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.385+0000 7f9575600640 1 -- 172.21.15.107:0/4091455338 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9568037020 con 0x7f95780745c0 2024-09-19T04:19:32.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.386+0000 7f9575600640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f955c0a3350 0x7f955c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.386+0000 7f9575600640 1 -- 172.21.15.107:0/4091455338 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f95680f07a0 con 0x7f95780745c0 2024-09-19T04:19:32.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.386+0000 7f956f400640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f955c0a3350 0x7f955c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.387+0000 7f956f400640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f955c0a3350 0x7f955c0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f95780ffe40 tx=0x7f9560005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.390+0000 7f9575600640 1 -- 172.21.15.107:0/4091455338 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f95680bd440 con 0x7f95780745c0 2024-09-19T04:19:32.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.573+0000 7f957ec00640 1 -- 172.21.15.107:0/4091455338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95781095b0 con 0x7f95780745c0 2024-09-19T04:19:32.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.576+0000 7f9575600640 1 -- 172.21.15.107:0/4091455338 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22735 (secure 0 0 0) 0x7f95680bcd40 con 0x7f95780745c0 2024-09-19T04:19:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:32.580 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":83,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:19:00.968552+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.578+0000 7f957ec00640 1 -- 172.21.15.107:0/4091455338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f955c0a3350 msgr2=0x7f955c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.578+0000 7f957ec00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f955c0a3350 0x7f955c0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f95780ffe40 tx=0x7f9560005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.578+0000 7f957ec00640 1 -- 172.21.15.107:0/4091455338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95780745c0 msgr2=0x7f957819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.578+0000 7f957ec00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95780745c0 0x7f957819a650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9568005cc0 tx=0x7f956802f420 comp rx=0 tx=0).stop 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.578+0000 7f957ec00640 1 -- 172.21.15.107:0/4091455338 shutdown_connections 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.578+0000 7f957ec00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f955c0a3350 0x7f955c0a5810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.578+0000 7f957ec00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f957810e430 0x7f957819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.578+0000 7f957ec00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9578074ee0 0x7f957819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.578+0000 7f957ec00640 1 --2- 172.21.15.107:0/4091455338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95780745c0 0x7f957819a650 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.578+0000 7f957ec00640 1 -- 172.21.15.107:0/4091455338 >> 172.21.15.107:0/4091455338 conn(0x7f95780fd8b0 msgr2=0x7f95780fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.579+0000 7f957ec00640 1 -- 172.21.15.107:0/4091455338 shutdown_connections 2024-09-19T04:19:32.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:32.579+0000 7f957ec00640 1 -- 172.21.15.107:0/4091455338 wait complete. 2024-09-19T04:19:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:32 smithi145 ceph-mon[35096]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:32 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/4091455338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:19:33.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:32 smithi107 ceph-mon[30659]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:33.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:32 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/4091455338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:19:33.313 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:19:33.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:32 smithi115 ceph-mon[35477]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:33.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:32 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/4091455338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:19:33.684 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:34.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.186+0000 7f2870200640 1 -- 172.21.15.107:0/400315027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f286810e3c0 msgr2=0x7f2868110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:34.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.186+0000 7f2870200640 1 --2- 172.21.15.107:0/400315027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f286810e3c0 0x7f2868110810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2860009ca0 tx=0x7f286002a950 comp rx=0 tx=0).stop 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.186+0000 7f2870200640 1 -- 172.21.15.107:0/400315027 shutdown_connections 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.186+0000 7f2870200640 1 --2- 172.21.15.107:0/400315027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f286810e3c0 0x7f2868110810 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.186+0000 7f2870200640 1 --2- 172.21.15.107:0/400315027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2868074f40 0x7f286810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.186+0000 7f2870200640 1 --2- 172.21.15.107:0/400315027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2868074620 0x7f2868074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.186+0000 7f2870200640 1 -- 172.21.15.107:0/400315027 >> 172.21.15.107:0/400315027 conn(0x7f28680fd8c0 msgr2=0x7f28680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.186+0000 7f2870200640 1 -- 172.21.15.107:0/400315027 shutdown_connections 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.186+0000 7f2870200640 1 -- 172.21.15.107:0/400315027 wait complete. 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.187+0000 7f2870200640 1 Processor -- start 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.188+0000 7f2870200640 1 -- start start 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.188+0000 7f2870200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2868074620 0x7f286819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.188+0000 7f2870200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2868074f40 0x7f286819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:34.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.188+0000 7f2870200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286810e3c0 0x7f286819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.188+0000 7f2870200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28681141a0 con 0x7f2868074620 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.188+0000 7f2870200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2868114020 con 0x7f286810e3c0 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.188+0000 7f2870200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2868114320 con 0x7f2868074f40 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.188+0000 7f286d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2868074620 0x7f286819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.188+0000 7f286d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2868074620 0x7f286819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56294/0 (socket says 172.21.15.107:56294) 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.188+0000 7f286d800640 1 -- 172.21.15.107:0/3006037316 learned_addr learned my addr 172.21.15.107:0/3006037316 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f286ce00640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2868074f40 0x7f286819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f286e200640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286810e3c0 0x7f286819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f286d800640 1 -- 172.21.15.107:0/3006037316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2868074f40 msgr2=0x7f286819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f286d800640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2868074f40 0x7f286819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f286d800640 1 -- 172.21.15.107:0/3006037316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286810e3c0 msgr2=0x7f286819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f286d800640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286810e3c0 0x7f286819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f286d800640 1 -- 172.21.15.107:0/3006037316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286819f5f0 con 0x7f2868074620 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f286e200640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286810e3c0 0x7f286819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f286d800640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2868074620 0x7f286819a640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f285c00e9f0 tx=0x7f285c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f2866000640 1 -- 172.21.15.107:0/3006037316 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c00cd80 con 0x7f2868074620 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f2866000640 1 -- 172.21.15.107:0/3006037316 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f285c010910 con 0x7f2868074620 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f2866000640 1 -- 172.21.15.107:0/3006037316 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c018990 con 0x7f2868074620 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f2870200640 1 -- 172.21.15.107:0/3006037316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286819f8e0 con 0x7f2868074620 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.189+0000 7f2870200640 1 -- 172.21.15.107:0/3006037316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28681a71c0 con 0x7f2868074620 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.190+0000 7f2870200640 1 -- 172.21.15.107:0/3006037316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2868074a00 con 0x7f2868074620 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.192+0000 7f2866000640 1 -- 172.21.15.107:0/3006037316 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f285c010ac0 con 0x7f2868074620 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.193+0000 7f2866000640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f28500a3350 0x7f28500a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:34.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.194+0000 7f2866000640 1 -- 172.21.15.107:0/3006037316 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f285c0cfde0 con 0x7f2868074620 2024-09-19T04:19:34.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.194+0000 7f286ce00640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f28500a3350 0x7f28500a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:34.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.194+0000 7f286ce00640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f28500a3350 0x7f28500a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f286810f750 tx=0x7f2854005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:34.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.197+0000 7f2866000640 1 -- 172.21.15.107:0/3006037316 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f285c09c970 con 0x7f2868074620 2024-09-19T04:19:34.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.407+0000 7f2870200640 1 -- 172.21.15.107:0/3006037316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f286819b9a0 con 0x7f2868074620 2024-09-19T04:19:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.408+0000 7f2866000640 1 -- 172.21.15.107:0/3006037316 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11992 (secure 0 0 0) 0x7f285c09c270 con 0x7f2868074620 2024-09-19T04:19:34.410 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:34.410 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"btime":"2024-09-19T04:18:25:571089+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:18:25.571069+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:19:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-09-19T04:19:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 -- 172.21.15.107:0/3006037316 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f28500a3350 msgr2=0x7f28500a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f28500a3350 0x7f28500a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f286810f750 tx=0x7f2854005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 -- 172.21.15.107:0/3006037316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2868074620 msgr2=0x7f286819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2868074620 0x7f286819a640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f285c00e9f0 tx=0x7f285c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 -- 172.21.15.107:0/3006037316 shutdown_connections 2024-09-19T04:19:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f28500a3350 0x7f28500a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286810e3c0 0x7f286819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2868074f40 0x7f286819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 --2- 172.21.15.107:0/3006037316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2868074620 0x7f286819a640 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 -- 172.21.15.107:0/3006037316 >> 172.21.15.107:0/3006037316 conn(0x7f28680fd8c0 msgr2=0x7f28680fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 -- 172.21.15.107:0/3006037316 shutdown_connections 2024-09-19T04:19:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:34.411+0000 7f2870200640 1 -- 172.21.15.107:0/3006037316 wait complete. 2024-09-19T04:19:35.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:34 smithi107 ceph-mon[30659]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:35.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:34 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3006037316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:35.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T04:18:13.349314+0000', 'modified': '2024-09-19T04:18:25.571069+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24517, 'mds_2': 24500, 'mds_3': 24496, 'mds_4': 24490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14499': {'gid': 14499, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6839/2243096646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 2243096646}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 2243096646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14511': {'gid': 14511, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6841/305932896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6840', 'nonce': 305932896}, {'type': 'v1', 'addr': '172.21.15.107:6841', 'nonce': 305932896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6833/1661283120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1661283120}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1661283120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6833/2248561847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6832', 'nonce': 2248561847}, {'type': 'v1', 'addr': '172.21.15.145:6833', 'nonce': 2248561847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/687464127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 687464127}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 687464127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.145:6835/4237714429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 4237714429}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 4237714429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.145:6837/1756017901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 1756017901}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 1756017901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.115:6837/191835918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 191835918}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 191835918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.145:6839/4220252383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 4220252383}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 4220252383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.115:6839/2907975805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 2907975805}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 2907975805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24517, 24539, 24500, 24496, 24490]} 2024-09-19T04:19:35.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T04:19:35.225 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T04:19:35.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:34 smithi145 ceph-mon[35096]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:35.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:34 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3006037316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:34 smithi115 ceph-mon[35477]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:34 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3006037316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:35.594 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:36.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.166+0000 7ff729c00640 1 -- 172.21.15.107:0/2988023822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff724106f10 msgr2=0x7ff72410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.166+0000 7ff729c00640 1 --2- 172.21.15.107:0/2988023822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff724106f10 0x7ff72410ea90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009900 tx=0x7ff71c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.166+0000 7ff729c00640 1 -- 172.21.15.107:0/2988023822 shutdown_connections 2024-09-19T04:19:36.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.166+0000 7ff729c00640 1 --2- 172.21.15.107:0/2988023822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff724106f10 0x7ff72410ea90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.166+0000 7ff729c00640 1 --2- 172.21.15.107:0/2988023822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff724102790 0x7ff7241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.166+0000 7ff729c00640 1 --2- 172.21.15.107:0/2988023822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff724101de0 0x7ff7241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.166+0000 7ff729c00640 1 -- 172.21.15.107:0/2988023822 >> 172.21.15.107:0/2988023822 conn(0x7ff7240fd8c0 msgr2=0x7ff7240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:36.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.166+0000 7ff729c00640 1 -- 172.21.15.107:0/2988023822 shutdown_connections 2024-09-19T04:19:36.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.166+0000 7ff729c00640 1 -- 172.21.15.107:0/2988023822 wait complete. 2024-09-19T04:19:36.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.167+0000 7ff729c00640 1 Processor -- start 2024-09-19T04:19:36.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.167+0000 7ff729c00640 1 -- start start 2024-09-19T04:19:36.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff729c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff724101de0 0x7ff72419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff729c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff724102790 0x7ff72419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff729c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff724106f10 0x7ff72419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff729c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff724111f70 con 0x7ff724102790 2024-09-19T04:19:36.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff729c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff724111df0 con 0x7ff724101de0 2024-09-19T04:19:36.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff729c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241120f0 con 0x7ff724106f10 2024-09-19T04:19:36.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff722000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff724102790 0x7ff72419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff722a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff724101de0 0x7ff72419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff722000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff724102790 0x7ff72419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56314/0 (socket says 172.21.15.107:56314) 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff722000640 1 -- 172.21.15.107:0/3086118840 learned_addr learned my addr 172.21.15.107:0/3086118840 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff722a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff724101de0 0x7ff72419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54742/0 (socket says 172.21.15.107:54742) 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.168+0000 7ff723400640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff724106f10 0x7ff72419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff722a00640 1 -- 172.21.15.107:0/3086118840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff724106f10 msgr2=0x7ff72419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff722a00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff724106f10 0x7ff72419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff722a00640 1 -- 172.21.15.107:0/3086118840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff724102790 msgr2=0x7ff72419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff722a00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff724102790 0x7ff72419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff722a00640 1 -- 172.21.15.107:0/3086118840 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72419f5b0 con 0x7ff724101de0 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff722000640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff724102790 0x7ff72419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff722a00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff724101de0 0x7ff72419a6a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff714009fd0 tx=0x7ff71400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:36.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff71b400640 1 -- 172.21.15.107:0/3086118840 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714004b20 con 0x7ff724101de0 2024-09-19T04:19:36.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff729c00640 1 -- 172.21.15.107:0/3086118840 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72419f8a0 con 0x7ff724101de0 2024-09-19T04:19:36.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff71b400640 1 -- 172.21.15.107:0/3086118840 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff714004cc0 con 0x7ff724101de0 2024-09-19T04:19:36.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff71b400640 1 -- 172.21.15.107:0/3086118840 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7140107b0 con 0x7ff724101de0 2024-09-19T04:19:36.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.169+0000 7ff729c00640 1 -- 172.21.15.107:0/3086118840 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7241a7180 con 0x7ff724101de0 2024-09-19T04:19:36.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.173+0000 7ff72c663740 1 Processor -- start 2024-09-19T04:19:36.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.173+0000 7ff72c663740 1 -- start start 2024-09-19T04:19:36.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff71b400640 1 -- 172.21.15.107:0/3086118840 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff714005020 con 0x7ff724101de0 2024-09-19T04:19:36.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff72c663740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff72c663740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff72c663740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 0x5648ba0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff72c663740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5648ba0280b0 con 0x5648ba0cd510 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff72c663740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5648b9fcfab0 con 0x5648ba0c2330 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff72c663740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5648ba024070 con 0x5648ba0c2c50 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff723400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff723400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54752/0 (socket says 172.21.15.107:54752) 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff722000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 0x5648ba0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff723400640 1 -- 172.21.15.107:0/3103888832 learned_addr learned my addr 172.21.15.107:0/3103888832 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.174+0000 7ff722a00640 1 --2- 172.21.15.107:0/3103888832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.175+0000 7ff723400640 1 -- 172.21.15.107:0/3103888832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 msgr2=0x5648ba0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.175+0000 7ff723400640 1 --2- 172.21.15.107:0/3103888832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.175+0000 7ff723400640 1 -- 172.21.15.107:0/3103888832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 msgr2=0x5648ba0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.175+0000 7ff723400640 1 --2- 172.21.15.107:0/3103888832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 0x5648ba0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.175+0000 7ff723400640 1 -- 172.21.15.107:0/3103888832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b9fdc4a0 con 0x5648ba0c2330 2024-09-19T04:19:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.175+0000 7ff722000640 1 --2- 172.21.15.107:0/3103888832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 0x5648ba0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.175+0000 7ff723400640 1 --2- 172.21.15.107:0/3103888832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0c2710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff71c033000 tx=0x7ff71c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9dd34c46bf49a24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.175+0000 7ff70fe00640 1 -- 172.21.15.107:0/3103888832 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c040070 con 0x5648ba0c2330 2024-09-19T04:19:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.175+0000 7ff70fe00640 1 -- 172.21.15.107:0/3103888832 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c0317b0 con 0x5648ba0c2330 2024-09-19T04:19:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.175+0000 7ff70fe00640 1 -- 172.21.15.107:0/3103888832 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c035040 con 0x5648ba0c2330 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff71b400640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7080a32d0 0x7ff7080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff72c663740 1 -- 172.21.15.107:0/3103888832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 msgr2=0x5648ba0c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff72c663740 1 --2- 172.21.15.107:0/3103888832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0c2710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff71c033000 tx=0x7ff71c005d50 comp rx=0 tx=0).stop 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff71b400640 1 -- 172.21.15.107:0/3086118840 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7ff7140d01f0 con 0x7ff724101de0 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff722a00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7080a32d0 0x7ff7080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff72c663740 1 -- 172.21.15.107:0/3103888832 shutdown_connections 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff72c663740 1 --2- 172.21.15.107:0/3103888832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 0x5648ba0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff72c663740 1 --2- 172.21.15.107:0/3103888832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff72c663740 1 --2- 172.21.15.107:0/3103888832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0c2710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff72c663740 1 -- 172.21.15.107:0/3103888832 >> 172.21.15.107:0/3103888832 conn(0x5648ba0be8c0 msgr2=0x5648ba0beca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff72c663740 1 -- 172.21.15.107:0/3103888832 shutdown_connections 2024-09-19T04:19:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.176+0000 7ff72c663740 1 -- 172.21.15.107:0/3103888832 wait complete. 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.177+0000 7ff722a00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7080a32d0 0x7ff7080a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5648ba0bf720 tx=0x7ff714005990 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.177+0000 7ff72c663740 1 -- start start 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.177+0000 7ff72c663740 1 Processor -- start 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff72c663740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff72c663740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff72c663740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 0x5648ba0cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff72c663740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5648ba0d0060 con 0x5648ba0cd510 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff72c663740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5648b9fb0920 con 0x5648ba0c2330 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff72c663740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5648ba0d01e0 con 0x5648ba0c2c50 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff72c663740 10 client.0 _ll_register_callbacks cb 0x5648b9fd48a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff723400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 0x5648ba0cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff722000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff723400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 0x5648ba0cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56324/0 (socket says 172.21.15.107:56324) 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff723400640 1 -- 172.21.15.107:0/3638072229 learned_addr learned my addr 172.21.15.107:0/3638072229 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff723400640 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff722000640 1 -- 172.21.15.107:0/3638072229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 msgr2=0x5648ba0ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff722000640 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff722000640 1 -- 172.21.15.107:0/3638072229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 msgr2=0x5648ba0cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff722000640 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 0x5648ba0cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff722000640 1 -- 172.21.15.107:0/3638072229 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b9d992b0 con 0x5648ba0c2c50 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.178+0000 7ff723400640 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.179+0000 7ff722000640 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0caac0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff72410eeb0 tx=0x7ff710012f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=827c470a5048f847 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.179+0000 7ff70cc00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:19:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.179+0000 7ff70cc00640 1 -- 172.21.15.107:0/3638072229 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71000fb00 con 0x5648ba0c2c50 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.179+0000 7ff72c663740 1 -- 172.21.15.107:0/3638072229 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff71c0317b0 con 0x5648ba0c2c50 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.179+0000 7ff70cc00640 1 -- 172.21.15.107:0/3638072229 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7ff71000fca0 con 0x5648ba0c2c50 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.179+0000 7ff70cc00640 1 -- 172.21.15.107:0/3638072229 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff710023070 con 0x5648ba0c2c50 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.179+0000 7ff72c663740 10 client.24880 fetch_fsmap learned FSMap version 27 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.179+0000 7ff72c663740 1 -- 172.21.15.107:0/3638072229 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648ba0dbec0 con 0x5648ba0c2c50 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.179+0000 7ff70cc00640 1 -- 172.21.15.107:0/3638072229 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c0317b0 con 0x5648ba0c2c50 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.180+0000 7ff70cc00640 1 -- 172.21.15.107:0/3638072229 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff710005430 con 0x5648ba0c2c50 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.180+0000 7ff72c663740 10 client.24880 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.180+0000 7ff72c663740 10 client.24880 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.180+0000 7ff72c663740 20 client.24880 populate_metadata read hostname 'smithi107' 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.180+0000 7ff72c663740 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5648ba0de4c0 0x5648ba0df930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.180+0000 7ff72c663740 4 client.24880 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.180+0000 7ff72c663740 1 -- 172.21.15.107:0/3638072229 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648ba035620 con 0x5648ba0de4c0 2024-09-19T04:19:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.180+0000 7ff723400640 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5648ba0de4c0 0x5648ba0df930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.181+0000 7ff723400640 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5648ba0de4c0 0x5648ba0df930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2729513c76841c75 server_cookie=47d04b83d8db6ec5 in_seq=0 out_seq=0 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.181+0000 7ff70cc00640 10 client.24880 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff70cc00640 1 -- 172.21.15.107:0/3638072229 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648ba035620 con 0x5648ba0de4c0 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff70cc00640 10 client.24880 handle_command_reply: tid=0 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff72c663740 1 client.24880 shutdown 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff72c663740 1 -- 172.21.15.107:0/3638072229 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 msgr2=0x5648ba0caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff72c663740 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0caac0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff72410eeb0 tx=0x7ff710012f70 comp rx=0 tx=0).stop 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff72c663740 1 -- 172.21.15.107:0/3638072229 shutdown_connections 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff72c663740 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5648ba0de4c0 0x5648ba0df930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff72c663740 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0cd510 0x5648ba0cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff70cc00640 0 client.24880 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:19:36.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff72c663740 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0caac0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff72c663740 1 --2- 172.21.15.107:0/3638072229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0c2330 0x5648ba0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff72c663740 1 -- 172.21.15.107:0/3638072229 >> 172.21.15.107:0/3638072229 conn(0x5648ba0be8c0 msgr2=0x5648ba0c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:36.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.182+0000 7ff72c663740 1 -- 172.21.15.107:0/3638072229 shutdown_connections 2024-09-19T04:19:36.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.183+0000 7ff72c663740 1 -- 172.21.15.107:0/3638072229 wait complete. 2024-09-19T04:19:36.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.183+0000 7ff72c663740 20 client.24880 trim_cache size 0 max 16384 2024-09-19T04:19:36.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.201+0000 7ff72c663740 1 Processor -- start 2024-09-19T04:19:36.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.201+0000 7ff72c663740 1 -- start start 2024-09-19T04:19:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.201+0000 7ff72c663740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.201+0000 7ff72c663740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.201+0000 7ff72c663740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0cd510 0x5648ba0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.201+0000 7ff72c663740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5648ba0ddc70 con 0x5648ba0c2330 2024-09-19T04:19:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.201+0000 7ff72c663740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5648ba0de340 con 0x5648ba0cd510 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.201+0000 7ff72c663740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5648ba0dea10 con 0x5648ba0c2c50 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff723400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff723400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56336/0 (socket says 172.21.15.107:56336) 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff723400640 1 -- 172.21.15.107:0/3115331265 learned_addr learned my addr 172.21.15.107:0/3115331265 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff722000640 1 --2- 172.21.15.107:0/3115331265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff723400640 1 --2- 172.21.15.107:0/3115331265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0cd510 0x5648ba0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff723400640 1 -- 172.21.15.107:0/3115331265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 msgr2=0x5648ba0c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff723400640 1 --2- 172.21.15.107:0/3115331265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff723400640 1 -- 172.21.15.107:0/3115331265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0cd510 msgr2=0x5648ba0c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff723400640 1 --2- 172.21.15.107:0/3115331265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0cd510 0x5648ba0c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff723400640 1 -- 172.21.15.107:0/3115331265 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ba0bf4c0 con 0x5648ba0c2330 2024-09-19T04:19:36.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.202+0000 7ff723400640 1 --2- 172.21.15.107:0/3115331265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0c8eb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff71c006010 tx=0x7ff71c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b179f9abb807b14f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:36.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff70f400640 1 -- 172.21.15.107:0/3115331265 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c035040 con 0x5648ba0c2330 2024-09-19T04:19:36.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff70f400640 1 -- 172.21.15.107:0/3115331265 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c004040 con 0x5648ba0c2330 2024-09-19T04:19:36.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff70f400640 1 -- 172.21.15.107:0/3115331265 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c040040 con 0x5648ba0c2330 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff72c663740 1 -- 172.21.15.107:0/3115331265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 msgr2=0x5648ba0c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff72c663740 1 --2- 172.21.15.107:0/3115331265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0c8eb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff71c006010 tx=0x7ff71c005be0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff72c663740 1 -- 172.21.15.107:0/3115331265 shutdown_connections 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff72c663740 1 --2- 172.21.15.107:0/3115331265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0cd510 0x5648ba0c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff72c663740 1 --2- 172.21.15.107:0/3115331265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff72c663740 1 --2- 172.21.15.107:0/3115331265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0c8eb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff72c663740 1 -- 172.21.15.107:0/3115331265 >> 172.21.15.107:0/3115331265 conn(0x5648ba0be8c0 msgr2=0x5648ba0c5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff72c663740 1 -- 172.21.15.107:0/3115331265 shutdown_connections 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.203+0000 7ff72c663740 1 -- 172.21.15.107:0/3115331265 wait complete. 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.204+0000 7ff72c663740 1 -- start start 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.204+0000 7ff72c663740 1 Processor -- start 2024-09-19T04:19:36.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.204+0000 7ff72c663740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0e8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.204+0000 7ff72c663740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.204+0000 7ff72c663740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0cd510 0x5648ba0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.204+0000 7ff72c663740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5648ba0df7b0 con 0x5648ba0c2330 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.204+0000 7ff72c663740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5648ba0df0e0 con 0x5648ba0cd510 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.204+0000 7ff72c663740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5648ba0dfe80 con 0x5648ba0c2c50 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.204+0000 7ff72c663740 10 client.0 _ll_register_callbacks cb 0x7ff710012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff723400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff723400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0e8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56338/0 (socket says 172.21.15.107:56338) 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff723400640 1 -- 172.21.15.107:0/1408987499 learned_addr learned my addr 172.21.15.107:0/1408987499 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff722000640 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff723400640 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0cd510 0x5648ba0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff723400640 1 -- 172.21.15.107:0/1408987499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 msgr2=0x5648ba0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff723400640 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff723400640 1 -- 172.21.15.107:0/1408987499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0cd510 msgr2=0x5648ba0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff723400640 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0cd510 0x5648ba0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff723400640 1 -- 172.21.15.107:0/1408987499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ba0cb040 con 0x5648ba0c2330 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff723400640 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0e8170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0055f0 tx=0x7ff71c004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e845ed6fecbab2c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.205+0000 7ff70cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.206+0000 7ff70cc00640 1 -- 172.21.15.107:0/1408987499 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c063060 con 0x5648ba0c2330 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.206+0000 7ff70cc00640 1 -- 172.21.15.107:0/1408987499 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7ff71c004470 con 0x5648ba0c2330 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.206+0000 7ff72c663740 1 -- 172.21.15.107:0/1408987499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff71c004040 con 0x5648ba0c2330 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.206+0000 7ff70cc00640 1 -- 172.21.15.107:0/1408987499 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff71c040070 con 0x5648ba0c2330 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.206+0000 7ff72c663740 10 client.14871 fetch_fsmap learned FSMap version 27 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.206+0000 7ff72c663740 1 -- 172.21.15.107:0/1408987499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648ba0e69b0 con 0x5648ba0c2330 2024-09-19T04:19:36.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.206+0000 7ff70cc00640 1 -- 172.21.15.107:0/1408987499 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c004040 con 0x5648ba0c2330 2024-09-19T04:19:36.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.207+0000 7ff70cc00640 1 -- 172.21.15.107:0/1408987499 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff71c0037b0 con 0x5648ba0c2330 2024-09-19T04:19:36.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.207+0000 7ff72c663740 10 client.14871 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:19:36.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.207+0000 7ff72c663740 10 client.14871 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:19:36.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.207+0000 7ff72c663740 20 client.14871 populate_metadata read hostname 'smithi107' 2024-09-19T04:19:36.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.207+0000 7ff72c663740 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5648ba14d010 0x5648ba14bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.207+0000 7ff72c663740 4 client.14871 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:19:36.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.207+0000 7ff72c663740 1 -- 172.21.15.107:0/1408987499 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "status"}) -- 0x5648ba02fb30 con 0x5648ba14d010 2024-09-19T04:19:36.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.207+0000 7ff722000640 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5648ba14d010 0x5648ba14bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff722000640 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5648ba14d010 0x5648ba14bd20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b05fae2bf1cb9ee server_cookie=54ff7b90ce1648b3 in_seq=0 out_seq=0 2024-09-19T04:19:36.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff70cc00640 10 client.14871 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:19:36.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff70cc00640 1 -- 172.21.15.107:0/1408987499 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5648ba02fb30 con 0x5648ba14d010 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff70cc00640 10 client.14871 handle_command_reply: tid=0 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 client.14871 shutdown 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 -- 172.21.15.107:0/1408987499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 msgr2=0x5648ba0e8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0e8170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0055f0 tx=0x7ff71c004890 comp rx=0 tx=0).stop 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 -- 172.21.15.107:0/1408987499 shutdown_connections 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5648ba14d010 0x5648ba14bd20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648ba0cd510 0x5648ba0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648ba0c2c50 0x5648ba0ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 --2- 172.21.15.107:0/1408987499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648ba0c2330 0x5648ba0e8170 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 -- 172.21.15.107:0/1408987499 >> 172.21.15.107:0/1408987499 conn(0x5648ba0be8c0 msgr2=0x5648ba0d8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff70cc00640 0 client.14871 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:19:36.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 -- 172.21.15.107:0/1408987499 shutdown_connections 2024-09-19T04:19:36.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 1 -- 172.21.15.107:0/1408987499 wait complete. 2024-09-19T04:19:36.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.208+0000 7ff72c663740 20 client.14871 trim_cache size 0 max 16384 2024-09-19T04:19:36.215 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:19:36.215 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:19:36.215 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 4, 2024-09-19T04:19:36.215 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24490, 2024-09-19T04:19:36.215 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-09-19T04:19:36.215 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-09-19T04:19:36.215 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-09-19T04:19:36.216 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 71.640821588999998, 2024-09-19T04:19:36.216 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-09-19T04:19:36.216 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 83, 2024-09-19T04:19:36.216 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T04:19:36.216 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 112.282373732 2024-09-19T04:19:36.216 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:19:36.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 -- 172.21.15.107:0/3086118840 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7080a32d0 msgr2=0x7ff7080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7080a32d0 0x7ff7080a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5648ba0bf720 tx=0x7ff714005990 comp rx=0 tx=0).stop 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 -- 172.21.15.107:0/3086118840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff724101de0 msgr2=0x7ff72419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff724101de0 0x7ff72419a6a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff714009fd0 tx=0x7ff71400ef90 comp rx=0 tx=0).stop 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 -- 172.21.15.107:0/3086118840 shutdown_connections 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7080a32d0 0x7ff7080a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff724106f10 0x7ff72419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff724102790 0x7ff72419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 --2- 172.21.15.107:0/3086118840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff724101de0 0x7ff72419a6a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 -- 172.21.15.107:0/3086118840 >> 172.21.15.107:0/3086118840 conn(0x7ff7240fd8c0 msgr2=0x7ff7240ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 -- 172.21.15.107:0/3086118840 shutdown_connections 2024-09-19T04:19:36.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:36.209+0000 7ff729c00640 1 -- 172.21.15.107:0/3086118840 wait complete. 2024-09-19T04:19:37.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T04:19:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:36 smithi145 ceph-mon[35096]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:36 smithi107 ceph-mon[30659]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:37.435 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:37.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:36 smithi115 ceph-mon[35477]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:38.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.052+0000 7f3b3a400640 1 -- 172.21.15.107:0/2808622990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b34074590 msgr2=0x7f3b34074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.052+0000 7f3b3a400640 1 --2- 172.21.15.107:0/2808622990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b34074590 0x7f3b34074970 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009900 tx=0x7f3b2402a480 comp rx=0 tx=0).stop 2024-09-19T04:19:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.052+0000 7f3b3a400640 1 -- 172.21.15.107:0/2808622990 shutdown_connections 2024-09-19T04:19:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.052+0000 7f3b3a400640 1 --2- 172.21.15.107:0/2808622990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b34112b90 0x7f3b34114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.052+0000 7f3b3a400640 1 --2- 172.21.15.107:0/2808622990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34074eb0 0x7f3b34112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.052+0000 7f3b3a400640 1 --2- 172.21.15.107:0/2808622990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b34074590 0x7f3b34074970 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.052+0000 7f3b3a400640 1 -- 172.21.15.107:0/2808622990 >> 172.21.15.107:0/2808622990 conn(0x7f3b340fd8c0 msgr2=0x7f3b340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.052+0000 7f3b3a400640 1 -- 172.21.15.107:0/2808622990 shutdown_connections 2024-09-19T04:19:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.052+0000 7f3b3a400640 1 -- 172.21.15.107:0/2808622990 wait complete. 2024-09-19T04:19:38.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.053+0000 7f3b3a400640 1 Processor -- start 2024-09-19T04:19:38.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.053+0000 7f3b3a400640 1 -- start start 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.053+0000 7f3b3a400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34074590 0x7f3b341a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.053+0000 7f3b3a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b34074eb0 0x7f3b341a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.053+0000 7f3b3a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b34112b90 0x7f3b3419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.053+0000 7f3b3a400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34118570 con 0x7f3b34074eb0 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.053+0000 7f3b3a400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341183f0 con 0x7f3b34074590 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.053+0000 7f3b3a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341186f0 con 0x7f3b34112b90 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.053+0000 7f3b32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b34074eb0 0x7f3b341a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.054+0000 7f3b32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b34074eb0 0x7f3b341a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56352/0 (socket says 172.21.15.107:56352) 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.054+0000 7f3b32a00640 1 -- 172.21.15.107:0/463993313 learned_addr learned my addr 172.21.15.107:0/463993313 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.054+0000 7f3b33e00640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b34112b90 0x7f3b3419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.054+0000 7f3b33400640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34074590 0x7f3b341a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.054+0000 7f3b32a00640 1 -- 172.21.15.107:0/463993313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b34112b90 msgr2=0x7f3b3419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.054+0000 7f3b32a00640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b34112b90 0x7f3b3419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.054+0000 7f3b32a00640 1 -- 172.21.15.107:0/463993313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34074590 msgr2=0x7f3b341a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.054+0000 7f3b32a00640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34074590 0x7f3b341a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.054+0000 7f3b32a00640 1 -- 172.21.15.107:0/463993313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3419eff0 con 0x7f3b34074eb0 2024-09-19T04:19:38.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.054+0000 7f3b32a00640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b34074eb0 0x7f3b341a4c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000e9f0 tx=0x7f3b2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.055+0000 7f3b2be00640 1 -- 172.21.15.107:0/463993313 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000cd80 con 0x7f3b34074eb0 2024-09-19T04:19:38.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.055+0000 7f3b2be00640 1 -- 172.21.15.107:0/463993313 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b20010910 con 0x7f3b34074eb0 2024-09-19T04:19:38.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.055+0000 7f3b3a400640 1 -- 172.21.15.107:0/463993313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3419f2e0 con 0x7f3b34074eb0 2024-09-19T04:19:38.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.055+0000 7f3b2be00640 1 -- 172.21.15.107:0/463993313 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20018990 con 0x7f3b34074eb0 2024-09-19T04:19:38.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.055+0000 7f3b3a400640 1 -- 172.21.15.107:0/463993313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341ab5d0 con 0x7f3b34074eb0 2024-09-19T04:19:38.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.058+0000 7f3b3d00b740 1 Processor -- start 2024-09-19T04:19:38.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.058+0000 7f3b3d00b740 1 -- start start 2024-09-19T04:19:38.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.058+0000 7f3b2be00640 1 -- 172.21.15.107:0/463993313 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3b20010ac0 con 0x7f3b34074eb0 2024-09-19T04:19:38.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.058+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004aedf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004aee9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aeec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56004ae450b0 con 0x56004aedfc50 2024-09-19T04:19:38.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56004adecab0 con 0x56004aeea510 2024-09-19T04:19:38.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56004ae41070 con 0x56004aedf330 2024-09-19T04:19:38.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004aedf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b33400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004aee9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004aedf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37588/0 (socket says 172.21.15.107:37588) 2024-09-19T04:19:38.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b33e00640 1 -- 172.21.15.107:0/220261412 learned_addr learned my addr 172.21.15.107:0/220261412 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:38.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b32a00640 1 --2- 172.21.15.107:0/220261412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aeec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b33e00640 1 -- 172.21.15.107:0/220261412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 msgr2=0x56004aeec900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b33e00640 1 --2- 172.21.15.107:0/220261412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aeec900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b33e00640 1 -- 172.21.15.107:0/220261412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 msgr2=0x56004aee9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b33e00640 1 --2- 172.21.15.107:0/220261412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004aee9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.059+0000 7f3b33e00640 1 -- 172.21.15.107:0/220261412 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56004adf94a0 con 0x56004aedf330 2024-09-19T04:19:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.060+0000 7f3b33e00640 1 --2- 172.21.15.107:0/220261412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004aedf710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009f40 tx=0x7f3b2c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fab14baf3117a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.060+0000 7f3b28c00640 1 -- 172.21.15.107:0/220261412 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c016070 con 0x56004aedf330 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.060+0000 7f3b28c00640 1 -- 172.21.15.107:0/220261412 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c004d20 con 0x56004aedf330 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.060+0000 7f3b28c00640 1 -- 172.21.15.107:0/220261412 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0128b0 con 0x56004aedf330 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.060+0000 7f3b2be00640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3b180a32d0 0x7f3b180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.060+0000 7f3b2be00640 1 -- 172.21.15.107:0/463993313 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f3b200cfde0 con 0x7f3b34074eb0 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.060+0000 7f3b33400640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3b180a32d0 0x7f3b180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.061+0000 7f3b3d00b740 1 -- 172.21.15.107:0/220261412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 msgr2=0x56004aedf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.061+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/220261412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004aedf710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009f40 tx=0x7f3b2c010800 comp rx=0 tx=0).stop 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.061+0000 7f3b3d00b740 1 -- 172.21.15.107:0/220261412 shutdown_connections 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.061+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/220261412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aeec900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.061+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/220261412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004aee9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.061+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/220261412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004aedf710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.061+0000 7f3b3d00b740 1 -- 172.21.15.107:0/220261412 >> 172.21.15.107:0/220261412 conn(0x56004aedb8c0 msgr2=0x56004aedbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.061+0000 7f3b3d00b740 1 -- 172.21.15.107:0/220261412 shutdown_connections 2024-09-19T04:19:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.061+0000 7f3b3d00b740 1 -- 172.21.15.107:0/220261412 wait complete. 2024-09-19T04:19:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.061+0000 7f3b33400640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3b180a32d0 0x7f3b180a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3b3419fa00 tx=0x7f3b240058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.062+0000 7f3b3d00b740 1 -- start start 2024-09-19T04:19:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.062+0000 7f3b3d00b740 1 Processor -- start 2024-09-19T04:19:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.062+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aedf330 0x56004aee7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.062+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004aee7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.062+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aeea510 0x56004aee8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.062+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56004aeed060 con 0x56004aedfc50 2024-09-19T04:19:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.062+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56004adcd920 con 0x56004aedf330 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.062+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56004aeed1e0 con 0x56004aeea510 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.062+0000 7f3b3d00b740 10 client.0 _ll_register_callbacks cb 0x56004adf18a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004aee7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004aee7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56370/0 (socket says 172.21.15.107:56370) 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b32a00640 1 -- 172.21.15.107:0/2965556944 learned_addr learned my addr 172.21.15.107:0/2965556944 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b33e00640 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aeea510 0x56004aee8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b33400640 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aedf330 0x56004aee7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b32a00640 1 -- 172.21.15.107:0/2965556944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aeea510 msgr2=0x56004aee8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b32a00640 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aeea510 0x56004aee8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b32a00640 1 -- 172.21.15.107:0/2965556944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aedf330 msgr2=0x56004aee7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b32a00640 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aedf330 0x56004aee7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b32a00640 1 -- 172.21.15.107:0/2965556944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56004abb62b0 con 0x56004aedfc50 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b33e00640 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aeea510 0x56004aee8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b32a00640 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004aee7ac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3b20098cf0 tx=0x7f3b200986e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa717c07f749c557 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b1d600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.063+0000 7f3b1d600640 1 -- 172.21.15.107:0/2965556944 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2009c320 con 0x56004aedfc50 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2965556944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b2c004d20 con 0x56004aedfc50 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b1d600640 1 -- 172.21.15.107:0/2965556944 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f3b200bc0b0 con 0x56004aedfc50 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b1d600640 1 -- 172.21.15.107:0/2965556944 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b200a0620 con 0x56004aedfc50 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b3d00b740 10 client.14883 fetch_fsmap learned FSMap version 27 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2965556944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56004aef8e60 con 0x56004aedfc50 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b1d600640 1 -- 172.21.15.107:0/2965556944 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c004d20 con 0x56004aedfc50 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b1d600640 1 -- 172.21.15.107:0/2965556944 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3b20018bc0 con 0x56004aedfc50 2024-09-19T04:19:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b3d00b740 10 client.14883 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:19:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b3d00b740 10 client.14883 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:19:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b3d00b740 20 client.14883 populate_metadata read hostname 'smithi107' 2024-09-19T04:19:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56004aefb410 0x56004aefc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b3d00b740 4 client.14883 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.064+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2965556944 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56004ae52620 con 0x56004aefb410 2024-09-19T04:19:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.065+0000 7f3b33400640 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56004aefb410 0x56004aefc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.065+0000 7f3b1d600640 10 client.14883 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:19:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.065+0000 7f3b33400640 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56004aefb410 0x56004aefc880 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5cc3d09b845b4533 server_cookie=756406f41e7b643 in_seq=0 out_seq=0 2024-09-19T04:19:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.066+0000 7f3b1d600640 1 -- 172.21.15.107:0/2965556944 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56004ae52620 con 0x56004aefb410 2024-09-19T04:19:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.066+0000 7f3b1d600640 10 client.14883 handle_command_reply: tid=0 2024-09-19T04:19:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.066+0000 7f3b3d00b740 1 client.14883 shutdown 2024-09-19T04:19:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2965556944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 msgr2=0x56004aee7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004aee7ac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3b20098cf0 tx=0x7f3b200986e0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2965556944 shutdown_connections 2024-09-19T04:19:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56004aefb410 0x56004aefc880 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aeea510 0x56004aee8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004aee7ac0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/2965556944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aedf330 0x56004aee7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b1d600640 0 client.14883 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:19:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2965556944 >> 172.21.15.107:0/2965556944 conn(0x56004aedb8c0 msgr2=0x56004aee15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2965556944 shutdown_connections 2024-09-19T04:19:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2965556944 wait complete. 2024-09-19T04:19:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.067+0000 7f3b3d00b740 20 client.14883 trim_cache size 0 max 16384 2024-09-19T04:19:38.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.085+0000 7f3b3d00b740 1 Processor -- start 2024-09-19T04:19:38.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.085+0000 7f3b3d00b740 1 -- start start 2024-09-19T04:19:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedf330 0x56004aee5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedfc50 0x56004aee6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aee4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56004aefabc0 con 0x56004aedf330 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56004aefb290 con 0x56004aeea510 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56004aefb960 con 0x56004aedfc50 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b33400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedf330 0x56004aee5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b33400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedf330 0x56004aee5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56374/0 (socket says 172.21.15.107:56374) 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b33400640 1 -- 172.21.15.107:0/2114441826 learned_addr learned my addr 172.21.15.107:0/2114441826 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b33e00640 1 --2- 172.21.15.107:0/2114441826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aee4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b32a00640 1 --2- 172.21.15.107:0/2114441826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedfc50 0x56004aee6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b33e00640 1 -- 172.21.15.107:0/2114441826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedfc50 msgr2=0x56004aee6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b33e00640 1 --2- 172.21.15.107:0/2114441826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedfc50 0x56004aee6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b33e00640 1 -- 172.21.15.107:0/2114441826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedf330 msgr2=0x56004aee5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b33e00640 1 --2- 172.21.15.107:0/2114441826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedf330 0x56004aee5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b33e00640 1 -- 172.21.15.107:0/2114441826 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56004aee5ad0 con 0x56004aeea510 2024-09-19T04:19:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.086+0000 7f3b33400640 1 --2- 172.21.15.107:0/2114441826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedf330 0x56004aee5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.087+0000 7f3b33e00640 1 --2- 172.21.15.107:0/2114441826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aee4800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005ad0 tx=0x7f3b2c012450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a3dc0848f96adf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.087+0000 7f3b1fe00640 1 -- 172.21.15.107:0/2114441826 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c016040 con 0x56004aeea510 2024-09-19T04:19:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.087+0000 7f3b1fe00640 1 -- 172.21.15.107:0/2114441826 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c003e50 con 0x56004aeea510 2024-09-19T04:19:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.087+0000 7f3b1fe00640 1 -- 172.21.15.107:0/2114441826 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0050b0 con 0x56004aeea510 2024-09-19T04:19:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.087+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2114441826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 msgr2=0x56004aee4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.087+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/2114441826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aee4800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005ad0 tx=0x7f3b2c012450 comp rx=0 tx=0).stop 2024-09-19T04:19:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.088+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2114441826 shutdown_connections 2024-09-19T04:19:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.088+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/2114441826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aee4800 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.088+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/2114441826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedfc50 0x56004aee6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.088+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/2114441826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedf330 0x56004aee5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.088+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2114441826 >> 172.21.15.107:0/2114441826 conn(0x56004aedb8c0 msgr2=0x56004aee2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.088+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2114441826 shutdown_connections 2024-09-19T04:19:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.088+0000 7f3b3d00b740 1 -- 172.21.15.107:0/2114441826 wait complete. 2024-09-19T04:19:38.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.088+0000 7f3b3d00b740 1 -- start start 2024-09-19T04:19:38.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.088+0000 7f3b3d00b740 1 Processor -- start 2024-09-19T04:19:38.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004af04d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004af05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b3d00b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aeffb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56004aefc700 con 0x56004aedfc50 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56004aefc030 con 0x56004aeea510 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b3d00b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56004aefcdd0 con 0x56004aedf330 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b3d00b740 10 client.0 _ll_register_callbacks cb 0x7f3b20009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004af05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aeffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aeffb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54810/0 (socket says 172.21.15.107:54810) 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b33e00640 1 -- 172.21.15.107:0/3524731847 learned_addr learned my addr 172.21.15.107:0/3524731847 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b33400640 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004af04d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b33e00640 1 -- 172.21.15.107:0/3524731847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 msgr2=0x56004af04d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b33e00640 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004af04d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b33e00640 1 -- 172.21.15.107:0/3524731847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 msgr2=0x56004af05270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b33e00640 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004af05270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b33e00640 1 -- 172.21.15.107:0/3524731847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56004af001f0 con 0x56004aeea510 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.089+0000 7f3b33400640 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004af04d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.090+0000 7f3b33e00640 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aeffb00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009640 tx=0x7f3b2c003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce8494eb8ab0a54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.090+0000 7f3b1d600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.090+0000 7f3b1d600640 1 -- 172.21.15.107:0/3524731847 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c02a020 con 0x56004aeea510 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.090+0000 7f3b3d00b740 1 -- 172.21.15.107:0/3524731847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b2c003e50 con 0x56004aeea510 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.090+0000 7f3b1d600640 1 -- 172.21.15.107:0/3524731847 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f3b2c0126d0 con 0x56004aeea510 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.090+0000 7f3b1d600640 1 -- 172.21.15.107:0/3524731847 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b2c016050 con 0x56004aeea510 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.090+0000 7f3b3d00b740 10 client.24905 fetch_fsmap learned FSMap version 27 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.090+0000 7f3b3d00b740 1 -- 172.21.15.107:0/3524731847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56004af664c0 con 0x56004aeea510 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.091+0000 7f3b1d600640 1 -- 172.21.15.107:0/3524731847 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c003e50 con 0x56004aeea510 2024-09-19T04:19:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.091+0000 7f3b1d600640 1 -- 172.21.15.107:0/3524731847 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3b2c00f040 con 0x56004aeea510 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.091+0000 7f3b3d00b740 10 client.24905 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.091+0000 7f3b3d00b740 10 client.24905 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.091+0000 7f3b3d00b740 20 client.24905 populate_metadata read hostname 'smithi107' 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.091+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56004af6a8e0 0x56004af695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.091+0000 7f3b3d00b740 4 client.24905 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.091+0000 7f3b3d00b740 1 -- 172.21.15.107:0/3524731847 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "status"}) -- 0x56004ae4cb30 con 0x56004af6a8e0 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b33400640 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56004af6a8e0 0x56004af695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b33400640 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56004af6a8e0 0x56004af695f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e8431f4c7c6ef1fe server_cookie=10a7400ee6208412 in_seq=0 out_seq=0 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b1d600640 10 client.24905 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b1d600640 1 -- 172.21.15.107:0/3524731847 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56004ae4cb30 con 0x56004af6a8e0 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b1d600640 10 client.24905 handle_command_reply: tid=0 2024-09-19T04:19:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b3d00b740 1 client.24905 shutdown 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b3d00b740 1 -- 172.21.15.107:0/3524731847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 msgr2=0x56004aeffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aeffb00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009640 tx=0x7f3b2c003c20 comp rx=0 tx=0).stop 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b3d00b740 1 -- 172.21.15.107:0/3524731847 shutdown_connections 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56004af6a8e0 0x56004af695f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56004aeea510 0x56004aeffb00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56004aedfc50 0x56004af05270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b1d600640 0 client.24905 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b3d00b740 1 --2- 172.21.15.107:0/3524731847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56004aedf330 0x56004af04d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.092+0000 7f3b3d00b740 1 -- 172.21.15.107:0/3524731847 >> 172.21.15.107:0/3524731847 conn(0x56004aedb8c0 msgr2=0x56004aee5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.093+0000 7f3b3d00b740 1 -- 172.21.15.107:0/3524731847 shutdown_connections 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.093+0000 7f3b3d00b740 1 -- 172.21.15.107:0/3524731847 wait complete. 2024-09-19T04:19:38.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.093+0000 7f3b3d00b740 20 client.24905 trim_cache size 0 max 16384 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 3, 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24496, 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 74.536292411000005, 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 83, 2024-09-19T04:19:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T04:19:38.101 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 111.644136608 2024-09-19T04:19:38.101 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:19:38.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.093+0000 7f3b3a400640 1 -- 172.21.15.107:0/463993313 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3b180a32d0 msgr2=0x7f3b180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.093+0000 7f3b3a400640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3b180a32d0 0x7f3b180a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3b3419fa00 tx=0x7f3b240058d0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.093+0000 7f3b3a400640 1 -- 172.21.15.107:0/463993313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b34074eb0 msgr2=0x7f3b341a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.093+0000 7f3b3a400640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b34074eb0 0x7f3b341a4c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000e9f0 tx=0x7f3b2000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.094+0000 7f3b3a400640 1 -- 172.21.15.107:0/463993313 shutdown_connections 2024-09-19T04:19:38.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.094+0000 7f3b3a400640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f3b180a32d0 0x7f3b180a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.094+0000 7f3b3a400640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b34112b90 0x7f3b3419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.094+0000 7f3b3a400640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b34074eb0 0x7f3b341a4c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.094+0000 7f3b3a400640 1 --2- 172.21.15.107:0/463993313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34074590 0x7f3b341a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.094+0000 7f3b3a400640 1 -- 172.21.15.107:0/463993313 >> 172.21.15.107:0/463993313 conn(0x7f3b340fd8c0 msgr2=0x7f3b340ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.094+0000 7f3b3a400640 1 -- 172.21.15.107:0/463993313 shutdown_connections 2024-09-19T04:19:38.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:38.094+0000 7f3b3a400640 1 -- 172.21.15.107:0/463993313 wait complete. 2024-09-19T04:19:38.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:38 smithi107 ceph-mon[30659]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:38.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:38 smithi115 ceph-mon[35477]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:38.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:38 smithi145 ceph-mon[35096]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:38.877 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T04:19:39.244 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:39.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.768+0000 7f1d39600640 1 -- 172.21.15.107:0/3824091317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d34101de0 msgr2=0x7f1d341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:39.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.768+0000 7f1d39600640 1 --2- 172.21.15.107:0/3824091317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d34101de0 0x7f1d341021c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009900 tx=0x7f1d2402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.768+0000 7f1d39600640 1 -- 172.21.15.107:0/3824091317 shutdown_connections 2024-09-19T04:19:39.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.768+0000 7f1d39600640 1 --2- 172.21.15.107:0/3824091317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d34106f10 0x7f1d3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.768+0000 7f1d39600640 1 --2- 172.21.15.107:0/3824091317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d34102790 0x7f1d341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.768+0000 7f1d39600640 1 --2- 172.21.15.107:0/3824091317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d34101de0 0x7f1d341021c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.768+0000 7f1d39600640 1 -- 172.21.15.107:0/3824091317 >> 172.21.15.107:0/3824091317 conn(0x7f1d340fd8c0 msgr2=0x7f1d340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:39.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.768+0000 7f1d39600640 1 -- 172.21.15.107:0/3824091317 shutdown_connections 2024-09-19T04:19:39.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.769+0000 7f1d39600640 1 -- 172.21.15.107:0/3824091317 wait complete. 2024-09-19T04:19:39.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.769+0000 7f1d39600640 1 Processor -- start 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d39600640 1 -- start start 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d39600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d34101de0 0x7f1d340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d39600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d34102790 0x7f1d34074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d39600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d34106f10 0x7f1d34074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d39600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34111f30 con 0x7f1d34106f10 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d39600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34111db0 con 0x7f1d34101de0 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d39600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1d341120b0 con 0x7f1d34102790 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d33400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d34106f10 0x7f1d34074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d33400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d34106f10 0x7f1d34074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46300/0 (socket says 172.21.15.107:46300) 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d33400640 1 -- 172.21.15.107:0/3045556976 learned_addr learned my addr 172.21.15.107:0/3045556976 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d32000640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d34102790 0x7f1d34074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.770+0000 7f1d32a00640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d34101de0 0x7f1d340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.771+0000 7f1d33400640 1 -- 172.21.15.107:0/3045556976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d34102790 msgr2=0x7f1d34074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:39.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.771+0000 7f1d33400640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d34102790 0x7f1d34074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.771+0000 7f1d33400640 1 -- 172.21.15.107:0/3045556976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d34101de0 msgr2=0x7f1d340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:39.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.771+0000 7f1d33400640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d34101de0 0x7f1d340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.771+0000 7f1d33400640 1 -- 172.21.15.107:0/3045556976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d341a6950 con 0x7f1d34106f10 2024-09-19T04:19:39.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.771+0000 7f1d32a00640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d34101de0 0x7f1d340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:39.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.771+0000 7f1d33400640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d34106f10 0x7f1d34074f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009fd0 tx=0x7f1d2c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.771+0000 7f1d2b400640 1 -- 172.21.15.107:0/3045556976 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c017070 con 0x7f1d34106f10 2024-09-19T04:19:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.771+0000 7f1d2b400640 1 -- 172.21.15.107:0/3045556976 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2c003e50 con 0x7f1d34106f10 2024-09-19T04:19:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.772+0000 7f1d39600640 1 -- 172.21.15.107:0/3045556976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d341a6c40 con 0x7f1d34106f10 2024-09-19T04:19:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.772+0000 7f1d2b400640 1 -- 172.21.15.107:0/3045556976 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0106c0 con 0x7f1d34106f10 2024-09-19T04:19:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.772+0000 7f1d39600640 1 -- 172.21.15.107:0/3045556976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d341a7180 con 0x7f1d34106f10 2024-09-19T04:19:39.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.775+0000 7f1d2b400640 1 -- 172.21.15.107:0/3045556976 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1d2c003970 con 0x7f1d34106f10 2024-09-19T04:19:39.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.775+0000 7f1d3c058740 1 Processor -- start 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.775+0000 7f1d3c058740 1 -- start start 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.775+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdf330 0x562911bdf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.775+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 0x562911be9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.775+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bea510 0x562911bec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d3c058740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562911b450b0 con 0x562911bea510 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d3c058740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562911aecab0 con 0x562911bdfc50 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d3c058740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562911b41070 con 0x562911bdf330 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 0x562911be9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 0x562911be9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35026/0 (socket says 172.21.15.107:35026) 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32a00640 1 -- 172.21.15.107:0/1983886332 learned_addr learned my addr 172.21.15.107:0/1983886332 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:39.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32000640 1 --2- 172.21.15.107:0/1983886332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdf330 0x562911bdf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32000640 1 --2- 172.21.15.107:0/1983886332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bea510 0x562911bec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32a00640 1 -- 172.21.15.107:0/1983886332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdf330 msgr2=0x562911bdf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:39.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32a00640 1 --2- 172.21.15.107:0/1983886332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdf330 0x562911bdf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32a00640 1 -- 172.21.15.107:0/1983886332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bea510 msgr2=0x562911bec900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:39.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32a00640 1 --2- 172.21.15.107:0/1983886332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bea510 0x562911bec900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32a00640 1 -- 172.21.15.107:0/1983886332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562911af94a0 con 0x562911bdfc50 2024-09-19T04:19:39.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d32000640 1 --2- 172.21.15.107:0/1983886332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bea510 0x562911bec900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:39.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.776+0000 7f1d2b400640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d180a3350 0x7f1d180a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.777+0000 7f1d32a00640 1 --2- 172.21.15.107:0/1983886332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 0x562911be9fd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009900 tx=0x7f1d24005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6402cb3a56d0e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:39.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.777+0000 7f1d2b400640 1 -- 172.21.15.107:0/3045556976 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f1d2c0cf6c0 con 0x7f1d34106f10 2024-09-19T04:19:39.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.777+0000 7f1d32000640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d180a3350 0x7f1d180a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.777+0000 7f1d1f400640 1 -- 172.21.15.107:0/1983886332 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24035040 con 0x562911bdfc50 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.777+0000 7f1d1f400640 1 -- 172.21.15.107:0/1983886332 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d240319f0 con 0x562911bdfc50 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.777+0000 7f1d1f400640 1 -- 172.21.15.107:0/1983886332 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24004a00 con 0x562911bdfc50 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d3c058740 1 -- 172.21.15.107:0/1983886332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 msgr2=0x562911be9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d3c058740 1 --2- 172.21.15.107:0/1983886332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 0x562911be9fd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009900 tx=0x7f1d24005e80 comp rx=0 tx=0).stop 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d32000640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d180a3350 0x7f1d180a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x562911bdcc40 tx=0x7f1d20004730 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d3c058740 1 -- 172.21.15.107:0/1983886332 shutdown_connections 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d3c058740 1 --2- 172.21.15.107:0/1983886332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bea510 0x562911bec900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d3c058740 1 --2- 172.21.15.107:0/1983886332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 0x562911be9fd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d3c058740 1 --2- 172.21.15.107:0/1983886332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdf330 0x562911bdf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d3c058740 1 -- 172.21.15.107:0/1983886332 >> 172.21.15.107:0/1983886332 conn(0x562911bdb8c0 msgr2=0x562911bdbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d3c058740 1 -- 172.21.15.107:0/1983886332 shutdown_connections 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d3c058740 1 -- 172.21.15.107:0/1983886332 wait complete. 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.778+0000 7f1d3c058740 1 -- start start 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d3c058740 1 Processor -- start 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 0x562911be79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bea510 0x562911be7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d3c058740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562911bed060 con 0x562911bdf330 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d3c058740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562911acd920 con 0x562911bdfc50 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d3c058740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562911bed1e0 con 0x562911bea510 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d3c058740 10 client.0 _ll_register_callbacks cb 0x562911af18a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46310/0 (socket says 172.21.15.107:46310) 2024-09-19T04:19:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.779+0000 7f1d32a00640 1 -- 172.21.15.107:0/1621570982 learned_addr learned my addr 172.21.15.107:0/1621570982 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d32000640 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 0x562911be79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d33400640 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bea510 0x562911be7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d32a00640 1 -- 172.21.15.107:0/1621570982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bea510 msgr2=0x562911be7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d32a00640 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bea510 0x562911be7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d32a00640 1 -- 172.21.15.107:0/1621570982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 msgr2=0x562911be79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d32a00640 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 0x562911be79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d32a00640 1 -- 172.21.15.107:0/1621570982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629118b62b0 con 0x562911bdf330 2024-09-19T04:19:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d32a00640 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be7460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1d2402a910 tx=0x7f1d24004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ca941256575e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d13e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:19:39.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d13e00640 1 -- 172.21.15.107:0/1621570982 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2404e070 con 0x562911bdf330 2024-09-19T04:19:39.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d13e00640 1 -- 172.21.15.107:0/1621570982 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f1d240035f0 con 0x562911bdf330 2024-09-19T04:19:39.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.780+0000 7f1d3c058740 1 -- 172.21.15.107:0/1621570982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d240319f0 con 0x562911bdf330 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d13e00640 1 -- 172.21.15.107:0/1621570982 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d2405a050 con 0x562911bdf330 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d3c058740 10 client.14898 fetch_fsmap learned FSMap version 27 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d3c058740 1 -- 172.21.15.107:0/1621570982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562911bf8ca0 con 0x562911bdf330 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d13e00640 1 -- 172.21.15.107:0/1621570982 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d240319f0 con 0x562911bdf330 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d13e00640 1 -- 172.21.15.107:0/1621570982 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1d24046cb0 con 0x562911bdf330 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d3c058740 10 client.14898 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d3c058740 10 client.14898 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d3c058740 20 client.14898 populate_metadata read hostname 'smithi107' 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d3c058740 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x562911bfae60 0x562911bfc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d3c058740 4 client.14898 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:39.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.781+0000 7f1d3c058740 1 -- 172.21.15.107:0/1621570982 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562911b52620 con 0x562911bfae60 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.782+0000 7f1d32000640 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x562911bfae60 0x562911bfc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.782+0000 7f1d32000640 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x562911bfae60 0x562911bfc2d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2516eb1a72ef2e81 server_cookie=874752a399e9e411 in_seq=0 out_seq=0 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.782+0000 7f1d13e00640 10 client.14898 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.783+0000 7f1d13e00640 1 -- 172.21.15.107:0/1621570982 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562911b52620 con 0x562911bfae60 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.783+0000 7f1d13e00640 10 client.14898 handle_command_reply: tid=0 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.783+0000 7f1d3c058740 1 client.14898 shutdown 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 1 -- 172.21.15.107:0/1621570982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 msgr2=0x562911be7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be7460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1d2402a910 tx=0x7f1d24004270 comp rx=0 tx=0).stop 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 1 -- 172.21.15.107:0/1621570982 shutdown_connections 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x562911bfae60 0x562911bfc2d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bea510 0x562911be7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bdfc50 0x562911be79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 1 --2- 172.21.15.107:0/1621570982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be7460 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 1 -- 172.21.15.107:0/1621570982 >> 172.21.15.107:0/1621570982 conn(0x562911bdb8c0 msgr2=0x562911be15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d13e00640 0 client.14898 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:19:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 1 -- 172.21.15.107:0/1621570982 shutdown_connections 2024-09-19T04:19:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 1 -- 172.21.15.107:0/1621570982 wait complete. 2024-09-19T04:19:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.784+0000 7f1d3c058740 20 client.14898 trim_cache size 0 max 16384 2024-09-19T04:19:39.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.802+0000 7f1d3c058740 1 Processor -- start 2024-09-19T04:19:39.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d3c058740 1 -- start start 2024-09-19T04:19:39.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdfc50 0x562911bdbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bea510 0x562911be7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d3c058740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562911bfa610 con 0x562911bdf330 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d3c058740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562911bface0 con 0x562911bea510 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d3c058740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562911bfb3b0 con 0x562911bdfc50 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46312/0 (socket says 172.21.15.107:46312) 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d32a00640 1 -- 172.21.15.107:0/3731793834 learned_addr learned my addr 172.21.15.107:0/3731793834 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d32000640 1 --2- 172.21.15.107:0/3731793834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdfc50 0x562911bdbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d33400640 1 --2- 172.21.15.107:0/3731793834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bea510 0x562911be7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d32a00640 1 -- 172.21.15.107:0/3731793834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdfc50 msgr2=0x562911bdbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d32a00640 1 --2- 172.21.15.107:0/3731793834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdfc50 0x562911bdbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d32a00640 1 -- 172.21.15.107:0/3731793834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bea510 msgr2=0x562911be7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d32a00640 1 --2- 172.21.15.107:0/3731793834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bea510 0x562911be7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.803+0000 7f1d32a00640 1 -- 172.21.15.107:0/3731793834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562911bdf160 con 0x562911bdf330 2024-09-19T04:19:39.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.804+0000 7f1d32a00640 1 --2- 172.21.15.107:0/3731793834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be1d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d24005c40 tx=0x7f1d24005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0382fe930c4d64c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:39.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.804+0000 7f1d1ea00640 1 -- 172.21.15.107:0/3731793834 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2404e040 con 0x562911bdf330 2024-09-19T04:19:39.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.804+0000 7f1d1ea00640 1 -- 172.21.15.107:0/3731793834 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d240317f0 con 0x562911bdf330 2024-09-19T04:19:39.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.804+0000 7f1d1ea00640 1 -- 172.21.15.107:0/3731793834 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24035040 con 0x562911bdf330 2024-09-19T04:19:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.805+0000 7f1d3c058740 1 -- 172.21.15.107:0/3731793834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 msgr2=0x562911be1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.805+0000 7f1d3c058740 1 --2- 172.21.15.107:0/3731793834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be1d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d24005c40 tx=0x7f1d24005e80 comp rx=0 tx=0).stop 2024-09-19T04:19:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.805+0000 7f1d3c058740 1 -- 172.21.15.107:0/3731793834 shutdown_connections 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.805+0000 7f1d3c058740 1 --2- 172.21.15.107:0/3731793834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bea510 0x562911be7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.805+0000 7f1d3c058740 1 --2- 172.21.15.107:0/3731793834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdfc50 0x562911bdbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.805+0000 7f1d3c058740 1 --2- 172.21.15.107:0/3731793834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdf330 0x562911be1d90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.805+0000 7f1d3c058740 1 -- 172.21.15.107:0/3731793834 >> 172.21.15.107:0/3731793834 conn(0x562911bdb8c0 msgr2=0x562911be6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.805+0000 7f1d3c058740 1 -- 172.21.15.107:0/3731793834 shutdown_connections 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.805+0000 7f1d3c058740 1 -- 172.21.15.107:0/3731793834 wait complete. 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.805+0000 7f1d3c058740 1 -- start start 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d3c058740 1 Processor -- start 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdf330 0x562911c04eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdfc50 0x562911bfe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d3c058740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bea510 0x562911bfe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d3c058740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562911bfc150 con 0x562911bdfc50 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d3c058740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562911bfba80 con 0x562911bea510 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d3c058740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562911bfc820 con 0x562911bdf330 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d3c058740 10 client.0 _ll_register_callbacks cb 0x7f1d24005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d32000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdfc50 0x562911bfe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d32000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdfc50 0x562911bfe1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46320/0 (socket says 172.21.15.107:46320) 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d32000640 1 -- 172.21.15.107:0/2304692102 learned_addr learned my addr 172.21.15.107:0/2304692102 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d33400640 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bea510 0x562911bfe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.806+0000 7f1d32a00640 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdf330 0x562911c04eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.807+0000 7f1d32000640 1 -- 172.21.15.107:0/2304692102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdf330 msgr2=0x562911c04eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.807+0000 7f1d32000640 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdf330 0x562911c04eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.807+0000 7f1d32000640 1 -- 172.21.15.107:0/2304692102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bea510 msgr2=0x562911bfe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.807+0000 7f1d32000640 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bea510 0x562911bfe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.807+0000 7f1d32000640 1 -- 172.21.15.107:0/2304692102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562911c029d0 con 0x562911bdfc50 2024-09-19T04:19:39.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.807+0000 7f1d32000640 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdfc50 0x562911bfe1e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d20005d00 tx=0x7f1d2000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae1185d548b3d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:39.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.807+0000 7f1d13e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:19:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.807+0000 7f1d13e00640 1 -- 172.21.15.107:0/2304692102 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2002a420 con 0x562911bdfc50 2024-09-19T04:19:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.807+0000 7f1d13e00640 1 -- 172.21.15.107:0/2304692102 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f1d2002a5c0 con 0x562911bdfc50 2024-09-19T04:19:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.807+0000 7f1d3c058740 1 -- 172.21.15.107:0/2304692102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d240317f0 con 0x562911bdfc50 2024-09-19T04:19:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d13e00640 1 -- 172.21.15.107:0/2304692102 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d20039070 con 0x562911bdfc50 2024-09-19T04:19:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d3c058740 10 client.14904 fetch_fsmap learned FSMap version 27 2024-09-19T04:19:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d3c058740 1 -- 172.21.15.107:0/2304692102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562911c657e0 con 0x562911bdfc50 2024-09-19T04:19:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d13e00640 1 -- 172.21.15.107:0/2304692102 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d240317f0 con 0x562911bdfc50 2024-09-19T04:19:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d13e00640 1 -- 172.21.15.107:0/2304692102 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1d2001db00 con 0x562911bdfc50 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d3c058740 10 client.14904 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d3c058740 10 client.14904 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d3c058740 20 client.14904 populate_metadata read hostname 'smithi107' 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d3c058740 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x562911c6a430 0x562911c69140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d3c058740 4 client.14904 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.808+0000 7f1d3c058740 1 -- 172.21.15.107:0/2304692102 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "status"}) -- 0x562911b4cb30 con 0x562911c6a430 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.809+0000 7f1d32a00640 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x562911c6a430 0x562911c69140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.809+0000 7f1d32a00640 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x562911c6a430 0x562911c69140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6d8fd73e46e6847 server_cookie=b295e02cd10c9862 in_seq=0 out_seq=0 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.809+0000 7f1d13e00640 10 client.14904 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.809+0000 7f1d13e00640 1 -- 172.21.15.107:0/2304692102 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562911b4cb30 con 0x562911c6a430 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.809+0000 7f1d13e00640 10 client.14904 handle_command_reply: tid=0 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.809+0000 7f1d3c058740 1 client.14904 shutdown 2024-09-19T04:19:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.809+0000 7f1d3c058740 1 -- 172.21.15.107:0/2304692102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdfc50 msgr2=0x562911bfe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.809+0000 7f1d3c058740 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdfc50 0x562911bfe1e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d20005d00 tx=0x7f1d2000b410 comp rx=0 tx=0).stop 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.809+0000 7f1d3c058740 1 -- 172.21.15.107:0/2304692102 shutdown_connections 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.809+0000 7f1d3c058740 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x562911c6a430 0x562911c69140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.810+0000 7f1d3c058740 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562911bea510 0x562911bfe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.810+0000 7f1d3c058740 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562911bdfc50 0x562911bfe1e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.810+0000 7f1d3c058740 1 --2- 172.21.15.107:0/2304692102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562911bdf330 0x562911c04eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.810+0000 7f1d13e00640 0 client.14904 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.810+0000 7f1d3c058740 1 -- 172.21.15.107:0/2304692102 >> 172.21.15.107:0/2304692102 conn(0x562911bdb8c0 msgr2=0x562911bda200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.810+0000 7f1d3c058740 1 -- 172.21.15.107:0/2304692102 shutdown_connections 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.810+0000 7f1d3c058740 1 -- 172.21.15.107:0/2304692102 wait complete. 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.810+0000 7f1d3c058740 20 client.14904 trim_cache size 0 max 16384 2024-09-19T04:19:39.815 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24500, 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 77.263818325000003, 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 83, 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 110.618119792 2024-09-19T04:19:39.816 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:19:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 -- 172.21.15.107:0/3045556976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d180a3350 msgr2=0x7f1d180a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d180a3350 0x7f1d180a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x562911bdcc40 tx=0x7f1d20004730 comp rx=0 tx=0).stop 2024-09-19T04:19:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 -- 172.21.15.107:0/3045556976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d34106f10 msgr2=0x7f1d34074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d34106f10 0x7f1d34074f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009fd0 tx=0x7f1d2c00b260 comp rx=0 tx=0).stop 2024-09-19T04:19:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 -- 172.21.15.107:0/3045556976 shutdown_connections 2024-09-19T04:19:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1d180a3350 0x7f1d180a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d34106f10 0x7f1d34074f70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d34102790 0x7f1d34074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 --2- 172.21.15.107:0/3045556976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d34101de0 0x7f1d340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:39.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 -- 172.21.15.107:0/3045556976 >> 172.21.15.107:0/3045556976 conn(0x7f1d340fd8c0 msgr2=0x7f1d340fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:39.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 -- 172.21.15.107:0/3045556976 shutdown_connections 2024-09-19T04:19:39.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:39.811+0000 7f1d39600640 1 -- 172.21.15.107:0/3045556976 wait complete. 2024-09-19T04:19:40.610 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T04:19:40.981 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:40 smithi115 ceph-mon[35477]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:40 smithi145 ceph-mon[35096]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:41.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:40 smithi107 ceph-mon[30659]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.553+0000 7fc51b200640 1 -- 172.21.15.107:0/1431814384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc51c1047e0 msgr2=0x7fc51c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.553+0000 7fc51b200640 1 --2- 172.21.15.107:0/1431814384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc51c1047e0 0x7fc51c108830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc504009900 tx=0x7fc50402a410 comp rx=0 tx=0).stop 2024-09-19T04:19:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.553+0000 7fc51b200640 1 -- 172.21.15.107:0/1431814384 shutdown_connections 2024-09-19T04:19:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.553+0000 7fc51b200640 1 --2- 172.21.15.107:0/1431814384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c108f60 0x7fc51c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.553+0000 7fc51b200640 1 --2- 172.21.15.107:0/1431814384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc51c1047e0 0x7fc51c108830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.553+0000 7fc51b200640 1 --2- 172.21.15.107:0/1431814384 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc51c103e30 0x7fc51c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.554+0000 7fc51b200640 1 -- 172.21.15.107:0/1431814384 >> 172.21.15.107:0/1431814384 conn(0x7fc51c0ff8d0 msgr2=0x7fc51c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.554+0000 7fc51b200640 1 -- 172.21.15.107:0/1431814384 shutdown_connections 2024-09-19T04:19:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.554+0000 7fc51b200640 1 -- 172.21.15.107:0/1431814384 wait complete. 2024-09-19T04:19:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.555+0000 7fc51b200640 1 Processor -- start 2024-09-19T04:19:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.555+0000 7fc51b200640 1 -- start start 2024-09-19T04:19:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.555+0000 7fc51b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc51c103e30 0x7fc51c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.555+0000 7fc51b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c1047e0 0x7fc51c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.555+0000 7fc51b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc51c108f60 0x7fc51c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.555+0000 7fc51b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c113f80 con 0x7fc51c108f60 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.555+0000 7fc51b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c113e00 con 0x7fc51c1047e0 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.555+0000 7fc51b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c114100 con 0x7fc51c103e30 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc511400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c1047e0 0x7fc51c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc511400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c1047e0 0x7fc51c19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35072/0 (socket says 172.21.15.107:35072) 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc511400640 1 -- 172.21.15.107:0/1313665560 learned_addr learned my addr 172.21.15.107:0/1313665560 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc519e00640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc51c103e30 0x7fc51c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc51a800640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc51c108f60 0x7fc51c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc511400640 1 -- 172.21.15.107:0/1313665560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc51c103e30 msgr2=0x7fc51c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc511400640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc51c103e30 0x7fc51c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc511400640 1 -- 172.21.15.107:0/1313665560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc51c108f60 msgr2=0x7fc51c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc511400640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc51c108f60 0x7fc51c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc511400640 1 -- 172.21.15.107:0/1313665560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c19f400 con 0x7fc51c1047e0 2024-09-19T04:19:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.556+0000 7fc511400640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c1047e0 0x7fc51c19a990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc504006010 tx=0x7fc5040042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:41.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.557+0000 7fc513400640 1 -- 172.21.15.107:0/1313665560 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50403f030 con 0x7fc51c1047e0 2024-09-19T04:19:41.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.557+0000 7fc51b200640 1 -- 172.21.15.107:0/1313665560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51c19f690 con 0x7fc51c1047e0 2024-09-19T04:19:41.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.557+0000 7fc513400640 1 -- 172.21.15.107:0/1313665560 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc504003ce0 con 0x7fc51c1047e0 2024-09-19T04:19:41.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.557+0000 7fc513400640 1 -- 172.21.15.107:0/1313665560 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504035070 con 0x7fc51c1047e0 2024-09-19T04:19:41.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.557+0000 7fc51b200640 1 -- 172.21.15.107:0/1313665560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51c1a6fd0 con 0x7fc51c1047e0 2024-09-19T04:19:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.560+0000 7fc523683740 1 Processor -- start 2024-09-19T04:19:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.560+0000 7fc513400640 1 -- 172.21.15.107:0/1313665560 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc504042030 con 0x7fc51c1047e0 2024-09-19T04:19:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.560+0000 7fc523683740 1 -- start start 2024-09-19T04:19:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.560+0000 7fc523683740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcec4e0 0x56366bcec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.560+0000 7fc523683740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcece00 0x56366bcf7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.560+0000 7fc523683740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcf76c0 0x56366bcf9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.560+0000 7fc523683740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56366bc520b0 con 0x56366bcf76c0 2024-09-19T04:19:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.560+0000 7fc523683740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56366bbf9ab0 con 0x56366bcec4e0 2024-09-19T04:19:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.560+0000 7fc523683740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56366bc4e070 con 0x56366bcece00 2024-09-19T04:19:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc51a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcec4e0 0x56366bcec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc51a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcec4e0 0x56366bcec8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35078/0 (socket says 172.21.15.107:35078) 2024-09-19T04:19:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc51a800640 1 -- 172.21.15.107:0/148919076 learned_addr learned my addr 172.21.15.107:0/148919076 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc519e00640 1 --2- 172.21.15.107:0/148919076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcece00 0x56366bcf7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc511400640 1 --2- 172.21.15.107:0/148919076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcf76c0 0x56366bcf9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc51a800640 1 -- 172.21.15.107:0/148919076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcece00 msgr2=0x56366bcf7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:41.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc51a800640 1 --2- 172.21.15.107:0/148919076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcece00 0x56366bcf7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc51a800640 1 -- 172.21.15.107:0/148919076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcf76c0 msgr2=0x56366bcf9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:41.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc51a800640 1 --2- 172.21.15.107:0/148919076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcf76c0 0x56366bcf9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc51a800640 1 -- 172.21.15.107:0/148919076 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366bc064a0 con 0x56366bcec4e0 2024-09-19T04:19:41.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc511400640 1 --2- 172.21.15.107:0/148919076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcf76c0 0x56366bcf9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:41.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc51a800640 1 --2- 172.21.15.107:0/148919076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcec4e0 0x56366bcec8c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc514009f50 tx=0x7fc5140118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6900c3b4be5fa3a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:41.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.561+0000 7fc513400640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5000a32d0 0x7fc5000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.562+0000 7fc50b000640 1 -- 172.21.15.107:0/148919076 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514004d20 con 0x56366bcec4e0 2024-09-19T04:19:41.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.562+0000 7fc50b000640 1 -- 172.21.15.107:0/148919076 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc514004050 con 0x56366bcec4e0 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.562+0000 7fc50b000640 1 -- 172.21.15.107:0/148919076 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514005220 con 0x56366bcec4e0 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.562+0000 7fc513400640 1 -- 172.21.15.107:0/1313665560 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fc5040effe0 con 0x7fc51c1047e0 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.562+0000 7fc519e00640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5000a32d0 0x7fc5000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.562+0000 7fc523683740 1 -- 172.21.15.107:0/148919076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcec4e0 msgr2=0x56366bcec8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.562+0000 7fc523683740 1 --2- 172.21.15.107:0/148919076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcec4e0 0x56366bcec8c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc514009f50 tx=0x7fc5140118d0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.562+0000 7fc519e00640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5000a32d0 0x7fc5000a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56366bce98d0 tx=0x7fc50c00b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.563+0000 7fc523683740 1 -- 172.21.15.107:0/148919076 shutdown_connections 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.563+0000 7fc523683740 1 --2- 172.21.15.107:0/148919076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcf76c0 0x56366bcf9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.563+0000 7fc523683740 1 --2- 172.21.15.107:0/148919076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcece00 0x56366bcf7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.563+0000 7fc523683740 1 --2- 172.21.15.107:0/148919076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcec4e0 0x56366bcec8c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.563+0000 7fc523683740 1 -- 172.21.15.107:0/148919076 >> 172.21.15.107:0/148919076 conn(0x56366bce8a70 msgr2=0x56366bce8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.563+0000 7fc523683740 1 -- 172.21.15.107:0/148919076 shutdown_connections 2024-09-19T04:19:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.563+0000 7fc523683740 1 -- 172.21.15.107:0/148919076 wait complete. 2024-09-19T04:19:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.563+0000 7fc523683740 1 -- start start 2024-09-19T04:19:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.563+0000 7fc523683740 1 Processor -- start 2024-09-19T04:19:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc523683740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 0x56366bcf4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc523683740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 0x56366bcf49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc523683740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 0x56366bcf4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc523683740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56366bcfa330 con 0x56366bcece00 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc523683740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56366bbda920 con 0x56366bcf76c0 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc523683740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56366bcfa4b0 con 0x56366bcec4e0 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc523683740 10 client.0 _ll_register_callbacks cb 0x56366bbfe8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc511400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 0x56366bcf49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc51a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 0x56366bcf4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc51a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 0x56366bcf4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35090/0 (socket says 172.21.15.107:35090) 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc511400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 0x56366bcf49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46346/0 (socket says 172.21.15.107:46346) 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc51a800640 1 -- 172.21.15.107:0/4130873219 learned_addr learned my addr 172.21.15.107:0/4130873219 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc519e00640 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 0x56366bcf4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc51a800640 1 -- 172.21.15.107:0/4130873219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 msgr2=0x56366bcf4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.564+0000 7fc51a800640 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 0x56366bcf4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc51a800640 1 -- 172.21.15.107:0/4130873219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 msgr2=0x56366bcf49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc51a800640 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 0x56366bcf49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc51a800640 1 -- 172.21.15.107:0/4130873219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366b9c32b0 con 0x56366bcf76c0 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc511400640 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 0x56366bcf49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc51a800640 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 0x56366bcf4f10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc514009250 tx=0x7fc5140044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e08a8743cb465c78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc4ff400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc4ff400640 1 -- 172.21.15.107:0/4130873219 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514003e40 con 0x56366bcf76c0 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc523683740 1 -- 172.21.15.107:0/4130873219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc514004050 con 0x56366bcf76c0 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc4ff400640 1 -- 172.21.15.107:0/4130873219 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fc51400f480 con 0x56366bcf76c0 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc4ff400640 1 -- 172.21.15.107:0/4130873219 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc51400f680 con 0x56366bcf76c0 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc523683740 10 client.24935 fetch_fsmap learned FSMap version 27 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.565+0000 7fc523683740 1 -- 172.21.15.107:0/4130873219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56366bd05f50 con 0x56366bcf76c0 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.566+0000 7fc4ff400640 1 -- 172.21.15.107:0/4130873219 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc514004050 con 0x56366bcf76c0 2024-09-19T04:19:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.566+0000 7fc4ff400640 1 -- 172.21.15.107:0/4130873219 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc514019990 con 0x56366bcf76c0 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.566+0000 7fc523683740 10 client.24935 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.566+0000 7fc523683740 10 client.24935 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.566+0000 7fc523683740 20 client.24935 populate_metadata read hostname 'smithi107' 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.566+0000 7fc523683740 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56366bd08290 0x56366bd09700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.566+0000 7fc523683740 4 client.24935 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.566+0000 7fc523683740 1 -- 172.21.15.107:0/4130873219 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56366bc5f620 con 0x56366bd08290 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.567+0000 7fc519e00640 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56366bd08290 0x56366bd09700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.567+0000 7fc519e00640 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56366bd08290 0x56366bd09700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4fe4d92bb349619f server_cookie=8f9b19c1e10616ed in_seq=0 out_seq=0 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.567+0000 7fc4ff400640 10 client.24935 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc4ff400640 1 -- 172.21.15.107:0/4130873219 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56366bc5f620 con 0x56366bd08290 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc4ff400640 10 client.24935 handle_command_reply: tid=0 2024-09-19T04:19:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 client.24935 shutdown 2024-09-19T04:19:41.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 -- 172.21.15.107:0/4130873219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 msgr2=0x56366bcf4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 0x56366bcf4f10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc514009250 tx=0x7fc5140044b0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 -- 172.21.15.107:0/4130873219 shutdown_connections 2024-09-19T04:19:41.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56366bd08290 0x56366bd09700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 0x56366bcf4f10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 0x56366bcf49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc4ff400640 0 client.24935 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:19:41.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 --2- 172.21.15.107:0/4130873219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 0x56366bcf4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 -- 172.21.15.107:0/4130873219 >> 172.21.15.107:0/4130873219 conn(0x56366bce8a70 msgr2=0x56366bcee750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:41.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 -- 172.21.15.107:0/4130873219 shutdown_connections 2024-09-19T04:19:41.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.568+0000 7fc523683740 1 -- 172.21.15.107:0/4130873219 wait complete. 2024-09-19T04:19:41.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.569+0000 7fc523683740 20 client.24935 trim_cache size 0 max 16384 2024-09-19T04:19:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.587+0000 7fc523683740 1 Processor -- start 2024-09-19T04:19:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.587+0000 7fc523683740 1 -- start start 2024-09-19T04:19:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.587+0000 7fc523683740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcec4e0 0x56366bd09030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.587+0000 7fc523683740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcece00 0x56366bcf4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.587+0000 7fc523683740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcf76c0 0x56366bcf16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.587+0000 7fc523683740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56366bd07a40 con 0x56366bcec4e0 2024-09-19T04:19:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.587+0000 7fc523683740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56366bd08110 con 0x56366bcece00 2024-09-19T04:19:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.587+0000 7fc523683740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56366bd087e0 con 0x56366bcf76c0 2024-09-19T04:19:41.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.587+0000 7fc519e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcec4e0 0x56366bd09030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc511400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcece00 0x56366bcf4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc519e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcec4e0 0x56366bd09030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46362/0 (socket says 172.21.15.107:46362) 2024-09-19T04:19:41.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc519e00640 1 -- 172.21.15.107:0/2136823847 learned_addr learned my addr 172.21.15.107:0/2136823847 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:41.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc51a800640 1 --2- 172.21.15.107:0/2136823847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcf76c0 0x56366bcf16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc511400640 1 -- 172.21.15.107:0/2136823847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcf76c0 msgr2=0x56366bcf16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc511400640 1 --2- 172.21.15.107:0/2136823847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcf76c0 0x56366bcf16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc511400640 1 -- 172.21.15.107:0/2136823847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcec4e0 msgr2=0x56366bd09030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc511400640 1 --2- 172.21.15.107:0/2136823847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcec4e0 0x56366bd09030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc511400640 1 -- 172.21.15.107:0/2136823847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366bce88a0 con 0x56366bcece00 2024-09-19T04:19:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc519e00640 1 --2- 172.21.15.107:0/2136823847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcec4e0 0x56366bd09030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc511400640 1 --2- 172.21.15.107:0/2136823847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcece00 0x56366bcf4d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc5040ed950 tx=0x7fc50402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5386a57c0bc8c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc50a600640 1 -- 172.21.15.107:0/2136823847 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504049070 con 0x56366bcece00 2024-09-19T04:19:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc50a600640 1 -- 172.21.15.107:0/2136823847 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5040bc600 con 0x56366bcece00 2024-09-19T04:19:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.588+0000 7fc50a600640 1 -- 172.21.15.107:0/2136823847 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5040c0900 con 0x56366bcece00 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.589+0000 7fc523683740 1 -- 172.21.15.107:0/2136823847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcece00 msgr2=0x56366bcf4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.589+0000 7fc523683740 1 --2- 172.21.15.107:0/2136823847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcece00 0x56366bcf4d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc5040ed950 tx=0x7fc50402a410 comp rx=0 tx=0).stop 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.589+0000 7fc523683740 1 -- 172.21.15.107:0/2136823847 shutdown_connections 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.589+0000 7fc523683740 1 --2- 172.21.15.107:0/2136823847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcf76c0 0x56366bcf16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.589+0000 7fc523683740 1 --2- 172.21.15.107:0/2136823847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcece00 0x56366bcf4d60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.589+0000 7fc523683740 1 --2- 172.21.15.107:0/2136823847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcec4e0 0x56366bd09030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.589+0000 7fc523683740 1 -- 172.21.15.107:0/2136823847 >> 172.21.15.107:0/2136823847 conn(0x56366bce8a70 msgr2=0x56366bcf3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.589+0000 7fc523683740 1 -- 172.21.15.107:0/2136823847 shutdown_connections 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.589+0000 7fc523683740 1 -- 172.21.15.107:0/2136823847 wait complete. 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc523683740 1 -- start start 2024-09-19T04:19:41.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc523683740 1 Processor -- start 2024-09-19T04:19:41.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc523683740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 0x56366bd11f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc523683740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 0x56366bd0da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc523683740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 0x56366bd0df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc523683740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56366bd09580 con 0x56366bcece00 2024-09-19T04:19:41.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc523683740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56366bd08eb0 con 0x56366bcf76c0 2024-09-19T04:19:41.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc523683740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56366bd09c50 con 0x56366bcec4e0 2024-09-19T04:19:41.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc523683740 10 client.0 _ll_register_callbacks cb 0x7fc514004de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc511400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 0x56366bd0da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.590+0000 7fc519e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 0x56366bd11f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc519e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 0x56366bd11f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37284/0 (socket says 172.21.15.107:37284) 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc519e00640 1 -- 172.21.15.107:0/2259980777 learned_addr learned my addr 172.21.15.107:0/2259980777 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc51a800640 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 0x56366bd0df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc519e00640 1 -- 172.21.15.107:0/2259980777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 msgr2=0x56366bd0df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc519e00640 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 0x56366bd0df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc519e00640 1 -- 172.21.15.107:0/2259980777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 msgr2=0x56366bd0da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc519e00640 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 0x56366bd0da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc519e00640 1 -- 172.21.15.107:0/2259980777 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366bce9b20 con 0x56366bcec4e0 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc519e00640 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 0x56366bd11f60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005ca0 tx=0x7fc50c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cee0acb4c555f004 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc4ff400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc4ff400640 1 -- 172.21.15.107:0/2259980777 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c01ee40 con 0x56366bcec4e0 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc4ff400640 1 -- 172.21.15.107:0/2259980777 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fc50c005020 con 0x56366bcec4e0 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.591+0000 7fc523683740 1 -- 172.21.15.107:0/2259980777 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5040bc600 con 0x56366bcec4e0 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc4ff400640 1 -- 172.21.15.107:0/2259980777 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc50c005220 con 0x56366bcec4e0 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc523683740 10 client.24937 fetch_fsmap learned FSMap version 27 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc523683740 1 -- 172.21.15.107:0/2259980777 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56366bcf9020 con 0x56366bcec4e0 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc4ff400640 1 -- 172.21.15.107:0/2259980777 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5040bc600 con 0x56366bcec4e0 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc4ff400640 1 -- 172.21.15.107:0/2259980777 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc50c02a4f0 con 0x56366bcec4e0 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc523683740 10 client.24937 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc523683740 10 client.24937 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc523683740 20 client.24937 populate_metadata read hostname 'smithi107' 2024-09-19T04:19:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc523683740 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56366bd77210 0x56366bd75f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc523683740 4 client.24937 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.592+0000 7fc523683740 1 -- 172.21.15.107:0/2259980777 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "status"}) -- 0x56366bc59b30 con 0x56366bd77210 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.593+0000 7fc511400640 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56366bd77210 0x56366bd75f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.593+0000 7fc511400640 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56366bd77210 0x56366bd75f20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=147b7b881be8a3f9 server_cookie=187a6f683a670878 in_seq=0 out_seq=0 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.593+0000 7fc4ff400640 10 client.24937 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.593+0000 7fc4ff400640 1 -- 172.21.15.107:0/2259980777 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56366bc59b30 con 0x56366bd77210 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.593+0000 7fc4ff400640 10 client.24937 handle_command_reply: tid=0 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 client.24937 shutdown 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 -- 172.21.15.107:0/2259980777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 msgr2=0x56366bd11f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 0x56366bd11f60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005ca0 tx=0x7fc50c0046c0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 -- 172.21.15.107:0/2259980777 shutdown_connections 2024-09-19T04:19:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56366bd77210 0x56366bd75f20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56366bcf76c0 0x56366bd0df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bcece00 0x56366bd0da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 --2- 172.21.15.107:0/2259980777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366bcec4e0 0x56366bd11f60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc4ff400640 0 client.24937 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:19:41.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 -- 172.21.15.107:0/2259980777 >> 172.21.15.107:0/2259980777 conn(0x56366bce8a70 msgr2=0x56366bce73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:41.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 -- 172.21.15.107:0/2259980777 shutdown_connections 2024-09-19T04:19:41.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 1 -- 172.21.15.107:0/2259980777 wait complete. 2024-09-19T04:19:41.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.594+0000 7fc523683740 20 client.24937 trim_cache size 0 max 16384 2024-09-19T04:19:41.600 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24517, 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 80.061372290999998, 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 83, 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 109.75568847 2024-09-19T04:19:41.601 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:19:41.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 -- 172.21.15.107:0/1313665560 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5000a32d0 msgr2=0x7fc5000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5000a32d0 0x7fc5000a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56366bce98d0 tx=0x7fc50c00b410 comp rx=0 tx=0).stop 2024-09-19T04:19:41.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 -- 172.21.15.107:0/1313665560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c1047e0 msgr2=0x7fc51c19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c1047e0 0x7fc51c19a990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc504006010 tx=0x7fc5040042f0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 -- 172.21.15.107:0/1313665560 shutdown_connections 2024-09-19T04:19:41.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fc5000a32d0 0x7fc5000a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc51c108f60 0x7fc51c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c1047e0 0x7fc51c19a990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 --2- 172.21.15.107:0/1313665560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc51c103e30 0x7fc51c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 -- 172.21.15.107:0/1313665560 >> 172.21.15.107:0/1313665560 conn(0x7fc51c0ff8d0 msgr2=0x7fc51c100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:41.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 -- 172.21.15.107:0/1313665560 shutdown_connections 2024-09-19T04:19:41.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:41.595+0000 7fc51b200640 1 -- 172.21.15.107:0/1313665560 wait complete. 2024-09-19T04:19:42.311 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T04:19:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:43.136 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:42 smithi107 ceph-mon[30659]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:42 smithi115 ceph-mon[35477]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:42 smithi145 ceph-mon[35096]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:43.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.244+0000 7f8b78000640 1 -- 172.21.15.107:0/2140619263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b70074620 msgr2=0x7f8b70074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.244+0000 7f8b78000640 1 --2- 172.21.15.107:0/2140619263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b70074620 0x7f8b70074a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009900 tx=0x7f8b6402a480 comp rx=0 tx=0).stop 2024-09-19T04:19:43.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.244+0000 7f8b78000640 1 -- 172.21.15.107:0/2140619263 shutdown_connections 2024-09-19T04:19:43.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.244+0000 7f8b78000640 1 --2- 172.21.15.107:0/2140619263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b7010e3c0 0x7f8b70110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.244+0000 7f8b78000640 1 --2- 172.21.15.107:0/2140619263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b70074f40 0x7f8b7010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.244+0000 7f8b78000640 1 --2- 172.21.15.107:0/2140619263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b70074620 0x7f8b70074a00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.244+0000 7f8b78000640 1 -- 172.21.15.107:0/2140619263 >> 172.21.15.107:0/2140619263 conn(0x7f8b700fd8c0 msgr2=0x7f8b700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:43.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.245+0000 7f8b78000640 1 -- 172.21.15.107:0/2140619263 shutdown_connections 2024-09-19T04:19:43.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.245+0000 7f8b78000640 1 -- 172.21.15.107:0/2140619263 wait complete. 2024-09-19T04:19:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.245+0000 7f8b78000640 1 Processor -- start 2024-09-19T04:19:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b78000640 1 -- start start 2024-09-19T04:19:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b78000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b70074620 0x7f8b7019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b78000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b70074f40 0x7f8b7019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b78000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7010e3c0 0x7f8b7019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b78000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701141c0 con 0x7f8b70074620 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b78000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70114040 con 0x7f8b7010e3c0 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b78000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70114340 con 0x7f8b70074f40 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b75600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b70074620 0x7f8b7019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b75600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b70074620 0x7f8b7019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46390/0 (socket says 172.21.15.107:46390) 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b75600640 1 -- 172.21.15.107:0/821149734 learned_addr learned my addr 172.21.15.107:0/821149734 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.246+0000 7f8b74c00640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b70074f40 0x7f8b7019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.247+0000 7f8b76000640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7010e3c0 0x7f8b7019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.247+0000 7f8b75600640 1 -- 172.21.15.107:0/821149734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b70074f40 msgr2=0x7f8b7019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.247+0000 7f8b75600640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b70074f40 0x7f8b7019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.247+0000 7f8b75600640 1 -- 172.21.15.107:0/821149734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7010e3c0 msgr2=0x7f8b7019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.247+0000 7f8b75600640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7010e3c0 0x7f8b7019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.247+0000 7f8b75600640 1 -- 172.21.15.107:0/821149734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7019f640 con 0x7f8b70074620 2024-09-19T04:19:43.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.247+0000 7f8b75600640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b70074620 0x7f8b7019a690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8b64004770 tx=0x7f8b64003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:43.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.247+0000 7f8b6e000640 1 -- 172.21.15.107:0/821149734 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6403f070 con 0x7f8b70074620 2024-09-19T04:19:43.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.247+0000 7f8b6e000640 1 -- 172.21.15.107:0/821149734 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b64031ca0 con 0x7f8b70074620 2024-09-19T04:19:43.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.247+0000 7f8b6e000640 1 -- 172.21.15.107:0/821149734 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64035030 con 0x7f8b70074620 2024-09-19T04:19:43.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.248+0000 7f8b78000640 1 -- 172.21.15.107:0/821149734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7019f8d0 con 0x7f8b70074620 2024-09-19T04:19:43.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.248+0000 7f8b78000640 1 -- 172.21.15.107:0/821149734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7019fd60 con 0x7f8b70074620 2024-09-19T04:19:43.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.250+0000 7f8b7aa1c740 1 Processor -- start 2024-09-19T04:19:43.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b7aa1c740 1 -- start start 2024-09-19T04:19:43.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b6e000640 1 -- 172.21.15.107:0/821149734 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8b64042070 con 0x7f8b70074620 2024-09-19T04:19:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 0x5613bc298710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc2a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc2a5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5613bc1fe0b0 con 0x5613bc2a3510 2024-09-19T04:19:43.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5613bc1a5ab0 con 0x5613bc298c50 2024-09-19T04:19:43.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5613bc1fa070 con 0x5613bc298330 2024-09-19T04:19:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b74c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc2a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b76000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 0x5613bc298710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b76000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 0x5613bc298710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37306/0 (socket says 172.21.15.107:37306) 2024-09-19T04:19:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b76000640 1 -- 172.21.15.107:0/2697367427 learned_addr learned my addr 172.21.15.107:0/2697367427 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b74c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc2a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46400/0 (socket says 172.21.15.107:46400) 2024-09-19T04:19:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.251+0000 7f8b75600640 1 --2- 172.21.15.107:0/2697367427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc2a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.252+0000 7f8b76000640 1 -- 172.21.15.107:0/2697367427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 msgr2=0x5613bc2a2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.252+0000 7f8b76000640 1 --2- 172.21.15.107:0/2697367427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc2a2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.252+0000 7f8b76000640 1 -- 172.21.15.107:0/2697367427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 msgr2=0x5613bc2a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.252+0000 7f8b76000640 1 --2- 172.21.15.107:0/2697367427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc2a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.252+0000 7f8b76000640 1 -- 172.21.15.107:0/2697367427 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613bc1b24a0 con 0x5613bc298330 2024-09-19T04:19:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.252+0000 7f8b74c00640 1 --2- 172.21.15.107:0/2697367427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc2a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.252+0000 7f8b76000640 1 --2- 172.21.15.107:0/2697367427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 0x5613bc298710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009f70 tx=0x7f8b68010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51ae5e8dd6cb596e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.252+0000 7f8b6e000640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8b580a3350 0x7f8b580a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.252+0000 7f8b62600640 1 -- 172.21.15.107:0/2697367427 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68016070 con 0x5613bc298330 2024-09-19T04:19:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b62600640 1 -- 172.21.15.107:0/2697367427 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b68004d20 con 0x5613bc298330 2024-09-19T04:19:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b62600640 1 -- 172.21.15.107:0/2697367427 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68012910 con 0x5613bc298330 2024-09-19T04:19:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b6e000640 1 -- 172.21.15.107:0/821149734 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8b640f0120 con 0x7f8b70074620 2024-09-19T04:19:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b75600640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8b580a3350 0x7f8b580a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/2697367427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 msgr2=0x5613bc298710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/2697367427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 0x5613bc298710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009f70 tx=0x7f8b68010800 comp rx=0 tx=0).stop 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b75600640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8b580a3350 0x7f8b580a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b640bc540 tx=0x7f8b640f3bd0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/2697367427 shutdown_connections 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/2697367427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc2a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/2697367427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc2a2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/2697367427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 0x5613bc298710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.253+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/2697367427 >> 172.21.15.107:0/2697367427 conn(0x5613bc2948c0 msgr2=0x5613bc294ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.254+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/2697367427 shutdown_connections 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.254+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/2697367427 wait complete. 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.254+0000 7f8b7aa1c740 1 -- start start 2024-09-19T04:19:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.254+0000 7f8b7aa1c740 1 Processor -- start 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298c50 0x5613bc2a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc2a3510 0x5613bc2a1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5613bc2a6060 con 0x5613bc298330 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5613bc186920 con 0x5613bc2a3510 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5613bc2a61e0 con 0x5613bc298c50 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b7aa1c740 10 client.0 _ll_register_callbacks cb 0x5613bc1aa8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b76000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b76000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46404/0 (socket says 172.21.15.107:46404) 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b76000640 1 -- 172.21.15.107:0/382785220 learned_addr learned my addr 172.21.15.107:0/382785220 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b74c00640 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298c50 0x5613bc2a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b76000640 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc2a3510 0x5613bc2a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b76000640 1 -- 172.21.15.107:0/382785220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298c50 msgr2=0x5613bc2a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b76000640 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298c50 0x5613bc2a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b76000640 1 -- 172.21.15.107:0/382785220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc2a3510 msgr2=0x5613bc2a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b76000640 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc2a3510 0x5613bc2a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b76000640 1 -- 172.21.15.107:0/382785220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613bbf6f2b0 con 0x5613bc298330 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.255+0000 7f8b74c00640 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298c50 0x5613bc2a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.256+0000 7f8b76000640 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2a05e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b68011010 tx=0x7f8b68012620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553c55fb8371eb3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.256+0000 7f8b56a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.256+0000 7f8b56a00640 1 -- 172.21.15.107:0/382785220 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68016070 con 0x5613bc298330 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.256+0000 7f8b56a00640 1 -- 172.21.15.107:0/382785220 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8b68012bc0 con 0x5613bc298330 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.256+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/382785220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b68004d20 con 0x5613bc298330 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.256+0000 7f8b56a00640 1 -- 172.21.15.107:0/382785220 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b68030020 con 0x5613bc298330 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.256+0000 7f8b7aa1c740 10 client.14934 fetch_fsmap learned FSMap version 27 2024-09-19T04:19:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.256+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/382785220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613bc2b1e80 con 0x5613bc298330 2024-09-19T04:19:43.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.257+0000 7f8b56a00640 1 -- 172.21.15.107:0/382785220 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b68004d20 con 0x5613bc298330 2024-09-19T04:19:43.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.257+0000 7f8b56a00640 1 -- 172.21.15.107:0/382785220 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8b6800f040 con 0x5613bc298330 2024-09-19T04:19:43.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.257+0000 7f8b7aa1c740 10 client.14934 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:19:43.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.257+0000 7f8b7aa1c740 10 client.14934 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:19:43.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.257+0000 7f8b7aa1c740 20 client.14934 populate_metadata read hostname 'smithi107' 2024-09-19T04:19:43.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.257+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613bc2b4100 0x5613bc2b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.257+0000 7f8b7aa1c740 4 client.14934 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:43.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.257+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/382785220 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613bc20b620 con 0x5613bc2b4100 2024-09-19T04:19:43.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.258+0000 7f8b74c00640 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613bc2b4100 0x5613bc2b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.258+0000 7f8b74c00640 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613bc2b4100 0x5613bc2b5570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82fcb6552e5c43e server_cookie=be6d34046d322752 in_seq=0 out_seq=0 2024-09-19T04:19:43.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.258+0000 7f8b56a00640 10 client.14934 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:19:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.259+0000 7f8b56a00640 1 -- 172.21.15.107:0/382785220 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613bc20b620 con 0x5613bc2b4100 2024-09-19T04:19:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.259+0000 7f8b56a00640 10 client.14934 handle_command_reply: tid=0 2024-09-19T04:19:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 client.14934 shutdown 2024-09-19T04:19:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/382785220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 msgr2=0x5613bc2a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2a05e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b68011010 tx=0x7f8b68012620 comp rx=0 tx=0).stop 2024-09-19T04:19:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/382785220 shutdown_connections 2024-09-19T04:19:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613bc2b4100 0x5613bc2b5570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc2a3510 0x5613bc2a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298c50 0x5613bc2a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/382785220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2a05e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b56a00640 0 client.14934 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:19:43.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/382785220 >> 172.21.15.107:0/382785220 conn(0x5613bc2948c0 msgr2=0x5613bc29a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:43.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/382785220 shutdown_connections 2024-09-19T04:19:43.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/382785220 wait complete. 2024-09-19T04:19:43.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.260+0000 7f8b7aa1c740 20 client.14934 trim_cache size 0 max 16384 2024-09-19T04:19:43.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.278+0000 7f8b7aa1c740 1 Processor -- start 2024-09-19T04:19:43.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b7aa1c740 1 -- start start 2024-09-19T04:19:43.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 0x5613bc29ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc29d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc29dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5613bc2b38b0 con 0x5613bc2a3510 2024-09-19T04:19:43.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5613bc2b3f80 con 0x5613bc298c50 2024-09-19T04:19:43.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5613bc2b4650 con 0x5613bc298330 2024-09-19T04:19:43.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b76000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc29dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b76000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc29dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46412/0 (socket says 172.21.15.107:46412) 2024-09-19T04:19:43.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b74c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc29d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b76000640 1 -- 172.21.15.107:0/2101068016 learned_addr learned my addr 172.21.15.107:0/2101068016 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:43.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.279+0000 7f8b76000640 1 --2- 172.21.15.107:0/2101068016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 0x5613bc29ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.280+0000 7f8b76000640 1 -- 172.21.15.107:0/2101068016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 msgr2=0x5613bc29ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:43.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.280+0000 7f8b76000640 1 --2- 172.21.15.107:0/2101068016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 0x5613bc29ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.280+0000 7f8b76000640 1 -- 172.21.15.107:0/2101068016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 msgr2=0x5613bc29d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.280+0000 7f8b76000640 1 --2- 172.21.15.107:0/2101068016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc29d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.280+0000 7f8b76000640 1 -- 172.21.15.107:0/2101068016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613bc298a80 con 0x5613bc2a3510 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.280+0000 7f8b74c00640 1 --2- 172.21.15.107:0/2101068016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc29d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.280+0000 7f8b76000640 1 --2- 172.21.15.107:0/2101068016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc29dda0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009a80 tx=0x7f8b68014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dab7675b14bea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.280+0000 7f8b61c00640 1 -- 172.21.15.107:0/2101068016 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68016040 con 0x5613bc2a3510 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.280+0000 7f8b61c00640 1 -- 172.21.15.107:0/2101068016 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6801f030 con 0x5613bc2a3510 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.280+0000 7f8b61c00640 1 -- 172.21.15.107:0/2101068016 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68004eb0 con 0x5613bc2a3510 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.281+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/2101068016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 msgr2=0x5613bc29dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.281+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/2101068016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc29dda0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009a80 tx=0x7f8b68014040 comp rx=0 tx=0).stop 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.281+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/2101068016 shutdown_connections 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.281+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/2101068016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc2a3510 0x5613bc29dda0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.281+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/2101068016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc29d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.281+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/2101068016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc298330 0x5613bc29ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.281+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/2101068016 >> 172.21.15.107:0/2101068016 conn(0x5613bc2948c0 msgr2=0x5613bc29eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:43.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.281+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/2101068016 shutdown_connections 2024-09-19T04:19:43.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.281+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/2101068016 wait complete. 2024-09-19T04:19:43.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.281+0000 7f8b7aa1c740 1 -- start start 2024-09-19T04:19:43.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b7aa1c740 1 Processor -- start 2024-09-19T04:19:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc29d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b7aa1c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc2a3510 0x5613bc29d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5613bc2b53f0 con 0x5613bc298330 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5613bc2b4d20 con 0x5613bc298c50 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b7aa1c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5613bc2b5ac0 con 0x5613bc2a3510 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b7aa1c740 10 client.0 _ll_register_callbacks cb 0x7f8b68009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b76000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b76000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2bdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46424/0 (socket says 172.21.15.107:46424) 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b76000640 1 -- 172.21.15.107:0/4019446613 learned_addr learned my addr 172.21.15.107:0/4019446613 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.282+0000 7f8b74c00640 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc29d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b76000640 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc2a3510 0x5613bc29d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b74c00640 1 -- 172.21.15.107:0/4019446613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc2a3510 msgr2=0x5613bc29d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b74c00640 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc2a3510 0x5613bc29d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b74c00640 1 -- 172.21.15.107:0/4019446613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 msgr2=0x5613bc2bdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b74c00640 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2bdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b74c00640 1 -- 172.21.15.107:0/4019446613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613bc295550 con 0x5613bc298c50 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b76000640 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2bdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b74c00640 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc29d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5613bc2b5d40 tx=0x7f8b5c00bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=881d66d6e4221650 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b56a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b56a00640 1 -- 172.21.15.107:0/4019446613 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c005220 con 0x5613bc298c50 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/4019446613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b6801f030 con 0x5613bc298c50 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.283+0000 7f8b56a00640 1 -- 172.21.15.107:0/4019446613 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8b5c0053c0 con 0x5613bc298c50 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b56a00640 1 -- 172.21.15.107:0/4019446613 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b5c005600 con 0x5613bc298c50 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b7aa1c740 10 client.24959 fetch_fsmap learned FSMap version 27 2024-09-19T04:19:43.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/4019446613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613bc2bc5f0 con 0x5613bc298c50 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b56a00640 1 -- 172.21.15.107:0/4019446613 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6801f030 con 0x5613bc298c50 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b56a00640 1 -- 172.21.15.107:0/4019446613 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8b5c011880 con 0x5613bc298c50 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b7aa1c740 10 client.24959 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b7aa1c740 10 client.24959 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b7aa1c740 20 client.24959 populate_metadata read hostname 'smithi107' 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613bc3230b0 0x5613bc321dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b7aa1c740 4 client.24959 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.284+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/4019446613 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "status"}) -- 0x5613bc205b30 con 0x5613bc3230b0 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.285+0000 7f8b76000640 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613bc3230b0 0x5613bc321dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.285+0000 7f8b76000640 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613bc3230b0 0x5613bc321dc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9951e91e3bfa6721 server_cookie=bb7d797c6772a0ff in_seq=0 out_seq=0 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.285+0000 7f8b56a00640 10 client.24959 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.285+0000 7f8b56a00640 1 -- 172.21.15.107:0/4019446613 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5613bc205b30 con 0x5613bc3230b0 2024-09-19T04:19:43.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.285+0000 7f8b56a00640 10 client.24959 handle_command_reply: tid=0 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.285+0000 7f8b7aa1c740 1 client.24959 shutdown 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.285+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/4019446613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 msgr2=0x5613bc29d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.285+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc29d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5613bc2b5d40 tx=0x7f8b5c00bf20 comp rx=0 tx=0).stop 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.285+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/4019446613 shutdown_connections 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.285+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613bc3230b0 0x5613bc321dc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613bc2a3510 0x5613bc29d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613bc298c50 0x5613bc29d1e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b56a00640 0 client.24959 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b7aa1c740 1 --2- 172.21.15.107:0/4019446613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613bc298330 0x5613bc2bdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/4019446613 >> 172.21.15.107:0/4019446613 conn(0x5613bc2948c0 msgr2=0x5613bc2ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/4019446613 shutdown_connections 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b7aa1c740 1 -- 172.21.15.107:0/4019446613 wait complete. 2024-09-19T04:19:43.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b7aa1c740 20 client.24959 trim_cache size 0 max 16384 2024-09-19T04:19:43.293 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:19:43.293 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:19:43.293 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-09-19T04:19:43.293 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24539, 2024-09-19T04:19:43.293 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-09-19T04:19:43.293 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-09-19T04:19:43.293 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-09-19T04:19:43.293 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 89.809115364999997, 2024-09-19T04:19:43.293 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-09-19T04:19:43.294 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 83, 2024-09-19T04:19:43.294 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T04:19:43.294 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 101.597507209 2024-09-19T04:19:43.294 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:19:43.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b78000640 1 -- 172.21.15.107:0/821149734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8b580a3350 msgr2=0x7f8b580a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b78000640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8b580a3350 0x7f8b580a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b640bc540 tx=0x7f8b640f3bd0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b78000640 1 -- 172.21.15.107:0/821149734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b70074620 msgr2=0x7f8b7019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.286+0000 7f8b78000640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b70074620 0x7f8b7019a690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8b64004770 tx=0x7f8b64003c10 comp rx=0 tx=0).stop 2024-09-19T04:19:43.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.287+0000 7f8b78000640 1 -- 172.21.15.107:0/821149734 shutdown_connections 2024-09-19T04:19:43.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.287+0000 7f8b78000640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f8b580a3350 0x7f8b580a5810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.287+0000 7f8b78000640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7010e3c0 0x7f8b7019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.287+0000 7f8b78000640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b70074f40 0x7f8b7019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.287+0000 7f8b78000640 1 --2- 172.21.15.107:0/821149734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b70074620 0x7f8b7019a690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.287+0000 7f8b78000640 1 -- 172.21.15.107:0/821149734 >> 172.21.15.107:0/821149734 conn(0x7f8b700fd8c0 msgr2=0x7f8b700fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:43.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.287+0000 7f8b78000640 1 -- 172.21.15.107:0/821149734 shutdown_connections 2024-09-19T04:19:43.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:43.287+0000 7f8b78000640 1 -- 172.21.15.107:0/821149734 wait complete. 2024-09-19T04:19:44.012 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T04:19:44.012 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T04:19:44.026 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-09-19T04:19:44.026 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T04:19:44.395 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:44.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.957+0000 7fe000c00640 1 -- 172.21.15.107:0/3018420974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdffc102680 msgr2=0x7fdffc104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:44.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.957+0000 7fe000c00640 1 --2- 172.21.15.107:0/3018420974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdffc102680 0x7fdffc104a70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009900 tx=0x7fdfec02a410 comp rx=0 tx=0).stop 2024-09-19T04:19:44.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.957+0000 7fe000c00640 1 -- 172.21.15.107:0/3018420974 shutdown_connections 2024-09-19T04:19:44.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.957+0000 7fe000c00640 1 --2- 172.21.15.107:0/3018420974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdffc1078e0 0x7fdffc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.957+0000 7fe000c00640 1 --2- 172.21.15.107:0/3018420974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdffc104fb0 0x7fdffc1073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.957+0000 7fe000c00640 1 --2- 172.21.15.107:0/3018420974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdffc102680 0x7fdffc104a70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.957+0000 7fe000c00640 1 -- 172.21.15.107:0/3018420974 >> 172.21.15.107:0/3018420974 conn(0x7fdffc0fc280 msgr2=0x7fdffc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:44.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.957+0000 7fe000c00640 1 -- 172.21.15.107:0/3018420974 shutdown_connections 2024-09-19T04:19:44.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.957+0000 7fe000c00640 1 -- 172.21.15.107:0/3018420974 wait complete. 2024-09-19T04:19:44.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.958+0000 7fe000c00640 1 Processor -- start 2024-09-19T04:19:44.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.958+0000 7fe000c00640 1 -- start start 2024-09-19T04:19:44.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fe000c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdffc102680 0x7fdffc19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:44.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fe000c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdffc104fb0 0x7fdffc19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:44.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fe000c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdffc1078e0 0x7fdffc1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:44.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fe000c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc10d890 con 0x7fdffc104fb0 2024-09-19T04:19:44.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fe000c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc10d710 con 0x7fdffc1078e0 2024-09-19T04:19:44.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fe000c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc10da10 con 0x7fdffc102680 2024-09-19T04:19:44.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fdff1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdffc104fb0 0x7fdffc19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:44.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fdff1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdffc104fb0 0x7fdffc19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46432/0 (socket says 172.21.15.107:46432) 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fdff1400640 1 -- 172.21.15.107:0/1569917956 learned_addr learned my addr 172.21.15.107:0/1569917956 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fdffa800640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdffc1078e0 0x7fdffc1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.959+0000 7fdff9e00640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdffc102680 0x7fdffc19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fdff1400640 1 -- 172.21.15.107:0/1569917956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdffc102680 msgr2=0x7fdffc19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fdff1400640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdffc102680 0x7fdffc19a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fdff1400640 1 -- 172.21.15.107:0/1569917956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdffc1078e0 msgr2=0x7fdffc1a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fdff1400640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdffc1078e0 0x7fdffc1a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fdff1400640 1 -- 172.21.15.107:0/1569917956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc1a1e20 con 0x7fdffc104fb0 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fdff1400640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdffc104fb0 0x7fdffc19a730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005fd0 tx=0x7fdfe400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fdff3400640 1 -- 172.21.15.107:0/1569917956 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4017070 con 0x7fdffc104fb0 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fdff3400640 1 -- 172.21.15.107:0/1569917956 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe4009c30 con 0x7fdffc104fb0 2024-09-19T04:19:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fdff3400640 1 -- 172.21.15.107:0/1569917956 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4010460 con 0x7fdffc104fb0 2024-09-19T04:19:44.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fe000c00640 1 -- 172.21.15.107:0/1569917956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdffc1a2110 con 0x7fdffc104fb0 2024-09-19T04:19:44.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.960+0000 7fe000c00640 1 -- 172.21.15.107:0/1569917956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdffc1a2600 con 0x7fdffc104fb0 2024-09-19T04:19:44.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.961+0000 7fe000c00640 1 -- 172.21.15.107:0/1569917956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdffc109430 con 0x7fdffc104fb0 2024-09-19T04:19:44.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.963+0000 7fdff3400640 1 -- 172.21.15.107:0/1569917956 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdfe4003e70 con 0x7fdffc104fb0 2024-09-19T04:19:44.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.964+0000 7fdff3400640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfe00a3300 0x7fdfe00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:44.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.965+0000 7fdff3400640 1 -- 172.21.15.107:0/1569917956 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fdfe40cf700 con 0x7fdffc104fb0 2024-09-19T04:19:44.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.965+0000 7fdff9e00640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfe00a3300 0x7fdfe00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:44.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.965+0000 7fdff9e00640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfe00a3300 0x7fdfe00a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdfec000c00 tx=0x7fdfec0315b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:44.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:44.969+0000 7fdff3400640 1 -- 172.21.15.107:0/1569917956 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdfe4020030 con 0x7fdffc104fb0 2024-09-19T04:19:45.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.158+0000 7fe000c00640 1 -- 172.21.15.107:0/1569917956 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fdffc062630 con 0x7fdfe00a3300 2024-09-19T04:19:45.160 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:44 smithi107 ceph-mon[30659]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:45.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.159+0000 7fdff3400640 1 -- 172.21.15.107:0/1569917956 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fdffc062630 con 0x7fdfe00a3300 2024-09-19T04:19:45.161 INFO:teuthology.orchestra.run.smithi107.stdout:Backend: cephadm 2024-09-19T04:19:45.161 INFO:teuthology.orchestra.run.smithi107.stdout:Available: Yes 2024-09-19T04:19:45.161 INFO:teuthology.orchestra.run.smithi107.stdout:Paused: No 2024-09-19T04:19:45.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.162+0000 7fe000c00640 1 -- 172.21.15.107:0/1569917956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfe00a3300 msgr2=0x7fdfe00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.162+0000 7fe000c00640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfe00a3300 0x7fdfe00a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdfec000c00 tx=0x7fdfec0315b0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.162+0000 7fe000c00640 1 -- 172.21.15.107:0/1569917956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdffc104fb0 msgr2=0x7fdffc19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.162+0000 7fe000c00640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdffc104fb0 0x7fdffc19a730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005fd0 tx=0x7fdfe400dde0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.162+0000 7fe000c00640 1 -- 172.21.15.107:0/1569917956 shutdown_connections 2024-09-19T04:19:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.162+0000 7fe000c00640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfe00a3300 0x7fdfe00a57c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.162+0000 7fe000c00640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdffc1078e0 0x7fdffc1a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.162+0000 7fe000c00640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdffc104fb0 0x7fdffc19a730 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.162+0000 7fe000c00640 1 --2- 172.21.15.107:0/1569917956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdffc102680 0x7fdffc19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.162+0000 7fe000c00640 1 -- 172.21.15.107:0/1569917956 >> 172.21.15.107:0/1569917956 conn(0x7fdffc0fc280 msgr2=0x7fdffc101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.163+0000 7fe000c00640 1 -- 172.21.15.107:0/1569917956 shutdown_connections 2024-09-19T04:19:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:45.163+0000 7fe000c00640 1 -- 172.21.15.107:0/1569917956 wait complete. 2024-09-19T04:19:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:44 smithi115 ceph-mon[35477]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:44 smithi145 ceph-mon[35096]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:45.913 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T04:19:46.281 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:46.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.838+0000 7f797fe00640 1 -- 172.21.15.107:0/2011475050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7980076ba0 msgr2=0x7f7980076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.838+0000 7f797fe00640 1 --2- 172.21.15.107:0/2011475050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7980076ba0 0x7f7980076fa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7970009900 tx=0x7f797002a410 comp rx=0 tx=0).stop 2024-09-19T04:19:46.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.838+0000 7f797fe00640 1 -- 172.21.15.107:0/2011475050 shutdown_connections 2024-09-19T04:19:46.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.838+0000 7f797fe00640 1 --2- 172.21.15.107:0/2011475050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7980078790 0x7f7980079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.838+0000 7f797fe00640 1 --2- 172.21.15.107:0/2011475050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7980077df0 0x7f7980078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.838+0000 7f797fe00640 1 --2- 172.21.15.107:0/2011475050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7980076ba0 0x7f7980076fa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.838+0000 7f797fe00640 1 -- 172.21.15.107:0/2011475050 >> 172.21.15.107:0/2011475050 conn(0x7f79800ff3e0 msgr2=0x7f7980101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:46.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.838+0000 7f797fe00640 1 -- 172.21.15.107:0/2011475050 shutdown_connections 2024-09-19T04:19:46.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.838+0000 7f797fe00640 1 -- 172.21.15.107:0/2011475050 wait complete. 2024-09-19T04:19:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.839+0000 7f797fe00640 1 Processor -- start 2024-09-19T04:19:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.839+0000 7f797fe00640 1 -- start start 2024-09-19T04:19:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.839+0000 7f797fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7980076ba0 0x7f798019e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.839+0000 7f797fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7980077df0 0x7f798019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.839+0000 7f797fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7980078790 0x7f79801a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.839+0000 7f797fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7980113d60 con 0x7f7980076ba0 2024-09-19T04:19:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.839+0000 7f797fe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7980113be0 con 0x7f7980078790 2024-09-19T04:19:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.839+0000 7f797fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7980113ee0 con 0x7f7980077df0 2024-09-19T04:19:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7980076ba0 0x7f798019e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7980076ba0 0x7f798019e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46458/0 (socket says 172.21.15.107:46458) 2024-09-19T04:19:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797ea00640 1 -- 172.21.15.107:0/1570476108 learned_addr learned my addr 172.21.15.107:0/1570476108 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797e000640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7980077df0 0x7f798019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797f400640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7980078790 0x7f79801a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797ea00640 1 -- 172.21.15.107:0/1570476108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7980077df0 msgr2=0x7f798019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797ea00640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7980077df0 0x7f798019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797ea00640 1 -- 172.21.15.107:0/1570476108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7980078790 msgr2=0x7f79801a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797ea00640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7980078790 0x7f79801a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797ea00640 1 -- 172.21.15.107:0/1570476108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79801a5f80 con 0x7f7980076ba0 2024-09-19T04:19:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.840+0000 7f797ea00640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7980076ba0 0x7f798019e350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7970031440 tx=0x7f7970032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.841+0000 7f7977400640 1 -- 172.21.15.107:0/1570476108 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797003f030 con 0x7f7980076ba0 2024-09-19T04:19:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.841+0000 7f7977400640 1 -- 172.21.15.107:0/1570476108 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7970003ce0 con 0x7f7980076ba0 2024-09-19T04:19:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.841+0000 7f797fe00640 1 -- 172.21.15.107:0/1570476108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79801a6210 con 0x7f7980076ba0 2024-09-19T04:19:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.841+0000 7f7977400640 1 -- 172.21.15.107:0/1570476108 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970035070 con 0x7f7980076ba0 2024-09-19T04:19:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.841+0000 7f797fe00640 1 -- 172.21.15.107:0/1570476108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79801a66f0 con 0x7f7980076ba0 2024-09-19T04:19:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.842+0000 7f797fe00640 1 -- 172.21.15.107:0/1570476108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79801105d0 con 0x7f7980076ba0 2024-09-19T04:19:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.844+0000 7f7977400640 1 -- 172.21.15.107:0/1570476108 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7970042030 con 0x7f7980076ba0 2024-09-19T04:19:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.845+0000 7f7977400640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79640a3300 0x7f79640a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.845+0000 7f7977400640 1 -- 172.21.15.107:0/1570476108 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f79700efef0 con 0x7f7980076ba0 2024-09-19T04:19:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.845+0000 7f797e000640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79640a3300 0x7f79640a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.846+0000 7f797e000640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79640a3300 0x7f79640a57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f798019f870 tx=0x7f796c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:46.849+0000 7f7977400640 1 -- 172.21.15.107:0/1570476108 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f79700bc430 con 0x7f7980076ba0 2024-09-19T04:19:46.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:46 smithi107 ceph-mon[30659]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:46.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:46 smithi107 ceph-mon[30659]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:47.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.035+0000 7f797fe00640 1 -- 172.21.15.107:0/1570476108 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7980062630 con 0x7f79640a3300 2024-09-19T04:19:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.047+0000 7f7977400640 1 -- 172.21.15.107:0/1570476108 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f7980062630 con 0x7f79640a3300 2024-09-19T04:19:47.049 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T04:19:47.049 INFO:teuthology.orchestra.run.smithi107.stdout:mds.a smithi107 running (108s) 24s ago 108s 15.5M - 19.1.1-361-g2f599a46 d2baa855c067 d992b5b44b7c 2024-09-19T04:19:47.049 INFO:teuthology.orchestra.run.smithi107.stdout:mds.b smithi115 running (105s) 36s ago 105s 29.0M - 19.1.1-361-g2f599a46 d2baa855c067 9d1512338f7a 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mds.c smithi145 running (2m) 36s ago 2m 18.2M - 19.1.1-361-g2f599a46 d2baa855c067 1e3bf76d9ee6 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mds.d smithi107 running (2m) 24s ago 2m 12.7M - 19.1.1-361-g2f599a46 d2baa855c067 937c61de8fef 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mds.e smithi115 running (2m) 36s ago 2m 17.9M - 19.1.1-361-g2f599a46 d2baa855c067 2e28e55b62ee 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mds.f smithi145 running (2m) 36s ago 2m 18.0M - 19.1.1-361-g2f599a46 d2baa855c067 8fe700ea027e 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mds.g smithi107 running (112s) 24s ago 111s 15.6M - 19.1.1-361-g2f599a46 d2baa855c067 64d4ddb63aea 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mds.h smithi115 running (118s) 36s ago 117s 18.0M - 19.1.1-361-g2f599a46 d2baa855c067 00340df01aee 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mds.i smithi145 running (115s) 36s ago 115s 20.6M - 19.1.1-361-g2f599a46 d2baa855c067 ce9d8167bf21 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mds.j smithi107 running (2m) 24s ago 2m 12.8M - 19.1.1-361-g2f599a46 d2baa855c067 2b706c582c80 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mds.k smithi115 running (2m) 36s ago 2m 17.0M - 19.1.1-361-g2f599a46 d2baa855c067 f6e1855e54d2 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mds.l smithi145 running (2m) 36s ago 2m 17.0M - 19.1.1-361-g2f599a46 d2baa855c067 59f8105a5daa 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.x smithi107 *:9283,8765 running (9m) 24s ago 9m 508M - 19.1.1-361-g2f599a46 d2baa855c067 c7ad715785fe 2024-09-19T04:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.y smithi115 *:8443,8765 running (7m) 36s ago 7m 442M - 19.1.1-361-g2f599a46 d2baa855c067 e28287cf2a5b 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.z smithi145 *:8443,8765 running (7m) 36s ago 7m 443M - 19.1.1-361-g2f599a46 d2baa855c067 42f6183e2e30 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:mon.a smithi107 running (9m) 24s ago 9m 48.8M 2048M 19.1.1-361-g2f599a46 d2baa855c067 7e651d195b90 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:mon.b smithi115 running (8m) 36s ago 8m 40.2M 2048M 19.1.1-361-g2f599a46 d2baa855c067 96e16a26aaa9 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:mon.c smithi145 running (8m) 36s ago 8m 42.0M 2048M 19.1.1-361-g2f599a46 d2baa855c067 63c8ed9438d4 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (7m) 24s ago 7m 54.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 38607649c50d 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi115 running (7m) 36s ago 7m 56.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 311ebaa95758 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi145 running (6m) 36s ago 6m 58.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 31538d34f8ed 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi107 running (6m) 24s ago 6m 51.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 281a3cebe535 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi115 running (5m) 36s ago 5m 55.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 b7995b9ee83a 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi145 running (5m) 36s ago 5m 43.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 1e202a27246c 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 smithi107 running (4m) 24s ago 4m 40.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 985a75a4ff67 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 smithi115 running (4m) 36s ago 4m 44.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 c714ebdca20a 2024-09-19T04:19:47.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 smithi145 running (3m) 36s ago 3m 53.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 bda2c4e8f785 2024-09-19T04:19:47.052 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 smithi107 running (3m) 24s ago 3m 50.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 c3f86b136dc7 2024-09-19T04:19:47.052 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 smithi115 running (3m) 36s ago 3m 55.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 b031c9130922 2024-09-19T04:19:47.052 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 smithi145 running (2m) 36s ago 2m 52.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 c30ed192c252 2024-09-19T04:19:47.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 -- 172.21.15.107:0/1570476108 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79640a3300 msgr2=0x7f79640a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:47.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79640a3300 0x7f79640a57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f798019f870 tx=0x7f796c005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 -- 172.21.15.107:0/1570476108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7980076ba0 msgr2=0x7f798019e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7980076ba0 0x7f798019e350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7970031440 tx=0x7f7970032ce0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 -- 172.21.15.107:0/1570476108 shutdown_connections 2024-09-19T04:19:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f79640a3300 0x7f79640a57c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7980078790 0x7f79801a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7980077df0 0x7f798019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 --2- 172.21.15.107:0/1570476108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7980076ba0 0x7f798019e350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 -- 172.21.15.107:0/1570476108 >> 172.21.15.107:0/1570476108 conn(0x7f79800ff3e0 msgr2=0x7f7980100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 -- 172.21.15.107:0/1570476108 shutdown_connections 2024-09-19T04:19:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:47.050+0000 7f797fe00640 1 -- 172.21.15.107:0/1570476108 wait complete. 2024-09-19T04:19:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:46 smithi115 ceph-mon[35477]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:46 smithi115 ceph-mon[35477]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:46 smithi145 ceph-mon[35096]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:46 smithi145 ceph-mon[35096]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:47.836 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T04:19:48.227 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:48.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.800+0000 7fa8ba200640 1 -- 172.21.15.107:0/1838403487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b4076ba0 msgr2=0x7fa8b4076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.800+0000 7fa8ba200640 1 --2- 172.21.15.107:0/1838403487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b4076ba0 0x7fa8b4076fa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009900 tx=0x7fa8a802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.800+0000 7fa8ba200640 1 -- 172.21.15.107:0/1838403487 shutdown_connections 2024-09-19T04:19:48.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.800+0000 7fa8ba200640 1 --2- 172.21.15.107:0/1838403487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4078790 0x7fa8b4079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.800+0000 7fa8ba200640 1 --2- 172.21.15.107:0/1838403487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b4077df0 0x7fa8b4078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.800+0000 7fa8ba200640 1 --2- 172.21.15.107:0/1838403487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b4076ba0 0x7fa8b4076fa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.800+0000 7fa8ba200640 1 -- 172.21.15.107:0/1838403487 >> 172.21.15.107:0/1838403487 conn(0x7fa8b40ff3e0 msgr2=0x7fa8b4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.800+0000 7fa8ba200640 1 -- 172.21.15.107:0/1838403487 shutdown_connections 2024-09-19T04:19:48.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.800+0000 7fa8ba200640 1 -- 172.21.15.107:0/1838403487 wait complete. 2024-09-19T04:19:48.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.801+0000 7fa8ba200640 1 Processor -- start 2024-09-19T04:19:48.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.801+0000 7fa8ba200640 1 -- start start 2024-09-19T04:19:48.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.801+0000 7fa8ba200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b4076ba0 0x7fa8b419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.801+0000 7fa8ba200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b4077df0 0x7fa8b419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.801+0000 7fa8ba200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4078790 0x7fa8b41a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.801+0000 7fa8ba200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b4113cf0 con 0x7fa8b4078790 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.801+0000 7fa8ba200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b4113b70 con 0x7fa8b4077df0 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.801+0000 7fa8ba200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b4113e70 con 0x7fa8b4076ba0 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.802+0000 7fa8b9800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4078790 0x7fa8b41a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.802+0000 7fa8b8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b4076ba0 0x7fa8b419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.802+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b4077df0 0x7fa8b419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.802+0000 7fa8b8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b4076ba0 0x7fa8b419e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37384/0 (socket says 172.21.15.107:37384) 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.802+0000 7fa8b8e00640 1 -- 172.21.15.107:0/1257299380 learned_addr learned my addr 172.21.15.107:0/1257299380 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.802+0000 7fa8b8e00640 1 -- 172.21.15.107:0/1257299380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b4077df0 msgr2=0x7fa8b419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.802+0000 7fa8b8e00640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b4077df0 0x7fa8b419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.802+0000 7fa8b8e00640 1 -- 172.21.15.107:0/1257299380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4078790 msgr2=0x7fa8b41a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.802+0000 7fa8b8e00640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4078790 0x7fa8b41a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.802+0000 7fa8b8e00640 1 -- 172.21.15.107:0/1257299380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b41a5f50 con 0x7fa8b4076ba0 2024-09-19T04:19:48.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.803+0000 7fa8b8e00640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b4076ba0 0x7fa8b419e320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009230 tx=0x7fa8a80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.803+0000 7fa8b1600640 1 -- 172.21.15.107:0/1257299380 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a803f030 con 0x7fa8b4076ba0 2024-09-19T04:19:48.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.803+0000 7fa8ba200640 1 -- 172.21.15.107:0/1257299380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b41a61e0 con 0x7fa8b4076ba0 2024-09-19T04:19:48.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.803+0000 7fa8b1600640 1 -- 172.21.15.107:0/1257299380 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a8003ce0 con 0x7fa8b4076ba0 2024-09-19T04:19:48.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.803+0000 7fa8b1600640 1 -- 172.21.15.107:0/1257299380 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8035070 con 0x7fa8b4076ba0 2024-09-19T04:19:48.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.803+0000 7fa8ba200640 1 -- 172.21.15.107:0/1257299380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b41a6710 con 0x7fa8b4076ba0 2024-09-19T04:19:48.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.804+0000 7fa8ba200640 1 -- 172.21.15.107:0/1257299380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b4110720 con 0x7fa8b4076ba0 2024-09-19T04:19:48.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.806+0000 7fa8b1600640 1 -- 172.21.15.107:0/1257299380 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa8a8042030 con 0x7fa8b4076ba0 2024-09-19T04:19:48.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.807+0000 7fa8b1600640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa89c0a3380 0x7fa89c0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.807+0000 7fa8b1600640 1 -- 172.21.15.107:0/1257299380 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fa8a80f0250 con 0x7fa8b4076ba0 2024-09-19T04:19:48.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.808+0000 7fa8b3e00640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa89c0a3380 0x7fa89c0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.808+0000 7fa8b3e00640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa89c0a3380 0x7fa89c0a5840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa8b419f840 tx=0x7fa8a0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.810+0000 7fa8b1600640 1 -- 172.21.15.107:0/1257299380 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8a80bc790 con 0x7fa8b4076ba0 2024-09-19T04:19:48.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:48.995+0000 7fa8ba200640 1 -- 172.21.15.107:0/1257299380 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fa8b4062630 con 0x7fa89c0a3380 2024-09-19T04:19:49.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.001+0000 7fa8b1600640 1 -- 172.21.15.107:0/1257299380 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fa8b4062630 con 0x7fa89c0a3380 2024-09-19T04:19:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T04:19:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:mds.all 12/12 38s ago 2m smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-09-19T04:19:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:mgr 3/3 38s ago 7m smithi107=x;smithi115=y;smithi145=z;count:3 2024-09-19T04:19:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:mon 3/3 38s ago 8m smithi107:172.21.15.107=a;smithi115:172.21.15.115=b;smithi145:172.21.15.145=c;count:3 2024-09-19T04:19:49.003 INFO:teuthology.orchestra.run.smithi107.stdout:osd 12 38s ago - 2024-09-19T04:19:49.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.003+0000 7fa8ba200640 1 -- 172.21.15.107:0/1257299380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa89c0a3380 msgr2=0x7fa89c0a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.003+0000 7fa8ba200640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa89c0a3380 0x7fa89c0a5840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa8b419f840 tx=0x7fa8a0005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.003+0000 7fa8ba200640 1 -- 172.21.15.107:0/1257299380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b4076ba0 msgr2=0x7fa8b419e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.003+0000 7fa8ba200640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b4076ba0 0x7fa8b419e320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009230 tx=0x7fa8a80042f0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.003+0000 7fa8ba200640 1 -- 172.21.15.107:0/1257299380 shutdown_connections 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.003+0000 7fa8ba200640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fa89c0a3380 0x7fa89c0a5840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.003+0000 7fa8ba200640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4078790 0x7fa8b41a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.003+0000 7fa8ba200640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b4077df0 0x7fa8b419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.003+0000 7fa8ba200640 1 --2- 172.21.15.107:0/1257299380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b4076ba0 0x7fa8b419e320 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.003+0000 7fa8ba200640 1 -- 172.21.15.107:0/1257299380 >> 172.21.15.107:0/1257299380 conn(0x7fa8b40ff3e0 msgr2=0x7fa8b4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.004+0000 7fa8ba200640 1 -- 172.21.15.107:0/1257299380 shutdown_connections 2024-09-19T04:19:49.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:49.004+0000 7fa8ba200640 1 -- 172.21.15.107:0/1257299380 wait complete. 2024-09-19T04:19:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:48 smithi115 ceph-mon[35477]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:48 smithi115 ceph-mon[35477]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:48 smithi145 ceph-mon[35096]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:48 smithi145 ceph-mon[35096]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:48 smithi107 ceph-mon[30659]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:48 smithi107 ceph-mon[30659]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:49.730 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T04:19:50.102 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:50.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.649+0000 7ff398c00640 1 -- 172.21.15.107:0/1648320516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff394101a40 msgr2=0x7ff394101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.649+0000 7ff398c00640 1 --2- 172.21.15.107:0/1648320516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff394101a40 0x7ff394101e40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff384009900 tx=0x7ff38402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.649+0000 7ff398c00640 1 -- 172.21.15.107:0/1648320516 shutdown_connections 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.649+0000 7ff398c00640 1 --2- 172.21.15.107:0/1648320516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394103600 0x7ff394109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.649+0000 7ff398c00640 1 --2- 172.21.15.107:0/1648320516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff394102c40 0x7ff3941030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.649+0000 7ff398c00640 1 --2- 172.21.15.107:0/1648320516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff394101a40 0x7ff394101e40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.649+0000 7ff398c00640 1 -- 172.21.15.107:0/1648320516 >> 172.21.15.107:0/1648320516 conn(0x7ff3940fd1f0 msgr2=0x7ff3940ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.650+0000 7ff398c00640 1 -- 172.21.15.107:0/1648320516 shutdown_connections 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.650+0000 7ff398c00640 1 -- 172.21.15.107:0/1648320516 wait complete. 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.650+0000 7ff398c00640 1 Processor -- start 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff398c00640 1 -- start start 2024-09-19T04:19:50.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff398c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff394101a40 0x7ff394075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff398c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394102c40 0x7ff3940745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff398c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff394103600 0x7ff394074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff398c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410d6c0 con 0x7ff394103600 2024-09-19T04:19:50.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff398c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410d540 con 0x7ff394102c40 2024-09-19T04:19:50.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff398c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410d840 con 0x7ff394101a40 2024-09-19T04:19:50.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff393400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff394101a40 0x7ff394075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff393e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff394103600 0x7ff394074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff393400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff394101a40 0x7ff394075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37192/0 (socket says 172.21.15.107:37192) 2024-09-19T04:19:50.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff393400640 1 -- 172.21.15.107:0/84248986 learned_addr learned my addr 172.21.15.107:0/84248986 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:50.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.651+0000 7ff392a00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394102c40 0x7ff3940745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff393e00640 1 -- 172.21.15.107:0/84248986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff394101a40 msgr2=0x7ff394075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff393e00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff394101a40 0x7ff394075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff393e00640 1 -- 172.21.15.107:0/84248986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394102c40 msgr2=0x7ff3940745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff393e00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394102c40 0x7ff3940745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff393e00640 1 -- 172.21.15.107:0/84248986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3940753e0 con 0x7ff394103600 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff393e00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff394103600 0x7ff394074b20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009fd0 tx=0x7ff38c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff38be00640 1 -- 172.21.15.107:0/84248986 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00ce20 con 0x7ff394103600 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff38be00640 1 -- 172.21.15.107:0/84248986 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38c005020 con 0x7ff394103600 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff398c00640 1 -- 172.21.15.107:0/84248986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3941a1ed0 con 0x7ff394103600 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff38be00640 1 -- 172.21.15.107:0/84248986 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c010720 con 0x7ff394103600 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.652+0000 7ff398c00640 1 -- 172.21.15.107:0/84248986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3941a2330 con 0x7ff394103600 2024-09-19T04:19:50.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.654+0000 7ff398c00640 1 -- 172.21.15.107:0/84248986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff394108de0 con 0x7ff394103600 2024-09-19T04:19:50.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.655+0000 7ff38be00640 1 -- 172.21.15.107:0/84248986 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff38c005540 con 0x7ff394103600 2024-09-19T04:19:50.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.657+0000 7ff38be00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3780a32d0 0x7ff3780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.657+0000 7ff38be00640 1 -- 172.21.15.107:0/84248986 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7ff38c0d0190 con 0x7ff394103600 2024-09-19T04:19:50.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.657+0000 7ff393400640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3780a32d0 0x7ff3780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.658+0000 7ff393400640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3780a32d0 0x7ff3780a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff384000c00 tx=0x7ff3840046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.660+0000 7ff38be00640 1 -- 172.21.15.107:0/84248986 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff38c09cd20 con 0x7ff394103600 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.842+0000 7ff398c00640 1 -- 172.21.15.107:0/84248986 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7ff394062630 con 0x7ff3780a32d0 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.844+0000 7ff38be00640 1 -- 172.21.15.107:0/84248986 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7ff394062630 con 0x7ff3780a32d0 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi107.stdout:HOST ADDR LABELS STATUS 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 172.21.15.107 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi107.stdout:smithi115 172.21.15.115 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi107.stdout:smithi145 172.21.15.145 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi107.stdout:3 hosts in cluster 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.847+0000 7ff398c00640 1 -- 172.21.15.107:0/84248986 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3780a32d0 msgr2=0x7ff3780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.847+0000 7ff398c00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3780a32d0 0x7ff3780a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff384000c00 tx=0x7ff3840046c0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.847+0000 7ff398c00640 1 -- 172.21.15.107:0/84248986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff394103600 msgr2=0x7ff394074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.847+0000 7ff398c00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff394103600 0x7ff394074b20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009fd0 tx=0x7ff38c00ef10 comp rx=0 tx=0).stop 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.847+0000 7ff398c00640 1 -- 172.21.15.107:0/84248986 shutdown_connections 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.847+0000 7ff398c00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff3780a32d0 0x7ff3780a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.847+0000 7ff398c00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff394103600 0x7ff394074b20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.847+0000 7ff398c00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394102c40 0x7ff3940745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.847+0000 7ff398c00640 1 --2- 172.21.15.107:0/84248986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff394101a40 0x7ff394075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.847+0000 7ff398c00640 1 -- 172.21.15.107:0/84248986 >> 172.21.15.107:0/84248986 conn(0x7ff3940fd1f0 msgr2=0x7ff394105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.848+0000 7ff398c00640 1 -- 172.21.15.107:0/84248986 shutdown_connections 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:50.848+0000 7ff398c00640 1 -- 172.21.15.107:0/84248986 wait complete. 2024-09-19T04:19:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:50 smithi115 ceph-mon[35477]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:50 smithi115 ceph-mon[35477]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:51.197 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:50 smithi107 ceph-mon[30659]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:51.197 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:50 smithi107 ceph-mon[30659]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:50 smithi145 ceph-mon[35096]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:50 smithi145 ceph-mon[35096]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:19:51.684 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T04:19:52.056 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:52.195 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:51 smithi107 ceph-mon[30659]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:52.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:51 smithi115 ceph-mon[35477]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:52.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:51 smithi145 ceph-mon[35096]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:52.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.582+0000 7efdeda00640 1 -- 172.21.15.107:0/2024208666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde8074c50 msgr2=0x7efde80750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:52.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.582+0000 7efdeda00640 1 --2- 172.21.15.107:0/2024208666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde8074c50 0x7efde80750b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009900 tx=0x7efdd002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:19:52.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.582+0000 7efdeda00640 1 -- 172.21.15.107:0/2024208666 shutdown_connections 2024-09-19T04:19:52.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.582+0000 7efdeda00640 1 --2- 172.21.15.107:0/2024208666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efde810e1e0 0x7efde81105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:52.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.582+0000 7efdeda00640 1 --2- 172.21.15.107:0/2024208666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde8074c50 0x7efde80750b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.582+0000 7efdeda00640 1 --2- 172.21.15.107:0/2024208666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efde8076220 0x7efde8074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.582+0000 7efdeda00640 1 -- 172.21.15.107:0/2024208666 >> 172.21.15.107:0/2024208666 conn(0x7efde80fd390 msgr2=0x7efde80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.582+0000 7efdeda00640 1 -- 172.21.15.107:0/2024208666 shutdown_connections 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.582+0000 7efdeda00640 1 -- 172.21.15.107:0/2024208666 wait complete. 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.583+0000 7efdeda00640 1 Processor -- start 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.583+0000 7efdeda00640 1 -- start start 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efdeda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efde8074c50 0x7efde819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efdeda00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde8076220 0x7efde819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efdeda00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efde810e1e0 0x7efde81a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efdeda00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efde8113ea0 con 0x7efde8076220 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efdeda00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efde8113d20 con 0x7efde810e1e0 2024-09-19T04:19:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efdeda00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efde8114020 con 0x7efde8074c50 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efde6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efde8074c50 0x7efde819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efdde000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde8076220 0x7efde819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efde6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efde8074c50 0x7efde819e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37206/0 (socket says 172.21.15.107:37206) 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efde6a00640 1 -- 172.21.15.107:0/4190277166 learned_addr learned my addr 172.21.15.107:0/4190277166 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efde7400640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efde810e1e0 0x7efde81a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efde7400640 1 -- 172.21.15.107:0/4190277166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efde8074c50 msgr2=0x7efde819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efde7400640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efde8074c50 0x7efde819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efde7400640 1 -- 172.21.15.107:0/4190277166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde8076220 msgr2=0x7efde819ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efde7400640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde8076220 0x7efde819ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.584+0000 7efde7400640 1 -- 172.21.15.107:0/4190277166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde81a6180 con 0x7efde810e1e0 2024-09-19T04:19:52.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.585+0000 7efde6a00640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efde8074c50 0x7efde819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:52.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.585+0000 7efde7400640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efde810e1e0 0x7efde81a5b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efde0009fd0 tx=0x7efde000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:52.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.585+0000 7efddfe00640 1 -- 172.21.15.107:0/4190277166 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde000ce20 con 0x7efde810e1e0 2024-09-19T04:19:52.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.585+0000 7efdeda00640 1 -- 172.21.15.107:0/4190277166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde81a6470 con 0x7efde810e1e0 2024-09-19T04:19:52.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.585+0000 7efddfe00640 1 -- 172.21.15.107:0/4190277166 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efde0005020 con 0x7efde810e1e0 2024-09-19T04:19:52.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.585+0000 7efddfe00640 1 -- 172.21.15.107:0/4190277166 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0010720 con 0x7efde810e1e0 2024-09-19T04:19:52.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.585+0000 7efdeda00640 1 -- 172.21.15.107:0/4190277166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde81a6980 con 0x7efde810e1e0 2024-09-19T04:19:52.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.586+0000 7efdeda00640 1 -- 172.21.15.107:0/4190277166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde8103b80 con 0x7efde810e1e0 2024-09-19T04:19:52.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.588+0000 7efddfe00640 1 -- 172.21.15.107:0/4190277166 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7efde0005540 con 0x7efde810e1e0 2024-09-19T04:19:52.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.590+0000 7efddfe00640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7efdcc0a3300 0x7efdcc0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:52.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.590+0000 7efddfe00640 1 -- 172.21.15.107:0/4190277166 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7efde00d02f0 con 0x7efde810e1e0 2024-09-19T04:19:52.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.590+0000 7efde6a00640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7efdcc0a3300 0x7efdcc0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:52.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.590+0000 7efde6a00640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7efdcc0a3300 0x7efdcc0a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005d00 tx=0x7efdd800b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:52.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.592+0000 7efddfe00640 1 -- 172.21.15.107:0/4190277166 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efde009ce80 con 0x7efde810e1e0 2024-09-19T04:19:52.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.775+0000 7efdeda00640 1 -- 172.21.15.107:0/4190277166 --> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7efde8062630 con 0x7efdcc0a3300 2024-09-19T04:19:52.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.777+0000 7efddfe00640 1 -- 172.21.15.107:0/4190277166 <== mgr.14146 v2:172.21.15.107:6800/3991722475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7efde8062630 con 0x7efdcc0a3300 2024-09-19T04:19:52.779 INFO:teuthology.orchestra.run.smithi107.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T04:19:52.779 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T04:19:52.779 INFO:teuthology.orchestra.run.smithi107.stdout:smithi115 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UU400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T04:19:52.779 INFO:teuthology.orchestra.run.smithi107.stdout:smithi145 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N9400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T04:19:52.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.779+0000 7efdeda00640 1 -- 172.21.15.107:0/4190277166 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7efdcc0a3300 msgr2=0x7efdcc0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:52.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.779+0000 7efdeda00640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7efdcc0a3300 0x7efdcc0a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005d00 tx=0x7efdd800b410 comp rx=0 tx=0).stop 2024-09-19T04:19:52.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.779+0000 7efdeda00640 1 -- 172.21.15.107:0/4190277166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efde810e1e0 msgr2=0x7efde81a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:52.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.779+0000 7efdeda00640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efde810e1e0 0x7efde81a5b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efde0009fd0 tx=0x7efde000ef10 comp rx=0 tx=0).stop 2024-09-19T04:19:52.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.779+0000 7efdeda00640 1 -- 172.21.15.107:0/4190277166 shutdown_connections 2024-09-19T04:19:52.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.779+0000 7efdeda00640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7efdcc0a3300 0x7efdcc0a57c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:52.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.779+0000 7efdeda00640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efde810e1e0 0x7efde81a5b10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:52.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.779+0000 7efdeda00640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde8076220 0x7efde819ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:52.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.779+0000 7efdeda00640 1 --2- 172.21.15.107:0/4190277166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efde8074c50 0x7efde819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:52.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.780+0000 7efdeda00640 1 -- 172.21.15.107:0/4190277166 >> 172.21.15.107:0/4190277166 conn(0x7efde80fd390 msgr2=0x7efde80ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:52.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.780+0000 7efdeda00640 1 -- 172.21.15.107:0/4190277166 shutdown_connections 2024-09-19T04:19:52.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:52.780+0000 7efdeda00640 1 -- 172.21.15.107:0/4190277166 wait complete. 2024-09-19T04:19:53.149 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:52 smithi107 ceph-mon[30659]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:52 smithi115 ceph-mon[35477]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:52 smithi145 ceph-mon[35096]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:53.652 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T04:19:53.662 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-09-19T04:19:53.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T04:19:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:54.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:53 smithi107 ceph-mon[30659]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:53 smithi115 ceph-mon[35477]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:54.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:53 smithi145 ceph-mon[35096]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:19:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.628+0000 7f6bbce00640 1 -- 172.21.15.107:0/1174129572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb8102e00 msgr2=0x7f6bb8103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.628+0000 7f6bbce00640 1 --2- 172.21.15.107:0/1174129572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb8102e00 0x7f6bb8103280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009900 tx=0x7f6ba002a400 comp rx=0 tx=0).stop 2024-09-19T04:19:54.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.628+0000 7f6bbce00640 1 -- 172.21.15.107:0/1174129572 shutdown_connections 2024-09-19T04:19:54.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.629+0000 7f6bbce00640 1 --2- 172.21.15.107:0/1174129572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bb81037c0 0x7f6bb810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.629+0000 7f6bbce00640 1 --2- 172.21.15.107:0/1174129572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb8102e00 0x7f6bb8103280 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.629+0000 7f6bbce00640 1 --2- 172.21.15.107:0/1174129572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bb8101c00 0x7f6bb8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.629+0000 7f6bbce00640 1 -- 172.21.15.107:0/1174129572 >> 172.21.15.107:0/1174129572 conn(0x7f6bb80fd390 msgr2=0x7f6bb80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:54.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.629+0000 7f6bbce00640 1 -- 172.21.15.107:0/1174129572 shutdown_connections 2024-09-19T04:19:54.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.629+0000 7f6bbce00640 1 -- 172.21.15.107:0/1174129572 wait complete. 2024-09-19T04:19:54.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.630+0000 7f6bbce00640 1 Processor -- start 2024-09-19T04:19:54.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.630+0000 7f6bbce00640 1 -- start start 2024-09-19T04:19:54.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bbce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bb8101c00 0x7f6bb8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:54.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bbce00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bb8102e00 0x7f6bb80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:54.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bbce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb81037c0 0x7f6bb8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:54.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bbce00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb810d8b0 con 0x7f6bb81037c0 2024-09-19T04:19:54.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bbce00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb810d730 con 0x7f6bb8102e00 2024-09-19T04:19:54.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bbce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb810da30 con 0x7f6bb8101c00 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bb5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bb8101c00 0x7f6bb8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bb5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bb8101c00 0x7f6bb8075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37218/0 (socket says 172.21.15.107:37218) 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bb5e00640 1 -- 172.21.15.107:0/1657320195 learned_addr learned my addr 172.21.15.107:0/1657320195 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bafe00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bb8102e00 0x7f6bb80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bb6800640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb81037c0 0x7f6bb8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bafe00640 1 -- 172.21.15.107:0/1657320195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bb8101c00 msgr2=0x7f6bb8075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bafe00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bb8101c00 0x7f6bb8075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bafe00640 1 -- 172.21.15.107:0/1657320195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb81037c0 msgr2=0x7f6bb8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bafe00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb81037c0 0x7f6bb8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bafe00640 1 -- 172.21.15.107:0/1657320195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb80753e0 con 0x7f6bb8102e00 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.631+0000 7f6bb5e00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bb8101c00 0x7f6bb8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:54.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.632+0000 7f6bafe00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bb8102e00 0x7f6bb80745e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ba002a920 tx=0x7f6ba0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:54.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.632+0000 7f6baea00640 1 -- 172.21.15.107:0/1657320195 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba003f030 con 0x7f6bb8102e00 2024-09-19T04:19:54.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.632+0000 7f6bbce00640 1 -- 172.21.15.107:0/1657320195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb81a2000 con 0x7f6bb8102e00 2024-09-19T04:19:54.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.632+0000 7f6bbce00640 1 -- 172.21.15.107:0/1657320195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb81a2500 con 0x7f6bb8102e00 2024-09-19T04:19:54.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.632+0000 7f6baea00640 1 -- 172.21.15.107:0/1657320195 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba0003ce0 con 0x7f6bb8102e00 2024-09-19T04:19:54.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.632+0000 7f6baea00640 1 -- 172.21.15.107:0/1657320195 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0049060 con 0x7f6bb8102e00 2024-09-19T04:19:54.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.634+0000 7f6bbce00640 1 -- 172.21.15.107:0/1657320195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb8102000 con 0x7f6bb8102e00 2024-09-19T04:19:54.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.635+0000 7f6baea00640 1 -- 172.21.15.107:0/1657320195 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6ba0042030 con 0x7f6bb8102e00 2024-09-19T04:19:54.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.637+0000 7f6baea00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f6b9c0a32d0 0x7f6b9c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:54.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.638+0000 7f6baea00640 1 -- 172.21.15.107:0/1657320195 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f6ba00efe00 con 0x7f6bb8102e00 2024-09-19T04:19:54.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.638+0000 7f6bb5e00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f6b9c0a32d0 0x7f6b9c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:54.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.638+0000 7f6bb5e00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f6b9c0a32d0 0x7f6b9c0a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80058c0 tx=0x7f6ba8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:54.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.639+0000 7f6baea00640 1 -- 172.21.15.107:0/1657320195 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ba00bc340 con 0x7f6bb8102e00 2024-09-19T04:19:54.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.852+0000 7f6bbce00640 1 -- 172.21.15.107:0/1657320195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6bb8108f80 con 0x7f6bb8102e00 2024-09-19T04:19:54.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.853+0000 7f6baea00640 1 -- 172.21.15.107:0/1657320195 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3141 (secure 0 0 0) 0x7f6ba00c0640 con 0x7f6bb8102e00 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout:e27 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout:btime 2024-09-19T04:18:25:571089+0000 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 27 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T04:19:54.855 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-09-19T04:18:13.349314+0000 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-09-19T04:18:25.571069+0000 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 300 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 5 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1,2,3,4 2024-09-19T04:19:54.856 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24539,1=24517,2=24500,3=24496,4=24490} 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 0 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 24539 members: 24517,24539,24500,24496,24490 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b{0:24539} state up:active seq 4 addr [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.e{0:24473} state up:standby-replay seq 1 addr [v2:172.21.15.115:6834/687464127,v1:172.21.15.115:6835/687464127] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.i{1:24517} state up:active seq 9 addr [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.857 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.l{1:24442} state up:standby-replay seq 1 addr [v2:172.21.15.145:6832/2248561847,v1:172.21.15.145:6833/2248561847] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.h{2:24500} state up:active seq 10 addr [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.k{2:24437} state up:standby-replay seq 1 addr [v2:172.21.15.115:6832/1661283120,v1:172.21.15.115:6833/1661283120] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c{3:24496} state up:active seq 11 addr [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a{3:14511} state up:standby-replay seq 1 addr [v2:172.21.15.107:6840/305932896,v1:172.21.15.107:6841/305932896] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.f{4:24490} state up:active seq 12 addr [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.g{4:14499} state up:standby-replay seq 1 addr [v2:172.21.15.107:6838/2243096646,v1:172.21.15.107:6839/2243096646] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.j{-1:14451} state up:standby seq 1 addr [v2:172.21.15.107:6834/3986650541,v1:172.21.15.107:6835/3986650541] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.858 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d{-1:14463} state up:standby seq 1 addr [v2:172.21.15.107:6836/2692745433,v1:172.21.15.107:6837/2692745433] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:19:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-09-19T04:19:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.855+0000 7f6bbce00640 1 -- 172.21.15.107:0/1657320195 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f6b9c0a32d0 msgr2=0x7f6b9c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.855+0000 7f6bbce00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f6b9c0a32d0 0x7f6b9c0a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80058c0 tx=0x7f6ba8005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.855+0000 7f6bbce00640 1 -- 172.21.15.107:0/1657320195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bb8102e00 msgr2=0x7f6bb80745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.855+0000 7f6bbce00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bb8102e00 0x7f6bb80745e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ba002a920 tx=0x7f6ba0032ce0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.855+0000 7f6bbce00640 1 -- 172.21.15.107:0/1657320195 shutdown_connections 2024-09-19T04:19:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.855+0000 7f6bbce00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f6b9c0a32d0 0x7f6b9c0a5790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.855+0000 7f6bbce00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb81037c0 0x7f6bb8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.855+0000 7f6bbce00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bb8102e00 0x7f6bb80745e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.855+0000 7f6bbce00640 1 --2- 172.21.15.107:0/1657320195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bb8101c00 0x7f6bb8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.855+0000 7f6bbce00640 1 -- 172.21.15.107:0/1657320195 >> 172.21.15.107:0/1657320195 conn(0x7f6bb80fd390 msgr2=0x7f6bb8106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.856+0000 7f6bbce00640 1 -- 172.21.15.107:0/1657320195 shutdown_connections 2024-09-19T04:19:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:54.856+0000 7f6bbce00640 1 -- 172.21.15.107:0/1657320195 wait complete. 2024-09-19T04:19:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:54 smithi115 ceph-mon[35477]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:54 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1657320195' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:19:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:54 smithi145 ceph-mon[35096]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:54 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1657320195' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:19:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:54 smithi107 ceph-mon[30659]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T04:19:55.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:54 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1657320195' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:19:55.702 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.629+0000 7fbc27a00640 1 -- 172.21.15.107:0/4209923358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc20105920 msgr2=0x7fbc2010c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.629+0000 7fbc27a00640 1 --2- 172.21.15.107:0/4209923358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc20105920 0x7fbc2010c240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009ca0 tx=0x7fbc1802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.629+0000 7fbc27a00640 1 -- 172.21.15.107:0/4209923358 shutdown_connections 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.629+0000 7fbc27a00640 1 --2- 172.21.15.107:0/4209923358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc20105920 0x7fbc2010c240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.629+0000 7fbc27a00640 1 --2- 172.21.15.107:0/4209923358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc20104f60 0x7fbc201053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.629+0000 7fbc27a00640 1 --2- 172.21.15.107:0/4209923358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc20103e00 0x7fbc20104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.629+0000 7fbc27a00640 1 -- 172.21.15.107:0/4209923358 >> 172.21.15.107:0/4209923358 conn(0x7fbc200ff5b0 msgr2=0x7fbc201019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.629+0000 7fbc27a00640 1 -- 172.21.15.107:0/4209923358 shutdown_connections 2024-09-19T04:19:56.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.629+0000 7fbc27a00640 1 -- 172.21.15.107:0/4209923358 wait complete. 2024-09-19T04:19:56.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.630+0000 7fbc27a00640 1 Processor -- start 2024-09-19T04:19:56.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.630+0000 7fbc27a00640 1 -- start start 2024-09-19T04:19:56.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc27a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc20103e00 0x7fbc2019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc27a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc20104f60 0x7fbc2019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc27a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc20105920 0x7fbc201a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc27a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2010fb00 con 0x7fbc20104f60 2024-09-19T04:19:56.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc27a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2010f980 con 0x7fbc20105920 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc27a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2010fc80 con 0x7fbc20103e00 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc17e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc20104f60 0x7fbc2019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc17e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc20104f60 0x7fbc2019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37320/0 (socket says 172.21.15.107:37320) 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc17e00640 1 -- 172.21.15.107:0/2680706126 learned_addr learned my addr 172.21.15.107:0/2680706126 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc25a00640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc20105920 0x7fbc201a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc25000640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc20103e00 0x7fbc2019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc17e00640 1 -- 172.21.15.107:0/2680706126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc20103e00 msgr2=0x7fbc2019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc17e00640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc20103e00 0x7fbc2019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc17e00640 1 -- 172.21.15.107:0/2680706126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc20105920 msgr2=0x7fbc201a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc17e00640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc20105920 0x7fbc201a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.631+0000 7fbc17e00640 1 -- 172.21.15.107:0/2680706126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc201a1c90 con 0x7fbc20104f60 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.632+0000 7fbc17e00640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc20104f60 0x7fbc2019a5a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800e9c0 tx=0x7fbc0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.632+0000 7fbc1e000640 1 -- 172.21.15.107:0/2680706126 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800cd80 con 0x7fbc20104f60 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.632+0000 7fbc1e000640 1 -- 172.21.15.107:0/2680706126 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc080035f0 con 0x7fbc20104f60 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.632+0000 7fbc1e000640 1 -- 172.21.15.107:0/2680706126 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08010690 con 0x7fbc20104f60 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.632+0000 7fbc27a00640 1 -- 172.21.15.107:0/2680706126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc201a1f80 con 0x7fbc20104f60 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.632+0000 7fbc27a00640 1 -- 172.21.15.107:0/2680706126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc201a23e0 con 0x7fbc20104f60 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.633+0000 7fbc27a00640 1 -- 172.21.15.107:0/2680706126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2010b0c0 con 0x7fbc20104f60 2024-09-19T04:19:56.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.635+0000 7fbc1e000640 1 -- 172.21.15.107:0/2680706126 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbc080037a0 con 0x7fbc20104f60 2024-09-19T04:19:56.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.637+0000 7fbc1e000640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbc040a3300 0x7fbc040a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.637+0000 7fbc1e000640 1 -- 172.21.15.107:0/2680706126 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fbc080cfde0 con 0x7fbc20104f60 2024-09-19T04:19:56.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.637+0000 7fbc25000640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbc040a3300 0x7fbc040a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.637+0000 7fbc25000640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbc040a3300 0x7fbc040a57c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009bd0 tx=0x7fbc10005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.640+0000 7fbc1e000640 1 -- 172.21.15.107:0/2680706126 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbc0809c970 con 0x7fbc20104f60 2024-09-19T04:19:56.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.817+0000 7fbc27a00640 1 -- 172.21.15.107:0/2680706126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fbc2002da10 con 0x7fbc20104f60 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.818+0000 7fbc1e000640 1 -- 172.21.15.107:0/2680706126 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5283 (secure 0 0 0) 0x7fbc0809c270 con 0x7fbc20104f60 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 83 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi107.stdout:fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-09-19T04:10:05.820654+0000 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-09-19T04:19:00.968552+0000 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi107.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi107.stdout:crush_version 26 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:full_ratio 0.95 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:backfillfull_ratio 0.9 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:nearfull_ratio 0.85 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:require_min_compat_client luminous 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:min_compat_client luminous 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:require_osd_release squid 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:stretch_mode_enabled false 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:max_osd 12 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6802/1501799384,v1:172.21.15.107:6803/1501799384] [v2:172.21.15.107:6804/1501799384,v1:172.21.15.107:6805/1501799384] exists,up 9218bb1e-b075-4d2f-9533-da54ea586a54 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/3371389240,v1:172.21.15.115:6801/3371389240] [v2:172.21.15.115:6802/3371389240,v1:172.21.15.115:6803/3371389240] exists,up 4bed3c76-ab42-4406-9064-a558f56ee785 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6800/2439686508,v1:172.21.15.145:6801/2439686508] [v2:172.21.15.145:6802/2439686508,v1:172.21.15.145:6803/2439686508] exists,up 9bfdcafa-f7f4-4f78-af4b-af1a16e24be7 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 up in weight 1 up_from 24 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6810/3212660448,v1:172.21.15.107:6811/3212660448] [v2:172.21.15.107:6812/3212660448,v1:172.21.15.107:6813/3212660448] exists,up 7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 up in weight 1 up_from 31 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/1123876846,v1:172.21.15.115:6809/1123876846] [v2:172.21.15.115:6810/1123876846,v1:172.21.15.115:6811/1123876846] exists,up 86388823-3a04-4923-8ece-d0e955be08e1 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 up in weight 1 up_from 36 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6808/1693312722,v1:172.21.15.145:6809/1693312722] [v2:172.21.15.145:6810/1693312722,v1:172.21.15.145:6811/1693312722] exists,up 47753716-2254-426c-b279-28ca4388d41f 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 up in weight 1 up_from 43 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6818/2695066565,v1:172.21.15.107:6819/2695066565] [v2:172.21.15.107:6820/2695066565,v1:172.21.15.107:6821/2695066565] exists,up 23e84324-77d0-460a-ae79-96ce61c8d0c0 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 up in weight 1 up_from 48 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/278000330,v1:172.21.15.115:6817/278000330] [v2:172.21.15.115:6818/278000330,v1:172.21.15.115:6819/278000330] exists,up eb0c2813-44ea-4a41-9a51-28e0e327a400 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 up in weight 1 up_from 55 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6816/2759275478,v1:172.21.15.145:6817/2759275478] [v2:172.21.15.145:6818/2759275478,v1:172.21.15.145:6819/2759275478] exists,up 8262ec4a-c805-449a-9e9c-b55b5b36ffed 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 up in weight 1 up_from 62 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6826/3784218099,v1:172.21.15.107:6827/3784218099] [v2:172.21.15.107:6828/3784218099,v1:172.21.15.107:6829/3784218099] exists,up 8277cde7-50c0-4a5b-9283-24f3813de2c5 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 up in weight 1 up_from 68 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/442507474,v1:172.21.15.115:6825/442507474] [v2:172.21.15.115:6826/442507474,v1:172.21.15.115:6827/442507474] exists,up 3df95b38-68a9-426e-83e4-7283c68a2ead 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 up in weight 1 up_from 75 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6824/399346726,v1:172.21.15.145:6825/399346726] [v2:172.21.15.145:6826/399346726,v1:172.21.15.145:6827/399346726] exists,up 5fa0931c-684b-448d-8d63-2bd10881d88f 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.a [2,11] 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.16 [2,4] 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.1d [8,11] 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.1 [4,9] 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.24 [4,9] 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.28 [5,8] 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.2c [4,9] 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/498138874 expires 2024-09-20T04:10:59.721218+0000 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/972711045 expires 2024-09-20T04:10:59.721218+0000 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/972711045 expires 2024-09-20T04:10:59.721218+0000 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/3237606700 expires 2024-09-20T04:10:59.721218+0000 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/1762826901 expires 2024-09-20T04:10:32.750566+0000 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/1481502377 expires 2024-09-20T04:10:59.721218+0000 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/2057620332 expires 2024-09-20T04:10:32.750566+0000 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/2057620332 expires 2024-09-20T04:10:32.750566+0000 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/1606203977 expires 2024-09-20T04:10:32.750566+0000 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/2144535101 expires 2024-09-20T04:10:32.750566+0000 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 -- 172.21.15.107:0/2680706126 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbc040a3300 msgr2=0x7fbc040a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbc040a3300 0x7fbc040a57c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009bd0 tx=0x7fbc10005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 -- 172.21.15.107:0/2680706126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc20104f60 msgr2=0x7fbc2019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc20104f60 0x7fbc2019a5a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800e9c0 tx=0x7fbc0800ee80 comp rx=0 tx=0).stop 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 -- 172.21.15.107:0/2680706126 shutdown_connections 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fbc040a3300 0x7fbc040a57c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc20105920 0x7fbc201a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc20104f60 0x7fbc2019a5a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 --2- 172.21.15.107:0/2680706126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc20103e00 0x7fbc2019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 -- 172.21.15.107:0/2680706126 >> 172.21.15.107:0/2680706126 conn(0x7fbc200ff5b0 msgr2=0x7fbc20108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.820+0000 7fbc27a00640 1 -- 172.21.15.107:0/2680706126 shutdown_connections 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:56.821+0000 7fbc27a00640 1 -- 172.21.15.107:0/2680706126 wait complete. 2024-09-19T04:19:57.152 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:56 smithi107 ceph-mon[30659]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:57.152 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:56 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2680706126' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:19:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:56 smithi115 ceph-mon[35477]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:56 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2680706126' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:19:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:56 smithi145 ceph-mon[35096]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:19:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:56 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2680706126' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:19:57.542 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T04:19:57.554 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:19:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.507+0000 7f9e2a800640 1 -- 172.21.15.107:0/1361775097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e24108f60 msgr2=0x7f9e24110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.507+0000 7f9e2a800640 1 --2- 172.21.15.107:0/1361775097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e24108f60 0x7f9e24110ae0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009ca0 tx=0x7f9e1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.507+0000 7f9e2a800640 1 -- 172.21.15.107:0/1361775097 shutdown_connections 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.507+0000 7f9e2a800640 1 --2- 172.21.15.107:0/1361775097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e24108f60 0x7f9e24110ae0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.507+0000 7f9e2a800640 1 --2- 172.21.15.107:0/1361775097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e241047e0 0x7f9e24108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.507+0000 7f9e2a800640 1 --2- 172.21.15.107:0/1361775097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e24103e30 0x7f9e24104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.507+0000 7f9e2a800640 1 -- 172.21.15.107:0/1361775097 >> 172.21.15.107:0/1361775097 conn(0x7f9e240ff8d0 msgr2=0x7f9e24101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.507+0000 7f9e2a800640 1 -- 172.21.15.107:0/1361775097 shutdown_connections 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.507+0000 7f9e2a800640 1 -- 172.21.15.107:0/1361775097 wait complete. 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.508+0000 7f9e2a800640 1 Processor -- start 2024-09-19T04:19:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e2a800640 1 -- start start 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e2a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e24103e30 0x7f9e2419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e2a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e241047e0 0x7f9e2419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e2a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e24108f60 0x7f9e2419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e2a800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e24114010 con 0x7f9e241047e0 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e2a800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e24113e90 con 0x7f9e24108f60 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e2a800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e24114190 con 0x7f9e24103e30 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e28a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e241047e0 0x7f9e2419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e28a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e241047e0 0x7f9e2419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37334/0 (socket says 172.21.15.107:37334) 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e28a00640 1 -- 172.21.15.107:0/2464948340 learned_addr learned my addr 172.21.15.107:0/2464948340 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e29400640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e24103e30 0x7f9e2419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.509+0000 7f9e29e00640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e24108f60 0x7f9e2419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e28a00640 1 -- 172.21.15.107:0/2464948340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e24103e30 msgr2=0x7f9e2419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e28a00640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e24103e30 0x7f9e2419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e28a00640 1 -- 172.21.15.107:0/2464948340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e24108f60 msgr2=0x7f9e2419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e28a00640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e24108f60 0x7f9e2419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e28a00640 1 -- 172.21.15.107:0/2464948340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2419f3c0 con 0x7f9e241047e0 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e29400640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e24103e30 0x7f9e2419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e28a00640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e241047e0 0x7f9e2419a950 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000e9f0 tx=0x7f9e1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e22000640 1 -- 172.21.15.107:0/2464948340 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000cd80 con 0x7f9e241047e0 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e22000640 1 -- 172.21.15.107:0/2464948340 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e10010910 con 0x7f9e241047e0 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e2a800640 1 -- 172.21.15.107:0/2464948340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2419f6b0 con 0x7f9e241047e0 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e22000640 1 -- 172.21.15.107:0/2464948340 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10018990 con 0x7f9e241047e0 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.510+0000 7f9e2a800640 1 -- 172.21.15.107:0/2464948340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e241a6f90 con 0x7f9e241047e0 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.512+0000 7f9e2a800640 1 -- 172.21.15.107:0/2464948340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e24104210 con 0x7f9e241047e0 2024-09-19T04:19:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.513+0000 7f9e22000640 1 -- 172.21.15.107:0/2464948340 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9e10010ac0 con 0x7f9e241047e0 2024-09-19T04:19:58.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.514+0000 7f9e22000640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9e0c0a32d0 0x7f9e0c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:58.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.515+0000 7f9e22000640 1 -- 172.21.15.107:0/2464948340 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f9e100cfd00 con 0x7f9e241047e0 2024-09-19T04:19:58.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.515+0000 7f9e29400640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9e0c0a32d0 0x7f9e0c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:58.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.515+0000 7f9e29400640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9e0c0a32d0 0x7f9e0c0a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009fc0 tx=0x7f9e18005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:58.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.518+0000 7f9e22000640 1 -- 172.21.15.107:0/2464948340 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9e1009c2a0 con 0x7f9e241047e0 2024-09-19T04:19:58.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.729+0000 7f9e2a800640 1 -- 172.21.15.107:0/2464948340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e241a71c0 con 0x7f9e241047e0 2024-09-19T04:19:58.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.729+0000 7f9e22000640 1 -- 172.21.15.107:0/2464948340 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11992 (secure 0 0 0) 0x7f9e100a05a0 con 0x7f9e241047e0 2024-09-19T04:19:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:19:58.732 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"btime":"2024-09-19T04:18:25:571089+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:18:25.571069+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:19:58.732 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-09-19T04:19:58.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.732+0000 7f9e2a800640 1 -- 172.21.15.107:0/2464948340 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9e0c0a32d0 msgr2=0x7f9e0c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:58.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.732+0000 7f9e2a800640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9e0c0a32d0 0x7f9e0c0a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009fc0 tx=0x7f9e18005f90 comp rx=0 tx=0).stop 2024-09-19T04:19:58.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.732+0000 7f9e2a800640 1 -- 172.21.15.107:0/2464948340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e241047e0 msgr2=0x7f9e2419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:58.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.732+0000 7f9e2a800640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e241047e0 0x7f9e2419a950 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000e9f0 tx=0x7f9e1000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.733+0000 7f9e2a800640 1 -- 172.21.15.107:0/2464948340 shutdown_connections 2024-09-19T04:19:58.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.733+0000 7f9e2a800640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9e0c0a32d0 0x7f9e0c0a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.733+0000 7f9e2a800640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e24108f60 0x7f9e2419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.733+0000 7f9e2a800640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e241047e0 0x7f9e2419a950 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.733+0000 7f9e2a800640 1 --2- 172.21.15.107:0/2464948340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e24103e30 0x7f9e2419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:58.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.733+0000 7f9e2a800640 1 -- 172.21.15.107:0/2464948340 >> 172.21.15.107:0/2464948340 conn(0x7f9e240ff8d0 msgr2=0x7f9e24100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:58.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.733+0000 7f9e2a800640 1 -- 172.21.15.107:0/2464948340 shutdown_connections 2024-09-19T04:19:58.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:19:58.733+0000 7f9e2a800640 1 -- 172.21.15.107:0/2464948340 wait complete. 2024-09-19T04:19:59.022 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:58 smithi107 ceph-mon[30659]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:59.022 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:19:58 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2464948340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:58 smithi115 ceph-mon[35477]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:19:58 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2464948340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:58 smithi145 ceph-mon[35096]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:19:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:19:58 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2464948340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:19:59.503 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:19:59.901 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.434+0000 7f0b70c00640 1 -- 172.21.15.107:0/1236665949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c074f40 msgr2=0x7f0b6c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.434+0000 7f0b70c00640 1 --2- 172.21.15.107:0/1236665949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c074f40 0x7f0b6c10de80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009900 tx=0x7f0b5802a480 comp rx=0 tx=0).stop 2024-09-19T04:20:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.434+0000 7f0b70c00640 1 -- 172.21.15.107:0/1236665949 shutdown_connections 2024-09-19T04:20:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.434+0000 7f0b70c00640 1 --2- 172.21.15.107:0/1236665949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b6c10e3c0 0x7f0b6c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.434+0000 7f0b70c00640 1 --2- 172.21.15.107:0/1236665949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c074f40 0x7f0b6c10de80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.434+0000 7f0b70c00640 1 --2- 172.21.15.107:0/1236665949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b6c074620 0x7f0b6c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.434+0000 7f0b70c00640 1 -- 172.21.15.107:0/1236665949 >> 172.21.15.107:0/1236665949 conn(0x7f0b6c0fd8c0 msgr2=0x7f0b6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.434+0000 7f0b70c00640 1 -- 172.21.15.107:0/1236665949 shutdown_connections 2024-09-19T04:20:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.434+0000 7f0b70c00640 1 -- 172.21.15.107:0/1236665949 wait complete. 2024-09-19T04:20:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b70c00640 1 Processor -- start 2024-09-19T04:20:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b70c00640 1 -- start start 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b70c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c074620 0x7f0b6c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b70c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b6c074f40 0x7f0b6c1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b70c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b6c10e3c0 0x7f0b6c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b70c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c114160 con 0x7f0b6c074620 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b70c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c113fe0 con 0x7f0b6c10e3c0 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b70c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c1142e0 con 0x7f0b6c074f40 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b69e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c074620 0x7f0b6c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b69e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c074620 0x7f0b6c1a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47752/0 (socket says 172.21.15.107:47752) 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.435+0000 7f0b69e00640 1 -- 172.21.15.107:0/661724746 learned_addr learned my addr 172.21.15.107:0/661724746 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b69400640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b6c074f40 0x7f0b6c1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b6a800640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b6c10e3c0 0x7f0b6c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b69e00640 1 -- 172.21.15.107:0/661724746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b6c074f40 msgr2=0x7f0b6c1a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b69e00640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b6c074f40 0x7f0b6c1a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b69e00640 1 -- 172.21.15.107:0/661724746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b6c10e3c0 msgr2=0x7f0b6c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b69e00640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b6c10e3c0 0x7f0b6c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b69e00640 1 -- 172.21.15.107:0/661724746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c19acd0 con 0x7f0b6c074620 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b69400640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b6c074f40 0x7f0b6c1a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b69e00640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c074620 0x7f0b6c1a03e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00d770 tx=0x7f0b5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b62a00640 1 -- 172.21.15.107:0/661724746 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c017070 con 0x7f0b6c074620 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b62a00640 1 -- 172.21.15.107:0/661724746 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b5c005500 con 0x7f0b6c074620 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b62a00640 1 -- 172.21.15.107:0/661724746 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c004e00 con 0x7f0b6c074620 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.436+0000 7f0b70c00640 1 -- 172.21.15.107:0/661724746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c19afc0 con 0x7f0b6c074620 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.437+0000 7f0b70c00640 1 -- 172.21.15.107:0/661724746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c1a72b0 con 0x7f0b6c074620 2024-09-19T04:20:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.437+0000 7f0b70c00640 1 -- 172.21.15.107:0/661724746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6c074a00 con 0x7f0b6c074620 2024-09-19T04:20:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.439+0000 7f0b62a00640 1 -- 172.21.15.107:0/661724746 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f0b5c004fa0 con 0x7f0b6c074620 2024-09-19T04:20:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.441+0000 7f0b62a00640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0b500a32c0 0x7f0b500a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.441+0000 7f0b62a00640 1 -- 172.21.15.107:0/661724746 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f0b5c0cf310 con 0x7f0b6c074620 2024-09-19T04:20:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.441+0000 7f0b69400640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0b500a32c0 0x7f0b500a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.442+0000 7f0b69400640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0b500a32c0 0x7f0b500a5780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c19b6e0 tx=0x7f0b58005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:00.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.445+0000 7f0b62a00640 1 -- 172.21.15.107:0/661724746 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b5c09c000 con 0x7f0b6c074620 2024-09-19T04:20:00.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.654+0000 7f0b70c00640 1 -- 172.21.15.107:0/661724746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b6c02d7c0 con 0x7f0b6c074620 2024-09-19T04:20:00.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.655+0000 7f0b62a00640 1 -- 172.21.15.107:0/661724746 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11992 (secure 0 0 0) 0x7f0b5c0afe20 con 0x7f0b6c074620 2024-09-19T04:20:00.657 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:20:00.657 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"btime":"2024-09-19T04:18:25:571089+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:18:25.571069+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:20:00.658 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-09-19T04:20:00.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.657+0000 7f0b70c00640 1 -- 172.21.15.107:0/661724746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0b500a32c0 msgr2=0x7f0b500a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:00.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.657+0000 7f0b70c00640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0b500a32c0 0x7f0b500a5780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c19b6e0 tx=0x7f0b58005a20 comp rx=0 tx=0).stop 2024-09-19T04:20:00.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.657+0000 7f0b70c00640 1 -- 172.21.15.107:0/661724746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c074620 msgr2=0x7f0b6c1a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:00.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.657+0000 7f0b70c00640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c074620 0x7f0b6c1a03e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00d770 tx=0x7f0b5c00dc30 comp rx=0 tx=0).stop 2024-09-19T04:20:00.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.658+0000 7f0b70c00640 1 -- 172.21.15.107:0/661724746 shutdown_connections 2024-09-19T04:20:00.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.658+0000 7f0b70c00640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f0b500a32c0 0x7f0b500a5780 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:00.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.658+0000 7f0b70c00640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b6c10e3c0 0x7f0b6c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:00.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.658+0000 7f0b70c00640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b6c074f40 0x7f0b6c1a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:00.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.658+0000 7f0b70c00640 1 --2- 172.21.15.107:0/661724746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c074620 0x7f0b6c1a03e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:00.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.658+0000 7f0b70c00640 1 -- 172.21.15.107:0/661724746 >> 172.21.15.107:0/661724746 conn(0x7f0b6c0fd8c0 msgr2=0x7f0b6c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:00.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.658+0000 7f0b70c00640 1 -- 172.21.15.107:0/661724746 shutdown_connections 2024-09-19T04:20:00.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:00.658+0000 7f0b70c00640 1 -- 172.21.15.107:0/661724746 wait complete. 2024-09-19T04:20:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:00 smithi115 ceph-mon[35477]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:20:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:00 smithi115 ceph-mon[35477]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T04:20:01.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:00 smithi115 ceph-mon[35477]: overall HEALTH_OK 2024-09-19T04:20:01.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:00 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/661724746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:00 smithi145 ceph-mon[35096]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:20:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:00 smithi145 ceph-mon[35096]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T04:20:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:00 smithi145 ceph-mon[35096]: overall HEALTH_OK 2024-09-19T04:20:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:00 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/661724746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:00 smithi107 ceph-mon[30659]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T04:20:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:00 smithi107 ceph-mon[30659]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T04:20:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:00 smithi107 ceph-mon[30659]: overall HEALTH_OK 2024-09-19T04:20:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:00 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/661724746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:01.587 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:20:01.961 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.511+0000 7ff7af200640 1 -- 172.21.15.107:0/1190362950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01047e0 msgr2=0x7ff7b0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.511+0000 7ff7af200640 1 --2- 172.21.15.107:0/1190362950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01047e0 0x7ff7b0108830 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff798009900 tx=0x7ff79802a410 comp rx=0 tx=0).stop 2024-09-19T04:20:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.511+0000 7ff7af200640 1 -- 172.21.15.107:0/1190362950 shutdown_connections 2024-09-19T04:20:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.511+0000 7ff7af200640 1 --2- 172.21.15.107:0/1190362950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7b0108f60 0x7ff7b0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.511+0000 7ff7af200640 1 --2- 172.21.15.107:0/1190362950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01047e0 0x7ff7b0108830 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.511+0000 7ff7af200640 1 --2- 172.21.15.107:0/1190362950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7b0103e30 0x7ff7b0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.511+0000 7ff7af200640 1 -- 172.21.15.107:0/1190362950 >> 172.21.15.107:0/1190362950 conn(0x7ff7b00ff8d0 msgr2=0x7ff7b0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.512+0000 7ff7af200640 1 -- 172.21.15.107:0/1190362950 shutdown_connections 2024-09-19T04:20:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.512+0000 7ff7af200640 1 -- 172.21.15.107:0/1190362950 wait complete. 2024-09-19T04:20:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.513+0000 7ff7af200640 1 Processor -- start 2024-09-19T04:20:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.513+0000 7ff7af200640 1 -- start start 2024-09-19T04:20:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7af200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7b0103e30 0x7ff7b019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7af200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01047e0 0x7ff7b019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7af200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7b0108f60 0x7ff7b019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7af200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0113fc0 con 0x7ff7b01047e0 2024-09-19T04:20:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7af200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0113e40 con 0x7ff7b0103e30 2024-09-19T04:20:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7af200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0114140 con 0x7ff7b0108f60 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7a5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01047e0 0x7ff7b019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7a5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01047e0 0x7ff7b019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47768/0 (socket says 172.21.15.107:47768) 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7a5400640 1 -- 172.21.15.107:0/3309081262 learned_addr learned my addr 172.21.15.107:0/3309081262 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7ae800640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7b0108f60 0x7ff7b019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7ade00640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7b0103e30 0x7ff7b019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7a5400640 1 -- 172.21.15.107:0/3309081262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7b0108f60 msgr2=0x7ff7b019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7a5400640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7b0108f60 0x7ff7b019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7a5400640 1 -- 172.21.15.107:0/3309081262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7b0103e30 msgr2=0x7ff7b019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7a5400640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7b0103e30 0x7ff7b019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.514+0000 7ff7a5400640 1 -- 172.21.15.107:0/3309081262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b019f450 con 0x7ff7b01047e0 2024-09-19T04:20:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.515+0000 7ff7a5400640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01047e0 0x7ff7b019a9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff798006010 tx=0x7ff7980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.515+0000 7ff7a7400640 1 -- 172.21.15.107:0/3309081262 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79803f030 con 0x7ff7b01047e0 2024-09-19T04:20:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.515+0000 7ff7af200640 1 -- 172.21.15.107:0/3309081262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b019f6e0 con 0x7ff7b01047e0 2024-09-19T04:20:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.515+0000 7ff7a7400640 1 -- 172.21.15.107:0/3309081262 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff798003ce0 con 0x7ff7b01047e0 2024-09-19T04:20:02.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.515+0000 7ff7a7400640 1 -- 172.21.15.107:0/3309081262 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798035070 con 0x7ff7b01047e0 2024-09-19T04:20:02.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.515+0000 7ff7af200640 1 -- 172.21.15.107:0/3309081262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b01a7020 con 0x7ff7b01047e0 2024-09-19T04:20:02.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.516+0000 7ff7af200640 1 -- 172.21.15.107:0/3309081262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b0104210 con 0x7ff7b01047e0 2024-09-19T04:20:02.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.518+0000 7ff7a7400640 1 -- 172.21.15.107:0/3309081262 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7ff798042030 con 0x7ff7b01047e0 2024-09-19T04:20:02.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.519+0000 7ff7a7400640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7940a32c0 0x7ff7940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:02.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.519+0000 7ff7a7400640 1 -- 172.21.15.107:0/3309081262 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7ff7980efdf0 con 0x7ff7b01047e0 2024-09-19T04:20:02.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.519+0000 7ff7ade00640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7940a32c0 0x7ff7940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:02.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.520+0000 7ff7ade00640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7940a32c0 0x7ff7940a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009bd0 tx=0x7ff7a0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:02.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.523+0000 7ff7a7400640 1 -- 172.21.15.107:0/3309081262 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff7980bc550 con 0x7ff7b01047e0 2024-09-19T04:20:02.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.698+0000 7ff7af200640 1 -- 172.21.15.107:0/3309081262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7b010c200 con 0x7ff7b01047e0 2024-09-19T04:20:02.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.700+0000 7ff7a7400640 1 -- 172.21.15.107:0/3309081262 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22735 (secure 0 0 0) 0x7ff7980c0850 con 0x7ff7b01047e0 2024-09-19T04:20:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:20:02.702 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":83,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:19:00.968552+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:20:02.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.702+0000 7ff7af200640 1 -- 172.21.15.107:0/3309081262 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7940a32c0 msgr2=0x7ff7940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:02.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.702+0000 7ff7af200640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7940a32c0 0x7ff7940a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009bd0 tx=0x7ff7a0005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:02.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.702+0000 7ff7af200640 1 -- 172.21.15.107:0/3309081262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01047e0 msgr2=0x7ff7b019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:02.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.702+0000 7ff7af200640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01047e0 0x7ff7b019a9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff798006010 tx=0x7ff7980042f0 comp rx=0 tx=0).stop 2024-09-19T04:20:02.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.703+0000 7ff7af200640 1 -- 172.21.15.107:0/3309081262 shutdown_connections 2024-09-19T04:20:02.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.703+0000 7ff7af200640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7ff7940a32c0 0x7ff7940a5780 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:02.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.703+0000 7ff7af200640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7b0108f60 0x7ff7b019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:02.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.703+0000 7ff7af200640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01047e0 0x7ff7b019a9e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:02.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.703+0000 7ff7af200640 1 --2- 172.21.15.107:0/3309081262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7b0103e30 0x7ff7b019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:02.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.703+0000 7ff7af200640 1 -- 172.21.15.107:0/3309081262 >> 172.21.15.107:0/3309081262 conn(0x7ff7b00ff8d0 msgr2=0x7ff7b0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:02.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.703+0000 7ff7af200640 1 -- 172.21.15.107:0/3309081262 shutdown_connections 2024-09-19T04:20:02.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:02.703+0000 7ff7af200640 1 -- 172.21.15.107:0/3309081262 wait complete. 2024-09-19T04:20:02.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:02 smithi107 ceph-mon[30659]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T04:20:02.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:02 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3309081262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:20:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:02 smithi115 ceph-mon[35477]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T04:20:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:02 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3309081262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:20:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:02 smithi145 ceph-mon[35096]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T04:20:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:02 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3309081262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:20:03.489 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T04:18:13.349314+0000', 'modified': '2024-09-19T04:18:25.571069+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24517, 'mds_2': 24500, 'mds_3': 24496, 'mds_4': 24490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14499': {'gid': 14499, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6839/2243096646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 2243096646}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 2243096646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14511': {'gid': 14511, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6841/305932896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6840', 'nonce': 305932896}, {'type': 'v1', 'addr': '172.21.15.107:6841', 'nonce': 305932896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6833/1661283120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1661283120}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1661283120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6833/2248561847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6832', 'nonce': 2248561847}, {'type': 'v1', 'addr': '172.21.15.145:6833', 'nonce': 2248561847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/687464127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 687464127}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 687464127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.145:6835/4237714429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 4237714429}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 4237714429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.145:6837/1756017901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 1756017901}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 1756017901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.115:6837/191835918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 191835918}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 191835918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.145:6839/4220252383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 4220252383}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 4220252383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.115:6839/2907975805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 2907975805}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 2907975805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24517, 24539, 24500, 24496, 24490]} 2024-09-19T04:20:03.489 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T04:20:03.489 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T04:20:03.857 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.421+0000 7fdfc9200640 1 -- 172.21.15.107:0/1891111704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc4112a50 msgr2=0x7fdfc4114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.421+0000 7fdfc9200640 1 --2- 172.21.15.107:0/1891111704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc4112a50 0x7fdfc4114e40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009ca0 tx=0x7fdfbc02a8b0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.421+0000 7fdfc9200640 1 -- 172.21.15.107:0/1891111704 shutdown_connections 2024-09-19T04:20:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.421+0000 7fdfc9200640 1 --2- 172.21.15.107:0/1891111704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc4112a50 0x7fdfc4114e40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.421+0000 7fdfc9200640 1 --2- 172.21.15.107:0/1891111704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc4077040 0x7fdfc4112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.421+0000 7fdfc9200640 1 --2- 172.21.15.107:0/1891111704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfc4076720 0x7fdfc4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.421+0000 7fdfc9200640 1 -- 172.21.15.107:0/1891111704 >> 172.21.15.107:0/1891111704 conn(0x7fdfc40ffa50 msgr2=0x7fdfc4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:04.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.421+0000 7fdfc9200640 1 -- 172.21.15.107:0/1891111704 shutdown_connections 2024-09-19T04:20:04.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.421+0000 7fdfc9200640 1 -- 172.21.15.107:0/1891111704 wait complete. 2024-09-19T04:20:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.422+0000 7fdfc9200640 1 Processor -- start 2024-09-19T04:20:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc9200640 1 -- start start 2024-09-19T04:20:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc9200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfc4076720 0x7fdfc41a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc4077040 0x7fdfc41a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc9200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc4112a50 0x7fdfc419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc9200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc41185e0 con 0x7fdfc4077040 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc9200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4118460 con 0x7fdfc4112a50 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc9200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4118760 con 0x7fdfc4076720 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc4077040 0x7fdfc41a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc4077040 0x7fdfc41a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47792/0 (socket says 172.21.15.107:47792) 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc1400640 1 -- 172.21.15.107:0/2450069125 learned_addr learned my addr 172.21.15.107:0/2450069125 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc1e00640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfc4076720 0x7fdfc41a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.423+0000 7fdfc2800640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc4112a50 0x7fdfc419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfc1400640 1 -- 172.21.15.107:0/2450069125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfc4076720 msgr2=0x7fdfc41a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfc1400640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfc4076720 0x7fdfc41a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfc1400640 1 -- 172.21.15.107:0/2450069125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc4112a50 msgr2=0x7fdfc419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfc1400640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc4112a50 0x7fdfc419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfc1400640 1 -- 172.21.15.107:0/2450069125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc419f020 con 0x7fdfc4077040 2024-09-19T04:20:04.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfc1400640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc4077040 0x7fdfc41a4c70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000d950 tx=0x7fdfb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:04.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfbaa00640 1 -- 172.21.15.107:0/2450069125 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0017070 con 0x7fdfc4077040 2024-09-19T04:20:04.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfbaa00640 1 -- 172.21.15.107:0/2450069125 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb0009c30 con 0x7fdfc4077040 2024-09-19T04:20:04.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfc9200640 1 -- 172.21.15.107:0/2450069125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc419f310 con 0x7fdfc4077040 2024-09-19T04:20:04.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfbaa00640 1 -- 172.21.15.107:0/2450069125 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb000f6b0 con 0x7fdfc4077040 2024-09-19T04:20:04.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.424+0000 7fdfc9200640 1 -- 172.21.15.107:0/2450069125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc41ab600 con 0x7fdfc4077040 2024-09-19T04:20:04.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.427+0000 7fdfcbc0f740 1 Processor -- start 2024-09-19T04:20:04.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfcbc0f740 1 -- start start 2024-09-19T04:20:04.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfbaa00640 1 -- 172.21.15.107:0/2450069125 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fdfb00109d0 con 0x7fdfc4077040 2024-09-19T04:20:04.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025330 0x56546e025710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 0x56546e02ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e030510 0x56546e032900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56546df8b0b0 con 0x56546e030510 2024-09-19T04:20:04.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56546df32ab0 con 0x56546e025c50 2024-09-19T04:20:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56546df87070 con 0x56546e025330 2024-09-19T04:20:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e030510 0x56546e032900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfc2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025330 0x56546e025710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e030510 0x56546e032900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47806/0 (socket says 172.21.15.107:47806) 2024-09-19T04:20:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfc1400640 1 -- 172.21.15.107:0/453488882 learned_addr learned my addr 172.21.15.107:0/453488882 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.428+0000 7fdfc1e00640 1 --2- 172.21.15.107:0/453488882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 0x56546e02ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.429+0000 7fdfc2800640 1 -- 172.21.15.107:0/453488882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 msgr2=0x56546e02ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.429+0000 7fdfc2800640 1 --2- 172.21.15.107:0/453488882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 0x56546e02ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.429+0000 7fdfc2800640 1 -- 172.21.15.107:0/453488882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e030510 msgr2=0x56546e032900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.429+0000 7fdfc2800640 1 --2- 172.21.15.107:0/453488882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e030510 0x56546e032900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.429+0000 7fdfc2800640 1 -- 172.21.15.107:0/453488882 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56546df3f4a0 con 0x56546e025330 2024-09-19T04:20:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.429+0000 7fdfc2800640 1 --2- 172.21.15.107:0/453488882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025330 0x56546e025710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc02aea0 tx=0x7fdfbc030470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2629025c8410aeae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.429+0000 7fdfaf400640 1 -- 172.21.15.107:0/453488882 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc003b70 con 0x56546e025330 2024-09-19T04:20:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.429+0000 7fdfbaa00640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfa80a3240 0x7fdfa80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfaf400640 1 -- 172.21.15.107:0/453488882 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc036030 con 0x56546e025330 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfaf400640 1 -- 172.21.15.107:0/453488882 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0044e0 con 0x56546e025330 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfbaa00640 1 -- 172.21.15.107:0/2450069125 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fdfb00cf3c0 con 0x7fdfc4077040 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfc1e00640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfa80a3240 0x7fdfa80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/453488882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025330 msgr2=0x56546e025710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/453488882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025330 0x56546e025710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc02aea0 tx=0x7fdfbc030470 comp rx=0 tx=0).stop 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/453488882 shutdown_connections 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/453488882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e030510 0x56546e032900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/453488882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 0x56546e02ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/453488882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025330 0x56546e025710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.430+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/453488882 >> 172.21.15.107:0/453488882 conn(0x56546e0218c0 msgr2=0x56546e021ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.431+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/453488882 shutdown_connections 2024-09-19T04:20:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.431+0000 7fdfc1e00640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfa80a3240 0x7fdfa80a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x56546e022720 tx=0x7fdfb400b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.431+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/453488882 wait complete. 2024-09-19T04:20:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.431+0000 7fdfcbc0f740 1 -- start start 2024-09-19T04:20:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.431+0000 7fdfcbc0f740 1 Processor -- start 2024-09-19T04:20:04.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 0x56546e02dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e030510 0x56546e02e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56546e033060 con 0x56546e025330 2024-09-19T04:20:04.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56546df13920 con 0x56546e025c50 2024-09-19T04:20:04.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56546e0331e0 con 0x56546e030510 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfcbc0f740 10 client.0 _ll_register_callbacks cb 0x56546df378a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfc1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfc1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47812/0 (socket says 172.21.15.107:47812) 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfc1e00640 1 -- 172.21.15.107:0/256224551 learned_addr learned my addr 172.21.15.107:0/256224551 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfc1400640 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 0x56546e02dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfc2800640 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e030510 0x56546e02e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfc1400640 1 -- 172.21.15.107:0/256224551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e030510 msgr2=0x56546e02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfc1400640 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e030510 0x56546e02e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfc1400640 1 -- 172.21.15.107:0/256224551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 msgr2=0x56546e02d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfc1400640 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.432+0000 7fdfc1400640 1 -- 172.21.15.107:0/256224551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56546dcfc2b0 con 0x56546e025c50 2024-09-19T04:20:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.433+0000 7fdfc1400640 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 0x56546e02dae0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009610 tx=0x7fdfb00d2a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f763fa07abede00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.433+0000 7fdfa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.433+0000 7fdfa3e00640 1 -- 172.21.15.107:0/256224551 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0020070 con 0x56546e025c50 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.433+0000 7fdfa3e00640 1 -- 172.21.15.107:0/256224551 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fdfb00860e0 con 0x56546e025c50 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.433+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/256224551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfbc036030 con 0x56546e025c50 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.433+0000 7fdfa3e00640 1 -- 172.21.15.107:0/256224551 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfb009b420 con 0x56546e025c50 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.433+0000 7fdfcbc0f740 10 client.25031 fetch_fsmap learned FSMap version 27 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.433+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/256224551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56546e03eea0 con 0x56546e025c50 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.433+0000 7fdfa3e00640 1 -- 172.21.15.107:0/256224551 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc036030 con 0x56546e025c50 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.434+0000 7fdfa3e00640 1 -- 172.21.15.107:0/256224551 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdfb009af10 con 0x56546e025c50 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.434+0000 7fdfcbc0f740 10 client.25031 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.434+0000 7fdfcbc0f740 10 client.25031 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.434+0000 7fdfcbc0f740 20 client.25031 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:04.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.434+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x56546e041460 0x56546e0428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.434+0000 7fdfcbc0f740 4 client.25031 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.434+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/256224551 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56546df98620 con 0x56546e041460 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.434+0000 7fdfc1e00640 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x56546e041460 0x56546e0428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.435+0000 7fdfc1e00640 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x56546e041460 0x56546e0428d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=67dd6d88550ff418 server_cookie=e9e5184cd7abedaa in_seq=0 out_seq=0 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.435+0000 7fdfa3e00640 10 client.25031 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfa3e00640 1 -- 172.21.15.107:0/256224551 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56546df98620 con 0x56546e041460 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfa3e00640 10 client.25031 handle_command_reply: tid=0 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 client.25031 shutdown 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/256224551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 msgr2=0x56546e02dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 0x56546e02dae0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009610 tx=0x7fdfb00d2a90 comp rx=0 tx=0).stop 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/256224551 shutdown_connections 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x56546e041460 0x56546e0428d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e030510 0x56546e02e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e025c50 0x56546e02dae0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfa3e00640 0 client.25031 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:20:04.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/256224551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/256224551 >> 172.21.15.107:0/256224551 conn(0x56546e0218c0 msgr2=0x56546e0275a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:04.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/256224551 shutdown_connections 2024-09-19T04:20:04.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/256224551 wait complete. 2024-09-19T04:20:04.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.436+0000 7fdfcbc0f740 20 client.25031 trim_cache size 0 max 16384 2024-09-19T04:20:04.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.455+0000 7fdfcbc0f740 1 Processor -- start 2024-09-19T04:20:04.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.455+0000 7fdfcbc0f740 1 -- start start 2024-09-19T04:20:04.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.455+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.455+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 0x56546e02be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.455+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 0x56546e02c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.455+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56546e040c10 con 0x56546e025330 2024-09-19T04:20:04.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.455+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56546e0412e0 con 0x56546e030510 2024-09-19T04:20:04.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.455+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56546e0419b0 con 0x56546e025c50 2024-09-19T04:20:04.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.455+0000 7fdfc1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.455+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 0x56546e02be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47814/0 (socket says 172.21.15.107:47814) 2024-09-19T04:20:04.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc1e00640 1 -- 172.21.15.107:0/3413991200 learned_addr learned my addr 172.21.15.107:0/3413991200 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc2800640 1 --2- 172.21.15.107:0/3413991200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 0x56546e02c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc1400640 1 -- 172.21.15.107:0/3413991200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 msgr2=0x56546e02c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc1400640 1 --2- 172.21.15.107:0/3413991200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 0x56546e02c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc1400640 1 -- 172.21.15.107:0/3413991200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 msgr2=0x56546e02ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc1400640 1 --2- 172.21.15.107:0/3413991200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc1400640 1 -- 172.21.15.107:0/3413991200 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56546e025a80 con 0x56546e025c50 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc2800640 1 --2- 172.21.15.107:0/3413991200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 0x56546e02c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc1e00640 1 --2- 172.21.15.107:0/3413991200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfc1400640 1 --2- 172.21.15.107:0/3413991200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 0x56546e02be70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00d2b00 tx=0x7fdfb0099fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a294fc2f3c39ceb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfaea00640 1 -- 172.21.15.107:0/3413991200 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0020070 con 0x56546e025c50 2024-09-19T04:20:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfaea00640 1 -- 172.21.15.107:0/3413991200 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb0009de0 con 0x56546e025c50 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.456+0000 7fdfaea00640 1 -- 172.21.15.107:0/3413991200 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb009aa70 con 0x56546e025c50 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.457+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/3413991200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 msgr2=0x56546e02be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.457+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/3413991200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 0x56546e02be70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00d2b00 tx=0x7fdfb0099fd0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.457+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/3413991200 shutdown_connections 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.457+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/3413991200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 0x56546e02c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.457+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/3413991200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 0x56546e02be70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.457+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/3413991200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e02ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.457+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/3413991200 >> 172.21.15.107:0/3413991200 conn(0x56546e0218c0 msgr2=0x56546e021ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.457+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/3413991200 shutdown_connections 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.457+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/3413991200 wait complete. 2024-09-19T04:20:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfcbc0f740 1 -- start start 2024-09-19T04:20:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfcbc0f740 1 Processor -- start 2024-09-19T04:20:04.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e04b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 0x56546e02a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfcbc0f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 0x56546e02ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56546e042750 con 0x56546e025330 2024-09-19T04:20:04.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56546e042080 con 0x56546e030510 2024-09-19T04:20:04.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfcbc0f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56546e042e20 con 0x56546e025c50 2024-09-19T04:20:04.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfcbc0f740 10 client.0 _ll_register_callbacks cb 0x7fdfb000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:04.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfc1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e04b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfc1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e04b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47830/0 (socket says 172.21.15.107:47830) 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.458+0000 7fdfc1e00640 1 -- 172.21.15.107:0/2915790741 learned_addr learned my addr 172.21.15.107:0/2915790741 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfc1400640 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 0x56546e02a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfc2800640 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 0x56546e02ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfc1e00640 1 -- 172.21.15.107:0/2915790741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 msgr2=0x56546e02a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfc1e00640 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 0x56546e02a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfc1e00640 1 -- 172.21.15.107:0/2915790741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 msgr2=0x56546e02ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfc1e00640 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 0x56546e02ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfc1e00640 1 -- 172.21.15.107:0/2915790741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56546e02b340 con 0x56546e025330 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfc1e00640 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e04b420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400b620 tx=0x7fdfb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bf6006d57821b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfa3e00640 1 -- 172.21.15.107:0/2915790741 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb402f070 con 0x56546e025330 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfa3e00640 1 -- 172.21.15.107:0/2915790741 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fdfb401ea50 con 0x56546e025330 2024-09-19T04:20:04.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.459+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/2915790741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb0009de0 con 0x56546e025330 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfa3e00640 1 -- 172.21.15.107:0/2915790741 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfb403b050 con 0x56546e025330 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfcbc0f740 10 client.15018 fetch_fsmap learned FSMap version 27 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/2915790741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56546e0abcc0 con 0x56546e025330 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfa3e00640 1 -- 172.21.15.107:0/2915790741 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb0009de0 con 0x56546e025330 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfa3e00640 1 -- 172.21.15.107:0/2915790741 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdfb40114d0 con 0x56546e025330 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfcbc0f740 10 client.15018 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfcbc0f740 10 client.15018 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfcbc0f740 20 client.15018 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x56546e0b08f0 0x56546e0af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfcbc0f740 4 client.15018 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.460+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/2915790741 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "status"}) -- 0x56546df92b30 con 0x56546e0b08f0 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.461+0000 7fdfc1400640 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x56546e0b08f0 0x56546e0af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.461+0000 7fdfc1400640 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x56546e0b08f0 0x56546e0af600 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4ce3961daf9bd27e server_cookie=c2e2291b906c4cd7 in_seq=0 out_seq=0 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.461+0000 7fdfa3e00640 10 client.15018 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:20:04.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.461+0000 7fdfa3e00640 1 -- 172.21.15.107:0/2915790741 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56546df92b30 con 0x56546e0b08f0 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.461+0000 7fdfa3e00640 10 client.15018 handle_command_reply: tid=0 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.461+0000 7fdfcbc0f740 1 client.15018 shutdown 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/2915790741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 msgr2=0x56546e04b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e04b420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400b620 tx=0x7fdfb40046c0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/2915790741 shutdown_connections 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x56546e0b08f0 0x56546e0af600 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e030510 0x56546e02ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56546e025c50 0x56546e02a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 1 --2- 172.21.15.107:0/2915790741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56546e025330 0x56546e04b420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/2915790741 >> 172.21.15.107:0/2915790741 conn(0x56546e0218c0 msgr2=0x56546e02bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfa3e00640 0 client.15018 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/2915790741 shutdown_connections 2024-09-19T04:20:04.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 1 -- 172.21.15.107:0/2915790741 wait complete. 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.462+0000 7fdfcbc0f740 20 client.15018 trim_cache size 0 max 16384 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 4, 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24490, 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 99.894421151000003, 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-09-19T04:20:04.469 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 83, 2024-09-19T04:20:04.470 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T04:20:04.470 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 140.53597262900001 2024-09-19T04:20:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:20:04.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 -- 172.21.15.107:0/2450069125 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfa80a3240 msgr2=0x7fdfa80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:04.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfa80a3240 0x7fdfa80a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x56546e022720 tx=0x7fdfb400b390 comp rx=0 tx=0).stop 2024-09-19T04:20:04.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 -- 172.21.15.107:0/2450069125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc4077040 msgr2=0x7fdfc41a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:04.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc4077040 0x7fdfc41a4c70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000d950 tx=0x7fdfb000de10 comp rx=0 tx=0).stop 2024-09-19T04:20:04.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 -- 172.21.15.107:0/2450069125 shutdown_connections 2024-09-19T04:20:04.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7fdfa80a3240 0x7fdfa80a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc4112a50 0x7fdfc419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc4077040 0x7fdfc41a4c70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 --2- 172.21.15.107:0/2450069125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfc4076720 0x7fdfc41a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:04.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 -- 172.21.15.107:0/2450069125 >> 172.21.15.107:0/2450069125 conn(0x7fdfc40ffa50 msgr2=0x7fdfc4113650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:04.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 -- 172.21.15.107:0/2450069125 shutdown_connections 2024-09-19T04:20:04.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:04.463+0000 7fdfc9200640 1 -- 172.21.15.107:0/2450069125 wait complete. 2024-09-19T04:20:05.152 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:04 smithi107 ceph-mon[30659]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T04:20:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:04 smithi115 ceph-mon[35477]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T04:20:05.204 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T04:20:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:04 smithi145 ceph-mon[35096]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T04:20:05.573 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:06.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.113+0000 7feb7b200640 1 -- 172.21.15.107:0/4266886025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74074f40 msgr2=0x7feb7410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:06.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.113+0000 7feb7b200640 1 --2- 172.21.15.107:0/4266886025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74074f40 0x7feb7410de80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009900 tx=0x7feb5c02a480 comp rx=0 tx=0).stop 2024-09-19T04:20:06.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.113+0000 7feb7b200640 1 -- 172.21.15.107:0/4266886025 shutdown_connections 2024-09-19T04:20:06.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.113+0000 7feb7b200640 1 --2- 172.21.15.107:0/4266886025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb7410e3c0 0x7feb74110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.113+0000 7feb7b200640 1 --2- 172.21.15.107:0/4266886025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74074f40 0x7feb7410de80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.113+0000 7feb7b200640 1 --2- 172.21.15.107:0/4266886025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb74074620 0x7feb74074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.113+0000 7feb7b200640 1 -- 172.21.15.107:0/4266886025 >> 172.21.15.107:0/4266886025 conn(0x7feb740fd8c0 msgr2=0x7feb740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:06.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.113+0000 7feb7b200640 1 -- 172.21.15.107:0/4266886025 shutdown_connections 2024-09-19T04:20:06.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.113+0000 7feb7b200640 1 -- 172.21.15.107:0/4266886025 wait complete. 2024-09-19T04:20:06.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.114+0000 7feb7b200640 1 Processor -- start 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.114+0000 7feb7b200640 1 -- start start 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.114+0000 7feb7b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb74074620 0x7feb7419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.114+0000 7feb7b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb74074f40 0x7feb7419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.114+0000 7feb7b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7410e3c0 0x7feb7419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.114+0000 7feb7b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb74114160 con 0x7feb7410e3c0 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.114+0000 7feb7b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb74113fe0 con 0x7feb74074620 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.114+0000 7feb7b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feb741142e0 con 0x7feb74074f40 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb73e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb74074620 0x7feb7419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb6b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb74074f40 0x7feb7419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb79200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7410e3c0 0x7feb7419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb6b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb74074f40 0x7feb7419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:34158/0 (socket says 172.21.15.107:34158) 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb6b400640 1 -- 172.21.15.107:0/859793080 learned_addr learned my addr 172.21.15.107:0/859793080 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb73e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb74074620 0x7feb7419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47896/0 (socket says 172.21.15.107:47896) 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb73e00640 1 -- 172.21.15.107:0/859793080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb74074f40 msgr2=0x7feb7419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb73e00640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb74074f40 0x7feb7419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb73e00640 1 -- 172.21.15.107:0/859793080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7410e3c0 msgr2=0x7feb7419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb73e00640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7410e3c0 0x7feb7419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb73e00640 1 -- 172.21.15.107:0/859793080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7419f6d0 con 0x7feb74074620 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb79200640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7410e3c0 0x7feb7419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.115+0000 7feb73e00640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb74074620 0x7feb7419a720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb6400e9f0 tx=0x7feb6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.116+0000 7feb71600640 1 -- 172.21.15.107:0/859793080 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6400ccc0 con 0x7feb74074620 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.116+0000 7feb71600640 1 -- 172.21.15.107:0/859793080 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb64005020 con 0x7feb74074620 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.116+0000 7feb7b200640 1 -- 172.21.15.107:0/859793080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7419f9c0 con 0x7feb74074620 2024-09-19T04:20:06.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.116+0000 7feb71600640 1 -- 172.21.15.107:0/859793080 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640049e0 con 0x7feb74074620 2024-09-19T04:20:06.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.116+0000 7feb7b200640 1 -- 172.21.15.107:0/859793080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb741a72a0 con 0x7feb74074620 2024-09-19T04:20:06.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.119+0000 7feb71600640 1 -- 172.21.15.107:0/859793080 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7feb64005560 con 0x7feb74074620 2024-09-19T04:20:06.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.119+0000 7feb7ddf2740 1 Processor -- start 2024-09-19T04:20:06.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.119+0000 7feb7ddf2740 1 -- start start 2024-09-19T04:20:06.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.119+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.119+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159c50 0x559c2a163fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.119+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a164510 0x559c2a166900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559c2a0bf0b0 con 0x559c2a159330 2024-09-19T04:20:06.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c2a066ab0 con 0x559c2a164510 2024-09-19T04:20:06.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559c2a0bb070 con 0x559c2a159c50 2024-09-19T04:20:06.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb6b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a164510 0x559c2a166900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb73e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159c50 0x559c2a163fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb79200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb6b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a164510 0x559c2a166900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47912/0 (socket says 172.21.15.107:47912) 2024-09-19T04:20:06.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb6b400640 1 -- 172.21.15.107:0/2598861141 learned_addr learned my addr 172.21.15.107:0/2598861141 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:06.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb79200640 1 -- 172.21.15.107:0/2598861141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159c50 msgr2=0x559c2a163fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:06.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb79200640 1 --2- 172.21.15.107:0/2598861141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159c50 0x559c2a163fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb79200640 1 -- 172.21.15.107:0/2598861141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a164510 msgr2=0x559c2a166900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:06.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb79200640 1 --2- 172.21.15.107:0/2598861141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a164510 0x559c2a166900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.120+0000 7feb79200640 1 -- 172.21.15.107:0/2598861141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2a0734a0 con 0x559c2a159330 2024-09-19T04:20:06.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.121+0000 7feb71600640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7feb580a32c0 0x7feb580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.121+0000 7feb79200640 1 --2- 172.21.15.107:0/2598861141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a159710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005fd0 tx=0x7feb6c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677072c8713f79ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:06.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.121+0000 7feb71600640 1 -- 172.21.15.107:0/859793080 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7feb640cf860 con 0x7feb74074620 2024-09-19T04:20:06.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.121+0000 7feb73e00640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7feb580a32c0 0x7feb580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.121+0000 7feb69600640 1 -- 172.21.15.107:0/2598861141 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c004d20 con 0x559c2a159330 2024-09-19T04:20:06.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.121+0000 7feb69600640 1 -- 172.21.15.107:0/2598861141 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb6c00f9a0 con 0x559c2a159330 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.121+0000 7feb69600640 1 -- 172.21.15.107:0/2598861141 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0045b0 con 0x559c2a159330 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.122+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2598861141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 msgr2=0x559c2a159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.122+0000 7feb73e00640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7feb580a32c0 0x7feb580a5780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb640a4590 tx=0x7feb640d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.122+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/2598861141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a159710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005fd0 tx=0x7feb6c011bf0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.122+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2598861141 shutdown_connections 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.122+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/2598861141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a164510 0x559c2a166900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.122+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/2598861141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159c50 0x559c2a163fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.122+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/2598861141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a159710 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.122+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2598861141 >> 172.21.15.107:0/2598861141 conn(0x559c2a1558c0 msgr2=0x559c2a155ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.122+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2598861141 shutdown_connections 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.122+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2598861141 wait complete. 2024-09-19T04:20:06.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.123+0000 7feb7ddf2740 1 -- start start 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.123+0000 7feb7ddf2740 1 Processor -- start 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.123+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159330 0x559c2a1615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.123+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159c50 0x559c2a161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.123+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a164510 0x559c2a162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.123+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559c2a167060 con 0x559c2a159c50 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.123+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c2a047920 con 0x559c2a159330 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.123+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559c2a1671e0 con 0x559c2a164510 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.123+0000 7feb7ddf2740 10 client.0 _ll_register_callbacks cb 0x559c2a06b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159c50 0x559c2a161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb79200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159330 0x559c2a1615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159c50 0x559c2a161af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47874/0 (socket says 172.21.15.107:47874) 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb6b400640 1 -- 172.21.15.107:0/969324401 learned_addr learned my addr 172.21.15.107:0/969324401 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:06.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb79200640 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a164510 0x559c2a162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb6b400640 1 -- 172.21.15.107:0/969324401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a164510 msgr2=0x559c2a162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:06.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb6b400640 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a164510 0x559c2a162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb6b400640 1 -- 172.21.15.107:0/969324401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159330 msgr2=0x559c2a1615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:06.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb6b400640 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159330 0x559c2a1615b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb6b400640 1 -- 172.21.15.107:0/969324401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c29e302b0 con 0x559c2a159c50 2024-09-19T04:20:06.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb6b400640 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159c50 0x559c2a161af0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feb5c005ab0 tx=0x7feb5c032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a69407d7e73612 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:06.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.124+0000 7feb62000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:20:06.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.125+0000 7feb62000640 1 -- 172.21.15.107:0/969324401 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c035040 con 0x559c2a159c50 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.125+0000 7feb62000640 1 -- 172.21.15.107:0/969324401 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7feb5c031830 con 0x559c2a159c50 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.125+0000 7feb7ddf2740 1 -- 172.21.15.107:0/969324401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb6c00f9a0 con 0x559c2a159c50 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.125+0000 7feb62000640 1 -- 172.21.15.107:0/969324401 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb5c04d070 con 0x559c2a159c50 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.125+0000 7feb7ddf2740 10 client.15030 fetch_fsmap learned FSMap version 27 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.125+0000 7feb7ddf2740 1 -- 172.21.15.107:0/969324401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c2a172e40 con 0x559c2a159c50 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.125+0000 7feb62000640 1 -- 172.21.15.107:0/969324401 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb6c00f9a0 con 0x559c2a159c50 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.125+0000 7feb62000640 1 -- 172.21.15.107:0/969324401 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feb5c004740 con 0x559c2a159c50 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.126+0000 7feb7ddf2740 10 client.15030 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.126+0000 7feb7ddf2740 10 client.15030 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.126+0000 7feb7ddf2740 20 client.15030 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.126+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559c2a175440 0x559c2a1768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.126+0000 7feb7ddf2740 4 client.15030 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:06.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.126+0000 7feb7ddf2740 1 -- 172.21.15.107:0/969324401 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c2a0cc620 con 0x559c2a175440 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.126+0000 7feb79200640 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559c2a175440 0x559c2a1768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.126+0000 7feb79200640 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559c2a175440 0x559c2a1768b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f09ff1976702823 server_cookie=a79f455af48a566f in_seq=0 out_seq=0 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.126+0000 7feb62000640 10 client.15030 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.127+0000 7feb62000640 1 -- 172.21.15.107:0/969324401 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c2a0cc620 con 0x559c2a175440 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.127+0000 7feb62000640 10 client.15030 handle_command_reply: tid=0 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.127+0000 7feb7ddf2740 1 client.15030 shutdown 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.127+0000 7feb7ddf2740 1 -- 172.21.15.107:0/969324401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159c50 msgr2=0x559c2a161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.127+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159c50 0x559c2a161af0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feb5c005ab0 tx=0x7feb5c032ed0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.128+0000 7feb7ddf2740 1 -- 172.21.15.107:0/969324401 shutdown_connections 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.128+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559c2a175440 0x559c2a1768b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.128+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a164510 0x559c2a162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.128+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159c50 0x559c2a161af0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.128+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/969324401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159330 0x559c2a1615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.128+0000 7feb62000640 0 client.15030 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:20:06.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.128+0000 7feb7ddf2740 1 -- 172.21.15.107:0/969324401 >> 172.21.15.107:0/969324401 conn(0x559c2a1558c0 msgr2=0x559c2a15b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:06.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.128+0000 7feb7ddf2740 1 -- 172.21.15.107:0/969324401 shutdown_connections 2024-09-19T04:20:06.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.128+0000 7feb7ddf2740 1 -- 172.21.15.107:0/969324401 wait complete. 2024-09-19T04:20:06.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.128+0000 7feb7ddf2740 20 client.15030 trim_cache size 0 max 16384 2024-09-19T04:20:06.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.146+0000 7feb7ddf2740 1 Processor -- start 2024-09-19T04:20:06.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.146+0000 7feb7ddf2740 1 -- start start 2024-09-19T04:20:06.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.146+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.146+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159c50 0x559c2a15e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.146+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a164510 0x559c2a15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.146+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559c2a174bf0 con 0x559c2a159330 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.146+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c2a1752c0 con 0x559c2a159c50 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.146+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559c2a175990 con 0x559c2a164510 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb79200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb79200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a161e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47890/0 (socket says 172.21.15.107:47890) 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb79200640 1 -- 172.21.15.107:0/3282782431 learned_addr learned my addr 172.21.15.107:0/3282782431 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb6b400640 1 --2- 172.21.15.107:0/3282782431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159c50 0x559c2a15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb79200640 1 --2- 172.21.15.107:0/3282782431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a164510 0x559c2a15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb79200640 1 -- 172.21.15.107:0/3282782431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a164510 msgr2=0x559c2a15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb79200640 1 --2- 172.21.15.107:0/3282782431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a164510 0x559c2a15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb79200640 1 -- 172.21.15.107:0/3282782431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159c50 msgr2=0x559c2a15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb79200640 1 --2- 172.21.15.107:0/3282782431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159c50 0x559c2a15e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb79200640 1 -- 172.21.15.107:0/3282782431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2a15fb20 con 0x559c2a159330 2024-09-19T04:20:06.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.147+0000 7feb79200640 1 --2- 172.21.15.107:0/3282782431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a161e80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0040c0 tx=0x7feb6c004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890dd966b442a341 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:06.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb68c00640 1 -- 172.21.15.107:0/3282782431 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c00f9a0 con 0x559c2a159330 2024-09-19T04:20:06.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb68c00640 1 -- 172.21.15.107:0/3282782431 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb6c00fe50 con 0x559c2a159330 2024-09-19T04:20:06.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb68c00640 1 -- 172.21.15.107:0/3282782431 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c027770 con 0x559c2a159330 2024-09-19T04:20:06.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb7ddf2740 1 -- 172.21.15.107:0/3282782431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 msgr2=0x559c2a161e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:06.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/3282782431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a161e80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0040c0 tx=0x7feb6c004230 comp rx=0 tx=0).stop 2024-09-19T04:20:06.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb7ddf2740 1 -- 172.21.15.107:0/3282782431 shutdown_connections 2024-09-19T04:20:06.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/3282782431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a164510 0x559c2a15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/3282782431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159c50 0x559c2a15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/3282782431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a159330 0x559c2a161e80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb7ddf2740 1 -- 172.21.15.107:0/3282782431 >> 172.21.15.107:0/3282782431 conn(0x559c2a1558c0 msgr2=0x559c2a15c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.148+0000 7feb7ddf2740 1 -- 172.21.15.107:0/3282782431 shutdown_connections 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.149+0000 7feb7ddf2740 1 -- 172.21.15.107:0/3282782431 wait complete. 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.149+0000 7feb7ddf2740 1 -- start start 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.149+0000 7feb7ddf2740 1 Processor -- start 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159330 0x559c2a17f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159c50 0x559c2a179ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb7ddf2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a164510 0x559c2a17a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559c2a176730 con 0x559c2a164510 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c2a176060 con 0x559c2a159c50 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb7ddf2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559c2a176e00 con 0x559c2a159330 2024-09-19T04:20:06.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb7ddf2740 10 client.0 _ll_register_callbacks cb 0x7feb5c005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb79200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159330 0x559c2a17f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb79200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159330 0x559c2a17f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:34210/0 (socket says 172.21.15.107:34210) 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb79200640 1 -- 172.21.15.107:0/2491705722 learned_addr learned my addr 172.21.15.107:0/2491705722 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.150+0000 7feb79200640 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a164510 0x559c2a17a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb6b400640 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159c50 0x559c2a179ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb79200640 1 -- 172.21.15.107:0/2491705722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159330 msgr2=0x559c2a17f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb79200640 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159330 0x559c2a17f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb79200640 1 -- 172.21.15.107:0/2491705722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159c50 msgr2=0x559c2a179ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb79200640 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159c50 0x559c2a179ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb79200640 1 -- 172.21.15.107:0/2491705722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c2a17a8a0 con 0x559c2a164510 2024-09-19T04:20:06.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb79200640 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a164510 0x559c2a17a020 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009fd0 tx=0x7feb6c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2420db1741cdfe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb62000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb62000640 1 -- 172.21.15.107:0/2491705722 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c030040 con 0x559c2a164510 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb62000640 1 -- 172.21.15.107:0/2491705722 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7feb6c0055b0 con 0x559c2a164510 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.151+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2491705722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb6c00fe50 con 0x559c2a164510 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb62000640 1 -- 172.21.15.107:0/2491705722 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb6c01e040 con 0x559c2a164510 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb7ddf2740 10 client.15036 fetch_fsmap learned FSMap version 27 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2491705722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c2a1e0bc0 con 0x559c2a164510 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb62000640 1 -- 172.21.15.107:0/2491705722 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb6c00fe50 con 0x559c2a164510 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb62000640 1 -- 172.21.15.107:0/2491705722 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feb6c036070 con 0x559c2a164510 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb7ddf2740 10 client.15036 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb7ddf2740 10 client.15036 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:20:06.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb7ddf2740 20 client.15036 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:06.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559c2a1e4fe0 0x559c2a1e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:06.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb7ddf2740 4 client.15036 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:20:06.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.152+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2491705722 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "status"}) -- 0x559c2a0c6b30 con 0x559c2a1e4fe0 2024-09-19T04:20:06.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb6b400640 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559c2a1e4fe0 0x559c2a1e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:06.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb6b400640 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559c2a1e4fe0 0x559c2a1e3cf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ded541e98a6340b4 server_cookie=8e2e9d99f421b9e7 in_seq=0 out_seq=0 2024-09-19T04:20:06.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb62000640 10 client.15036 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:20:06.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb62000640 1 -- 172.21.15.107:0/2491705722 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559c2a0c6b30 con 0x559c2a1e4fe0 2024-09-19T04:20:06.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb62000640 10 client.15036 handle_command_reply: tid=0 2024-09-19T04:20:06.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb7ddf2740 1 client.15036 shutdown 2024-09-19T04:20:06.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2491705722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a164510 msgr2=0x559c2a17a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a164510 0x559c2a17a020 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009fd0 tx=0x7feb6c0040f0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2491705722 shutdown_connections 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559c2a1e4fe0 0x559c2a1e3cf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c2a164510 0x559c2a17a020 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c2a159c50 0x559c2a179ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.153+0000 7feb7ddf2740 1 --2- 172.21.15.107:0/2491705722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c2a159330 0x559c2a17f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.154+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2491705722 >> 172.21.15.107:0/2491705722 conn(0x559c2a1558c0 msgr2=0x559c2a16f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.154+0000 7feb62000640 0 client.15036 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.154+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2491705722 shutdown_connections 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.154+0000 7feb7ddf2740 1 -- 172.21.15.107:0/2491705722 wait complete. 2024-09-19T04:20:06.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.154+0000 7feb7ddf2740 20 client.15036 trim_cache size 0 max 16384 2024-09-19T04:20:06.161 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:20:06.161 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:20:06.161 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 3, 2024-09-19T04:20:06.161 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24496, 2024-09-19T04:20:06.161 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-09-19T04:20:06.161 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-09-19T04:20:06.161 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-09-19T04:20:06.161 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 102.597283977, 2024-09-19T04:20:06.161 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-09-19T04:20:06.161 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 83, 2024-09-19T04:20:06.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T04:20:06.162 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 139.70512715300001 2024-09-19T04:20:06.162 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:20:06.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.154+0000 7feb7b200640 1 -- 172.21.15.107:0/859793080 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7feb580a32c0 msgr2=0x7feb580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:06.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.154+0000 7feb7b200640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7feb580a32c0 0x7feb580a5780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb640a4590 tx=0x7feb640d5530 comp rx=0 tx=0).stop 2024-09-19T04:20:06.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.154+0000 7feb7b200640 1 -- 172.21.15.107:0/859793080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb74074620 msgr2=0x7feb7419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:06.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.155+0000 7feb7b200640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb74074620 0x7feb7419a720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb6400e9f0 tx=0x7feb6400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.155+0000 7feb7b200640 1 -- 172.21.15.107:0/859793080 shutdown_connections 2024-09-19T04:20:06.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.155+0000 7feb7b200640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7feb580a32c0 0x7feb580a5780 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.155+0000 7feb7b200640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7410e3c0 0x7feb7419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.155+0000 7feb7b200640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb74074f40 0x7feb7419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.155+0000 7feb7b200640 1 --2- 172.21.15.107:0/859793080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb74074620 0x7feb7419a720 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:06.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.155+0000 7feb7b200640 1 -- 172.21.15.107:0/859793080 >> 172.21.15.107:0/859793080 conn(0x7feb740fd8c0 msgr2=0x7feb740fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:06.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.155+0000 7feb7b200640 1 -- 172.21.15.107:0/859793080 shutdown_connections 2024-09-19T04:20:06.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:06.155+0000 7feb7b200640 1 -- 172.21.15.107:0/859793080 wait complete. 2024-09-19T04:20:07.050 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T04:20:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:06 smithi115 ceph-mon[35477]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:20:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:06 smithi145 ceph-mon[35096]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:20:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:06 smithi107 ceph-mon[30659]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:20:07.418 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.965+0000 7f80b0800640 1 -- 172.21.15.107:0/1472460147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a8074620 msgr2=0x7f80a8074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.965+0000 7f80b0800640 1 --2- 172.21.15.107:0/1472460147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a8074620 0x7f80a8074a00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f809c009900 tx=0x7f809c02a480 comp rx=0 tx=0).stop 2024-09-19T04:20:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.965+0000 7f80b0800640 1 -- 172.21.15.107:0/1472460147 shutdown_connections 2024-09-19T04:20:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.965+0000 7f80b0800640 1 --2- 172.21.15.107:0/1472460147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a810e3c0 0x7f80a8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.965+0000 7f80b0800640 1 --2- 172.21.15.107:0/1472460147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a8074f40 0x7f80a810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.965+0000 7f80b0800640 1 --2- 172.21.15.107:0/1472460147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a8074620 0x7f80a8074a00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.965+0000 7f80b0800640 1 -- 172.21.15.107:0/1472460147 >> 172.21.15.107:0/1472460147 conn(0x7f80a80fd8c0 msgr2=0x7f80a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.965+0000 7f80b0800640 1 -- 172.21.15.107:0/1472460147 shutdown_connections 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.965+0000 7f80b0800640 1 -- 172.21.15.107:0/1472460147 wait complete. 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.966+0000 7f80b0800640 1 Processor -- start 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.966+0000 7f80b0800640 1 -- start start 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80b0800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a8074620 0x7f80a819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80b0800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a8074f40 0x7f80a819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80b0800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a810e3c0 0x7f80a819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80b0800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f80a8114150 con 0x7f80a810e3c0 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80b0800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80a8113fd0 con 0x7f80a8074f40 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80b0800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f80a81142d0 con 0x7f80a8074620 2024-09-19T04:20:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a810e3c0 0x7f80a819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80ade00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a8074620 0x7f80a819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a810e3c0 0x7f80a819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47920/0 (socket says 172.21.15.107:47920) 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80ae800640 1 -- 172.21.15.107:0/4279576484 learned_addr learned my addr 172.21.15.107:0/4279576484 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80ad400640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a8074f40 0x7f80a819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80ae800640 1 -- 172.21.15.107:0/4279576484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a8074620 msgr2=0x7f80a819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80ae800640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a8074620 0x7f80a819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80ae800640 1 -- 172.21.15.107:0/4279576484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a8074f40 msgr2=0x7f80a819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80ae800640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a8074f40 0x7f80a819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.967+0000 7f80ae800640 1 -- 172.21.15.107:0/4279576484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a819f6a0 con 0x7f80a810e3c0 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.968+0000 7f80ae800640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a810e3c0 0x7f80a819efc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f80a400eaa0 tx=0x7f80a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.968+0000 7f80a2a00640 1 -- 172.21.15.107:0/4279576484 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a400cdb0 con 0x7f80a810e3c0 2024-09-19T04:20:07.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.968+0000 7f80a2a00640 1 -- 172.21.15.107:0/4279576484 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a4005020 con 0x7f80a810e3c0 2024-09-19T04:20:07.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.968+0000 7f80a2a00640 1 -- 172.21.15.107:0/4279576484 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a40116b0 con 0x7f80a810e3c0 2024-09-19T04:20:07.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.968+0000 7f80b0800640 1 -- 172.21.15.107:0/4279576484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a819f990 con 0x7f80a810e3c0 2024-09-19T04:20:07.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.968+0000 7f80b0800640 1 -- 172.21.15.107:0/4279576484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a81a7270 con 0x7f80a810e3c0 2024-09-19T04:20:07.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.971+0000 7f80b33f9740 1 Processor -- start 2024-09-19T04:20:07.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.971+0000 7f80a2a00640 1 -- 172.21.15.107:0/4279576484 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f80a4005560 con 0x7f80a810e3c0 2024-09-19T04:20:07.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.971+0000 7f80b33f9740 1 -- start start 2024-09-19T04:20:07.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.971+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a3f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a49fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a4a510 0x555d36a4c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80b33f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555d369a50b0 con 0x555d36a3fc50 2024-09-19T04:20:07.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80b33f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555d3694cab0 con 0x555d36a3f330 2024-09-19T04:20:07.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80b33f9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555d369a1070 con 0x555d36a4a510 2024-09-19T04:20:07.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ade00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a49fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ade00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a49fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47932/0 (socket says 172.21.15.107:47932) 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ade00640 1 -- 172.21.15.107:0/1933665127 learned_addr learned my addr 172.21.15.107:0/1933665127 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ad400640 1 --2- 172.21.15.107:0/1933665127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a3f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ad400640 1 --2- 172.21.15.107:0/1933665127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a4a510 0x555d36a4c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ade00640 1 -- 172.21.15.107:0/1933665127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a4a510 msgr2=0x555d36a4c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ade00640 1 --2- 172.21.15.107:0/1933665127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a4a510 0x555d36a4c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ade00640 1 -- 172.21.15.107:0/1933665127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 msgr2=0x555d36a3f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ade00640 1 --2- 172.21.15.107:0/1933665127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a3f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ade00640 1 -- 172.21.15.107:0/1933665127 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d369594a0 con 0x555d36a3fc50 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.972+0000 7f80ad400640 1 --2- 172.21.15.107:0/1933665127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a3f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.973+0000 7f80a2a00640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f80900a32c0 0x7f80900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.973+0000 7f80ade00640 1 --2- 172.21.15.107:0/1933665127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a49fd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f809c009900 tx=0x7f809c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fff47360179362 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:07.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.973+0000 7f80a2a00640 1 -- 172.21.15.107:0/4279576484 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f80a40cfa10 con 0x7f80a810e3c0 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.973+0000 7f80ad400640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f80900a32c0 0x7f80900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.973+0000 7f8097000640 1 -- 172.21.15.107:0/1933665127 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c040070 con 0x555d36a3fc50 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.973+0000 7f8097000640 1 -- 172.21.15.107:0/1933665127 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f809c005020 con 0x555d36a3fc50 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.973+0000 7f8097000640 1 -- 172.21.15.107:0/1933665127 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c035040 con 0x555d36a3fc50 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.974+0000 7f80b33f9740 1 -- 172.21.15.107:0/1933665127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 msgr2=0x555d36a49fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.974+0000 7f80ad400640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f80900a32c0 0x7f80900a5780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x555d36a3cc40 tx=0x7f809800a640 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.974+0000 7f80b33f9740 1 --2- 172.21.15.107:0/1933665127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a49fd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f809c009900 tx=0x7f809c031630 comp rx=0 tx=0).stop 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.974+0000 7f80b33f9740 1 -- 172.21.15.107:0/1933665127 shutdown_connections 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.974+0000 7f80b33f9740 1 --2- 172.21.15.107:0/1933665127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a4a510 0x555d36a4c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.974+0000 7f80b33f9740 1 --2- 172.21.15.107:0/1933665127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a49fd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.974+0000 7f80b33f9740 1 --2- 172.21.15.107:0/1933665127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a3f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.974+0000 7f80b33f9740 1 -- 172.21.15.107:0/1933665127 >> 172.21.15.107:0/1933665127 conn(0x555d36a3b8c0 msgr2=0x555d36a3bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.974+0000 7f80b33f9740 1 -- 172.21.15.107:0/1933665127 shutdown_connections 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.974+0000 7f80b33f9740 1 -- 172.21.15.107:0/1933665127 wait complete. 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.975+0000 7f80b33f9740 1 -- start start 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.975+0000 7f80b33f9740 1 Processor -- start 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a47570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a47ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a4a510 0x555d36a47ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80b33f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555d36a4d060 con 0x555d36a3fc50 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80b33f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555d3692d920 con 0x555d36a3f330 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80b33f9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555d36a4d1e0 con 0x555d36a4a510 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80b33f9740 10 client.0 _ll_register_callbacks cb 0x555d369518a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a47ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ade00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a47570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ade00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a47570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47942/0 (socket says 172.21.15.107:47942) 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ade00640 1 -- 172.21.15.107:0/4230660935 learned_addr learned my addr 172.21.15.107:0/4230660935 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ae800640 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a4a510 0x555d36a47ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ad400640 1 -- 172.21.15.107:0/4230660935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a4a510 msgr2=0x555d36a47ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ad400640 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a4a510 0x555d36a47ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ad400640 1 -- 172.21.15.107:0/4230660935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 msgr2=0x555d36a47570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ad400640 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a47570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ad400640 1 -- 172.21.15.107:0/4230660935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d367162b0 con 0x555d36a3fc50 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.976+0000 7f80ade00640 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a47570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.977+0000 7f80ad400640 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a47ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f809800afd0 tx=0x7f8098004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ddcd226816ff10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.977+0000 7f808b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:20:07.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.977+0000 7f808b400640 1 -- 172.21.15.107:0/4230660935 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098021e30 con 0x555d36a3fc50 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.977+0000 7f808b400640 1 -- 172.21.15.107:0/4230660935 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8098004020 con 0x555d36a3fc50 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.977+0000 7f80b33f9740 1 -- 172.21.15.107:0/4230660935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f809c005020 con 0x555d36a3fc50 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.977+0000 7f808b400640 1 -- 172.21.15.107:0/4230660935 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8098004220 con 0x555d36a3fc50 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.977+0000 7f80b33f9740 10 client.15048 fetch_fsmap learned FSMap version 27 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.977+0000 7f80b33f9740 1 -- 172.21.15.107:0/4230660935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d36a58e90 con 0x555d36a3fc50 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.978+0000 7f808b400640 1 -- 172.21.15.107:0/4230660935 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f809c005020 con 0x555d36a3fc50 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.978+0000 7f808b400640 1 -- 172.21.15.107:0/4230660935 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8098022430 con 0x555d36a3fc50 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.978+0000 7f80b33f9740 10 client.15048 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.978+0000 7f80b33f9740 10 client.15048 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.978+0000 7f80b33f9740 20 client.15048 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.978+0000 7f80b33f9740 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x555d36a5b520 0x555d36a5c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:07.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.978+0000 7f80b33f9740 4 client.15048 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:07.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.978+0000 7f80b33f9740 1 -- 172.21.15.107:0/4230660935 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d369b2620 con 0x555d36a5b520 2024-09-19T04:20:07.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.978+0000 7f80ade00640 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x555d36a5b520 0x555d36a5c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:07.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.979+0000 7f80ade00640 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x555d36a5b520 0x555d36a5c990 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b11402d26d539aa server_cookie=bda3a1178cbea8a7 in_seq=0 out_seq=0 2024-09-19T04:20:07.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.979+0000 7f808b400640 10 client.15048 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:20:07.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f808b400640 1 -- 172.21.15.107:0/4230660935 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d369b2620 con 0x555d36a5b520 2024-09-19T04:20:07.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f808b400640 10 client.15048 handle_command_reply: tid=0 2024-09-19T04:20:07.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f80b33f9740 1 client.15048 shutdown 2024-09-19T04:20:07.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f80b33f9740 1 -- 172.21.15.107:0/4230660935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 msgr2=0x555d36a47ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:07.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f80b33f9740 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a47ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f809800afd0 tx=0x7f8098004730 comp rx=0 tx=0).stop 2024-09-19T04:20:07.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f80b33f9740 1 -- 172.21.15.107:0/4230660935 shutdown_connections 2024-09-19T04:20:07.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f80b33f9740 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x555d36a5b520 0x555d36a5c990 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f80b33f9740 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a4a510 0x555d36a47ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f80b33f9740 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a3fc50 0x555d36a47ab0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f80b33f9740 1 --2- 172.21.15.107:0/4230660935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3f330 0x555d36a47570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:07.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f80b33f9740 1 -- 172.21.15.107:0/4230660935 >> 172.21.15.107:0/4230660935 conn(0x555d36a3b8c0 msgr2=0x555d36a415f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:07.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.980+0000 7f808b400640 0 client.15048 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:20:07.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.981+0000 7f80b33f9740 1 -- 172.21.15.107:0/4230660935 shutdown_connections 2024-09-19T04:20:07.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.981+0000 7f80b33f9740 1 -- 172.21.15.107:0/4230660935 wait complete. 2024-09-19T04:20:07.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.981+0000 7f80b33f9740 20 client.15048 trim_cache size 0 max 16384 2024-09-19T04:20:08.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.999+0000 7f80b33f9740 1 Processor -- start 2024-09-19T04:20:08.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.999+0000 7f80b33f9740 1 -- start start 2024-09-19T04:20:08.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.999+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 0x555d36a47e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:08.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.999+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 0x555d36a41d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:08.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.999+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a494e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:08.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.999+0000 7f80b33f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555d36a5acd0 con 0x555d36a4a510 2024-09-19T04:20:08.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.999+0000 7f80b33f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555d36a5b3a0 con 0x555d36a3fc50 2024-09-19T04:20:08.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:07.999+0000 7f80b33f9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555d36a5ba70 con 0x555d36a3f330 2024-09-19T04:20:08.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:08.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47946/0 (socket says 172.21.15.107:47946) 2024-09-19T04:20:08.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ae800640 1 -- 172.21.15.107:0/1666503701 learned_addr learned my addr 172.21.15.107:0/1666503701 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:08.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ade00640 1 --2- 172.21.15.107:0/1666503701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 0x555d36a47e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:08.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ad400640 1 --2- 172.21.15.107:0/1666503701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 0x555d36a41d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:08.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ae800640 1 -- 172.21.15.107:0/1666503701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 msgr2=0x555d36a47e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:08.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ae800640 1 --2- 172.21.15.107:0/1666503701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 0x555d36a47e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ae800640 1 -- 172.21.15.107:0/1666503701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 msgr2=0x555d36a41d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:08.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ae800640 1 --2- 172.21.15.107:0/1666503701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 0x555d36a41d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ae800640 1 -- 172.21.15.107:0/1666503701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d36a5ccc0 con 0x555d36a4a510 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f80ae800640 1 --2- 172.21.15.107:0/1666503701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a494e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f80a40984c0 tx=0x7f80a40092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccdaaef11ee641e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.000+0000 7f8096600640 1 -- 172.21.15.107:0/1666503701 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a401f070 con 0x555d36a4a510 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f8096600640 1 -- 172.21.15.107:0/1666503701 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a40bc0c0 con 0x555d36a4a510 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f8096600640 1 -- 172.21.15.107:0/1666503701 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a409b500 con 0x555d36a4a510 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f80b33f9740 1 -- 172.21.15.107:0/1666503701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 msgr2=0x555d36a494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f80b33f9740 1 --2- 172.21.15.107:0/1666503701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a494e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f80a40984c0 tx=0x7f80a40092b0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f80b33f9740 1 -- 172.21.15.107:0/1666503701 shutdown_connections 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f80b33f9740 1 --2- 172.21.15.107:0/1666503701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a494e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f80b33f9740 1 --2- 172.21.15.107:0/1666503701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 0x555d36a41d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f80b33f9740 1 --2- 172.21.15.107:0/1666503701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 0x555d36a47e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f80b33f9740 1 -- 172.21.15.107:0/1666503701 >> 172.21.15.107:0/1666503701 conn(0x555d36a3b8c0 msgr2=0x555d36a48dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:08.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f80b33f9740 1 -- 172.21.15.107:0/1666503701 shutdown_connections 2024-09-19T04:20:08.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.001+0000 7f80b33f9740 1 -- 172.21.15.107:0/1666503701 wait complete. 2024-09-19T04:20:08.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.002+0000 7f80b33f9740 1 -- start start 2024-09-19T04:20:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.002+0000 7f80b33f9740 1 Processor -- start 2024-09-19T04:20:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 0x555d36a44780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 0x555d36a44cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80b33f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a45200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80b33f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555d36a5c810 con 0x555d36a4a510 2024-09-19T04:20:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80b33f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555d36a5c140 con 0x555d36a3fc50 2024-09-19T04:20:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80b33f9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555d36a5cee0 con 0x555d36a3f330 2024-09-19T04:20:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80b33f9740 10 client.0 _ll_register_callbacks cb 0x7f809800fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ad400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 0x555d36a44cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a45200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ade00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 0x555d36a44780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ade00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 0x555d36a44780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:34272/0 (socket says 172.21.15.107:34272) 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a45200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47954/0 (socket says 172.21.15.107:47954) 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ad400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 0x555d36a44cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47960/0 (socket says 172.21.15.107:47960) 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ade00640 1 -- 172.21.15.107:0/3482492141 learned_addr learned my addr 172.21.15.107:0/3482492141 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ade00640 1 -- 172.21.15.107:0/3482492141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 msgr2=0x555d36a44cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ade00640 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 0x555d36a44cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ade00640 1 -- 172.21.15.107:0/3482492141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 msgr2=0x555d36a45200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ade00640 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a45200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ade00640 1 -- 172.21.15.107:0/3482492141 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d36a635a0 con 0x555d36a3f330 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ae800640 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a45200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.003+0000 7f80ad400640 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 0x555d36a44cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.004+0000 7f80ade00640 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 0x555d36a44780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f809c009900 tx=0x7f809c005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603fa06be17d0c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.004+0000 7f808b400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.004+0000 7f808b400640 1 -- 172.21.15.107:0/3482492141 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c053070 con 0x555d36a3f330 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.004+0000 7f808b400640 1 -- 172.21.15.107:0/3482492141 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f809c04f3a0 con 0x555d36a3f330 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.004+0000 7f80b33f9740 1 -- 172.21.15.107:0/3482492141 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a40bc0c0 con 0x555d36a3f330 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.004+0000 7f808b400640 1 -- 172.21.15.107:0/3482492141 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f809c05f050 con 0x555d36a3f330 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.004+0000 7f80b33f9740 10 client.25066 fetch_fsmap learned FSMap version 27 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.004+0000 7f80b33f9740 1 -- 172.21.15.107:0/3482492141 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d36ace8b0 con 0x555d36a3f330 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.005+0000 7f808b400640 1 -- 172.21.15.107:0/3482492141 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a40bc0c0 con 0x555d36a3f330 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.005+0000 7f808b400640 1 -- 172.21.15.107:0/3482492141 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f809c004b90 con 0x555d36a3f330 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.005+0000 7f80b33f9740 10 client.25066 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.005+0000 7f80b33f9740 10 client.25066 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:20:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.005+0000 7f80b33f9740 20 client.25066 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.005+0000 7f80b33f9740 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x555d36ad3060 0x555d36ad1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.005+0000 7f80b33f9740 4 client.25066 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:20:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.005+0000 7f80b33f9740 1 -- 172.21.15.107:0/3482492141 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "status"}) -- 0x555d369acb30 con 0x555d36ad3060 2024-09-19T04:20:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.005+0000 7f80ad400640 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x555d36ad3060 0x555d36ad1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80ad400640 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x555d36ad3060 0x555d36ad1d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d72ad2e31c64165 server_cookie=aa413992975357ad in_seq=0 out_seq=0 2024-09-19T04:20:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f808b400640 10 client.25066 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:20:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f808b400640 1 -- 172.21.15.107:0/3482492141 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555d369acb30 con 0x555d36ad3060 2024-09-19T04:20:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f808b400640 10 client.25066 handle_command_reply: tid=0 2024-09-19T04:20:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80b33f9740 1 client.25066 shutdown 2024-09-19T04:20:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80b33f9740 1 -- 172.21.15.107:0/3482492141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 msgr2=0x555d36a44780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80b33f9740 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 0x555d36a44780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f809c009900 tx=0x7f809c005a20 comp rx=0 tx=0).stop 2024-09-19T04:20:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80b33f9740 1 -- 172.21.15.107:0/3482492141 shutdown_connections 2024-09-19T04:20:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80b33f9740 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x555d36ad3060 0x555d36ad1d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80b33f9740 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d36a4a510 0x555d36a45200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80b33f9740 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d36a3fc50 0x555d36a44cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f808b400640 0 client.25066 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:20:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80b33f9740 1 --2- 172.21.15.107:0/3482492141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d36a3f330 0x555d36a44780 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80b33f9740 1 -- 172.21.15.107:0/3482492141 >> 172.21.15.107:0/3482492141 conn(0x555d36a3b8c0 msgr2=0x555d36a3aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.006+0000 7f80b33f9740 1 -- 172.21.15.107:0/3482492141 shutdown_connections 2024-09-19T04:20:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b33f9740 1 -- 172.21.15.107:0/3482492141 wait complete. 2024-09-19T04:20:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b33f9740 20 client.25066 trim_cache size 0 max 16384 2024-09-19T04:20:08.012 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24500, 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 105.460571066, 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 83, 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 138.81487254800001 2024-09-19T04:20:08.013 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:20:08.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b0800640 1 -- 172.21.15.107:0/4279576484 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f80900a32c0 msgr2=0x7f80900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:08.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b0800640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f80900a32c0 0x7f80900a5780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x555d36a3cc40 tx=0x7f809800a640 comp rx=0 tx=0).stop 2024-09-19T04:20:08.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b0800640 1 -- 172.21.15.107:0/4279576484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a810e3c0 msgr2=0x7f80a819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:08.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b0800640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a810e3c0 0x7f80a819efc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f80a400eaa0 tx=0x7f80a400ef60 comp rx=0 tx=0).stop 2024-09-19T04:20:08.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b0800640 1 -- 172.21.15.107:0/4279576484 shutdown_connections 2024-09-19T04:20:08.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b0800640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f80900a32c0 0x7f80900a5780 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b0800640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a810e3c0 0x7f80a819efc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b0800640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a8074f40 0x7f80a819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b0800640 1 --2- 172.21.15.107:0/4279576484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a8074620 0x7f80a819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:08.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.007+0000 7f80b0800640 1 -- 172.21.15.107:0/4279576484 >> 172.21.15.107:0/4279576484 conn(0x7f80a80fd8c0 msgr2=0x7f80a80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:08.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.008+0000 7f80b0800640 1 -- 172.21.15.107:0/4279576484 shutdown_connections 2024-09-19T04:20:08.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:08.008+0000 7f80b0800640 1 -- 172.21.15.107:0/4279576484 wait complete. 2024-09-19T04:20:08.723 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T04:20:09.090 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:08 smithi115 ceph-mon[35477]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:20:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:08 smithi145 ceph-mon[35096]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:20:09.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:08 smithi107 ceph-mon[30659]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:20:09.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.644+0000 7f1c8ac00640 1 -- 172.21.15.107:0/407688969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c84109140 msgr2=0x7f1c84110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:09.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.644+0000 7f1c8ac00640 1 --2- 172.21.15.107:0/407688969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c84109140 0x7f1c84110cc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009ca0 tx=0x7f1c7c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.644+0000 7f1c8ac00640 1 -- 172.21.15.107:0/407688969 shutdown_connections 2024-09-19T04:20:09.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.644+0000 7f1c8ac00640 1 --2- 172.21.15.107:0/407688969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c84109140 0x7f1c84110cc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.644+0000 7f1c8ac00640 1 --2- 172.21.15.107:0/407688969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c841049c0 0x7f1c84108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.644+0000 7f1c8ac00640 1 --2- 172.21.15.107:0/407688969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c84104010 0x7f1c841043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.644+0000 7f1c8ac00640 1 -- 172.21.15.107:0/407688969 >> 172.21.15.107:0/407688969 conn(0x7f1c840ffa50 msgr2=0x7f1c84101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:09.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.644+0000 7f1c8ac00640 1 -- 172.21.15.107:0/407688969 shutdown_connections 2024-09-19T04:20:09.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.644+0000 7f1c8ac00640 1 -- 172.21.15.107:0/407688969 wait complete. 2024-09-19T04:20:09.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.645+0000 7f1c8ac00640 1 Processor -- start 2024-09-19T04:20:09.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.645+0000 7f1c8ac00640 1 -- start start 2024-09-19T04:20:09.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.645+0000 7f1c8ac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c84104010 0x7f1c8419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.645+0000 7f1c8ac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c841049c0 0x7f1c8419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.645+0000 7f1c8ac00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c84109140 0x7f1c8419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.645+0000 7f1c8ac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841141a0 con 0x7f1c841049c0 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.645+0000 7f1c8ac00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84114020 con 0x7f1c84109140 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.645+0000 7f1c8ac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84114320 con 0x7f1c84104010 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c83400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c841049c0 0x7f1c8419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c83400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c841049c0 0x7f1c8419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52168/0 (socket says 172.21.15.107:52168) 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c83400640 1 -- 172.21.15.107:0/3055568392 learned_addr learned my addr 172.21.15.107:0/3055568392 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c88c00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c84109140 0x7f1c8419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c83e00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c84104010 0x7f1c8419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c83400640 1 -- 172.21.15.107:0/3055568392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c84104010 msgr2=0x7f1c8419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c83400640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c84104010 0x7f1c8419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c83400640 1 -- 172.21.15.107:0/3055568392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c84109140 msgr2=0x7f1c8419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:09.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c83400640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c84109140 0x7f1c8419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c83400640 1 -- 172.21.15.107:0/3055568392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8419f650 con 0x7f1c841049c0 2024-09-19T04:20:09.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.646+0000 7f1c83400640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c841049c0 0x7f1c8419abe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000d950 tx=0x7f1c7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:09.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.647+0000 7f1c80c00640 1 -- 172.21.15.107:0/3055568392 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70017030 con 0x7f1c841049c0 2024-09-19T04:20:09.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.647+0000 7f1c8ac00640 1 -- 172.21.15.107:0/3055568392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8419f940 con 0x7f1c841049c0 2024-09-19T04:20:09.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.647+0000 7f1c80c00640 1 -- 172.21.15.107:0/3055568392 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c70004020 con 0x7f1c841049c0 2024-09-19T04:20:09.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.647+0000 7f1c80c00640 1 -- 172.21.15.107:0/3055568392 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c700047a0 con 0x7f1c841049c0 2024-09-19T04:20:09.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.647+0000 7f1c8ac00640 1 -- 172.21.15.107:0/3055568392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c841a7220 con 0x7f1c841049c0 2024-09-19T04:20:09.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.651+0000 7f1c8d679740 1 Processor -- start 2024-09-19T04:20:09.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.651+0000 7f1c8d679740 1 -- start start 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.651+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9330 0x55be58bd9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.651+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9c50 0x55be58be3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.651+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58be4510 0x55be58be6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.651+0000 7f1c8d679740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55be58b3f0b0 con 0x55be58bd9330 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.651+0000 7f1c8d679740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55be58ae6ab0 con 0x55be58be4510 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.651+0000 7f1c8d679740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55be58b3b070 con 0x55be58bd9c50 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.651+0000 7f1c80c00640 1 -- 172.21.15.107:0/3055568392 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f1c700041d0 con 0x7f1c841049c0 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.651+0000 7f1c88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9330 0x55be58bd9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9330 0x55be58bd9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52178/0 (socket says 172.21.15.107:52178) 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c88c00640 1 -- 172.21.15.107:0/3674298992 learned_addr learned my addr 172.21.15.107:0/3674298992 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c83400640 1 --2- 172.21.15.107:0/3674298992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58be4510 0x55be58be6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c83e00640 1 --2- 172.21.15.107:0/3674298992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9c50 0x55be58be3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c88c00640 1 -- 172.21.15.107:0/3674298992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9c50 msgr2=0x55be58be3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:09.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c88c00640 1 --2- 172.21.15.107:0/3674298992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9c50 0x55be58be3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c88c00640 1 -- 172.21.15.107:0/3674298992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58be4510 msgr2=0x55be58be6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:09.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c88c00640 1 --2- 172.21.15.107:0/3674298992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58be4510 0x55be58be6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c88c00640 1 -- 172.21.15.107:0/3674298992 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be58af34a0 con 0x55be58bd9330 2024-09-19T04:20:09.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c83400640 1 --2- 172.21.15.107:0/3674298992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58be4510 0x55be58be6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:09.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.652+0000 7f1c88c00640 1 --2- 172.21.15.107:0/3674298992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9330 0x55be58bd9710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009ca0 tx=0x7f1c7c02f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bd82d5d8bc5f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:09.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.653+0000 7f1c79200640 1 -- 172.21.15.107:0/3674298992 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c003970 con 0x55be58bd9330 2024-09-19T04:20:09.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.653+0000 7f1c79200640 1 -- 172.21.15.107:0/3674298992 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7c036030 con 0x55be58bd9330 2024-09-19T04:20:09.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.653+0000 7f1c79200640 1 -- 172.21.15.107:0/3674298992 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c004360 con 0x55be58bd9330 2024-09-19T04:20:09.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.653+0000 7f1c80c00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1c680a3240 0x7f1c680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.653+0000 7f1c80c00640 1 -- 172.21.15.107:0/3055568392 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f1c700cf1d0 con 0x7f1c841049c0 2024-09-19T04:20:09.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.653+0000 7f1c83e00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1c680a3240 0x7f1c680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.653+0000 7f1c8d679740 1 -- 172.21.15.107:0/3674298992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9330 msgr2=0x55be58bd9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:09.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.653+0000 7f1c8d679740 1 --2- 172.21.15.107:0/3674298992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9330 0x55be58bd9710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009ca0 tx=0x7f1c7c02f3c0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.654+0000 7f1c8d679740 1 -- 172.21.15.107:0/3674298992 shutdown_connections 2024-09-19T04:20:09.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.654+0000 7f1c8d679740 1 --2- 172.21.15.107:0/3674298992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58be4510 0x55be58be6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.654+0000 7f1c8d679740 1 --2- 172.21.15.107:0/3674298992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9c50 0x55be58be3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.654+0000 7f1c8d679740 1 --2- 172.21.15.107:0/3674298992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9330 0x55be58bd9710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.654+0000 7f1c8d679740 1 -- 172.21.15.107:0/3674298992 >> 172.21.15.107:0/3674298992 conn(0x55be58bd58c0 msgr2=0x55be58bd5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:09.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.654+0000 7f1c8d679740 1 -- 172.21.15.107:0/3674298992 shutdown_connections 2024-09-19T04:20:09.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.654+0000 7f1c8d679740 1 -- 172.21.15.107:0/3674298992 wait complete. 2024-09-19T04:20:09.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.654+0000 7f1c83e00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1c680a3240 0x7f1c680a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55be58bd6720 tx=0x7f1c74005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:09.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.655+0000 7f1c8d679740 1 -- start start 2024-09-19T04:20:09.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.655+0000 7f1c8d679740 1 Processor -- start 2024-09-19T04:20:09.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.655+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 0x55be58be13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.655+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58be1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.655+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 0x55be58be1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.655+0000 7f1c8d679740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55be58be7000 con 0x55be58be4510 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.655+0000 7f1c8d679740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55be58ac7920 con 0x55be58bd9c50 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.655+0000 7f1c8d679740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55be58be7180 con 0x55be58bd9330 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.655+0000 7f1c8d679740 10 client.0 _ll_register_callbacks cb 0x55be58aeb8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 0x55be58be1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c83e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 0x55be58be13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c83400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58be1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c83e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 0x55be58be13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:40258/0 (socket says 172.21.15.107:40258) 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c83e00640 1 -- 172.21.15.107:0/1403693091 learned_addr learned my addr 172.21.15.107:0/1403693091 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 0x55be58be1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52194/0 (socket says 172.21.15.107:52194) 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c83e00640 1 -- 172.21.15.107:0/1403693091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 msgr2=0x55be58be1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c83e00640 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58be1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c83e00640 1 -- 172.21.15.107:0/1403693091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 msgr2=0x55be58be1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c83e00640 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 0x55be58be1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c83e00640 1 -- 172.21.15.107:0/1403693091 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be588b02b0 con 0x55be58bd9330 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.656+0000 7f1c83400640 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58be1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.657+0000 7f1c83e00640 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 0x55be58be13e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c74004620 tx=0x7f1c74004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb795b17eaeef2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.657+0000 7f1c6d600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.657+0000 7f1c6d600640 1 -- 172.21.15.107:0/1403693091 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74029440 con 0x55be58bd9330 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.657+0000 7f1c8d679740 1 -- 172.21.15.107:0/1403693091 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7c036030 con 0x55be58bd9330 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.657+0000 7f1c6d600640 1 -- 172.21.15.107:0/1403693091 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f1c740295e0 con 0x55be58bd9330 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.657+0000 7f1c6d600640 1 -- 172.21.15.107:0/1403693091 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c74030070 con 0x55be58bd9330 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.657+0000 7f1c8d679740 10 client.25078 fetch_fsmap learned FSMap version 27 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.657+0000 7f1c8d679740 1 -- 172.21.15.107:0/1403693091 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be58bf2be0 con 0x55be58bd9330 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.657+0000 7f1c6d600640 1 -- 172.21.15.107:0/1403693091 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7c036030 con 0x55be58bd9330 2024-09-19T04:20:09.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.658+0000 7f1c6d600640 1 -- 172.21.15.107:0/1403693091 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1c74027020 con 0x55be58bd9330 2024-09-19T04:20:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.658+0000 7f1c8d679740 10 client.25078 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:20:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.658+0000 7f1c8d679740 10 client.25078 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:20:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.658+0000 7f1c8d679740 20 client.25078 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.658+0000 7f1c8d679740 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55be58bf51f0 0x55be58bf6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.658+0000 7f1c8d679740 4 client.25078 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.658+0000 7f1c8d679740 1 -- 172.21.15.107:0/1403693091 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be58b4c620 con 0x55be58bf51f0 2024-09-19T04:20:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.658+0000 7f1c83400640 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55be58bf51f0 0x55be58bf6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.659+0000 7f1c83400640 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55be58bf51f0 0x55be58bf6660 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7835c231c1eb7c01 server_cookie=894e22acac56a3bf in_seq=0 out_seq=0 2024-09-19T04:20:09.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.659+0000 7f1c6d600640 10 client.25078 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:20:09.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c6d600640 1 -- 172.21.15.107:0/1403693091 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be58b4c620 con 0x55be58bf51f0 2024-09-19T04:20:09.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c6d600640 10 client.25078 handle_command_reply: tid=0 2024-09-19T04:20:09.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 client.25078 shutdown 2024-09-19T04:20:09.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 -- 172.21.15.107:0/1403693091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 msgr2=0x55be58be13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 0x55be58be13e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c74004620 tx=0x7f1c74004730 comp rx=0 tx=0).stop 2024-09-19T04:20:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 -- 172.21.15.107:0/1403693091 shutdown_connections 2024-09-19T04:20:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55be58bf51f0 0x55be58bf6660 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 0x55be58be1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58be1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 --2- 172.21.15.107:0/1403693091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 0x55be58be13e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 -- 172.21.15.107:0/1403693091 >> 172.21.15.107:0/1403693091 conn(0x55be58bd58c0 msgr2=0x55be58bdb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:09.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c6d600640 0 client.25078 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:20:09.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 -- 172.21.15.107:0/1403693091 shutdown_connections 2024-09-19T04:20:09.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.660+0000 7f1c8d679740 1 -- 172.21.15.107:0/1403693091 wait complete. 2024-09-19T04:20:09.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.661+0000 7f1c8d679740 20 client.25078 trim_cache size 0 max 16384 2024-09-19T04:20:09.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.679+0000 7f1c8d679740 1 Processor -- start 2024-09-19T04:20:09.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.679+0000 7f1c8d679740 1 -- start start 2024-09-19T04:20:09.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.679+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 0x55be58bf51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.679+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58bd5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.679+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 0x55be58be1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.679+0000 7f1c8d679740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55be58bf49a0 con 0x55be58be4510 2024-09-19T04:20:09.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.679+0000 7f1c8d679740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55be58bf5070 con 0x55be58bd9c50 2024-09-19T04:20:09.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.679+0000 7f1c8d679740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55be58bf5740 con 0x55be58bd9330 2024-09-19T04:20:09.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.679+0000 7f1c88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 0x55be58be1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.679+0000 7f1c83400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58bd5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c83400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58bd5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37492/0 (socket says 172.21.15.107:37492) 2024-09-19T04:20:09.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c83400640 1 -- 172.21.15.107:0/1668136208 learned_addr learned my addr 172.21.15.107:0/1668136208 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:09.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c83e00640 1 --2- 172.21.15.107:0/1668136208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 0x55be58bf51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c83400640 1 -- 172.21.15.107:0/1668136208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 msgr2=0x55be58bf51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:09.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c83400640 1 --2- 172.21.15.107:0/1668136208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 0x55be58bf51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c83400640 1 -- 172.21.15.107:0/1668136208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 msgr2=0x55be58be1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:09.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c83400640 1 --2- 172.21.15.107:0/1668136208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 0x55be58be1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c83400640 1 -- 172.21.15.107:0/1668136208 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be58bdf930 con 0x55be58bd9c50 2024-09-19T04:20:09.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c83400640 1 --2- 172.21.15.107:0/1668136208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58bd5ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c7009b4c0 tx=0x7f1c70086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56eda07939cfef63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:09.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c6fe00640 1 -- 172.21.15.107:0/1668136208 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7009b300 con 0x55be58bd9c50 2024-09-19T04:20:09.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c6fe00640 1 -- 172.21.15.107:0/1668136208 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7009bec0 con 0x55be58bd9c50 2024-09-19T04:20:09.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.680+0000 7f1c6fe00640 1 -- 172.21.15.107:0/1668136208 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7009b120 con 0x55be58bd9c50 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.681+0000 7f1c8d679740 1 -- 172.21.15.107:0/1668136208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 msgr2=0x55be58bd5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.681+0000 7f1c8d679740 1 --2- 172.21.15.107:0/1668136208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58bd5ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c7009b4c0 tx=0x7f1c70086040 comp rx=0 tx=0).stop 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.681+0000 7f1c8d679740 1 -- 172.21.15.107:0/1668136208 shutdown_connections 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.681+0000 7f1c8d679740 1 --2- 172.21.15.107:0/1668136208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58be4510 0x55be58be1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.681+0000 7f1c8d679740 1 --2- 172.21.15.107:0/1668136208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9c50 0x55be58bd5ca0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.681+0000 7f1c8d679740 1 --2- 172.21.15.107:0/1668136208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58bd9330 0x55be58bf51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.681+0000 7f1c8d679740 1 -- 172.21.15.107:0/1668136208 >> 172.21.15.107:0/1668136208 conn(0x55be58bd58c0 msgr2=0x55be58bdfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.681+0000 7f1c8d679740 1 -- 172.21.15.107:0/1668136208 shutdown_connections 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.681+0000 7f1c8d679740 1 -- 172.21.15.107:0/1668136208 wait complete. 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.681+0000 7f1c8d679740 1 -- start start 2024-09-19T04:20:09.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c8d679740 1 Processor -- start 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9330 0x55be58bfeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9c50 0x55be58bff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c8d679740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58be4510 0x55be58bdde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c8d679740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55be58bf64e0 con 0x55be58bd9c50 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c8d679740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55be58bf5e10 con 0x55be58bd9330 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c8d679740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55be58bf6bb0 con 0x55be58be4510 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c8d679740 10 client.0 _ll_register_callbacks cb 0x7f1c740044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c83e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9330 0x55be58bfeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c83400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9c50 0x55be58bff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c83e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9330 0x55be58bfeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37494/0 (socket says 172.21.15.107:37494) 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.682+0000 7f1c83e00640 1 -- 172.21.15.107:0/3482796953 learned_addr learned my addr 172.21.15.107:0/3482796953 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c88c00640 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58be4510 0x55be58bdde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c83e00640 1 -- 172.21.15.107:0/3482796953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58be4510 msgr2=0x55be58bdde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c83e00640 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58be4510 0x55be58bdde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c83e00640 1 -- 172.21.15.107:0/3482796953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9c50 msgr2=0x55be58bff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c83e00640 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9c50 0x55be58bff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c83e00640 1 -- 172.21.15.107:0/3482796953 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be58bde690 con 0x55be58bd9330 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c88c00640 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58be4510 0x55be58bdde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c83e00640 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9330 0x55be58bfeae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c74012cb0 tx=0x7f1c74009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138a70cbfdbd54aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c6d600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c6d600640 1 -- 172.21.15.107:0/3482796953 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74040680 con 0x55be58bd9330 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c8d679740 1 -- 172.21.15.107:0/3482796953 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7009bec0 con 0x55be58bd9330 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.683+0000 7f1c6d600640 1 -- 172.21.15.107:0/3482796953 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f1c74035480 con 0x55be58bd9330 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c6d600640 1 -- 172.21.15.107:0/3482796953 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c74040820 con 0x55be58bd9330 2024-09-19T04:20:09.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c8d679740 10 client.25091 fetch_fsmap learned FSMap version 27 2024-09-19T04:20:09.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c8d679740 1 -- 172.21.15.107:0/3482796953 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be58bfd100 con 0x55be58bd9330 2024-09-19T04:20:09.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c6d600640 1 -- 172.21.15.107:0/3482796953 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7009bec0 con 0x55be58bd9330 2024-09-19T04:20:09.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c6d600640 1 -- 172.21.15.107:0/3482796953 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1c74027040 con 0x55be58bd9330 2024-09-19T04:20:09.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c8d679740 10 client.25091 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:20:09.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c8d679740 10 client.25091 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:20:09.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c8d679740 20 client.25091 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:09.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c8d679740 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55be58c64020 0x55be58c62d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:09.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c8d679740 4 client.25091 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:20:09.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.684+0000 7f1c8d679740 1 -- 172.21.15.107:0/3482796953 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "status"}) -- 0x55be58b46b30 con 0x55be58c64020 2024-09-19T04:20:09.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.685+0000 7f1c83400640 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55be58c64020 0x55be58c62d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:09.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.685+0000 7f1c83400640 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55be58c64020 0x55be58c62d30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0e645bfd844e0ae server_cookie=161948535351faf5 in_seq=0 out_seq=0 2024-09-19T04:20:09.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.685+0000 7f1c6d600640 10 client.25091 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.685+0000 7f1c6d600640 1 -- 172.21.15.107:0/3482796953 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55be58b46b30 con 0x55be58c64020 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.685+0000 7f1c6d600640 10 client.25091 handle_command_reply: tid=0 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.685+0000 7f1c8d679740 1 client.25091 shutdown 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 1 -- 172.21.15.107:0/3482796953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9330 msgr2=0x55be58bfeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9330 0x55be58bfeae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c74012cb0 tx=0x7f1c74009420 comp rx=0 tx=0).stop 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 1 -- 172.21.15.107:0/3482796953 shutdown_connections 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55be58c64020 0x55be58c62d30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be58be4510 0x55be58bdde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be58bd9c50 0x55be58bff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 1 --2- 172.21.15.107:0/3482796953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be58bd9330 0x55be58bfeae0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c6d600640 0 client.25091 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:20:09.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 1 -- 172.21.15.107:0/3482796953 >> 172.21.15.107:0/3482796953 conn(0x55be58bd58c0 msgr2=0x55be58bd4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 1 -- 172.21.15.107:0/3482796953 shutdown_connections 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 1 -- 172.21.15.107:0/3482796953 wait complete. 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.686+0000 7f1c8d679740 20 client.25091 trim_cache size 0 max 16384 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24517, 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-09-19T04:20:09.692 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 108.153373571, 2024-09-19T04:20:09.693 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-09-19T04:20:09.693 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 83, 2024-09-19T04:20:09.693 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T04:20:09.693 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 137.84768578399999 2024-09-19T04:20:09.693 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:20:09.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 -- 172.21.15.107:0/3055568392 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1c680a3240 msgr2=0x7f1c680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:09.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1c680a3240 0x7f1c680a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55be58bd6720 tx=0x7f1c74005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 -- 172.21.15.107:0/3055568392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c841049c0 msgr2=0x7f1c8419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c841049c0 0x7f1c8419abe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000d950 tx=0x7f1c7000de10 comp rx=0 tx=0).stop 2024-09-19T04:20:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 -- 172.21.15.107:0/3055568392 shutdown_connections 2024-09-19T04:20:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f1c680a3240 0x7f1c680a5700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c84109140 0x7f1c8419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c841049c0 0x7f1c8419abe0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 --2- 172.21.15.107:0/3055568392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c84104010 0x7f1c8419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 -- 172.21.15.107:0/3055568392 >> 172.21.15.107:0/3055568392 conn(0x7f1c840ffa50 msgr2=0x7f1c84100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 -- 172.21.15.107:0/3055568392 shutdown_connections 2024-09-19T04:20:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:09.687+0000 7f1c8ac00640 1 -- 172.21.15.107:0/3055568392 wait complete. 2024-09-19T04:20:10.467 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T04:20:10.835 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:10 smithi115 ceph-mon[35477]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-19T04:20:11.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:10 smithi107 ceph-mon[30659]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-19T04:20:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:10 smithi145 ceph-mon[35096]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-19T04:20:11.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.458+0000 7f76f8c00640 1 -- 172.21.15.107:0/3334756497 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76f4101de0 msgr2=0x7f76f41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:11.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.458+0000 7f76f8c00640 1 --2- 172.21.15.107:0/3334756497 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76f4101de0 0x7f76f41021c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009900 tx=0x7f76e402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.458+0000 7f76f8c00640 1 -- 172.21.15.107:0/3334756497 shutdown_connections 2024-09-19T04:20:11.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.458+0000 7f76f8c00640 1 --2- 172.21.15.107:0/3334756497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76f4106f10 0x7f76f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.458+0000 7f76f8c00640 1 --2- 172.21.15.107:0/3334756497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f4102790 0x7f76f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.458+0000 7f76f8c00640 1 --2- 172.21.15.107:0/3334756497 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76f4101de0 0x7f76f41021c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.458+0000 7f76f8c00640 1 -- 172.21.15.107:0/3334756497 >> 172.21.15.107:0/3334756497 conn(0x7f76f40fd8c0 msgr2=0x7f76f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:11.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.458+0000 7f76f8c00640 1 -- 172.21.15.107:0/3334756497 shutdown_connections 2024-09-19T04:20:11.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.458+0000 7f76f8c00640 1 -- 172.21.15.107:0/3334756497 wait complete. 2024-09-19T04:20:11.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.459+0000 7f76f8c00640 1 Processor -- start 2024-09-19T04:20:11.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.459+0000 7f76f8c00640 1 -- start start 2024-09-19T04:20:11.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.459+0000 7f76f8c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f4101de0 0x7f76f419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.459+0000 7f76f8c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76f4102790 0x7f76f419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.459+0000 7f76f8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76f4106f10 0x7f76f419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.459+0000 7f76f8c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4111f70 con 0x7f76f4102790 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.459+0000 7f76f8c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4111df0 con 0x7f76f4101de0 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.459+0000 7f76f8c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41120f0 con 0x7f76f4106f10 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76f4102790 0x7f76f419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f4101de0 0x7f76f419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76f4102790 0x7f76f419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52224/0 (socket says 172.21.15.107:52224) 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f1400640 1 -- 172.21.15.107:0/3478069055 learned_addr learned my addr 172.21.15.107:0/3478069055 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f2800640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76f4106f10 0x7f76f419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f1e00640 1 -- 172.21.15.107:0/3478069055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76f4106f10 msgr2=0x7f76f419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f1e00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76f4106f10 0x7f76f419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f1e00640 1 -- 172.21.15.107:0/3478069055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76f4102790 msgr2=0x7f76f419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:11.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f1e00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76f4102790 0x7f76f419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f1e00640 1 -- 172.21.15.107:0/3478069055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f419f5d0 con 0x7f76f4101de0 2024-09-19T04:20:11.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.460+0000 7f76f1e00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f4101de0 0x7f76f419a620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005940 tx=0x7f76e40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:11.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.461+0000 7f76eaa00640 1 -- 172.21.15.107:0/3478069055 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e403f030 con 0x7f76f4101de0 2024-09-19T04:20:11.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.461+0000 7f76f8c00640 1 -- 172.21.15.107:0/3478069055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f419f860 con 0x7f76f4101de0 2024-09-19T04:20:11.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.461+0000 7f76eaa00640 1 -- 172.21.15.107:0/3478069055 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e40044c0 con 0x7f76f4101de0 2024-09-19T04:20:11.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.461+0000 7f76eaa00640 1 -- 172.21.15.107:0/3478069055 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4035070 con 0x7f76f4101de0 2024-09-19T04:20:11.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.461+0000 7f76f8c00640 1 -- 172.21.15.107:0/3478069055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f41a71a0 con 0x7f76f4101de0 2024-09-19T04:20:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.464+0000 7f76fb60a740 1 Processor -- start 2024-09-19T04:20:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.464+0000 7f76eaa00640 1 -- 172.21.15.107:0/3478069055 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f76e4042030 con 0x7f76f4101de0 2024-09-19T04:20:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.464+0000 7f76fb60a740 1 -- start start 2024-09-19T04:20:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.464+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998330 0x55d71f998710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.464+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.464+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f9a3510 0x55d71f9a5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.464+0000 7f76fb60a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d71f8fe0b0 con 0x55d71f998c50 2024-09-19T04:20:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.464+0000 7f76fb60a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d71f8a5ab0 con 0x55d71f9a3510 2024-09-19T04:20:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.464+0000 7f76fb60a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d71f8fa070 con 0x55d71f998330 2024-09-19T04:20:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.464+0000 7f76f1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76f1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f9a3510 0x55d71f9a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76f2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998330 0x55d71f998710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76f1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9a2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52238/0 (socket says 172.21.15.107:52238) 2024-09-19T04:20:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76f1e00640 1 -- 172.21.15.107:0/3321052106 learned_addr learned my addr 172.21.15.107:0/3321052106 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76f2800640 1 -- 172.21.15.107:0/3321052106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f9a3510 msgr2=0x55d71f9a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76f2800640 1 --2- 172.21.15.107:0/3321052106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f9a3510 0x55d71f9a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76f2800640 1 -- 172.21.15.107:0/3321052106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 msgr2=0x55d71f9a2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76f2800640 1 --2- 172.21.15.107:0/3321052106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9a2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76f2800640 1 -- 172.21.15.107:0/3321052106 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d71f8b24a0 con 0x55d71f998330 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76f2800640 1 --2- 172.21.15.107:0/3321052106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998330 0x55d71f998710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009f00 tx=0x7f76ec011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2146f7ecef9b01a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.465+0000 7f76df000640 1 -- 172.21.15.107:0/3321052106 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004d20 con 0x55d71f998330 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.466+0000 7f76df000640 1 -- 172.21.15.107:0/3321052106 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ec004050 con 0x55d71f998330 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.466+0000 7f76df000640 1 -- 172.21.15.107:0/3321052106 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec005280 con 0x55d71f998330 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.466+0000 7f76eaa00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76d80a3240 0x7f76d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.466+0000 7f76eaa00640 1 -- 172.21.15.107:0/3478069055 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f76e40efbe0 con 0x7f76f4101de0 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.466+0000 7f76f1e00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76d80a3240 0x7f76d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.466+0000 7f76fb60a740 1 -- 172.21.15.107:0/3321052106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998330 msgr2=0x55d71f998710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.466+0000 7f76fb60a740 1 --2- 172.21.15.107:0/3321052106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998330 0x55d71f998710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009f00 tx=0x7f76ec011810 comp rx=0 tx=0).stop 2024-09-19T04:20:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.467+0000 7f76fb60a740 1 -- 172.21.15.107:0/3321052106 shutdown_connections 2024-09-19T04:20:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.467+0000 7f76fb60a740 1 --2- 172.21.15.107:0/3321052106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f9a3510 0x55d71f9a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.467+0000 7f76fb60a740 1 --2- 172.21.15.107:0/3321052106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9a2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.467+0000 7f76fb60a740 1 --2- 172.21.15.107:0/3321052106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998330 0x55d71f998710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.467+0000 7f76fb60a740 1 -- 172.21.15.107:0/3321052106 >> 172.21.15.107:0/3321052106 conn(0x55d71f9948c0 msgr2=0x55d71f994ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.467+0000 7f76fb60a740 1 -- 172.21.15.107:0/3321052106 shutdown_connections 2024-09-19T04:20:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.467+0000 7f76fb60a740 1 -- 172.21.15.107:0/3321052106 wait complete. 2024-09-19T04:20:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.467+0000 7f76f1e00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76d80a3240 0x7f76d80a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76e402a910 tx=0x7f76e40f3620 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.467+0000 7f76fb60a740 1 -- start start 2024-09-19T04:20:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.467+0000 7f76fb60a740 1 Processor -- start 2024-09-19T04:20:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998330 0x55d71f9a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998c50 0x55d71f9a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f9a3510 0x55d71f9a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76fb60a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d71f9a6060 con 0x55d71f998330 2024-09-19T04:20:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76fb60a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d71f886920 con 0x55d71f9a3510 2024-09-19T04:20:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76fb60a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d71f9a61e0 con 0x55d71f998c50 2024-09-19T04:20:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76fb60a740 10 client.0 _ll_register_callbacks cb 0x55d71f8aa8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76f2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998330 0x55d71f9a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76f1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998c50 0x55d71f9a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76f2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998330 0x55d71f9a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52240/0 (socket says 172.21.15.107:52240) 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76f2800640 1 -- 172.21.15.107:0/2725081811 learned_addr learned my addr 172.21.15.107:0/2725081811 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.468+0000 7f76f2800640 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f9a3510 0x55d71f9a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76f1400640 1 -- 172.21.15.107:0/2725081811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f9a3510 msgr2=0x55d71f9a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76f1400640 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f9a3510 0x55d71f9a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76f1400640 1 -- 172.21.15.107:0/2725081811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998330 msgr2=0x55d71f9a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76f1400640 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998330 0x55d71f9a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76f1400640 1 -- 172.21.15.107:0/2725081811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d71f66f2b0 con 0x55d71f998c50 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76f2800640 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998330 0x55d71f9a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:11.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76f1400640 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998c50 0x55d71f9a0b00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009620 tx=0x7f76e0012bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff93666bce08b783 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76d3400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76d3400640 1 -- 172.21.15.107:0/2725081811 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0004050 con 0x55d71f998c50 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76fb60a740 1 -- 172.21.15.107:0/2725081811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76ec004050 con 0x55d71f998c50 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.469+0000 7f76d3400640 1 -- 172.21.15.107:0/2725081811 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f76e00041f0 con 0x55d71f998c50 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76d3400640 1 -- 172.21.15.107:0/2725081811 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f76e0023070 con 0x55d71f998c50 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76fb60a740 10 client.25096 fetch_fsmap learned FSMap version 27 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76fb60a740 1 -- 172.21.15.107:0/2725081811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d71f9ba720 con 0x55d71f998c50 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76d3400640 1 -- 172.21.15.107:0/2725081811 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ec004050 con 0x55d71f998c50 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76d3400640 1 -- 172.21.15.107:0/2725081811 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f76e0003a40 con 0x55d71f998c50 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76fb60a740 10 client.25096 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76fb60a740 10 client.25096 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76fb60a740 20 client.25096 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76fb60a740 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55d71f9bcd40 0x55d71f9be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76fb60a740 4 client.25096 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.470+0000 7f76fb60a740 1 -- 172.21.15.107:0/2725081811 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d71f90b620 con 0x55d71f9bcd40 2024-09-19T04:20:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.471+0000 7f76f2800640 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55d71f9bcd40 0x55d71f9be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.471+0000 7f76f2800640 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55d71f9bcd40 0x55d71f9be1b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5861ae63887bfcd server_cookie=10e04b35a75270ed in_seq=0 out_seq=0 2024-09-19T04:20:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.471+0000 7f76d3400640 10 client.25096 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76d3400640 1 -- 172.21.15.107:0/2725081811 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d71f90b620 con 0x55d71f9bcd40 2024-09-19T04:20:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76d3400640 10 client.25096 handle_command_reply: tid=0 2024-09-19T04:20:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76fb60a740 1 client.25096 shutdown 2024-09-19T04:20:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76fb60a740 1 -- 172.21.15.107:0/2725081811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998c50 msgr2=0x55d71f9a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76fb60a740 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998c50 0x55d71f9a0b00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009620 tx=0x7f76e0012bd0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76fb60a740 1 -- 172.21.15.107:0/2725081811 shutdown_connections 2024-09-19T04:20:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76fb60a740 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55d71f9bcd40 0x55d71f9be1b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76fb60a740 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f9a3510 0x55d71f9a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76fb60a740 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f998c50 0x55d71f9a0b00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76fb60a740 1 --2- 172.21.15.107:0/2725081811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998330 0x55d71f9a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76d3400640 0 client.25096 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:11.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.472+0000 7f76fb60a740 1 -- 172.21.15.107:0/2725081811 >> 172.21.15.107:0/2725081811 conn(0x55d71f9948c0 msgr2=0x55d71f99a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:11.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.473+0000 7f76fb60a740 1 -- 172.21.15.107:0/2725081811 shutdown_connections 2024-09-19T04:20:11.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.473+0000 7f76fb60a740 1 -- 172.21.15.107:0/2725081811 wait complete. 2024-09-19T04:20:11.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.473+0000 7f76fb60a740 20 client.25096 trim_cache size 0 max 16384 2024-09-19T04:20:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.492+0000 7f76fb60a740 1 Processor -- start 2024-09-19T04:20:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76fb60a740 1 -- start start 2024-09-19T04:20:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f998330 0x55d71f9a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f99f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 0x55d71f99d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76fb60a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d71f9bc4f0 con 0x55d71f998c50 2024-09-19T04:20:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76fb60a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d71f9bcbc0 con 0x55d71f998330 2024-09-19T04:20:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76fb60a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d71f9bd290 con 0x55d71f9a3510 2024-09-19T04:20:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76f1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f99f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76f1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f99f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52242/0 (socket says 172.21.15.107:52242) 2024-09-19T04:20:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76f1400640 1 -- 172.21.15.107:0/3166249491 learned_addr learned my addr 172.21.15.107:0/3166249491 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76f2800640 1 --2- 172.21.15.107:0/3166249491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f998330 0x55d71f9a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.493+0000 7f76f2800640 1 --2- 172.21.15.107:0/3166249491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 0x55d71f99d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.494+0000 7f76f1400640 1 -- 172.21.15.107:0/3166249491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 msgr2=0x55d71f99d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.494+0000 7f76f1400640 1 --2- 172.21.15.107:0/3166249491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 0x55d71f99d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.494+0000 7f76f1400640 1 -- 172.21.15.107:0/3166249491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f998330 msgr2=0x55d71f9a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.494+0000 7f76f1400640 1 --2- 172.21.15.107:0/3166249491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f998330 0x55d71f9a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.494+0000 7f76f1400640 1 -- 172.21.15.107:0/3166249491 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d71f998a80 con 0x55d71f998c50 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.494+0000 7f76f2800640 1 --2- 172.21.15.107:0/3166249491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 0x55d71f99d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.494+0000 7f76f1400640 1 --2- 172.21.15.107:0/3166249491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f99f5c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f76e0003e30 tx=0x7f76e0010790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9039dccd4cf55d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.494+0000 7f76de600640 1 -- 172.21.15.107:0/3166249491 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e00109e0 con 0x55d71f998c50 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.494+0000 7f76de600640 1 -- 172.21.15.107:0/3166249491 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e00055f0 con 0x55d71f998c50 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.494+0000 7f76de600640 1 -- 172.21.15.107:0/3166249491 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e000f790 con 0x55d71f998c50 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.495+0000 7f76fb60a740 1 -- 172.21.15.107:0/3166249491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 msgr2=0x55d71f99f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.495+0000 7f76fb60a740 1 --2- 172.21.15.107:0/3166249491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f99f5c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f76e0003e30 tx=0x7f76e0010790 comp rx=0 tx=0).stop 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.495+0000 7f76fb60a740 1 -- 172.21.15.107:0/3166249491 shutdown_connections 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.495+0000 7f76fb60a740 1 --2- 172.21.15.107:0/3166249491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 0x55d71f99d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.495+0000 7f76fb60a740 1 --2- 172.21.15.107:0/3166249491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f99f5c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.495+0000 7f76fb60a740 1 --2- 172.21.15.107:0/3166249491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f998330 0x55d71f9a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.495+0000 7f76fb60a740 1 -- 172.21.15.107:0/3166249491 >> 172.21.15.107:0/3166249491 conn(0x55d71f9948c0 msgr2=0x55d71f99eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.495+0000 7f76fb60a740 1 -- 172.21.15.107:0/3166249491 shutdown_connections 2024-09-19T04:20:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.495+0000 7f76fb60a740 1 -- 172.21.15.107:0/3166249491 wait complete. 2024-09-19T04:20:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.496+0000 7f76fb60a740 1 -- start start 2024-09-19T04:20:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.496+0000 7f76fb60a740 1 Processor -- start 2024-09-19T04:20:11.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.496+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f998330 0x55d71f9c0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.496+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.496+0000 7f76fb60a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 0x55d71f9c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.496+0000 7f76fb60a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d71f9be030 con 0x55d71f998c50 2024-09-19T04:20:11.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.496+0000 7f76fb60a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d71f9bd960 con 0x55d71f998330 2024-09-19T04:20:11.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.496+0000 7f76fb60a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d71f9be700 con 0x55d71f9a3510 2024-09-19T04:20:11.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76fb60a740 10 client.0 _ll_register_callbacks cb 0x7f76e0009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9c0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52258/0 (socket says 172.21.15.107:52258) 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f1400640 1 -- 172.21.15.107:0/1566913790 learned_addr learned my addr 172.21.15.107:0/1566913790 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f2800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f998330 0x55d71f9c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f2800640 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 0x55d71f9c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f1400640 1 -- 172.21.15.107:0/1566913790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 msgr2=0x55d71f9c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f1400640 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 0x55d71f9c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f1400640 1 -- 172.21.15.107:0/1566913790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f998330 msgr2=0x55d71f9c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f1400640 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f998330 0x55d71f9c0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f1400640 1 -- 172.21.15.107:0/1566913790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d71f9c53b0 con 0x55d71f998c50 2024-09-19T04:20:11.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76f1400640 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9c0da0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009620 tx=0x7f76e0004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a8340bad414196 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:11.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.497+0000 7f76d3400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:20:11.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.498+0000 7f76d3400640 1 -- 172.21.15.107:0/1566913790 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0003be0 con 0x55d71f998c50 2024-09-19T04:20:11.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.498+0000 7f76d3400640 1 -- 172.21.15.107:0/1566913790 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f76e0005110 con 0x55d71f998c50 2024-09-19T04:20:11.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.498+0000 7f76fb60a740 1 -- 172.21.15.107:0/1566913790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e00055f0 con 0x55d71f998c50 2024-09-19T04:20:11.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.498+0000 7f76d3400640 1 -- 172.21.15.107:0/1566913790 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f76e0005310 con 0x55d71f998c50 2024-09-19T04:20:11.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.498+0000 7f76fb60a740 10 client.15090 fetch_fsmap learned FSMap version 27 2024-09-19T04:20:11.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.498+0000 7f76fb60a740 1 -- 172.21.15.107:0/1566913790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d71fa27bf0 con 0x55d71f998c50 2024-09-19T04:20:11.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.498+0000 7f76d3400640 1 -- 172.21.15.107:0/1566913790 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e00055f0 con 0x55d71f998c50 2024-09-19T04:20:11.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.498+0000 7f76d3400640 1 -- 172.21.15.107:0/1566913790 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f76e000fcc0 con 0x55d71f998c50 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.498+0000 7f76fb60a740 10 client.15090 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.499+0000 7f76fb60a740 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.499+0000 7f76fb60a740 20 client.15090 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.499+0000 7f76fb60a740 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55d71fa2c3d0 0x55d71fa2b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.499+0000 7f76fb60a740 4 client.15090 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.499+0000 7f76fb60a740 1 -- 172.21.15.107:0/1566913790 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "status"}) -- 0x55d71f905b30 con 0x55d71fa2c3d0 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.499+0000 7f76f2800640 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55d71fa2c3d0 0x55d71fa2b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.499+0000 7f76f2800640 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55d71fa2c3d0 0x55d71fa2b0e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ad32a104f9af7c server_cookie=a3d0a8a9a0a232b8 in_seq=0 out_seq=0 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.499+0000 7f76d3400640 10 client.15090 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76d3400640 1 -- 172.21.15.107:0/1566913790 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55d71f905b30 con 0x55d71fa2c3d0 2024-09-19T04:20:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76d3400640 10 client.15090 handle_command_reply: tid=0 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 client.15090 shutdown 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 -- 172.21.15.107:0/1566913790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 msgr2=0x55d71f9c0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9c0da0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009620 tx=0x7f76e0004ef0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 -- 172.21.15.107:0/1566913790 shutdown_connections 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55d71fa2c3d0 0x55d71fa2b0e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d71f9a3510 0x55d71f9c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d71f998c50 0x55d71f9c0da0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 --2- 172.21.15.107:0/1566913790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d71f998330 0x55d71f9c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 -- 172.21.15.107:0/1566913790 >> 172.21.15.107:0/1566913790 conn(0x55d71f9948c0 msgr2=0x55d71f993a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76d3400640 0 client.15090 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 -- 172.21.15.107:0/1566913790 shutdown_connections 2024-09-19T04:20:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 1 -- 172.21.15.107:0/1566913790 wait complete. 2024-09-19T04:20:11.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.500+0000 7f76fb60a740 20 client.15090 trim_cache size 0 max 16384 2024-09-19T04:20:11.507 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:20:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "e2ec5b0e-763c-11ef-bceb-c7b262605968", 2024-09-19T04:20:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-09-19T04:20:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24539, 2024-09-19T04:20:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-09-19T04:20:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-09-19T04:20:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-09-19T04:20:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 118.023435085, 2024-09-19T04:20:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-09-19T04:20:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 83, 2024-09-19T04:20:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T04:20:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 129.811826518 2024-09-19T04:20:11.508 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:20:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 -- 172.21.15.107:0/3478069055 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76d80a3240 msgr2=0x7f76d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76d80a3240 0x7f76d80a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76e402a910 tx=0x7f76e40f3620 comp rx=0 tx=0).stop 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 -- 172.21.15.107:0/3478069055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f4101de0 msgr2=0x7f76f419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f4101de0 0x7f76f419a620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005940 tx=0x7f76e40040c0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 -- 172.21.15.107:0/3478069055 shutdown_connections 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f76d80a3240 0x7f76d80a5700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76f4106f10 0x7f76f419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76f4102790 0x7f76f419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 --2- 172.21.15.107:0/3478069055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f4101de0 0x7f76f419a620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 -- 172.21.15.107:0/3478069055 >> 172.21.15.107:0/3478069055 conn(0x7f76f40fd8c0 msgr2=0x7f76f40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 -- 172.21.15.107:0/3478069055 shutdown_connections 2024-09-19T04:20:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:11.501+0000 7f76f8c00640 1 -- 172.21.15.107:0/3478069055 wait complete. 2024-09-19T04:20:12.225 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T04:20:12.236 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T04:20:12.236 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T04:20:12.236 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T04:20:12.236 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T04:20:12.237 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:20:12.237 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-09-19T04:20:12.238 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T04:20:12.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.388+0000 7f2df6000640 1 -- 172.21.15.107:0/3299421159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df0103b70 msgr2=0x7f2df0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:12.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.388+0000 7f2df6000640 1 --2- 172.21.15.107:0/3299421159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df0103b70 0x7f2df0103f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009900 tx=0x7f2de402a410 comp rx=0 tx=0).stop 2024-09-19T04:20:12.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.388+0000 7f2df6000640 1 -- 172.21.15.107:0/3299421159 shutdown_connections 2024-09-19T04:20:12.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.388+0000 7f2df6000640 1 --2- 172.21.15.107:0/3299421159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2df0105720 0x7f2df010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.388+0000 7f2df6000640 1 --2- 172.21.15.107:0/3299421159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df0104d60 0x7f2df01051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.388+0000 7f2df6000640 1 --2- 172.21.15.107:0/3299421159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df0103b70 0x7f2df0103f70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.388+0000 7f2df6000640 1 -- 172.21.15.107:0/3299421159 >> 172.21.15.107:0/3299421159 conn(0x7f2df00ff2f0 msgr2=0x7f2df0101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:12.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.388+0000 7f2df6000640 1 -- 172.21.15.107:0/3299421159 shutdown_connections 2024-09-19T04:20:12.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.388+0000 7f2df6000640 1 -- 172.21.15.107:0/3299421159 wait complete. 2024-09-19T04:20:12.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.389+0000 7f2df6000640 1 Processor -- start 2024-09-19T04:20:12.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.389+0000 7f2df6000640 1 -- start start 2024-09-19T04:20:12.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.389+0000 7f2df6000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2df0103b70 0x7f2df0199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:12.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2df6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df0104d60 0x7f2df019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:12.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2df6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df0105720 0x7f2df01a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:12.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2df6000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010f7d0 con 0x7f2df0104d60 2024-09-19T04:20:12.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2df6000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010f650 con 0x7f2df0103b70 2024-09-19T04:20:12.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2df6000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010f950 con 0x7f2df0105720 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2defe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df0104d60 0x7f2df019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2df4c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2df0103b70 0x7f2df0199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2defe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df0104d60 0x7f2df019a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52270/0 (socket says 172.21.15.107:52270) 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2defe00640 1 -- 172.21.15.107:0/1891683270 learned_addr learned my addr 172.21.15.107:0/1891683270 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2df5600640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df0105720 0x7f2df01a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2defe00640 1 -- 172.21.15.107:0/1891683270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df0105720 msgr2=0x7f2df01a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2defe00640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df0105720 0x7f2df01a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2defe00640 1 -- 172.21.15.107:0/1891683270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2df0103b70 msgr2=0x7f2df0199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2defe00640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2df0103b70 0x7f2df0199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2defe00640 1 -- 172.21.15.107:0/1891683270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01a1aa0 con 0x7f2df0104d60 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.390+0000 7f2defe00640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df0104d60 0x7f2df019a3c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00e9f0 tx=0x7f2ddc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.391+0000 7f2ded600640 1 -- 172.21.15.107:0/1891683270 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00cd80 con 0x7f2df0104d60 2024-09-19T04:20:12.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.391+0000 7f2ded600640 1 -- 172.21.15.107:0/1891683270 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc010910 con 0x7f2df0104d60 2024-09-19T04:20:12.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.391+0000 7f2df6000640 1 -- 172.21.15.107:0/1891683270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df01a1d90 con 0x7f2df0104d60 2024-09-19T04:20:12.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.391+0000 7f2ded600640 1 -- 172.21.15.107:0/1891683270 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc018990 con 0x7f2df0104d60 2024-09-19T04:20:12.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.391+0000 7f2df6000640 1 -- 172.21.15.107:0/1891683270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01a2260 con 0x7f2df0104d60 2024-09-19T04:20:12.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.392+0000 7f2df6000640 1 -- 172.21.15.107:0/1891683270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df010b000 con 0x7f2df0104d60 2024-09-19T04:20:12.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.394+0000 7f2ded600640 1 -- 172.21.15.107:0/1891683270 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f2ddc010ac0 con 0x7f2df0104d60 2024-09-19T04:20:12.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.396+0000 7f2ded600640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2dd80a32c0 0x7f2dd80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:12.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.397+0000 7f2ded600640 1 -- 172.21.15.107:0/1891683270 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f2ddc0cfc60 con 0x7f2df0104d60 2024-09-19T04:20:12.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.397+0000 7f2df4c00640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2dd80a32c0 0x7f2dd80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:12.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.397+0000 7f2df4c00640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2dd80a32c0 0x7f2dd80a5780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2de4005ab0 tx=0x7f2de4000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:12.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.401+0000 7f2ded600640 1 -- 172.21.15.107:0/1891683270 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2ddc09c950 con 0x7f2df0104d60 2024-09-19T04:20:12.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.626+0000 7f2df6000640 1 -- 172.21.15.107:0/1891683270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f2df0062e90 con 0x7f2df0104d60 2024-09-19T04:20:12.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.953+0000 7f2ded600640 1 -- 172.21.15.107:0/1891683270 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f2ddc09c250 con 0x7f2df0104d60 2024-09-19T04:20:12.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.955+0000 7f2df6000640 1 -- 172.21.15.107:0/1891683270 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2dd80a32c0 msgr2=0x7f2dd80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:12.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.955+0000 7f2df6000640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2dd80a32c0 0x7f2dd80a5780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2de4005ab0 tx=0x7f2de4000bc0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.955+0000 7f2df6000640 1 -- 172.21.15.107:0/1891683270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df0104d60 msgr2=0x7f2df019a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:12.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.955+0000 7f2df6000640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df0104d60 0x7f2df019a3c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00e9f0 tx=0x7f2ddc00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.955+0000 7f2df6000640 1 -- 172.21.15.107:0/1891683270 shutdown_connections 2024-09-19T04:20:12.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.955+0000 7f2df6000640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f2dd80a32c0 0x7f2dd80a5780 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.955+0000 7f2df6000640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df0105720 0x7f2df01a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.955+0000 7f2df6000640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df0104d60 0x7f2df019a3c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.955+0000 7f2df6000640 1 --2- 172.21.15.107:0/1891683270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2df0103b70 0x7f2df0199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:12.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.955+0000 7f2df6000640 1 -- 172.21.15.107:0/1891683270 >> 172.21.15.107:0/1891683270 conn(0x7f2df00ff2f0 msgr2=0x7f2df01081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:12.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.956+0000 7f2df6000640 1 -- 172.21.15.107:0/1891683270 shutdown_connections 2024-09-19T04:20:12.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:12.957+0000 7f2df6000640 1 -- 172.21.15.107:0/1891683270 wait complete. 2024-09-19T04:20:12.981 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T04:20:13.006 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T04:20:13.006 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T04:20:13.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:13.007 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-09-19T04:20:13.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:13.045 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-09-19T04:20:13.146 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2024-09-19T04:20:13.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:20:13.150 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-19T04:20:13.150 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T04:20:13.150 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:20:13.150 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T04:20:13.201 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: ignoring --setuser ceph since I am not root 2024-09-19T04:20:13.201 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: ignoring --setgroup ceph since I am not root 2024-09-19T04:20:13.201 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:12 smithi107 ceph-mon[30659]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:20:13.201 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:12 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1891683270' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:20:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:12 smithi145 ceph-mon[35096]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:20:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:12 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1891683270' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:20:13.224 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: ignoring --setuser ceph since I am not root 2024-09-19T04:20:13.224 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: ignoring --setgroup ceph since I am not root 2024-09-19T04:20:13.224 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-mgr[36917]: -- 172.21.15.145:0/3757325829 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563731fa70e0 con 0x563731f84800 2024-09-19T04:20:13.225 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:13.112+0000 7fa066a69040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:20:13.225 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:13.169+0000 7fa066a69040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:20:13.312 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:12 smithi115 ceph-mon[35477]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:20:13.313 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:12 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1891683270' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T04:20:13.313 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: ignoring --setuser ceph since I am not root 2024-09-19T04:20:13.313 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: ignoring --setgroup ceph since I am not root 2024-09-19T04:20:13.313 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-mgr[36843]: -- 172.21.15.115:0/3732309004 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c5e8ab0e0 con 0x562c5e889000 2024-09-19T04:20:13.313 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:13.107+0000 7f8b727c7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:20:13.313 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:13.163+0000 7f8b727c7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:20:13.313 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:13.311+0000 7f8b727c7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:20:13.480 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:13.206+0000 7fd9dc638040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:20:13.480 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:13.279+0000 7fd9dc638040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:20:13.544 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:13.669 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:13.378+0000 7f8b727c7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:20:13.669 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:13.525+0000 7f8b727c7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:20:13.673 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:13.317+0000 7fa066a69040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:20:13.673 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:13.383+0000 7fa066a69040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:20:13.673 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:13.530+0000 7fa066a69040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:20:13.673 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:13.671+0000 7fa066a69040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:20:13.756 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:13.480+0000 7fd9dc638040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:20:13.757 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:13.568+0000 7fd9dc638040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:20:13.946 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:13.667+0000 7f8b727c7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:20:14.008 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:13.754+0000 7fd9dc638040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:20:14.009 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:13.926+0000 7fd9dc638040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:20:14.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:13 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1891683270' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T04:20:14.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:13 smithi107 ceph-mon[30659]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T04:20:14.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.112+0000 7fb017a00640 1 -- 172.21.15.107:0/3271475370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010101df0 msgr2=0x7fb0101021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.112+0000 7fb017a00640 1 --2- 172.21.15.107:0/3271475370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010101df0 0x7fb0101021d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb004009900 tx=0x7fb00402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.112+0000 7fb017a00640 1 -- 172.21.15.107:0/3271475370 shutdown_connections 2024-09-19T04:20:14.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.112+0000 7fb017a00640 1 --2- 172.21.15.107:0/3271475370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb010106f20 0x7fb01010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.112+0000 7fb017a00640 1 --2- 172.21.15.107:0/3271475370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0101027a0 0x7fb0101067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.112+0000 7fb017a00640 1 --2- 172.21.15.107:0/3271475370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010101df0 0x7fb0101021d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.112+0000 7fb017a00640 1 -- 172.21.15.107:0/3271475370 >> 172.21.15.107:0/3271475370 conn(0x7fb0100fd8b0 msgr2=0x7fb0100ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.112+0000 7fb017a00640 1 -- 172.21.15.107:0/3271475370 shutdown_connections 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.112+0000 7fb017a00640 1 -- 172.21.15.107:0/3271475370 wait complete. 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.113+0000 7fb017a00640 1 Processor -- start 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb017a00640 1 -- start start 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb017a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb010101df0 0x7fb01019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb017a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0101027a0 0x7fb01019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb017a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106f20 0x7fb01019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb017a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb010111fd0 con 0x7fb0101027a0 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb017a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb010111e50 con 0x7fb010106f20 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb017a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb010112150 con 0x7fb010101df0 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb015000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb010101df0 0x7fb01019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb015000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb010101df0 0x7fb01019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:40382/0 (socket says 172.21.15.107:40382) 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb00fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0101027a0 0x7fb01019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb015000640 1 -- 172.21.15.107:0/3678055989 learned_addr learned my addr 172.21.15.107:0/3678055989 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.114+0000 7fb015a00640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106f20 0x7fb01019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb015000640 1 -- 172.21.15.107:0/3678055989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106f20 msgr2=0x7fb01019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb015000640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106f20 0x7fb01019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb015000640 1 -- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0101027a0 msgr2=0x7fb01019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb015000640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0101027a0 0x7fb01019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb015000640 1 -- 172.21.15.107:0/3678055989 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01019f5f0 con 0x7fb010101df0 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb00fe00640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0101027a0 0x7fb01019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb015000640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb010101df0 0x7fb01019a640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb004004770 tx=0x7fb004003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb00d600640 1 -- 172.21.15.107:0/3678055989 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00403f070 con 0x7fb010101df0 2024-09-19T04:20:14.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb017a00640 1 -- 172.21.15.107:0/3678055989 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01019f880 con 0x7fb010101df0 2024-09-19T04:20:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb00d600640 1 -- 172.21.15.107:0/3678055989 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb004031ca0 con 0x7fb010101df0 2024-09-19T04:20:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.115+0000 7fb00d600640 1 -- 172.21.15.107:0/3678055989 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004035030 con 0x7fb010101df0 2024-09-19T04:20:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.116+0000 7fb017a00640 1 -- 172.21.15.107:0/3678055989 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0101a71c0 con 0x7fb010101df0 2024-09-19T04:20:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.117+0000 7fb017a00640 1 -- 172.21.15.107:0/3678055989 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0101021d0 con 0x7fb010101df0 2024-09-19T04:20:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.118+0000 7fb00d600640 1 -- 172.21.15.107:0/3678055989 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7fb004042070 con 0x7fb010101df0 2024-09-19T04:20:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.120+0000 7fb00d600640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7faff80a3290 0x7faff80a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.120+0000 7fb00fe00640 1 -- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7faff80a3290 msgr2=0x7faff80a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3991722475 2024-09-19T04:20:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.120+0000 7fb00d600640 1 -- 172.21.15.107:0/3678055989 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fb0040eff80 con 0x7fb010101df0 2024-09-19T04:20:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.120+0000 7fb00fe00640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7faff80a3290 0x7faff80a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:20:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.123+0000 7fb00d600640 1 -- 172.21.15.107:0/3678055989 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb0040bc6e0 con 0x7fb010101df0 2024-09-19T04:20:14.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1891683270' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T04:20:14.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-mon[35096]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T04:20:14.224 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:13.983+0000 7fa066a69040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:20:14.224 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:14.043+0000 7fa066a69040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:20:14.228 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1891683270' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T04:20:14.228 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-mon[35477]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T04:20:14.228 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:13.982+0000 7f8b727c7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:20:14.228 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:14 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:14.043+0000 7f8b727c7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:20:14.228 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:14 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:14.226+0000 7f8b727c7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:20:14.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.321+0000 7fb00fe00640 1 -- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7faff80a3290 msgr2=0x7faff80a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3991722475 2024-09-19T04:20:14.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.321+0000 7fb00fe00640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7faff80a3290 0x7faff80a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:20:14.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.375+0000 7fb017a00640 1 -- 172.21.15.107:0/3678055989 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb0101a7390 con 0x7fb010101df0 2024-09-19T04:20:14.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.375+0000 7fb00d600640 1 -- 172.21.15.107:0/3678055989 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fb0040dc020 con 0x7fb010101df0 2024-09-19T04:20:14.377 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T04:20:14.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 -- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7faff80a3290 msgr2=0x7faff80a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7faff80a3290 0x7faff80a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 -- 172.21.15.107:0/3678055989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb010101df0 msgr2=0x7fb01019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb010101df0 0x7fb01019a640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb004004770 tx=0x7fb004003c10 comp rx=0 tx=0).stop 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 -- 172.21.15.107:0/3678055989 shutdown_connections 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7faff80a3290 0x7faff80a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106f20 0x7fb01019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0101027a0 0x7fb01019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 --2- 172.21.15.107:0/3678055989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb010101df0 0x7fb01019a640 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 -- 172.21.15.107:0/3678055989 >> 172.21.15.107:0/3678055989 conn(0x7fb0100fd8b0 msgr2=0x7fb0100fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 -- 172.21.15.107:0/3678055989 shutdown_connections 2024-09-19T04:20:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:14.378+0000 7fb017a00640 1 -- 172.21.15.107:0/3678055989 wait complete. 2024-09-19T04:20:14.624 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:14.330+0000 7fd9dc638040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:20:14.624 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:14.406+0000 7fd9dc638040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:20:14.724 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:14.226+0000 7fa066a69040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:20:14.957 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:14.622+0000 7fd9dc638040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:20:15.112 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:20:15.112 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:20:15.112 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-09-19T04:20:15.112 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:20:15.112 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:20:15.113 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:20:15.113 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:20:15.113 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:20:15.113 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:20:15.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:15.113 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-09-19T04:20:15.149 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:20:15.149 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:20:15.149 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:20:15.149 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:20:15.149 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-09-19T04:20:15.149 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:20:15.149 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:20:15.149 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42089sec preferred_lft 42089sec 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link noprefixroute 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:20:15.151 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T04:20:15.151 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:20:15.151 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-09-19T04:20:15.151 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-09-19T04:20:15.151 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-09-19T04:20:15.151 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-09-19T04:20:15.152 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T04:20:15.152 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-09-19T04:20:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:14 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3678055989' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:20:15.224 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:15 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:15.103+0000 7fa066a69040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:20:15.224 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:15 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:15.170+0000 7fa066a69040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:20:15.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:14 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3678055989' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:20:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:20:15.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:14 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3678055989' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:20:15.446 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:15.108+0000 7f8b727c7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:20:15.446 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:15.176+0000 7f8b727c7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:20:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:15.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:15.572 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:20:15.669 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-09-19T04:20:15.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:15.673 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-09-19T04:20:15.726 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-09-19T04:20:15.726 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:20:15.726 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:20:15.726 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:20:15.726 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:20:15.727 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:20:15.727 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-09-19T04:20:15.727 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:20:15.727 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:20:15.727 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:20:15.727 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-09-19T04:20:15.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:20:15.824 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:15 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:15.775+0000 7fd9dc638040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:20:16.089 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:15 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:15.859+0000 7fd9dc638040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:20:16.146 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:15.887+0000 7f8b727c7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:20:16.146 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.002+0000 7f8b727c7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:20:16.185 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:15 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:15.915+0000 7fa066a69040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:20:16.185 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.035+0000 7fa066a69040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:20:16.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:16.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:16.353 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-09-19T04:20:16.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:16.391 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-09-19T04:20:16.421 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.144+0000 7f8b727c7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:20:16.421 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.206+0000 7f8b727c7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:20:16.421 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.353+0000 7f8b727c7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:20:16.421 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.419+0000 7f8b727c7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:20:16.467 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:20:16.467 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-09-19T04:20:16.468 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:20:16.468 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T04:20:16.468 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-09-19T04:20:16.474 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.183+0000 7fa066a69040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:20:16.474 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.247+0000 7fa066a69040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:20:16.474 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.402+0000 7fa066a69040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:20:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:20:16.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:16.662 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T04:20:16.663 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:20:16.663 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-09-19T04:20:16.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T04:20:16.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T04:20:16.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T04:20:16.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T04:20:16.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T04:20:16.663 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-09-19T04:20:16.696 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.479+0000 7f8b727c7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:20:16.696 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.637+0000 7f8b727c7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:20:16.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:20:16.760 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.473+0000 7fa066a69040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:20:16.760 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.534+0000 7fa066a69040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:20:16.760 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.693+0000 7fa066a69040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:20:16.760 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.758+0000 7fa066a69040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:20:16.871 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:16 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:16.833+0000 7fd9dc638040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:20:16.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:16.918 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:20:16.918 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-09-19T04:20:16.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-09-19T04:20:16.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T04:20:16.918 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-09-19T04:20:16.973 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.703+0000 7f8b727c7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:20:16.973 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.770+0000 7f8b727c7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:20:16.973 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.853+0000 7f8b727c7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:20:16.973 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.913+0000 7f8b727c7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:20:17.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:20:17.028 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.826+0000 7fa066a69040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:20:17.028 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.909+0000 7fa066a69040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:20:17.028 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:16.968+0000 7fa066a69040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:20:17.028 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:17.027+0000 7fa066a69040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:20:17.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:17.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:20:17.096 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T04:20:17.131 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:20:17.132 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:20:17.132 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:20:17.156 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:16 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:16.980+0000 7fd9dc638040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:20:17.192 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T04:20:17.192 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T04:20:17.193 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/26bf594d-d66e-4c83-a2f3-faba049e9da5 2024-09-19T04:20:17.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:17.193 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/26bf594d-d66e-4c83-a2f3-faba049e9da5 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T04:20:17.345 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T04:20:17.345 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-19T04:20:17.413 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:17.154+0000 7fd9dc638040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:20:17.413 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:17.230+0000 7fd9dc638040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:20:17.719 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:17.411+0000 7fd9dc638040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:20:17.719 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:17.499+0000 7fd9dc638040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:20:17.720 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:17.575+0000 7fd9dc638040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:20:18.018 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:17.765+0000 7fd9dc638040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:20:18.018 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:17.849+0000 7fd9dc638040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:20:18.018 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:17.931+0000 7fd9dc638040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:20:18.019 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-mon[30659]: Standby manager daemon y restarted 2024-09-19T04:20:18.019 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-mon[30659]: Standby manager daemon y started 2024-09-19T04:20:18.019 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:20:18.019 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:20:18.019 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:20:18.020 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:17 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:20:18.026 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T04:20:18.026 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-19T04:20:18.027 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.107:6789,172.21.15.115:6789,172.21.15.145:6789" 2024-09-19T04:20:18.027 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with new device syntax: 0@e2ec5b0e-763c-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/26bf594d-d66e-4c83-a2f3-faba049e9da5 2024-09-19T04:20:18.027 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.107:6789/172.21.15.115:6789/172.21.15.145:6789" will pass to kernel 2024-09-19T04:20:18.030 INFO:teuthology.orchestra.run.smithi107.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T04:20:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: You just mounted a file system that supports labels which does not 2024-09-19T04:20:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T04:20:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T04:20:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T04:20:18.033 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T04:20:18.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:18.033 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:20:18.093 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T04:20:18.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:20:18.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:18.187 INFO:teuthology.orchestra.run.smithi107.stdout:client15123 2024-09-19T04:20:18.187 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:20:18.187 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:20:18.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:16.971+0000 7f8b727c7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:20:18.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:20:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y[36839]: 2024-09-19T04:20:17.756+0000 7f8b727c7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:20:18.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:17 smithi115 ceph-mon[35477]: Standby manager daemon y restarted 2024-09-19T04:20:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:17 smithi115 ceph-mon[35477]: Standby manager daemon y started 2024-09-19T04:20:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:17 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:20:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:17 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:20:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:17 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:20:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:17 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:20:18.225 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T04:20:18.225 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:20:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z[36913]: 2024-09-19T04:20:17.814+0000 7fa066a69040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:20:18.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:17 smithi145 ceph-mon[35096]: Standby manager daemon y restarted 2024-09-19T04:20:18.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:17 smithi145 ceph-mon[35096]: Standby manager daemon y started 2024-09-19T04:20:18.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:17 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:20:18.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:17 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:20:18.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:17 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:20:18.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:17 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.115:0/878660993' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:20:18.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:20:18.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:18.357 INFO:teuthology.orchestra.run.smithi107.stdout:client15123 2024-09-19T04:20:18.357 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T04:20:18.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:20:18.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:18.448 INFO:teuthology.orchestra.run.smithi107.stdout:client15123 2024-09-19T04:20:18.448 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968.client15123/status') 2024-09-19T04:20:18.480 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:18.036+0000 7fd9dc638040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:20:18.480 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:18.113+0000 7fd9dc638040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:20:18.481 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:18.189+0000 7fd9dc638040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:20:18.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:20:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:instance: client.15123 (3)192.168.0.1:0/1912512800 2024-09-19T04:20:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:blocklisted: false 2024-09-19T04:20:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records in 2024-09-19T04:20:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records out 2024-09-19T04:20:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:70 bytes copied, 7.0611e-05 s, 991 kB/s 2024-09-19T04:20:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:18.561 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T04:20:18.570 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:20:18.570 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-09-19T04:20:18.570 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-19T04:20:18.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.756+0000 7f9fef000640 1 -- 172.21.15.107:0/837501502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fe8102d50 msgr2=0x7f9fe81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:18.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.756+0000 7f9fef000640 1 --2- 172.21.15.107:0/837501502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fe8102d50 0x7f9fe81031d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009900 tx=0x7f9fd402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.756+0000 7f9fef000640 1 -- 172.21.15.107:0/837501502 shutdown_connections 2024-09-19T04:20:18.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.756+0000 7f9fef000640 1 --2- 172.21.15.107:0/837501502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe8103710 0x7f9fe8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.756+0000 7f9fef000640 1 --2- 172.21.15.107:0/837501502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fe8102d50 0x7f9fe81031d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.756+0000 7f9fef000640 1 --2- 172.21.15.107:0/837501502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8101b60 0x7f9fe8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.756+0000 7f9fef000640 1 -- 172.21.15.107:0/837501502 >> 172.21.15.107:0/837501502 conn(0x7f9fe80fd2e0 msgr2=0x7f9fe80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:18.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.756+0000 7f9fef000640 1 -- 172.21.15.107:0/837501502 shutdown_connections 2024-09-19T04:20:18.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.756+0000 7f9fef000640 1 -- 172.21.15.107:0/837501502 wait complete. 2024-09-19T04:20:18.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.757+0000 7f9fef000640 1 Processor -- start 2024-09-19T04:20:18.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.757+0000 7f9fef000640 1 -- start start 2024-09-19T04:20:18.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fef000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fe8101b60 0x7f9fe8199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:18.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fef000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8102d50 0x7f9fe819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:18.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fef000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe8103710 0x7f9fe81a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:18.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fef000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe810d7c0 con 0x7f9fe8101b60 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fef000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe810d640 con 0x7f9fe8103710 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fef000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe810d940 con 0x7f9fe8102d50 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fe7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fe8101b60 0x7f9fe8199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fe7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fe8101b60 0x7f9fe8199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52310/0 (socket says 172.21.15.107:52310) 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fe7e00640 1 -- 172.21.15.107:0/3225367855 learned_addr learned my addr 172.21.15.107:0/3225367855 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fed000640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe8103710 0x7f9fe81a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fe7400640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8102d50 0x7f9fe819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fe7e00640 1 -- 172.21.15.107:0/3225367855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8102d50 msgr2=0x7f9fe819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fe7e00640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8102d50 0x7f9fe819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fe7e00640 1 -- 172.21.15.107:0/3225367855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe8103710 msgr2=0x7f9fe81a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fe7e00640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe8103710 0x7f9fe81a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fe7e00640 1 -- 172.21.15.107:0/3225367855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe81a1be0 con 0x7f9fe8101b60 2024-09-19T04:20:18.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.758+0000 7f9fed000640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe8103710 0x7f9fe81a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:18.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.759+0000 7f9fe7e00640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fe8101b60 0x7f9fe8199fc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800e9c0 tx=0x7f9fd800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:18.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.759+0000 7f9fe4c00640 1 -- 172.21.15.107:0/3225367855 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd800cd80 con 0x7f9fe8101b60 2024-09-19T04:20:18.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.759+0000 7f9fe4c00640 1 -- 172.21.15.107:0/3225367855 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd80035f0 con 0x7f9fe8101b60 2024-09-19T04:20:18.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.759+0000 7f9fe4c00640 1 -- 172.21.15.107:0/3225367855 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8010690 con 0x7f9fe8101b60 2024-09-19T04:20:18.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.759+0000 7f9fef000640 1 -- 172.21.15.107:0/3225367855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe81a1ed0 con 0x7f9fe8101b60 2024-09-19T04:20:18.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.759+0000 7f9fef000640 1 -- 172.21.15.107:0/3225367855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe81a2330 con 0x7f9fe8101b60 2024-09-19T04:20:18.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.760+0000 7f9fef000640 1 -- 172.21.15.107:0/3225367855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe81a25c0 con 0x7f9fe8101b60 2024-09-19T04:20:18.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.762+0000 7f9fe4c00640 1 -- 172.21.15.107:0/3225367855 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 138373+0+0 (secure 0 0 0) 0x7f9fd80037a0 con 0x7f9fe8101b60 2024-09-19T04:20:18.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.763+0000 7f9fe4c00640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9fcc0a3340 0x7f9fcc0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:18.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.763+0000 7f9fe7400640 1 -- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9fcc0a3340 msgr2=0x7f9fcc0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3991722475 2024-09-19T04:20:18.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.763+0000 7f9fe4c00640 1 -- 172.21.15.107:0/3225367855 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f9fd80cfee0 con 0x7f9fe8101b60 2024-09-19T04:20:18.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.763+0000 7f9fe7400640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9fcc0a3340 0x7f9fcc0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:20:18.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.767+0000 7f9fe4c00640 1 -- 172.21.15.107:0/3225367855 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9fd809cbd0 con 0x7f9fe8101b60 2024-09-19T04:20:18.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.825+0000 7f9fe4c00640 1 -- 172.21.15.107:0/3225367855 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 21) ==== 138373+0+0 (secure 0 0 0) 0x7f9fd8098b40 con 0x7f9fe8101b60 2024-09-19T04:20:18.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.964+0000 7f9fe7400640 1 -- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9fcc0a3340 msgr2=0x7f9fcc0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3991722475 2024-09-19T04:20:18.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.964+0000 7f9fe7400640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9fcc0a3340 0x7f9fcc0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:20:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:18.992+0000 7f9fef000640 1 -- 172.21.15.107:0/3225367855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f9fe81a2790 con 0x7f9fe8101b60 2024-09-19T04:20:19.150 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:18 smithi107 ceph-mon[30659]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-19T04:20:19.150 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:18 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:20:19.150 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:18 smithi107 ceph-mon[30659]: Standby manager daemon z restarted 2024-09-19T04:20:19.150 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:18 smithi107 ceph-mon[30659]: Standby manager daemon z started 2024-09-19T04:20:19.150 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:18 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:20:19.150 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:18 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:20:19.150 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:18 smithi107 ceph-mon[30659]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:20:19.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.163+0000 7f9fe4c00640 1 -- 172.21.15.107:0/3225367855 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 22) ==== 138040+0+0 (secure 0 0 0) 0x7f9fd80cda30 con 0x7f9fe8101b60 2024-09-19T04:20:19.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.163+0000 7f9fe4c00640 1 -- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9fcc0a3340 msgr2=0x7f9fcc0a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:20:19.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.163+0000 7f9fe4c00640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9fcc0a3340 0x7f9fcc0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:18 smithi115 ceph-mon[35477]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-19T04:20:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:18 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:20:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:18 smithi115 ceph-mon[35477]: Standby manager daemon z restarted 2024-09-19T04:20:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:18 smithi115 ceph-mon[35477]: Standby manager daemon z started 2024-09-19T04:20:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:18 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:20:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:18 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:20:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:18 smithi115 ceph-mon[35477]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:20:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:18 smithi145 ceph-mon[35096]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-19T04:20:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:18 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:20:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:18 smithi145 ceph-mon[35096]: Standby manager daemon z restarted 2024-09-19T04:20:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:18 smithi145 ceph-mon[35096]: Standby manager daemon z started 2024-09-19T04:20:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:18 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:20:19.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:18 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:20:19.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:18 smithi145 ceph-mon[35096]: from='mgr.? 172.21.15.145:0/599546279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:20:19.405 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.148+0000 7fd9dc638040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:20:19.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.446+0000 7fd934e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:20:19.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.446+0000 7fd934e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:20:19.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.446+0000 7fd934e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:20:19.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.446+0000 7fd934e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:20:19.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.446+0000 7fd934e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:20:19.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.449+0000 7fd939400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:20:19.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.449+0000 7fd939400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:20:19.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.449+0000 7fd939400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:20:19.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.449+0000 7fd939400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:20:19.732 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:19.449+0000 7fd939400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:20:19.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.828+0000 7f9fe4c00640 1 -- 172.21.15.107:0/3225367855 <== mon.0 v2:172.21.15.107:3300/0 9 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f9fd80cbc60 con 0x7f9fe8101b60 2024-09-19T04:20:19.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.832+0000 7f9fef000640 1 -- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fe8101b60 msgr2=0x7f9fe8199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.832+0000 7f9fef000640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fe8101b60 0x7f9fe8199fc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800e9c0 tx=0x7f9fd800ee80 comp rx=0 tx=0).stop 2024-09-19T04:20:19.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.832+0000 7f9fef000640 1 -- 172.21.15.107:0/3225367855 shutdown_connections 2024-09-19T04:20:19.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.832+0000 7f9fef000640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:6800/3991722475,v1:172.21.15.107:6801/3991722475] conn(0x7f9fcc0a3340 0x7f9fcc0a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.832+0000 7f9fef000640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe8103710 0x7f9fe81a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.832+0000 7f9fef000640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8102d50 0x7f9fe819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.832+0000 7f9fef000640 1 --2- 172.21.15.107:0/3225367855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fe8101b60 0x7f9fe8199fc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.832+0000 7f9fef000640 1 -- 172.21.15.107:0/3225367855 >> 172.21.15.107:0/3225367855 conn(0x7f9fe80fd2e0 msgr2=0x7f9fe8074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:19.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.832+0000 7f9fef000640 1 -- 172.21.15.107:0/3225367855 shutdown_connections 2024-09-19T04:20:19.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:19.832+0000 7f9fef000640 1 -- 172.21.15.107:0/3225367855 wait complete. 2024-09-19T04:20:19.857 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T04:20:19.868 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T04:20:19.912 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:20:20.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.071+0000 7f342c400640 1 -- 172.21.15.107:0/986244584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3424077df0 msgr2=0x7f3424078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.071+0000 7f342c400640 1 --2- 172.21.15.107:0/986244584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3424077df0 0x7f3424078250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3410009900 tx=0x7f341002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.071+0000 7f342c400640 1 -- 172.21.15.107:0/986244584 shutdown_connections 2024-09-19T04:20:20.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.071+0000 7f342c400640 1 --2- 172.21.15.107:0/986244584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3424078790 0x7f3424079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.071+0000 7f342c400640 1 --2- 172.21.15.107:0/986244584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3424077df0 0x7f3424078250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.071+0000 7f342c400640 1 --2- 172.21.15.107:0/986244584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3424076bb0 0x7f3424076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.071+0000 7f342c400640 1 -- 172.21.15.107:0/986244584 >> 172.21.15.107:0/986244584 conn(0x7f34240ff4b0 msgr2=0x7f34241018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:20.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.071+0000 7f342c400640 1 -- 172.21.15.107:0/986244584 shutdown_connections 2024-09-19T04:20:20.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.071+0000 7f342c400640 1 -- 172.21.15.107:0/986244584 wait complete. 2024-09-19T04:20:20.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342c400640 1 Processor -- start 2024-09-19T04:20:20.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342c400640 1 -- start start 2024-09-19T04:20:20.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342c400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3424076bb0 0x7f342419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342c400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3424077df0 0x7f342419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342c400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3424078790 0x7f34241a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342c400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3424113e10 con 0x7f3424076bb0 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342c400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3424113c90 con 0x7f3424077df0 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342c400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3424113f90 con 0x7f3424078790 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f3429a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3424076bb0 0x7f342419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3424078790 0x7f34241a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f3429000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3424077df0 0x7f342419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3424078790 0x7f34241a5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46450/0 (socket says 172.21.15.107:46450) 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.072+0000 7f342a400640 1 -- 172.21.15.107:0/836682285 learned_addr learned my addr 172.21.15.107:0/836682285 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.073+0000 7f342a400640 1 -- 172.21.15.107:0/836682285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3424077df0 msgr2=0x7f342419e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.073+0000 7f342a400640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3424077df0 0x7f342419e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.073+0000 7f342a400640 1 -- 172.21.15.107:0/836682285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3424076bb0 msgr2=0x7f342419e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.073+0000 7f342a400640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3424076bb0 0x7f342419e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.073+0000 7f342a400640 1 -- 172.21.15.107:0/836682285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34241a60a0 con 0x7f3424078790 2024-09-19T04:20:20.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.073+0000 7f342a400640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3424078790 0x7f34241a5a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f341c00eb10 tx=0x7f341c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:20.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.073+0000 7f3422000640 1 -- 172.21.15.107:0/836682285 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c004b20 con 0x7f3424078790 2024-09-19T04:20:20.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.073+0000 7f342c400640 1 -- 172.21.15.107:0/836682285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34241a6390 con 0x7f3424078790 2024-09-19T04:20:20.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.073+0000 7f3422000640 1 -- 172.21.15.107:0/836682285 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f341c004cc0 con 0x7f3424078790 2024-09-19T04:20:20.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.073+0000 7f3422000640 1 -- 172.21.15.107:0/836682285 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c010760 con 0x7f3424078790 2024-09-19T04:20:20.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.074+0000 7f342c400640 1 -- 172.21.15.107:0/836682285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34241a6900 con 0x7f3424078790 2024-09-19T04:20:20.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.075+0000 7f342c400640 1 -- 172.21.15.107:0/836682285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3424110840 con 0x7f3424078790 2024-09-19T04:20:20.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.078+0000 7f3422000640 1 -- 172.21.15.107:0/836682285 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 22) ==== 138040+0+0 (secure 0 0 0) 0x7f341c005020 con 0x7f3424078790 2024-09-19T04:20:20.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.079+0000 7f3422000640 1 -- 172.21.15.107:0/836682285 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f341c0cfaf0 con 0x7f3424078790 2024-09-19T04:20:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.084+0000 7f3422000640 1 -- 172.21.15.107:0/836682285 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f341c09cb40 con 0x7f3424078790 2024-09-19T04:20:20.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.174+0000 7f3422000640 1 -- 172.21.15.107:0/836682285 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mgrmap(e 23) ==== 138167+0+0 (secure 0 0 0) 0x7f341c098190 con 0x7f3424078790 2024-09-19T04:20:20.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.175+0000 7f3422000640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f340c0a9ec0 0x7f340c0ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.179+0000 7f3429a00640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f340c0a9ec0 0x7f340c0ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.184+0000 7f3429a00640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f340c0a9ec0 0x7f340c0ac2b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3418009bd0 tx=0x7f3418005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:20.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3225367855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: Active manager daemon x restarted 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: Activating manager daemon x 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: mgrmap e22: x(active, starting, since 0.0134713s), standbys: z, y 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:20:20.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:20:20.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:20:20.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:20:20.200 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:20:20.200 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:20:20.200 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:20:20.200 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:20:20.200 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: Manager daemon x is now available 2024-09-19T04:20:20.200 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:20.200 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:20:20.200 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:19 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:20:20.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3225367855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: Active manager daemon x restarted 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: Activating manager daemon x 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: mgrmap e22: x(active, starting, since 0.0134713s), standbys: z, y 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:20:20.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:20:20.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: Manager daemon x is now available 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:20.227 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:20:20.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:19 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3225367855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: Active manager daemon x restarted 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: Activating manager daemon x 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: mgrmap e22: x(active, starting, since 0.0134713s), standbys: z, y 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:20:20.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:20:20.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:20:20.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:20:20.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:20:20.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:20:20.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: Manager daemon x is now available 2024-09-19T04:20:20.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:20.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:20:20.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:19 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:20:20.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.304+0000 7f342c400640 1 -- 172.21.15.107:0/836682285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3424062df0 con 0x7f3424078790 2024-09-19T04:20:20.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.305+0000 7f3422000640 1 -- 172.21.15.107:0/836682285 <== mon.1 v2:172.21.15.145:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+23119 (secure 0 0 0) 0x7f341c057f60 con 0x7f3424078790 2024-09-19T04:20:20.307 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:20:20.307 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":84,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:20:19.150785+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6800/3991722475":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/3866113757":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/1307215535":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/3489252098":"2024-09-20T04:20:19.150743+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/3991722475":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/3889998535":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 -- 172.21.15.107:0/836682285 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f340c0a9ec0 msgr2=0x7f340c0ac2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f340c0a9ec0 0x7f340c0ac2b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3418009bd0 tx=0x7f3418005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 -- 172.21.15.107:0/836682285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3424078790 msgr2=0x7f34241a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3424078790 0x7f34241a5a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f341c00eb10 tx=0x7f341c00c2d0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 -- 172.21.15.107:0/836682285 shutdown_connections 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f340c0a9ec0 0x7f340c0ac2b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3424078790 0x7f34241a5a30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3424077df0 0x7f342419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 --2- 172.21.15.107:0/836682285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3424076bb0 0x7f342419e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 -- 172.21.15.107:0/836682285 >> 172.21.15.107:0/836682285 conn(0x7f34240ff4b0 msgr2=0x7f3424100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:20.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 -- 172.21.15.107:0/836682285 shutdown_connections 2024-09-19T04:20:20.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.308+0000 7f342c400640 1 -- 172.21.15.107:0/836682285 wait complete. 2024-09-19T04:20:20.329 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:13:18.308041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T04:18:09.031386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T04:18:11.455271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:20:20.330 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:20:20.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.505+0000 7f09e6e00640 1 -- 172.21.15.107:0/2049950482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09e0074c50 msgr2=0x7f09e00750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.505+0000 7f09e6e00640 1 --2- 172.21.15.107:0/2049950482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09e0074c50 0x7f09e00750b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009900 tx=0x7f09cc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.505+0000 7f09e6e00640 1 -- 172.21.15.107:0/2049950482 shutdown_connections 2024-09-19T04:20:20.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.505+0000 7f09e6e00640 1 --2- 172.21.15.107:0/2049950482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e010dfd0 0x7f09e01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.505+0000 7f09e6e00640 1 --2- 172.21.15.107:0/2049950482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09e0074c50 0x7f09e00750b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.505+0000 7f09e6e00640 1 --2- 172.21.15.107:0/2049950482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e0076230 0x7f09e0074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.505+0000 7f09e6e00640 1 -- 172.21.15.107:0/2049950482 >> 172.21.15.107:0/2049950482 conn(0x7f09e00fd180 msgr2=0x7f09e00ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:20.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.505+0000 7f09e6e00640 1 -- 172.21.15.107:0/2049950482 shutdown_connections 2024-09-19T04:20:20.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.505+0000 7f09e6e00640 1 -- 172.21.15.107:0/2049950482 wait complete. 2024-09-19T04:20:20.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e6e00640 1 Processor -- start 2024-09-19T04:20:20.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e6e00640 1 -- start start 2024-09-19T04:20:20.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e6e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e0074c50 0x7f09e019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e6e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09e0076230 0x7f09e019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e6e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e010dfd0 0x7f09e01a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e6e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0113c90 con 0x7f09e010dfd0 2024-09-19T04:20:20.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e6e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0113b10 con 0x7f09e0076230 2024-09-19T04:20:20.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e6e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0113e10 con 0x7f09e0074c50 2024-09-19T04:20:20.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e0074c50 0x7f09e019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e0074c50 0x7f09e019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46468/0 (socket says 172.21.15.107:46468) 2024-09-19T04:20:20.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e5a00640 1 -- 172.21.15.107:0/4036024664 learned_addr learned my addr 172.21.15.107:0/4036024664 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:20.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.506+0000 7f09e5000640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09e0076230 0x7f09e019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09e6400640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e010dfd0 0x7f09e01a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09e5a00640 1 -- 172.21.15.107:0/4036024664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09e0076230 msgr2=0x7f09e019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09e5a00640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09e0076230 0x7f09e019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09e5a00640 1 -- 172.21.15.107:0/4036024664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e010dfd0 msgr2=0x7f09e01a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09e5a00640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e010dfd0 0x7f09e01a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09e5a00640 1 -- 172.21.15.107:0/4036024664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e01a5f20 con 0x7f09e0074c50 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09e5a00640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e0074c50 0x7f09e019e300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f09d400e9f0 tx=0x7f09d400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09de000640 1 -- 172.21.15.107:0/4036024664 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d400cd80 con 0x7f09e0074c50 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09e6e00640 1 -- 172.21.15.107:0/4036024664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e01a6210 con 0x7f09e0074c50 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09de000640 1 -- 172.21.15.107:0/4036024664 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d4005500 con 0x7f09e0074c50 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.507+0000 7f09de000640 1 -- 172.21.15.107:0/4036024664 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4010690 con 0x7f09e0074c50 2024-09-19T04:20:20.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.508+0000 7f09e6e00640 1 -- 172.21.15.107:0/4036024664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e01a6730 con 0x7f09e0074c50 2024-09-19T04:20:20.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.509+0000 7f09e6e00640 1 -- 172.21.15.107:0/4036024664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e0104080 con 0x7f09e0074c50 2024-09-19T04:20:20.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.511+0000 7f09de000640 1 -- 172.21.15.107:0/4036024664 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 23) ==== 138167+0+0 (secure 0 0 0) 0x7f09d40046c0 con 0x7f09e0074c50 2024-09-19T04:20:20.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.512+0000 7f09de000640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09c80a3050 0x7f09c80a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.513+0000 7f09de000640 1 -- 172.21.15.107:0/4036024664 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f09d40cfdb0 con 0x7f09e0074c50 2024-09-19T04:20:20.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.513+0000 7f09e5000640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09c80a3050 0x7f09c80a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.514+0000 7f09e5000640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09c80a3050 0x7f09c80a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0314e0 tx=0x7f09cc000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:20.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.514+0000 7f09de000640 1 -- 172.21.15.107:0/4036024664 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f09d409ccb0 con 0x7f09e0074c50 2024-09-19T04:20:20.668 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:20.513+0000 7fd91d200640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:20:20] ENGINE Error in HTTPServer.serve 2024-09-19T04:20:20.668 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: Traceback (most recent call last): 2024-09-19T04:20:20.668 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:20:20.668 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self._connections.run(self.expiration_interval) 2024-09-19T04:20:20.668 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:20:20.668 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self._run(expiration_interval) 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: s = self.context.wrap_socket( 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: return self.sslsocket_class._create( 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self.do_handshake() 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: self._sslobj.do_handshake() 2024-09-19T04:20:20.669 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:20:20.670 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:20:20.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.699+0000 7f09e6e00640 1 -- 172.21.15.107:0/4036024664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09e002da70 con 0x7f09e0074c50 2024-09-19T04:20:20.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.699+0000 7f09de000640 1 -- 172.21.15.107:0/4036024664 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f09d409c5b0 con 0x7f09e0074c50 2024-09-19T04:20:20.701 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-09-19T04:20:20.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.702+0000 7f09e6e00640 1 -- 172.21.15.107:0/4036024664 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09c80a3050 msgr2=0x7f09c80a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09c80a3050 0x7f09c80a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0314e0 tx=0x7f09cc000bc0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 -- 172.21.15.107:0/4036024664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e0074c50 msgr2=0x7f09e019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e0074c50 0x7f09e019e300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f09d400e9f0 tx=0x7f09d400edf0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 -- 172.21.15.107:0/4036024664 shutdown_connections 2024-09-19T04:20:20.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09c80a3050 0x7f09c80a5510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e010dfd0 0x7f09e01a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09e0076230 0x7f09e019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 --2- 172.21.15.107:0/4036024664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e0074c50 0x7f09e019e300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 -- 172.21.15.107:0/4036024664 >> 172.21.15.107:0/4036024664 conn(0x7f09e00fd180 msgr2=0x7f09e0101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:20.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 -- 172.21.15.107:0/4036024664 shutdown_connections 2024-09-19T04:20:20.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.703+0000 7f09e6e00640 1 -- 172.21.15.107:0/4036024664 wait complete. 2024-09-19T04:20:20.727 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T04:20:20.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.891+0000 7fa7b7200640 1 -- 172.21.15.107:0/3579524904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7b81035b0 msgr2=0x7fa7b8109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.891+0000 7fa7b7200640 1 --2- 172.21.15.107:0/3579524904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7b81035b0 0x7fa7b8109e40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009ca0 tx=0x7fa7b002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.891+0000 7fa7b7200640 1 -- 172.21.15.107:0/3579524904 shutdown_connections 2024-09-19T04:20:20.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.891+0000 7fa7b7200640 1 --2- 172.21.15.107:0/3579524904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7b81035b0 0x7fa7b8109e40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.891+0000 7fa7b7200640 1 --2- 172.21.15.107:0/3579524904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7b8102bf0 0x7fa7b8103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.891+0000 7fa7b7200640 1 --2- 172.21.15.107:0/3579524904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b8101a00 0x7fa7b8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.891+0000 7fa7b7200640 1 -- 172.21.15.107:0/3579524904 >> 172.21.15.107:0/3579524904 conn(0x7fa7b80fd180 msgr2=0x7fa7b80ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:20.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.891+0000 7fa7b7200640 1 -- 172.21.15.107:0/3579524904 shutdown_connections 2024-09-19T04:20:20.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.891+0000 7fa7b7200640 1 -- 172.21.15.107:0/3579524904 wait complete. 2024-09-19T04:20:20.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.892+0000 7fa7b7200640 1 Processor -- start 2024-09-19T04:20:20.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.892+0000 7fa7b7200640 1 -- start start 2024-09-19T04:20:20.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.892+0000 7fa7b7200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b8101a00 0x7fa7b8197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.892+0000 7fa7b7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7b8102bf0 0x7fa7b8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.892+0000 7fa7b7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7b81035b0 0x7fa7b819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.892+0000 7fa7b7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810d6d0 con 0x7fa7b8102bf0 2024-09-19T04:20:20.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.892+0000 7fa7b7200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810d550 con 0x7fa7b8101a00 2024-09-19T04:20:20.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.892+0000 7fa7b7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810d850 con 0x7fa7b81035b0 2024-09-19T04:20:20.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.892+0000 7fa7afe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7b8102bf0 0x7fa7b8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7b5e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b8101a00 0x7fa7b8197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7b6800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7b81035b0 0x7fa7b819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7afe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7b8102bf0 0x7fa7b8198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54134/0 (socket says 172.21.15.107:54134) 2024-09-19T04:20:20.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7afe00640 1 -- 172.21.15.107:0/2272006671 learned_addr learned my addr 172.21.15.107:0/2272006671 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:20.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7b5e00640 1 -- 172.21.15.107:0/2272006671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7b81035b0 msgr2=0x7fa7b819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7b5e00640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7b81035b0 0x7fa7b819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7b5e00640 1 -- 172.21.15.107:0/2272006671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7b8102bf0 msgr2=0x7fa7b8198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7b5e00640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7b8102bf0 0x7fa7b8198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7b5e00640 1 -- 172.21.15.107:0/2272006671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b819f900 con 0x7fa7b8101a00 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7b6800640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7b81035b0 0x7fa7b819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7b5e00640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b8101a00 0x7fa7b8197cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800e9c0 tx=0x7fa7a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.893+0000 7fa7aea00640 1 -- 172.21.15.107:0/2272006671 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800ccc0 con 0x7fa7b8101a00 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.894+0000 7fa7b7200640 1 -- 172.21.15.107:0/2272006671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b819fbf0 con 0x7fa7b8101a00 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.894+0000 7fa7aea00640 1 -- 172.21.15.107:0/2272006671 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a80035f0 con 0x7fa7b8101a00 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.894+0000 7fa7aea00640 1 -- 172.21.15.107:0/2272006671 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800ce60 con 0x7fa7b8101a00 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.894+0000 7fa7b7200640 1 -- 172.21.15.107:0/2272006671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b81a0140 con 0x7fa7b8101a00 2024-09-19T04:20:20.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.895+0000 7fa7b7200640 1 -- 172.21.15.107:0/2272006671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b8108d20 con 0x7fa7b8101a00 2024-09-19T04:20:20.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.897+0000 7fa7aea00640 1 -- 172.21.15.107:0/2272006671 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 138167+0+0 (secure 0 0 0) 0x7fa7a80037a0 con 0x7fa7b8101a00 2024-09-19T04:20:20.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.898+0000 7fa7aea00640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa79c0a3050 0x7fa79c0a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:20.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.898+0000 7fa7aea00640 1 -- 172.21.15.107:0/2272006671 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fa7a80cfe00 con 0x7fa7b8101a00 2024-09-19T04:20:20.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.898+0000 7fa7afe00640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa79c0a3050 0x7fa79c0a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:20.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.899+0000 7fa7afe00640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa79c0a3050 0x7fa79c0a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa7b81991c0 tx=0x7fa7a0005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:20.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:20.900+0000 7fa7aea00640 1 -- 172.21.15.107:0/2272006671 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa7a809cd00 con 0x7fa7b8101a00 2024-09-19T04:20:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:20 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3225367855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T04:20:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:20 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:20:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:20 smithi115 ceph-mon[35477]: mgrmap e23: x(active, since 1.02298s), standbys: z, y 2024-09-19T04:20:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:20 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/836682285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:20:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:20 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/4036024664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:20:20.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3225367855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T04:20:20.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:20:20.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-mon[30659]: mgrmap e23: x(active, since 1.02298s), standbys: z, y 2024-09-19T04:20:20.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/836682285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:20:20.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:20 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/4036024664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:20:21.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.080+0000 7fa7b7200640 1 -- 172.21.15.107:0/2272006671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa7b802d9c0 con 0x7fa7b8101a00 2024-09-19T04:20:21.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.080+0000 7fa7aea00640 1 -- 172.21.15.107:0/2272006671 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7fa7a809c600 con 0x7fa7b8101a00 2024-09-19T04:20:21.082 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-09-19T04:20:21.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 -- 172.21.15.107:0/2272006671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa79c0a3050 msgr2=0x7fa79c0a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa79c0a3050 0x7fa79c0a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa7b81991c0 tx=0x7fa7a0005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 -- 172.21.15.107:0/2272006671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b8101a00 msgr2=0x7fa7b8197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b8101a00 0x7fa7b8197cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800e9c0 tx=0x7fa7a800ee80 comp rx=0 tx=0).stop 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 -- 172.21.15.107:0/2272006671 shutdown_connections 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa79c0a3050 0x7fa79c0a5510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7b81035b0 0x7fa7b819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7b8102bf0 0x7fa7b8198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 --2- 172.21.15.107:0/2272006671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b8101a00 0x7fa7b8197cc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 -- 172.21.15.107:0/2272006671 >> 172.21.15.107:0/2272006671 conn(0x7fa7b80fd180 msgr2=0x7fa7b8105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 -- 172.21.15.107:0/2272006671 shutdown_connections 2024-09-19T04:20:21.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.083+0000 7fa7b7200640 1 -- 172.21.15.107:0/2272006671 wait complete. 2024-09-19T04:20:21.103 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T04:20:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:20 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3225367855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T04:20:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:20 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:20:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:20 smithi145 ceph-mon[35096]: mgrmap e23: x(active, since 1.02298s), standbys: z, y 2024-09-19T04:20:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:20 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/836682285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:20:21.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:20 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/4036024664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:20:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.286+0000 7fcfa3e00640 1 -- 172.21.15.107:0/4145866552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfa4109890 msgr2=0x7fcfa410bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.286+0000 7fcfa3e00640 1 --2- 172.21.15.107:0/4145866552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfa4109890 0x7fcfa410bd50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009ca0 tx=0x7fcf9c02a950 comp rx=0 tx=0).stop 2024-09-19T04:20:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.286+0000 7fcfa3e00640 1 -- 172.21.15.107:0/4145866552 shutdown_connections 2024-09-19T04:20:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.286+0000 7fcfa3e00640 1 --2- 172.21.15.107:0/4145866552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfa4109890 0x7fcfa410bd50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.286+0000 7fcfa3e00640 1 --2- 172.21.15.107:0/4145866552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa4074bf0 0x7fcfa4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.286+0000 7fcfa3e00640 1 --2- 172.21.15.107:0/4145866552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa40761d0 0x7fcfa4074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.286+0000 7fcfa3e00640 1 -- 172.21.15.107:0/4145866552 >> 172.21.15.107:0/4145866552 conn(0x7fcfa40fd180 msgr2=0x7fcfa40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.286+0000 7fcfa3e00640 1 -- 172.21.15.107:0/4145866552 shutdown_connections 2024-09-19T04:20:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.286+0000 7fcfa3e00640 1 -- 172.21.15.107:0/4145866552 wait complete. 2024-09-19T04:20:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.287+0000 7fcfa3e00640 1 Processor -- start 2024-09-19T04:20:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.287+0000 7fcfa3e00640 1 -- start start 2024-09-19T04:20:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.287+0000 7fcfa3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa4074bf0 0x7fcfa4199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.287+0000 7fcfa3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa40761d0 0x7fcfa419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.287+0000 7fcfa3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfa4109890 0x7fcfa41a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.287+0000 7fcfa3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa410f890 con 0x7fcfa40761d0 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.287+0000 7fcfa3e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa410f710 con 0x7fcfa4109890 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.287+0000 7fcfa3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa410fa10 con 0x7fcfa4074bf0 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa4074bf0 0x7fcfa4199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfa4109890 0x7fcfa41a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa4074bf0 0x7fcfa4199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46516/0 (socket says 172.21.15.107:46516) 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa2a00640 1 -- 172.21.15.107:0/3151414318 learned_addr learned my addr 172.21.15.107:0/3151414318 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcf9be00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa40761d0 0x7fcfa419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa2a00640 1 -- 172.21.15.107:0/3151414318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfa4109890 msgr2=0x7fcfa41a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa2a00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfa4109890 0x7fcfa41a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa2a00640 1 -- 172.21.15.107:0/3151414318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa40761d0 msgr2=0x7fcfa419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa2a00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa40761d0 0x7fcfa419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa2a00640 1 -- 172.21.15.107:0/3151414318 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa41a1ad0 con 0x7fcfa4074bf0 2024-09-19T04:20:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa3400640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfa4109890 0x7fcfa41a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.288+0000 7fcfa2a00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa4074bf0 0x7fcfa4199eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400e9f0 tx=0x7fcf9400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.289+0000 7fcf9b400640 1 -- 172.21.15.107:0/3151414318 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9400cd80 con 0x7fcfa4074bf0 2024-09-19T04:20:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.289+0000 7fcf9b400640 1 -- 172.21.15.107:0/3151414318 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf94010910 con 0x7fcfa4074bf0 2024-09-19T04:20:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.289+0000 7fcfa3e00640 1 -- 172.21.15.107:0/3151414318 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa41a1dc0 con 0x7fcfa4074bf0 2024-09-19T04:20:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.289+0000 7fcf9b400640 1 -- 172.21.15.107:0/3151414318 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94018990 con 0x7fcfa4074bf0 2024-09-19T04:20:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.289+0000 7fcfa3e00640 1 -- 172.21.15.107:0/3151414318 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa41a2220 con 0x7fcfa4074bf0 2024-09-19T04:20:21.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.290+0000 7fcfa3e00640 1 -- 172.21.15.107:0/3151414318 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa4108650 con 0x7fcfa4074bf0 2024-09-19T04:20:21.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.292+0000 7fcf9b400640 1 -- 172.21.15.107:0/3151414318 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 23) ==== 138167+0+0 (secure 0 0 0) 0x7fcf94010ac0 con 0x7fcfa4074bf0 2024-09-19T04:20:21.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.293+0000 7fcf9b400640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcf880a3050 0x7fcf880a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.293+0000 7fcf9b400640 1 -- 172.21.15.107:0/3151414318 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fcf940cfcb0 con 0x7fcfa4074bf0 2024-09-19T04:20:21.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.293+0000 7fcf9be00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcf880a3050 0x7fcf880a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.294+0000 7fcf9be00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcf880a3050 0x7fcf880a5510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcfa419b3c0 tx=0x7fcf8c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.295+0000 7fcf9b400640 1 -- 172.21.15.107:0/3151414318 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcf9409c560 con 0x7fcfa4074bf0 2024-09-19T04:20:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.497+0000 7fcfa3e00640 1 -- 172.21.15.107:0/3151414318 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcfa4062e90 con 0x7fcfa4074bf0 2024-09-19T04:20:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.498+0000 7fcf9b400640 1 -- 172.21.15.107:0/3151414318 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7fcf940a0860 con 0x7fcfa4074bf0 2024-09-19T04:20:21.500 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.500+0000 7fcfa3e00640 1 -- 172.21.15.107:0/3151414318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcf880a3050 msgr2=0x7fcf880a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.500+0000 7fcfa3e00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcf880a3050 0x7fcf880a5510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcfa419b3c0 tx=0x7fcf8c005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.500+0000 7fcfa3e00640 1 -- 172.21.15.107:0/3151414318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa4074bf0 msgr2=0x7fcfa4199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.500+0000 7fcfa3e00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa4074bf0 0x7fcfa4199eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400e9f0 tx=0x7fcf9400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.500+0000 7fcfa3e00640 1 -- 172.21.15.107:0/3151414318 shutdown_connections 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.500+0000 7fcfa3e00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcf880a3050 0x7fcf880a5510 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.500+0000 7fcfa3e00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfa4109890 0x7fcfa41a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.500+0000 7fcfa3e00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa40761d0 0x7fcfa419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.500+0000 7fcfa3e00640 1 --2- 172.21.15.107:0/3151414318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa4074bf0 0x7fcfa4199eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.500+0000 7fcfa3e00640 1 -- 172.21.15.107:0/3151414318 >> 172.21.15.107:0/3151414318 conn(0x7fcfa40fd180 msgr2=0x7fcfa4105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.501+0000 7fcfa3e00640 1 -- 172.21.15.107:0/3151414318 shutdown_connections 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:21.501+0000 7fcfa3e00640 1 -- 172.21.15.107:0/3151414318 wait complete. 2024-09-19T04:20:21.518 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T04:20:21.518 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:20:21.941 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:22.116 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: [19/Sep/2024:04:20:20] ENGINE Bus STARTING 2024-09-19T04:20:22.116 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: [19/Sep/2024:04:20:20] ENGINE Serving on https://172.21.15.107:7150 2024-09-19T04:20:22.116 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: [19/Sep/2024:04:20:20] ENGINE Error in HTTPServer.serve 2024-09-19T04:20:22.116 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-09-19T04:20:22.117 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:20:22.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-09-19T04:20:22.118 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:20:22.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: [19/Sep/2024:04:20:20] ENGINE Serving on http://172.21.15.107:8765 2024-09-19T04:20:22.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: [19/Sep/2024:04:20:20] ENGINE Bus STARTED 2024-09-19T04:20:22.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2272006671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:20:22.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:21 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3151414318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:20:22.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: [19/Sep/2024:04:20:20] ENGINE Bus STARTING 2024-09-19T04:20:22.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: [19/Sep/2024:04:20:20] ENGINE Serving on https://172.21.15.107:7150 2024-09-19T04:20:22.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: [19/Sep/2024:04:20:20] ENGINE Error in HTTPServer.serve 2024-09-19T04:20:22.196 INFO:journalctl@ceph.mon.b.smithi115.stdout: Traceback (most recent call last): 2024-09-19T04:20:22.196 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: self._run(expiration_interval) 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: s = self.context.wrap_socket( 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: return self.sslsocket_class._create( 2024-09-19T04:20:22.197 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout: self.do_handshake() 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout: self._sslobj.do_handshake() 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: [19/Sep/2024:04:20:20] ENGINE Serving on http://172.21.15.107:8765 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: [19/Sep/2024:04:20:20] ENGINE Bus STARTED 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2272006671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:21 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3151414318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:20:22.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: [19/Sep/2024:04:20:20] ENGINE Bus STARTING 2024-09-19T04:20:22.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: [19/Sep/2024:04:20:20] ENGINE Serving on https://172.21.15.107:7150 2024-09-19T04:20:22.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: [19/Sep/2024:04:20:20] ENGINE Error in HTTPServer.serve 2024-09-19T04:20:22.224 INFO:journalctl@ceph.mon.c.smithi145.stdout: Traceback (most recent call last): 2024-09-19T04:20:22.224 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:20:22.224 INFO:journalctl@ceph.mon.c.smithi145.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:20:22.224 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: self._run(expiration_interval) 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: s = self.context.wrap_socket( 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: return self.sslsocket_class._create( 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: self.do_handshake() 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:20:22.225 INFO:journalctl@ceph.mon.c.smithi145.stdout: self._sslobj.do_handshake() 2024-09-19T04:20:22.226 INFO:journalctl@ceph.mon.c.smithi145.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:20:22.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: [19/Sep/2024:04:20:20] ENGINE Serving on http://172.21.15.107:8765 2024-09-19T04:20:22.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: [19/Sep/2024:04:20:20] ENGINE Bus STARTED 2024-09-19T04:20:22.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2272006671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:20:22.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:22.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:21 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3151414318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:20:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.497+0000 7f71b3600640 1 -- 172.21.15.107:0/1851399309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ac103ff0 msgr2=0x7f71ac1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.497+0000 7f71b3600640 1 --2- 172.21.15.107:0/1851399309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ac103ff0 0x7f71ac1043d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009900 tx=0x7f71a002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.497+0000 7f71b3600640 1 -- 172.21.15.107:0/1851399309 shutdown_connections 2024-09-19T04:20:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.497+0000 7f71b3600640 1 --2- 172.21.15.107:0/1851399309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71ac109120 0x7f71ac110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.497+0000 7f71b3600640 1 --2- 172.21.15.107:0/1851399309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71ac1049a0 0x7f71ac1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.497+0000 7f71b3600640 1 --2- 172.21.15.107:0/1851399309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ac103ff0 0x7f71ac1043d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.497+0000 7f71b3600640 1 -- 172.21.15.107:0/1851399309 >> 172.21.15.107:0/1851399309 conn(0x7f71ac0ffa90 msgr2=0x7f71ac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.497+0000 7f71b3600640 1 -- 172.21.15.107:0/1851399309 shutdown_connections 2024-09-19T04:20:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.497+0000 7f71b3600640 1 -- 172.21.15.107:0/1851399309 wait complete. 2024-09-19T04:20:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.498+0000 7f71b3600640 1 Processor -- start 2024-09-19T04:20:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.498+0000 7f71b3600640 1 -- start start 2024-09-19T04:20:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71b3600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ac103ff0 0x7f71ac19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71b3600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71ac1049a0 0x7f71ac19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71b3600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71ac109120 0x7f71ac19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71b3600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac114140 con 0x7f71ac1049a0 2024-09-19T04:20:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71b3600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac113fc0 con 0x7f71ac109120 2024-09-19T04:20:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71b3600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1142c0 con 0x7f71ac103ff0 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71ac1049a0 0x7f71ac19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71b0c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ac103ff0 0x7f71ac19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71ac1049a0 0x7f71ac19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54194/0 (socket says 172.21.15.107:54194) 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71b1600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71ac109120 0x7f71ac19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.499+0000 7f71abe00640 1 -- 172.21.15.107:0/2882544619 learned_addr learned my addr 172.21.15.107:0/2882544619 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.500+0000 7f71b0c00640 1 -- 172.21.15.107:0/2882544619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71ac109120 msgr2=0x7f71ac19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.500+0000 7f71b0c00640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71ac109120 0x7f71ac19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.500+0000 7f71b0c00640 1 -- 172.21.15.107:0/2882544619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71ac1049a0 msgr2=0x7f71ac19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.500+0000 7f71b0c00640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71ac1049a0 0x7f71ac19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.500+0000 7f71b0c00640 1 -- 172.21.15.107:0/2882544619 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac19f5b0 con 0x7f71ac103ff0 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.500+0000 7f71b0c00640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ac103ff0 0x7f71ac19a600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71a0005940 tx=0x7f71a00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.500+0000 7f71a9600640 1 -- 172.21.15.107:0/2882544619 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a003f030 con 0x7f71ac103ff0 2024-09-19T04:20:22.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.500+0000 7f71a9600640 1 -- 172.21.15.107:0/2882544619 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a00044c0 con 0x7f71ac103ff0 2024-09-19T04:20:22.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.500+0000 7f71b3600640 1 -- 172.21.15.107:0/2882544619 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac19f840 con 0x7f71ac103ff0 2024-09-19T04:20:22.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.501+0000 7f71a9600640 1 -- 172.21.15.107:0/2882544619 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0035070 con 0x7f71ac103ff0 2024-09-19T04:20:22.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.501+0000 7f71b3600640 1 -- 172.21.15.107:0/2882544619 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac1a7180 con 0x7f71ac103ff0 2024-09-19T04:20:22.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.502+0000 7f71b3600640 1 -- 172.21.15.107:0/2882544619 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ac1043d0 con 0x7f71ac103ff0 2024-09-19T04:20:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.503+0000 7f71a9600640 1 -- 172.21.15.107:0/2882544619 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 138476+0+0 (secure 0 0 0) 0x7f71a0042030 con 0x7f71ac103ff0 2024-09-19T04:20:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.504+0000 7f71a9600640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71940a3430 0x7f71940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.505+0000 7f71a9600640 1 -- 172.21.15.107:0/2882544619 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f71a00eff60 con 0x7f71ac103ff0 2024-09-19T04:20:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.505+0000 7f71abe00640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71940a3430 0x7f71940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.505+0000 7f71abe00640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71940a3430 0x7f71940a58f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0fff20 tx=0x7f7198005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.508+0000 7f71a9600640 1 -- 172.21.15.107:0/2882544619 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f71a00bc2e0 con 0x7f71ac103ff0 2024-09-19T04:20:22.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.732+0000 7f71b3600640 1 -- 172.21.15.107:0/2882544619 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71ac10c300 con 0x7f71ac103ff0 2024-09-19T04:20:22.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.733+0000 7f71a9600640 1 -- 172.21.15.107:0/2882544619 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11992 (secure 0 0 0) 0x7f71a00c05e0 con 0x7f71ac103ff0 2024-09-19T04:20:22.735 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:20:22.735 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"btime":"2024-09-19T04:20:19:828247+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:20:18.993568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24490,24496,24500,24539,24517]},"id":1}]} 2024-09-19T04:20:22.735 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-09-19T04:20:22.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.735+0000 7f71b3600640 1 -- 172.21.15.107:0/2882544619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71940a3430 msgr2=0x7f71940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.735+0000 7f71b3600640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71940a3430 0x7f71940a58f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0fff20 tx=0x7f7198005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:22.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.735+0000 7f71b3600640 1 -- 172.21.15.107:0/2882544619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ac103ff0 msgr2=0x7f71ac19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.735+0000 7f71b3600640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ac103ff0 0x7f71ac19a600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71a0005940 tx=0x7f71a00040c0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.735+0000 7f71b3600640 1 -- 172.21.15.107:0/2882544619 shutdown_connections 2024-09-19T04:20:22.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.735+0000 7f71b3600640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71940a3430 0x7f71940a58f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.735+0000 7f71b3600640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71ac109120 0x7f71ac19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.735+0000 7f71b3600640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71ac1049a0 0x7f71ac19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.735+0000 7f71b3600640 1 --2- 172.21.15.107:0/2882544619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ac103ff0 0x7f71ac19a600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.735+0000 7f71b3600640 1 -- 172.21.15.107:0/2882544619 >> 172.21.15.107:0/2882544619 conn(0x7f71ac0ffa90 msgr2=0x7f71ac100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.736+0000 7f71b3600640 1 -- 172.21.15.107:0/2882544619 shutdown_connections 2024-09-19T04:20:22.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:22.736+0000 7f71b3600640 1 -- 172.21.15.107:0/2882544619 wait complete. 2024-09-19T04:20:23.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:20:23.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-09-19T04:20:23.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-19T04:20:23.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-19T04:20:23.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: Unable to set osd_memory_target on smithi145 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-19T04:20:23.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: Unable to set osd_memory_target on smithi115 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:20:23.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:22 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2882544619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:20:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-19T04:20:23.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-19T04:20:23.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: Unable to set osd_memory_target on smithi145 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-19T04:20:23.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: Unable to set osd_memory_target on smithi115 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:20:23.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:22 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2882544619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:20:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-09-19T04:20:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: Unable to set osd_memory_target on smithi145 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-19T04:20:23.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: Unable to set osd_memory_target on smithi115 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:20:23.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:23.226 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:22 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2882544619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:23.426 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T04:20:23.426 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T04:20:23.427 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:20:23.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.595+0000 7f8a37600640 1 -- 172.21.15.107:0/3892235509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a30103d30 msgr2=0x7f8a30104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.595+0000 7f8a37600640 1 --2- 172.21.15.107:0/3892235509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a30103d30 0x7f8a30104130 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009900 tx=0x7f8a2002a410 comp rx=0 tx=0).stop 2024-09-19T04:20:23.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.595+0000 7f8a37600640 1 -- 172.21.15.107:0/3892235509 shutdown_connections 2024-09-19T04:20:23.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.595+0000 7f8a37600640 1 --2- 172.21.15.107:0/3892235509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a301058e0 0x7f8a3010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.595+0000 7f8a37600640 1 --2- 172.21.15.107:0/3892235509 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a30104f20 0x7f8a301053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.595+0000 7f8a37600640 1 --2- 172.21.15.107:0/3892235509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a30103d30 0x7f8a30104130 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.595+0000 7f8a37600640 1 -- 172.21.15.107:0/3892235509 >> 172.21.15.107:0/3892235509 conn(0x7f8a300ff4b0 msgr2=0x7f8a301018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.595+0000 7f8a37600640 1 -- 172.21.15.107:0/3892235509 shutdown_connections 2024-09-19T04:20:23.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.595+0000 7f8a37600640 1 -- 172.21.15.107:0/3892235509 wait complete. 2024-09-19T04:20:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.596+0000 7f8a37600640 1 Processor -- start 2024-09-19T04:20:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.596+0000 7f8a37600640 1 -- start start 2024-09-19T04:20:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.596+0000 7f8a37600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a30103d30 0x7f8a3019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.596+0000 7f8a37600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a30104f20 0x7f8a3019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.596+0000 7f8a37600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a301058e0 0x7f8a301a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.596+0000 7f8a37600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3010fa00 con 0x7f8a30103d30 2024-09-19T04:20:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.596+0000 7f8a37600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3010f880 con 0x7f8a30104f20 2024-09-19T04:20:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.596+0000 7f8a37600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3010fb80 con 0x7f8a301058e0 2024-09-19T04:20:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a35600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a301058e0 0x7f8a301a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a27e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a30104f20 0x7f8a3019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a34c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a30103d30 0x7f8a3019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a35600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a301058e0 0x7f8a301a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46544/0 (socket says 172.21.15.107:46544) 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a35600640 1 -- 172.21.15.107:0/1176084240 learned_addr learned my addr 172.21.15.107:0/1176084240 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a27e00640 1 -- 172.21.15.107:0/1176084240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a301058e0 msgr2=0x7f8a301a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a27e00640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a301058e0 0x7f8a301a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a27e00640 1 -- 172.21.15.107:0/1176084240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a30103d30 msgr2=0x7f8a3019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a27e00640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a30103d30 0x7f8a3019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a27e00640 1 -- 172.21.15.107:0/1176084240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a301a1c50 con 0x7f8a30104f20 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a27e00640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a30104f20 0x7f8a3019a570 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800e9f0 tx=0x7f8a1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a2e000640 1 -- 172.21.15.107:0/1176084240 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1800ccc0 con 0x7f8a30104f20 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a2e000640 1 -- 172.21.15.107:0/1176084240 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a18005020 con 0x7f8a30104f20 2024-09-19T04:20:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.597+0000 7f8a37600640 1 -- 172.21.15.107:0/1176084240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a301a1f40 con 0x7f8a30104f20 2024-09-19T04:20:23.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.598+0000 7f8a2e000640 1 -- 172.21.15.107:0/1176084240 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a180049e0 con 0x7f8a30104f20 2024-09-19T04:20:23.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.598+0000 7f8a37600640 1 -- 172.21.15.107:0/1176084240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a301a23b0 con 0x7f8a30104f20 2024-09-19T04:20:23.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.599+0000 7f8a37600640 1 -- 172.21.15.107:0/1176084240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3010b170 con 0x7f8a30104f20 2024-09-19T04:20:23.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.601+0000 7f8a2e000640 1 -- 172.21.15.107:0/1176084240 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 138476+0+0 (secure 0 0 0) 0x7f8a18005560 con 0x7f8a30104f20 2024-09-19T04:20:23.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.602+0000 7f8a2e000640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a140a33c0 0x7f8a140a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.602+0000 7f8a2e000640 1 -- 172.21.15.107:0/1176084240 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f8a180cfe90 con 0x7f8a30104f20 2024-09-19T04:20:23.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.602+0000 7f8a34c00640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a140a33c0 0x7f8a140a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.603+0000 7f8a34c00640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a140a33c0 0x7f8a140a5880 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a2002aa00 tx=0x7f8a200315b0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.604+0000 7f8a2e000640 1 -- 172.21.15.107:0/1176084240 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8a180051d0 con 0x7f8a30104f20 2024-09-19T04:20:23.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.789+0000 7f8a37600640 1 -- 172.21.15.107:0/1176084240 --> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a30062630 con 0x7f8a140a33c0 2024-09-19T04:20:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.797+0000 7f8a2e000640 1 -- 172.21.15.107:0/1176084240 <== mgr.15114 v2:172.21.15.107:6800/1724794486 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258046 (secure 0 0 0) 0x7f8a30062630 con 0x7f8a140a33c0 2024-09-19T04:20:23.802 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:20:23.804 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-19T04:20:23.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.801+0000 7f8a37600640 1 -- 172.21.15.107:0/1176084240 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a140a33c0 msgr2=0x7f8a140a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.801+0000 7f8a37600640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a140a33c0 0x7f8a140a5880 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a2002aa00 tx=0x7f8a200315b0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.801+0000 7f8a37600640 1 -- 172.21.15.107:0/1176084240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a30104f20 msgr2=0x7f8a3019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.801+0000 7f8a37600640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a30104f20 0x7f8a3019a570 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800e9f0 tx=0x7f8a1800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.801+0000 7f8a37600640 1 -- 172.21.15.107:0/1176084240 shutdown_connections 2024-09-19T04:20:23.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.801+0000 7f8a37600640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a140a33c0 0x7f8a140a5880 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.801+0000 7f8a37600640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a301058e0 0x7f8a301a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.801+0000 7f8a37600640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a30104f20 0x7f8a3019a570 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.801+0000 7f8a37600640 1 --2- 172.21.15.107:0/1176084240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a30103d30 0x7f8a3019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.801+0000 7f8a37600640 1 -- 172.21.15.107:0/1176084240 >> 172.21.15.107:0/1176084240 conn(0x7f8a300ff4b0 msgr2=0x7f8a30108340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.802+0000 7f8a37600640 1 -- 172.21.15.107:0/1176084240 shutdown_connections 2024-09-19T04:20:23.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:23.802+0000 7f8a37600640 1 -- 172.21.15.107:0/1176084240 wait complete. 2024-09-19T04:20:23.822 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T04:20:23.171159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192060,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1427,"num_read_kb":881,"num_write":564,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":946,"ondisk_log_size":946,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377448,"kb_used_data":11516,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564376,"statfs":{"total":1151940427776,"available":1151553921024,"internally_reserved":0,"allocated":11792384,"data_stored":8885574,"data_compressed":214973,"data_compressed_allocated":278528,"data_compressed_original":3354624,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.980758"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.467136+0000","last_change":"2024-09-19T04:18:12.499720+0000","last_active":"2024-09-19T04:20:19.467136+0000","last_peered":"2024-09-19T04:20:19.467136+0000","last_clean":"2024-09-19T04:20:19.467136+0000","last_became_active":"2024-09-19T04:18:12.499381+0000","last_became_peered":"2024-09-19T04:18:12.499381+0000","last_unstale":"2024-09-19T04:20:19.467136+0000","last_undegraded":"2024-09-19T04:20:19.467136+0000","last_fullsized":"2024-09-19T04:20:19.467136+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:46.708870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235809+0000","last_change":"2024-09-19T04:18:12.498163+0000","last_active":"2024-09-19T04:20:19.235809+0000","last_peered":"2024-09-19T04:20:19.235809+0000","last_clean":"2024-09-19T04:20:19.235809+0000","last_became_active":"2024-09-19T04:18:12.495706+0000","last_became_peered":"2024-09-19T04:18:12.495706+0000","last_unstale":"2024-09-19T04:20:19.235809+0000","last_undegraded":"2024-09-19T04:20:19.235809+0000","last_fullsized":"2024-09-19T04:20:19.235809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:50.604410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235810+0000","last_change":"2024-09-19T04:18:12.493903+0000","last_active":"2024-09-19T04:20:19.235810+0000","last_peered":"2024-09-19T04:20:19.235810+0000","last_clean":"2024-09-19T04:20:19.235810+0000","last_became_active":"2024-09-19T04:18:12.493443+0000","last_became_peered":"2024-09-19T04:18:12.493443+0000","last_unstale":"2024-09-19T04:20:19.235810+0000","last_undegraded":"2024-09-19T04:20:19.235810+0000","last_fullsized":"2024-09-19T04:20:19.235810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:51.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.458868+0000","last_change":"2024-09-19T04:18:12.500460+0000","last_active":"2024-09-19T04:20:19.458868+0000","last_peered":"2024-09-19T04:20:19.458868+0000","last_clean":"2024-09-19T04:20:19.458868+0000","last_became_active":"2024-09-19T04:18:12.494675+0000","last_became_peered":"2024-09-19T04:18:12.494675+0000","last_unstale":"2024-09-19T04:20:19.458868+0000","last_undegraded":"2024-09-19T04:20:19.458868+0000","last_fullsized":"2024-09-19T04:20:19.458868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:13:32.863156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173788+0000","last_change":"2024-09-19T04:18:12.494936+0000","last_active":"2024-09-19T04:20:19.173788+0000","last_peered":"2024-09-19T04:20:19.173788+0000","last_clean":"2024-09-19T04:20:19.173788+0000","last_became_active":"2024-09-19T04:18:12.494556+0000","last_became_peered":"2024-09-19T04:18:12.494556+0000","last_unstale":"2024-09-19T04:20:19.173788+0000","last_undegraded":"2024-09-19T04:20:19.173788+0000","last_fullsized":"2024-09-19T04:20:19.173788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:06.001529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158827+0000","last_change":"2024-09-19T04:18:12.495880+0000","last_active":"2024-09-19T04:20:19.158827+0000","last_peered":"2024-09-19T04:20:19.158827+0000","last_clean":"2024-09-19T04:20:19.158827+0000","last_became_active":"2024-09-19T04:18:12.495562+0000","last_became_peered":"2024-09-19T04:18:12.495562+0000","last_unstale":"2024-09-19T04:20:19.158827+0000","last_undegraded":"2024-09-19T04:20:19.158827+0000","last_fullsized":"2024-09-19T04:20:19.158827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:20.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454525+0000","last_change":"2024-09-19T04:18:12.497991+0000","last_active":"2024-09-19T04:20:19.454525+0000","last_peered":"2024-09-19T04:20:19.454525+0000","last_clean":"2024-09-19T04:20:19.454525+0000","last_became_active":"2024-09-19T04:18:12.497644+0000","last_became_peered":"2024-09-19T04:18:12.497644+0000","last_unstale":"2024-09-19T04:20:19.454525+0000","last_undegraded":"2024-09-19T04:20:19.454525+0000","last_fullsized":"2024-09-19T04:20:19.454525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.851930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259408+0000","last_change":"2024-09-19T04:18:12.495789+0000","last_active":"2024-09-19T04:20:19.259408+0000","last_peered":"2024-09-19T04:20:19.259408+0000","last_clean":"2024-09-19T04:20:19.259408+0000","last_became_active":"2024-09-19T04:18:12.494637+0000","last_became_peered":"2024-09-19T04:18:12.494637+0000","last_unstale":"2024-09-19T04:20:19.259408+0000","last_undegraded":"2024-09-19T04:20:19.259408+0000","last_fullsized":"2024-09-19T04:20:19.259408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:14.245180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.457103+0000","last_change":"2024-09-19T04:18:12.491433+0000","last_active":"2024-09-19T04:20:19.457103+0000","last_peered":"2024-09-19T04:20:19.457103+0000","last_clean":"2024-09-19T04:20:19.457103+0000","last_became_active":"2024-09-19T04:18:12.490444+0000","last_became_peered":"2024-09-19T04:18:12.490444+0000","last_unstale":"2024-09-19T04:20:19.457103+0000","last_undegraded":"2024-09-19T04:20:19.457103+0000","last_fullsized":"2024-09-19T04:20:19.457103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:43.068355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159407+0000","last_change":"2024-09-19T04:18:12.493188+0000","last_active":"2024-09-19T04:20:19.159407+0000","last_peered":"2024-09-19T04:20:19.159407+0000","last_clean":"2024-09-19T04:20:19.159407+0000","last_became_active":"2024-09-19T04:18:12.492788+0000","last_became_peered":"2024-09-19T04:18:12.492788+0000","last_unstale":"2024-09-19T04:20:19.159407+0000","last_undegraded":"2024-09-19T04:20:19.159407+0000","last_fullsized":"2024-09-19T04:20:19.159407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:17.912820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454786+0000","last_change":"2024-09-19T04:18:12.490616+0000","last_active":"2024-09-19T04:20:19.454786+0000","last_peered":"2024-09-19T04:20:19.454786+0000","last_clean":"2024-09-19T04:20:19.454786+0000","last_became_active":"2024-09-19T04:18:12.490155+0000","last_became_peered":"2024-09-19T04:18:12.490155+0000","last_unstale":"2024-09-19T04:20:19.454786+0000","last_undegraded":"2024-09-19T04:20:19.454786+0000","last_fullsized":"2024-09-19T04:20:19.454786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:47.082082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.458913+0000","last_change":"2024-09-19T04:18:12.494191+0000","last_active":"2024-09-19T04:20:19.458913+0000","last_peered":"2024-09-19T04:20:19.458913+0000","last_clean":"2024-09-19T04:20:19.458913+0000","last_became_active":"2024-09-19T04:18:12.493786+0000","last_became_peered":"2024-09-19T04:18:12.493786+0000","last_unstale":"2024-09-19T04:20:19.458913+0000","last_undegraded":"2024-09-19T04:20:19.458913+0000","last_fullsized":"2024-09-19T04:20:19.458913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:29.691556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157025+0000","last_change":"2024-09-19T04:18:12.482167+0000","last_active":"2024-09-19T04:20:19.157025+0000","last_peered":"2024-09-19T04:20:19.157025+0000","last_clean":"2024-09-19T04:20:19.157025+0000","last_became_active":"2024-09-19T04:18:12.481314+0000","last_became_peered":"2024-09-19T04:18:12.481314+0000","last_unstale":"2024-09-19T04:20:19.157025+0000","last_undegraded":"2024-09-19T04:20:19.157025+0000","last_fullsized":"2024-09-19T04:20:19.157025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:28.081641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235565+0000","last_change":"2024-09-19T04:18:12.491711+0000","last_active":"2024-09-19T04:20:19.235565+0000","last_peered":"2024-09-19T04:20:19.235565+0000","last_clean":"2024-09-19T04:20:19.235565+0000","last_became_active":"2024-09-19T04:18:12.490964+0000","last_became_peered":"2024-09-19T04:18:12.490964+0000","last_unstale":"2024-09-19T04:20:19.235565+0000","last_undegraded":"2024-09-19T04:20:19.235565+0000","last_fullsized":"2024-09-19T04:20:19.235565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:16.607679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235461+0000","last_change":"2024-09-19T04:18:12.504737+0000","last_active":"2024-09-19T04:20:19.235461+0000","last_peered":"2024-09-19T04:20:19.235461+0000","last_clean":"2024-09-19T04:20:19.235461+0000","last_became_active":"2024-09-19T04:18:12.497583+0000","last_became_peered":"2024-09-19T04:18:12.497583+0000","last_unstale":"2024-09-19T04:20:19.235461+0000","last_undegraded":"2024-09-19T04:20:19.235461+0000","last_fullsized":"2024-09-19T04:20:19.235461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:48.456738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456542+0000","last_change":"2024-09-19T04:18:12.499045+0000","last_active":"2024-09-19T04:20:19.456542+0000","last_peered":"2024-09-19T04:20:19.456542+0000","last_clean":"2024-09-19T04:20:19.456542+0000","last_became_active":"2024-09-19T04:18:12.498713+0000","last_became_peered":"2024-09-19T04:18:12.498713+0000","last_unstale":"2024-09-19T04:20:19.456542+0000","last_undegraded":"2024-09-19T04:20:19.456542+0000","last_fullsized":"2024-09-19T04:20:19.456542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:54.337769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.467108+0000","last_change":"2024-09-19T04:18:12.495989+0000","last_active":"2024-09-19T04:20:19.467108+0000","last_peered":"2024-09-19T04:20:19.467108+0000","last_clean":"2024-09-19T04:20:19.467108+0000","last_became_active":"2024-09-19T04:18:12.495633+0000","last_became_peered":"2024-09-19T04:18:12.495633+0000","last_unstale":"2024-09-19T04:20:19.467108+0000","last_undegraded":"2024-09-19T04:20:19.467108+0000","last_fullsized":"2024-09-19T04:20:19.467108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:17.225861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.255995+0000","last_change":"2024-09-19T04:18:12.488532+0000","last_active":"2024-09-19T04:20:19.255995+0000","last_peered":"2024-09-19T04:20:19.255995+0000","last_clean":"2024-09-19T04:20:19.255995+0000","last_became_active":"2024-09-19T04:18:12.488117+0000","last_became_peered":"2024-09-19T04:18:12.488117+0000","last_unstale":"2024-09-19T04:20:19.255995+0000","last_undegraded":"2024-09-19T04:20:19.255995+0000","last_fullsized":"2024-09-19T04:20:19.255995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:45.318852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455765+0000","last_change":"2024-09-19T04:18:12.496784+0000","last_active":"2024-09-19T04:20:19.455765+0000","last_peered":"2024-09-19T04:20:19.455765+0000","last_clean":"2024-09-19T04:20:19.455765+0000","last_became_active":"2024-09-19T04:18:12.496443+0000","last_became_peered":"2024-09-19T04:18:12.496443+0000","last_unstale":"2024-09-19T04:20:19.455765+0000","last_undegraded":"2024-09-19T04:20:19.455765+0000","last_fullsized":"2024-09-19T04:20:19.455765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:25:46.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466725+0000","last_change":"2024-09-19T04:19:00.980557+0000","last_active":"2024-09-19T04:20:19.466725+0000","last_peered":"2024-09-19T04:20:19.466725+0000","last_clean":"2024-09-19T04:20:19.466725+0000","last_became_active":"2024-09-19T04:19:00.980149+0000","last_became_peered":"2024-09-19T04:19:00.980149+0000","last_unstale":"2024-09-19T04:20:19.466725+0000","last_undegraded":"2024-09-19T04:20:19.466725+0000","last_fullsized":"2024-09-19T04:20:19.466725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:06.140085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157083+0000","last_change":"2024-09-19T04:18:12.493731+0000","last_active":"2024-09-19T04:20:19.157083+0000","last_peered":"2024-09-19T04:20:19.157083+0000","last_clean":"2024-09-19T04:20:19.157083+0000","last_became_active":"2024-09-19T04:18:12.493412+0000","last_became_peered":"2024-09-19T04:18:12.493412+0000","last_unstale":"2024-09-19T04:20:19.157083+0000","last_undegraded":"2024-09-19T04:20:19.157083+0000","last_fullsized":"2024-09-19T04:20:19.157083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:41.089637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456785+0000","last_change":"2024-09-19T04:18:12.497033+0000","last_active":"2024-09-19T04:20:19.456785+0000","last_peered":"2024-09-19T04:20:19.456785+0000","last_clean":"2024-09-19T04:20:19.456785+0000","last_became_active":"2024-09-19T04:18:12.496710+0000","last_became_peered":"2024-09-19T04:18:12.496710+0000","last_unstale":"2024-09-19T04:20:19.456785+0000","last_undegraded":"2024-09-19T04:20:19.456785+0000","last_fullsized":"2024-09-19T04:20:19.456785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:19.193869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259529+0000","last_change":"2024-09-19T04:18:12.499213+0000","last_active":"2024-09-19T04:20:19.259529+0000","last_peered":"2024-09-19T04:20:19.259529+0000","last_clean":"2024-09-19T04:20:19.259529+0000","last_became_active":"2024-09-19T04:18:12.498728+0000","last_became_peered":"2024-09-19T04:18:12.498728+0000","last_unstale":"2024-09-19T04:20:19.259529+0000","last_undegraded":"2024-09-19T04:20:19.259529+0000","last_fullsized":"2024-09-19T04:20:19.259529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:37.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173675+0000","last_change":"2024-09-19T04:19:00.982647+0000","last_active":"2024-09-19T04:20:19.173675+0000","last_peered":"2024-09-19T04:20:19.173675+0000","last_clean":"2024-09-19T04:20:19.173675+0000","last_became_active":"2024-09-19T04:19:00.982246+0000","last_became_peered":"2024-09-19T04:19:00.982246+0000","last_unstale":"2024-09-19T04:20:19.173675+0000","last_undegraded":"2024-09-19T04:20:19.173675+0000","last_fullsized":"2024-09-19T04:20:19.173675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:48.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260612+0000","last_change":"2024-09-19T04:18:12.506679+0000","last_active":"2024-09-19T04:20:19.260612+0000","last_peered":"2024-09-19T04:20:19.260612+0000","last_clean":"2024-09-19T04:20:19.260612+0000","last_became_active":"2024-09-19T04:18:12.505953+0000","last_became_peered":"2024-09-19T04:18:12.505953+0000","last_unstale":"2024-09-19T04:20:19.260612+0000","last_undegraded":"2024-09-19T04:20:19.260612+0000","last_fullsized":"2024-09-19T04:20:19.260612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:53.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455913+0000","last_change":"2024-09-19T04:18:12.490511+0000","last_active":"2024-09-19T04:20:19.455913+0000","last_peered":"2024-09-19T04:20:19.455913+0000","last_clean":"2024-09-19T04:20:19.455913+0000","last_became_active":"2024-09-19T04:18:12.490051+0000","last_became_peered":"2024-09-19T04:18:12.490051+0000","last_unstale":"2024-09-19T04:20:19.455913+0000","last_undegraded":"2024-09-19T04:20:19.455913+0000","last_fullsized":"2024-09-19T04:20:19.455913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:32.283312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.467072+0000","last_change":"2024-09-19T04:18:12.491465+0000","last_active":"2024-09-19T04:20:19.467072+0000","last_peered":"2024-09-19T04:20:19.467072+0000","last_clean":"2024-09-19T04:20:19.467072+0000","last_became_active":"2024-09-19T04:18:12.491136+0000","last_became_peered":"2024-09-19T04:18:12.491136+0000","last_unstale":"2024-09-19T04:20:19.467072+0000","last_undegraded":"2024-09-19T04:20:19.467072+0000","last_fullsized":"2024-09-19T04:20:19.467072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:29.498909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157299+0000","last_change":"2024-09-19T04:19:01.599944+0000","last_active":"2024-09-19T04:20:19.157299+0000","last_peered":"2024-09-19T04:20:19.157299+0000","last_clean":"2024-09-19T04:20:19.157299+0000","last_became_active":"2024-09-19T04:19:01.599571+0000","last_became_peered":"2024-09-19T04:19:01.599571+0000","last_unstale":"2024-09-19T04:20:19.157299+0000","last_undegraded":"2024-09-19T04:20:19.157299+0000","last_fullsized":"2024-09-19T04:20:19.157299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:17.929785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235780+0000","last_change":"2024-09-19T04:18:12.504364+0000","last_active":"2024-09-19T04:20:19.235780+0000","last_peered":"2024-09-19T04:20:19.235780+0000","last_clean":"2024-09-19T04:20:19.235780+0000","last_became_active":"2024-09-19T04:18:12.504006+0000","last_became_peered":"2024-09-19T04:18:12.504006+0000","last_unstale":"2024-09-19T04:20:19.235780+0000","last_undegraded":"2024-09-19T04:20:19.235780+0000","last_fullsized":"2024-09-19T04:20:19.235780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:53.309963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455576+0000","last_change":"2024-09-19T04:18:12.498223+0000","last_active":"2024-09-19T04:20:19.455576+0000","last_peered":"2024-09-19T04:20:19.455576+0000","last_clean":"2024-09-19T04:20:19.455576+0000","last_became_active":"2024-09-19T04:18:12.497862+0000","last_became_peered":"2024-09-19T04:18:12.497862+0000","last_unstale":"2024-09-19T04:20:19.455576+0000","last_undegraded":"2024-09-19T04:20:19.455576+0000","last_fullsized":"2024-09-19T04:20:19.455576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:28.858620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455732+0000","last_change":"2024-09-19T04:18:12.502335+0000","last_active":"2024-09-19T04:20:19.455732+0000","last_peered":"2024-09-19T04:20:19.455732+0000","last_clean":"2024-09-19T04:20:19.455732+0000","last_became_active":"2024-09-19T04:18:12.501686+0000","last_became_peered":"2024-09-19T04:18:12.501686+0000","last_unstale":"2024-09-19T04:20:19.455732+0000","last_undegraded":"2024-09-19T04:20:19.455732+0000","last_fullsized":"2024-09-19T04:20:19.455732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:22.348745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466807+0000","last_change":"2024-09-19T04:18:12.490905+0000","last_active":"2024-09-19T04:20:19.466807+0000","last_peered":"2024-09-19T04:20:19.466807+0000","last_clean":"2024-09-19T04:20:19.466807+0000","last_became_active":"2024-09-19T04:18:12.490489+0000","last_became_peered":"2024-09-19T04:18:12.490489+0000","last_unstale":"2024-09-19T04:20:19.466807+0000","last_undegraded":"2024-09-19T04:20:19.466807+0000","last_fullsized":"2024-09-19T04:20:19.466807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.946474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173739+0000","last_change":"2024-09-19T04:18:12.494607+0000","last_active":"2024-09-19T04:20:19.173739+0000","last_peered":"2024-09-19T04:20:19.173739+0000","last_clean":"2024-09-19T04:20:19.173739+0000","last_became_active":"2024-09-19T04:18:12.494288+0000","last_became_peered":"2024-09-19T04:18:12.494288+0000","last_unstale":"2024-09-19T04:20:19.173739+0000","last_undegraded":"2024-09-19T04:20:19.173739+0000","last_fullsized":"2024-09-19T04:20:19.173739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:42.824951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":153,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738608+0000","last_change":"2024-09-19T04:18:10.103108+0000","last_active":"2024-09-19T04:20:19.738608+0000","last_peered":"2024-09-19T04:20:19.738608+0000","last_clean":"2024-09-19T04:20:19.738608+0000","last_became_active":"2024-09-19T04:18:10.102783+0000","last_became_peered":"2024-09-19T04:18:10.102783+0000","last_unstale":"2024-09-19T04:20:19.738608+0000","last_undegraded":"2024-09-19T04:20:19.738608+0000","last_fullsized":"2024-09-19T04:20:19.738608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:25.521953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158921+0000","last_change":"2024-09-19T04:18:12.495453+0000","last_active":"2024-09-19T04:20:19.158921+0000","last_peered":"2024-09-19T04:20:19.158921+0000","last_clean":"2024-09-19T04:20:19.158921+0000","last_became_active":"2024-09-19T04:18:12.494744+0000","last_became_peered":"2024-09-19T04:18:12.494744+0000","last_unstale":"2024-09-19T04:20:19.158921+0000","last_undegraded":"2024-09-19T04:20:19.158921+0000","last_fullsized":"2024-09-19T04:20:19.158921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:41.071723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158885+0000","last_change":"2024-09-19T04:18:10.102675+0000","last_active":"2024-09-19T04:20:19.158885+0000","last_peered":"2024-09-19T04:20:19.158885+0000","last_clean":"2024-09-19T04:20:19.158885+0000","last_became_active":"2024-09-19T04:18:10.102368+0000","last_became_peered":"2024-09-19T04:18:10.102368+0000","last_unstale":"2024-09-19T04:20:19.158885+0000","last_undegraded":"2024-09-19T04:20:19.158885+0000","last_fullsized":"2024-09-19T04:20:19.158885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:45.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.156851+0000","last_change":"2024-09-19T04:18:10.099932+0000","last_active":"2024-09-19T04:20:19.156851+0000","last_peered":"2024-09-19T04:20:19.156851+0000","last_clean":"2024-09-19T04:20:19.156851+0000","last_became_active":"2024-09-19T04:18:10.099637+0000","last_became_peered":"2024-09-19T04:18:10.099637+0000","last_unstale":"2024-09-19T04:20:19.156851+0000","last_undegraded":"2024-09-19T04:20:19.156851+0000","last_fullsized":"2024-09-19T04:20:19.156851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:50.594382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259565+0000","last_change":"2024-09-19T04:18:12.502229+0000","last_active":"2024-09-19T04:20:19.259565+0000","last_peered":"2024-09-19T04:20:19.259565+0000","last_clean":"2024-09-19T04:20:19.259565+0000","last_became_active":"2024-09-19T04:18:12.501921+0000","last_became_peered":"2024-09-19T04:18:12.501921+0000","last_unstale":"2024-09-19T04:20:19.259565+0000","last_undegraded":"2024-09-19T04:20:19.259565+0000","last_fullsized":"2024-09-19T04:20:19.259565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:55.412103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158774+0000","last_change":"2024-09-19T04:18:10.103961+0000","last_active":"2024-09-19T04:20:19.158774+0000","last_peered":"2024-09-19T04:20:19.158774+0000","last_clean":"2024-09-19T04:20:19.158774+0000","last_became_active":"2024-09-19T04:18:10.102737+0000","last_became_peered":"2024-09-19T04:18:10.102737+0000","last_unstale":"2024-09-19T04:20:19.158774+0000","last_undegraded":"2024-09-19T04:20:19.158774+0000","last_fullsized":"2024-09-19T04:20:19.158774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:31:01.806124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.255887+0000","last_change":"2024-09-19T04:18:12.502385+0000","last_active":"2024-09-19T04:20:19.255887+0000","last_peered":"2024-09-19T04:20:19.255887+0000","last_clean":"2024-09-19T04:20:19.255887+0000","last_became_active":"2024-09-19T04:18:12.502025+0000","last_became_peered":"2024-09-19T04:18:12.502025+0000","last_unstale":"2024-09-19T04:20:19.255887+0000","last_undegraded":"2024-09-19T04:20:19.255887+0000","last_fullsized":"2024-09-19T04:20:19.255887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:24.650911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'8","reported_seq":262,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.737999+0000","last_change":"2024-09-19T04:18:10.103526+0000","last_active":"2024-09-19T04:20:19.737999+0000","last_peered":"2024-09-19T04:20:19.737999+0000","last_clean":"2024-09-19T04:20:19.737999+0000","last_became_active":"2024-09-19T04:18:10.103191+0000","last_became_peered":"2024-09-19T04:18:10.103191+0000","last_unstale":"2024-09-19T04:20:19.737999+0000","last_undegraded":"2024-09-19T04:20:19.737999+0000","last_fullsized":"2024-09-19T04:20:19.737999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:07.695715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456912+0000","last_change":"2024-09-19T04:18:12.493169+0000","last_active":"2024-09-19T04:20:19.456912+0000","last_peered":"2024-09-19T04:20:19.456912+0000","last_clean":"2024-09-19T04:20:19.456912+0000","last_became_active":"2024-09-19T04:18:12.492669+0000","last_became_peered":"2024-09-19T04:18:12.492669+0000","last_unstale":"2024-09-19T04:20:19.456912+0000","last_undegraded":"2024-09-19T04:20:19.456912+0000","last_fullsized":"2024-09-19T04:20:19.456912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:04.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454874+0000","last_change":"2024-09-19T04:19:01.007275+0000","last_active":"2024-09-19T04:20:19.454874+0000","last_peered":"2024-09-19T04:20:19.454874+0000","last_clean":"2024-09-19T04:20:19.454874+0000","last_became_active":"2024-09-19T04:19:01.003361+0000","last_became_peered":"2024-09-19T04:19:01.003361+0000","last_unstale":"2024-09-19T04:20:19.454874+0000","last_undegraded":"2024-09-19T04:20:19.454874+0000","last_fullsized":"2024-09-19T04:20:19.454874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:07:11.907462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.236090+0000","last_change":"2024-09-19T04:18:12.504348+0000","last_active":"2024-09-19T04:20:19.236090+0000","last_peered":"2024-09-19T04:20:19.236090+0000","last_clean":"2024-09-19T04:20:19.236090+0000","last_became_active":"2024-09-19T04:18:12.503180+0000","last_became_peered":"2024-09-19T04:18:12.503180+0000","last_unstale":"2024-09-19T04:20:19.236090+0000","last_undegraded":"2024-09-19T04:20:19.236090+0000","last_fullsized":"2024-09-19T04:20:19.236090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:03.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159013+0000","last_change":"2024-09-19T04:18:10.365136+0000","last_active":"2024-09-19T04:20:19.159013+0000","last_peered":"2024-09-19T04:20:19.159013+0000","last_clean":"2024-09-19T04:20:19.159013+0000","last_became_active":"2024-09-19T04:18:10.364594+0000","last_became_peered":"2024-09-19T04:18:10.364594+0000","last_unstale":"2024-09-19T04:20:19.159013+0000","last_undegraded":"2024-09-19T04:20:19.159013+0000","last_fullsized":"2024-09-19T04:20:19.159013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:52.252157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456842+0000","last_change":"2024-09-19T04:18:12.502051+0000","last_active":"2024-09-19T04:20:19.456842+0000","last_peered":"2024-09-19T04:20:19.456842+0000","last_clean":"2024-09-19T04:20:19.456842+0000","last_became_active":"2024-09-19T04:18:12.501578+0000","last_became_peered":"2024-09-19T04:18:12.501578+0000","last_unstale":"2024-09-19T04:20:19.456842+0000","last_undegraded":"2024-09-19T04:20:19.456842+0000","last_fullsized":"2024-09-19T04:20:19.456842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:59.693175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'37","reported_seq":303,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738014+0000","last_change":"2024-09-19T04:18:10.101507+0000","last_active":"2024-09-19T04:20:19.738014+0000","last_peered":"2024-09-19T04:20:19.738014+0000","last_clean":"2024-09-19T04:20:19.738014+0000","last_became_active":"2024-09-19T04:18:10.101152+0000","last_became_peered":"2024-09-19T04:18:10.101152+0000","last_unstale":"2024-09-19T04:20:19.738014+0000","last_undegraded":"2024-09-19T04:20:19.738014+0000","last_fullsized":"2024-09-19T04:20:19.738014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:01.397605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66483,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":186,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174032+0000","last_change":"2024-09-19T04:18:12.491611+0000","last_active":"2024-09-19T04:20:19.174032+0000","last_peered":"2024-09-19T04:20:19.174032+0000","last_clean":"2024-09-19T04:20:19.174032+0000","last_became_active":"2024-09-19T04:18:12.491279+0000","last_became_peered":"2024-09-19T04:18:12.491279+0000","last_unstale":"2024-09-19T04:20:19.174032+0000","last_undegraded":"2024-09-19T04:20:19.174032+0000","last_fullsized":"2024-09-19T04:20:19.174032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:01.414713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455643+0000","last_change":"2024-09-19T04:18:10.105574+0000","last_active":"2024-09-19T04:20:19.455643+0000","last_peered":"2024-09-19T04:20:19.455643+0000","last_clean":"2024-09-19T04:20:19.455643+0000","last_became_active":"2024-09-19T04:18:10.105203+0000","last_became_peered":"2024-09-19T04:18:10.105203+0000","last_unstale":"2024-09-19T04:20:19.455643+0000","last_undegraded":"2024-09-19T04:20:19.455643+0000","last_fullsized":"2024-09-19T04:20:19.455643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:00.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.156878+0000","last_change":"2024-09-19T04:18:12.497242+0000","last_active":"2024-09-19T04:20:19.156878+0000","last_peered":"2024-09-19T04:20:19.156878+0000","last_clean":"2024-09-19T04:20:19.156878+0000","last_became_active":"2024-09-19T04:18:12.496732+0000","last_became_peered":"2024-09-19T04:18:12.496732+0000","last_unstale":"2024-09-19T04:20:19.156878+0000","last_undegraded":"2024-09-19T04:20:19.156878+0000","last_fullsized":"2024-09-19T04:20:19.156878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:56.520341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158716+0000","last_change":"2024-09-19T04:18:10.103905+0000","last_active":"2024-09-19T04:20:19.158716+0000","last_peered":"2024-09-19T04:20:19.158716+0000","last_clean":"2024-09-19T04:20:19.158716+0000","last_became_active":"2024-09-19T04:18:10.103517+0000","last_became_peered":"2024-09-19T04:18:10.103517+0000","last_unstale":"2024-09-19T04:20:19.158716+0000","last_undegraded":"2024-09-19T04:20:19.158716+0000","last_fullsized":"2024-09-19T04:20:19.158716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:48.292878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454822+0000","last_change":"2024-09-19T04:18:12.499380+0000","last_active":"2024-09-19T04:20:19.454822+0000","last_peered":"2024-09-19T04:20:19.454822+0000","last_clean":"2024-09-19T04:20:19.454822+0000","last_became_active":"2024-09-19T04:18:12.498995+0000","last_became_peered":"2024-09-19T04:18:12.498995+0000","last_unstale":"2024-09-19T04:20:19.454822+0000","last_undegraded":"2024-09-19T04:20:19.454822+0000","last_fullsized":"2024-09-19T04:20:19.454822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:20.653923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738043+0000","last_change":"2024-09-19T04:18:10.105619+0000","last_active":"2024-09-19T04:20:19.738043+0000","last_peered":"2024-09-19T04:20:19.738043+0000","last_clean":"2024-09-19T04:20:19.738043+0000","last_became_active":"2024-09-19T04:18:10.105082+0000","last_became_peered":"2024-09-19T04:18:10.105082+0000","last_unstale":"2024-09-19T04:20:19.738043+0000","last_undegraded":"2024-09-19T04:20:19.738043+0000","last_fullsized":"2024-09-19T04:20:19.738043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:56.843068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454480+0000","last_change":"2024-09-19T04:18:12.498473+0000","last_active":"2024-09-19T04:20:19.454480+0000","last_peered":"2024-09-19T04:20:19.454480+0000","last_clean":"2024-09-19T04:20:19.454480+0000","last_became_active":"2024-09-19T04:18:12.496904+0000","last_became_peered":"2024-09-19T04:18:12.496904+0000","last_unstale":"2024-09-19T04:20:19.454480+0000","last_undegraded":"2024-09-19T04:20:19.454480+0000","last_fullsized":"2024-09-19T04:20:19.454480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:03.155899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173775+0000","last_change":"2024-09-19T04:18:10.365182+0000","last_active":"2024-09-19T04:20:19.173775+0000","last_peered":"2024-09-19T04:20:19.173775+0000","last_clean":"2024-09-19T04:20:19.173775+0000","last_became_active":"2024-09-19T04:18:10.364717+0000","last_became_peered":"2024-09-19T04:18:10.364717+0000","last_unstale":"2024-09-19T04:20:19.173775+0000","last_undegraded":"2024-09-19T04:20:19.173775+0000","last_fullsized":"2024-09-19T04:20:19.173775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:32.063066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158550+0000","last_change":"2024-09-19T04:18:12.493585+0000","last_active":"2024-09-19T04:20:19.158550+0000","last_peered":"2024-09-19T04:20:19.158550+0000","last_clean":"2024-09-19T04:20:19.158550+0000","last_became_active":"2024-09-19T04:18:12.493255+0000","last_became_peered":"2024-09-19T04:18:12.493255+0000","last_unstale":"2024-09-19T04:20:19.158550+0000","last_undegraded":"2024-09-19T04:20:19.158550+0000","last_fullsized":"2024-09-19T04:20:19.158550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:33.139727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174154+0000","last_change":"2024-09-19T04:18:10.101537+0000","last_active":"2024-09-19T04:20:19.174154+0000","last_peered":"2024-09-19T04:20:19.174154+0000","last_clean":"2024-09-19T04:20:19.174154+0000","last_became_active":"2024-09-19T04:18:10.100951+0000","last_became_peered":"2024-09-19T04:18:10.100951+0000","last_unstale":"2024-09-19T04:20:19.174154+0000","last_undegraded":"2024-09-19T04:20:19.174154+0000","last_fullsized":"2024-09-19T04:20:19.174154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:08.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466903+0000","last_change":"2024-09-19T04:18:12.495530+0000","last_active":"2024-09-19T04:20:19.466903+0000","last_peered":"2024-09-19T04:20:19.466903+0000","last_clean":"2024-09-19T04:20:19.466903+0000","last_became_active":"2024-09-19T04:18:12.495176+0000","last_became_peered":"2024-09-19T04:18:12.495176+0000","last_unstale":"2024-09-19T04:20:19.466903+0000","last_undegraded":"2024-09-19T04:20:19.466903+0000","last_fullsized":"2024-09-19T04:20:19.466903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:24.308047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157053+0000","last_change":"2024-09-19T04:18:10.102539+0000","last_active":"2024-09-19T04:20:19.157053+0000","last_peered":"2024-09-19T04:20:19.157053+0000","last_clean":"2024-09-19T04:20:19.157053+0000","last_became_active":"2024-09-19T04:18:10.102109+0000","last_became_peered":"2024-09-19T04:18:10.102109+0000","last_unstale":"2024-09-19T04:20:19.157053+0000","last_undegraded":"2024-09-19T04:20:19.157053+0000","last_fullsized":"2024-09-19T04:20:19.157053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:58.852767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455095+0000","last_change":"2024-09-19T04:18:12.491769+0000","last_active":"2024-09-19T04:20:19.455095+0000","last_peered":"2024-09-19T04:20:19.455095+0000","last_clean":"2024-09-19T04:20:19.455095+0000","last_became_active":"2024-09-19T04:18:12.491435+0000","last_became_peered":"2024-09-19T04:18:12.491435+0000","last_unstale":"2024-09-19T04:20:19.455095+0000","last_undegraded":"2024-09-19T04:20:19.455095+0000","last_fullsized":"2024-09-19T04:20:19.455095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:54.880533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'41","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.411367+0000","last_change":"2024-09-19T04:18:10.103559+0000","last_active":"2024-09-19T04:20:19.411367+0000","last_peered":"2024-09-19T04:20:19.411367+0000","last_clean":"2024-09-19T04:20:19.411367+0000","last_became_active":"2024-09-19T04:18:10.103208+0000","last_became_peered":"2024-09-19T04:18:10.103208+0000","last_unstale":"2024-09-19T04:20:19.411367+0000","last_undegraded":"2024-09-19T04:20:19.411367+0000","last_fullsized":"2024-09-19T04:20:19.411367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:16.564916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454709+0000","last_change":"2024-09-19T04:18:12.495616+0000","last_active":"2024-09-19T04:20:19.454709+0000","last_peered":"2024-09-19T04:20:19.454709+0000","last_clean":"2024-09-19T04:20:19.454709+0000","last_became_active":"2024-09-19T04:18:12.495243+0000","last_became_peered":"2024-09-19T04:18:12.495243+0000","last_unstale":"2024-09-19T04:20:19.454709+0000","last_undegraded":"2024-09-19T04:20:19.454709+0000","last_fullsized":"2024-09-19T04:20:19.454709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:12.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'8","reported_seq":152,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738564+0000","last_change":"2024-09-19T04:18:10.626874+0000","last_active":"2024-09-19T04:20:19.738564+0000","last_peered":"2024-09-19T04:20:19.738564+0000","last_clean":"2024-09-19T04:20:19.738564+0000","last_became_active":"2024-09-19T04:18:10.626506+0000","last_became_peered":"2024-09-19T04:18:10.626506+0000","last_unstale":"2024-09-19T04:20:19.738564+0000","last_undegraded":"2024-09-19T04:20:19.738564+0000","last_fullsized":"2024-09-19T04:20:19.738564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:48.725143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454917+0000","last_change":"2024-09-19T04:18:12.496350+0000","last_active":"2024-09-19T04:20:19.454917+0000","last_peered":"2024-09-19T04:20:19.454917+0000","last_clean":"2024-09-19T04:20:19.454917+0000","last_became_active":"2024-09-19T04:18:12.495904+0000","last_became_peered":"2024-09-19T04:18:12.495904+0000","last_unstale":"2024-09-19T04:20:19.454917+0000","last_undegraded":"2024-09-19T04:20:19.454917+0000","last_fullsized":"2024-09-19T04:20:19.454917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:08.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454676+0000","last_change":"2024-09-19T04:16:56.164450+0000","last_active":"2024-09-19T04:20:19.454676+0000","last_peered":"2024-09-19T04:20:19.454676+0000","last_clean":"2024-09-19T04:20:19.454676+0000","last_became_active":"2024-09-19T04:16:56.164008+0000","last_became_peered":"2024-09-19T04:16:56.164008+0000","last_unstale":"2024-09-19T04:20:19.454676+0000","last_undegraded":"2024-09-19T04:20:19.454676+0000","last_fullsized":"2024-09-19T04:20:19.454676+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_clean_scrub_stamp":"2024-09-19T04:13:18.308718+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:14.697725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.459005+0000","last_change":"2024-09-19T04:18:10.101598+0000","last_active":"2024-09-19T04:20:19.459005+0000","last_peered":"2024-09-19T04:20:19.459005+0000","last_clean":"2024-09-19T04:20:19.459005+0000","last_became_active":"2024-09-19T04:18:10.101210+0000","last_became_peered":"2024-09-19T04:18:10.101210+0000","last_unstale":"2024-09-19T04:20:19.459005+0000","last_undegraded":"2024-09-19T04:20:19.459005+0000","last_fullsized":"2024-09-19T04:20:19.459005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:10.653481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260008+0000","last_change":"2024-09-19T04:18:12.495034+0000","last_active":"2024-09-19T04:20:19.260008+0000","last_peered":"2024-09-19T04:20:19.260008+0000","last_clean":"2024-09-19T04:20:19.260008+0000","last_became_active":"2024-09-19T04:18:12.494611+0000","last_became_peered":"2024-09-19T04:18:12.494611+0000","last_unstale":"2024-09-19T04:20:19.260008+0000","last_undegraded":"2024-09-19T04:20:19.260008+0000","last_fullsized":"2024-09-19T04:20:19.260008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:28.448927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454656+0000","last_change":"2024-09-19T04:18:10.105212+0000","last_active":"2024-09-19T04:20:19.454656+0000","last_peered":"2024-09-19T04:20:19.454656+0000","last_clean":"2024-09-19T04:20:19.454656+0000","last_became_active":"2024-09-19T04:18:10.104710+0000","last_became_peered":"2024-09-19T04:18:10.104710+0000","last_unstale":"2024-09-19T04:20:19.454656+0000","last_undegraded":"2024-09-19T04:20:19.454656+0000","last_fullsized":"2024-09-19T04:20:19.454656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:58.007773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.156921+0000","last_change":"2024-09-19T04:19:00.980419+0000","last_active":"2024-09-19T04:20:19.156921+0000","last_peered":"2024-09-19T04:20:19.156921+0000","last_clean":"2024-09-19T04:20:19.156921+0000","last_became_active":"2024-09-19T04:19:00.980008+0000","last_became_peered":"2024-09-19T04:19:00.980008+0000","last_unstale":"2024-09-19T04:20:19.156921+0000","last_undegraded":"2024-09-19T04:20:19.156921+0000","last_fullsized":"2024-09-19T04:20:19.156921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:18.396611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":147,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.737581+0000","last_change":"2024-09-19T04:18:10.364530+0000","last_active":"2024-09-19T04:20:19.737581+0000","last_peered":"2024-09-19T04:20:19.737581+0000","last_clean":"2024-09-19T04:20:19.737581+0000","last_became_active":"2024-09-19T04:18:10.364204+0000","last_became_peered":"2024-09-19T04:18:10.364204+0000","last_unstale":"2024-09-19T04:20:19.737581+0000","last_undegraded":"2024-09-19T04:20:19.737581+0000","last_fullsized":"2024-09-19T04:20:19.737581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:04.229530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158567+0000","last_change":"2024-09-19T04:18:12.493730+0000","last_active":"2024-09-19T04:20:19.158567+0000","last_peered":"2024-09-19T04:20:19.158567+0000","last_clean":"2024-09-19T04:20:19.158567+0000","last_became_active":"2024-09-19T04:18:12.493153+0000","last_became_peered":"2024-09-19T04:18:12.493153+0000","last_unstale":"2024-09-19T04:20:19.158567+0000","last_undegraded":"2024-09-19T04:20:19.158567+0000","last_fullsized":"2024-09-19T04:20:19.158567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.758490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'7","reported_seq":150,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.739821+0000","last_change":"2024-09-19T04:18:10.102073+0000","last_active":"2024-09-19T04:20:19.739821+0000","last_peered":"2024-09-19T04:20:19.739821+0000","last_clean":"2024-09-19T04:20:19.739821+0000","last_became_active":"2024-09-19T04:18:10.101755+0000","last_became_peered":"2024-09-19T04:18:10.101755+0000","last_unstale":"2024-09-19T04:20:19.739821+0000","last_undegraded":"2024-09-19T04:20:19.739821+0000","last_fullsized":"2024-09-19T04:20:19.739821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:21.489146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1292,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159049+0000","last_change":"2024-09-19T04:18:12.494560+0000","last_active":"2024-09-19T04:20:19.159049+0000","last_peered":"2024-09-19T04:20:19.159049+0000","last_clean":"2024-09-19T04:20:19.159049+0000","last_became_active":"2024-09-19T04:18:12.494203+0000","last_became_peered":"2024-09-19T04:18:12.494203+0000","last_unstale":"2024-09-19T04:20:19.159049+0000","last_undegraded":"2024-09-19T04:20:19.159049+0000","last_fullsized":"2024-09-19T04:20:19.159049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:35:17.129649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.256225+0000","last_change":"2024-09-19T04:18:10.365781+0000","last_active":"2024-09-19T04:20:19.256225+0000","last_peered":"2024-09-19T04:20:19.256225+0000","last_clean":"2024-09-19T04:20:19.256225+0000","last_became_active":"2024-09-19T04:18:10.365417+0000","last_became_peered":"2024-09-19T04:18:10.365417+0000","last_unstale":"2024-09-19T04:20:19.256225+0000","last_undegraded":"2024-09-19T04:20:19.256225+0000","last_fullsized":"2024-09-19T04:20:19.256225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:20.097572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159240+0000","last_change":"2024-09-19T04:18:12.495570+0000","last_active":"2024-09-19T04:20:19.159240+0000","last_peered":"2024-09-19T04:20:19.159240+0000","last_clean":"2024-09-19T04:20:19.159240+0000","last_became_active":"2024-09-19T04:18:12.495242+0000","last_became_peered":"2024-09-19T04:18:12.495242+0000","last_unstale":"2024-09-19T04:20:19.159240+0000","last_undegraded":"2024-09-19T04:20:19.159240+0000","last_fullsized":"2024-09-19T04:20:19.159240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:57.448182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174114+0000","last_change":"2024-09-19T04:18:10.365134+0000","last_active":"2024-09-19T04:20:19.174114+0000","last_peered":"2024-09-19T04:20:19.174114+0000","last_clean":"2024-09-19T04:20:19.174114+0000","last_became_active":"2024-09-19T04:18:10.364726+0000","last_became_peered":"2024-09-19T04:18:10.364726+0000","last_unstale":"2024-09-19T04:20:19.174114+0000","last_undegraded":"2024-09-19T04:20:19.174114+0000","last_fullsized":"2024-09-19T04:20:19.174114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:44.207855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157194+0000","last_change":"2024-09-19T04:18:12.488836+0000","last_active":"2024-09-19T04:20:19.157194+0000","last_peered":"2024-09-19T04:20:19.157194+0000","last_clean":"2024-09-19T04:20:19.157194+0000","last_became_active":"2024-09-19T04:18:12.488507+0000","last_became_peered":"2024-09-19T04:18:12.488507+0000","last_unstale":"2024-09-19T04:20:19.157194+0000","last_undegraded":"2024-09-19T04:20:19.157194+0000","last_fullsized":"2024-09-19T04:20:19.157194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:31.308114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'52","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.417279+0000","last_change":"2024-09-19T04:18:10.103623+0000","last_active":"2024-09-19T04:20:19.417279+0000","last_peered":"2024-09-19T04:20:19.417279+0000","last_clean":"2024-09-19T04:20:19.417279+0000","last_became_active":"2024-09-19T04:18:10.103340+0000","last_became_peered":"2024-09-19T04:18:10.103340+0000","last_unstale":"2024-09-19T04:20:19.417279+0000","last_undegraded":"2024-09-19T04:20:19.417279+0000","last_fullsized":"2024-09-19T04:20:19.417279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:11.724013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260722+0000","last_change":"2024-09-19T04:18:12.498167+0000","last_active":"2024-09-19T04:20:19.260722+0000","last_peered":"2024-09-19T04:20:19.260722+0000","last_clean":"2024-09-19T04:20:19.260722+0000","last_became_active":"2024-09-19T04:18:12.496924+0000","last_became_peered":"2024-09-19T04:18:12.496924+0000","last_unstale":"2024-09-19T04:20:19.260722+0000","last_undegraded":"2024-09-19T04:20:19.260722+0000","last_fullsized":"2024-09-19T04:20:19.260722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:13.952030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.459180+0000","last_change":"2024-09-19T04:18:10.627106+0000","last_active":"2024-09-19T04:20:19.459180+0000","last_peered":"2024-09-19T04:20:19.459180+0000","last_clean":"2024-09-19T04:20:19.459180+0000","last_became_active":"2024-09-19T04:18:10.626708+0000","last_became_peered":"2024-09-19T04:18:10.626708+0000","last_unstale":"2024-09-19T04:20:19.459180+0000","last_undegraded":"2024-09-19T04:20:19.459180+0000","last_fullsized":"2024-09-19T04:20:19.459180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:46.049699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235881+0000","last_change":"2024-09-19T04:18:12.506995+0000","last_active":"2024-09-19T04:20:19.235881+0000","last_peered":"2024-09-19T04:20:19.235881+0000","last_clean":"2024-09-19T04:20:19.235881+0000","last_became_active":"2024-09-19T04:18:12.500607+0000","last_became_peered":"2024-09-19T04:18:12.500607+0000","last_unstale":"2024-09-19T04:20:19.235881+0000","last_undegraded":"2024-09-19T04:20:19.235881+0000","last_fullsized":"2024-09-19T04:20:19.235881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:37.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456633+0000","last_change":"2024-09-19T04:18:10.365150+0000","last_active":"2024-09-19T04:20:19.456633+0000","last_peered":"2024-09-19T04:20:19.456633+0000","last_clean":"2024-09-19T04:20:19.456633+0000","last_became_active":"2024-09-19T04:18:10.364755+0000","last_became_peered":"2024-09-19T04:18:10.364755+0000","last_unstale":"2024-09-19T04:20:19.456633+0000","last_undegraded":"2024-09-19T04:20:19.456633+0000","last_fullsized":"2024-09-19T04:20:19.456633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:11.639532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173701+0000","last_change":"2024-09-19T04:18:12.484588+0000","last_active":"2024-09-19T04:20:19.173701+0000","last_peered":"2024-09-19T04:20:19.173701+0000","last_clean":"2024-09-19T04:20:19.173701+0000","last_became_active":"2024-09-19T04:18:12.484218+0000","last_became_peered":"2024-09-19T04:18:12.484218+0000","last_unstale":"2024-09-19T04:20:19.173701+0000","last_undegraded":"2024-09-19T04:20:19.173701+0000","last_fullsized":"2024-09-19T04:20:19.173701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:55.634557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259785+0000","last_change":"2024-09-19T04:18:10.100832+0000","last_active":"2024-09-19T04:20:19.259785+0000","last_peered":"2024-09-19T04:20:19.259785+0000","last_clean":"2024-09-19T04:20:19.259785+0000","last_became_active":"2024-09-19T04:18:10.100518+0000","last_became_peered":"2024-09-19T04:18:10.100518+0000","last_unstale":"2024-09-19T04:20:19.259785+0000","last_undegraded":"2024-09-19T04:20:19.259785+0000","last_fullsized":"2024-09-19T04:20:19.259785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:37.028120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466793+0000","last_change":"2024-09-19T04:18:12.496914+0000","last_active":"2024-09-19T04:20:19.466793+0000","last_peered":"2024-09-19T04:20:19.466793+0000","last_clean":"2024-09-19T04:20:19.466793+0000","last_became_active":"2024-09-19T04:18:12.496556+0000","last_became_peered":"2024-09-19T04:18:12.496556+0000","last_unstale":"2024-09-19T04:20:19.466793+0000","last_undegraded":"2024-09-19T04:20:19.466793+0000","last_fullsized":"2024-09-19T04:20:19.466793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:52.391412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466731+0000","last_change":"2024-09-19T04:19:00.993461+0000","last_active":"2024-09-19T04:20:19.466731+0000","last_peered":"2024-09-19T04:20:19.466731+0000","last_clean":"2024-09-19T04:20:19.466731+0000","last_became_active":"2024-09-19T04:19:00.980704+0000","last_became_peered":"2024-09-19T04:19:00.980704+0000","last_unstale":"2024-09-19T04:20:19.466731+0000","last_undegraded":"2024-09-19T04:20:19.466731+0000","last_fullsized":"2024-09-19T04:20:19.466731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:56.090092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260090+0000","last_change":"2024-09-19T04:18:12.496852+0000","last_active":"2024-09-19T04:20:19.260090+0000","last_peered":"2024-09-19T04:20:19.260090+0000","last_clean":"2024-09-19T04:20:19.260090+0000","last_became_active":"2024-09-19T04:18:12.495365+0000","last_became_peered":"2024-09-19T04:18:12.495365+0000","last_unstale":"2024-09-19T04:20:19.260090+0000","last_undegraded":"2024-09-19T04:20:19.260090+0000","last_fullsized":"2024-09-19T04:20:19.260090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:14:57.012618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173834+0000","last_change":"2024-09-19T04:18:10.104641+0000","last_active":"2024-09-19T04:20:19.173834+0000","last_peered":"2024-09-19T04:20:19.173834+0000","last_clean":"2024-09-19T04:20:19.173834+0000","last_became_active":"2024-09-19T04:18:10.104295+0000","last_became_peered":"2024-09-19T04:18:10.104295+0000","last_unstale":"2024-09-19T04:20:19.173834+0000","last_undegraded":"2024-09-19T04:20:19.173834+0000","last_fullsized":"2024-09-19T04:20:19.173834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:58.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454750+0000","last_change":"2024-09-19T04:18:12.488396+0000","last_active":"2024-09-19T04:20:19.454750+0000","last_peered":"2024-09-19T04:20:19.454750+0000","last_clean":"2024-09-19T04:20:19.454750+0000","last_became_active":"2024-09-19T04:18:12.488048+0000","last_became_peered":"2024-09-19T04:18:12.488048+0000","last_unstale":"2024-09-19T04:20:19.454750+0000","last_undegraded":"2024-09-19T04:20:19.454750+0000","last_fullsized":"2024-09-19T04:20:19.454750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:27.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158938+0000","last_change":"2024-09-19T04:18:12.493722+0000","last_active":"2024-09-19T04:20:19.158938+0000","last_peered":"2024-09-19T04:20:19.158938+0000","last_clean":"2024-09-19T04:20:19.158938+0000","last_became_active":"2024-09-19T04:18:12.492483+0000","last_became_peered":"2024-09-19T04:18:12.492483+0000","last_unstale":"2024-09-19T04:20:19.158938+0000","last_undegraded":"2024-09-19T04:20:19.158938+0000","last_fullsized":"2024-09-19T04:20:19.158938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:33.492623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174205+0000","last_change":"2024-09-19T04:18:10.101326+0000","last_active":"2024-09-19T04:20:19.174205+0000","last_peered":"2024-09-19T04:20:19.174205+0000","last_clean":"2024-09-19T04:20:19.174205+0000","last_became_active":"2024-09-19T04:18:10.100955+0000","last_became_peered":"2024-09-19T04:18:10.100955+0000","last_unstale":"2024-09-19T04:20:19.174205+0000","last_undegraded":"2024-09-19T04:20:19.174205+0000","last_fullsized":"2024-09-19T04:20:19.174205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:41.939741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456660+0000","last_change":"2024-09-19T04:18:12.492750+0000","last_active":"2024-09-19T04:20:19.456660+0000","last_peered":"2024-09-19T04:20:19.456660+0000","last_clean":"2024-09-19T04:20:19.456660+0000","last_became_active":"2024-09-19T04:18:12.492417+0000","last_became_peered":"2024-09-19T04:18:12.492417+0000","last_unstale":"2024-09-19T04:20:19.456660+0000","last_undegraded":"2024-09-19T04:20:19.456660+0000","last_fullsized":"2024-09-19T04:20:19.456660+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:25.279452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466779+0000","last_change":"2024-09-19T04:19:00.989741+0000","last_active":"2024-09-19T04:20:19.466779+0000","last_peered":"2024-09-19T04:20:19.466779+0000","last_clean":"2024-09-19T04:20:19.466779+0000","last_became_active":"2024-09-19T04:19:00.977530+0000","last_became_peered":"2024-09-19T04:19:00.977530+0000","last_unstale":"2024-09-19T04:20:19.466779+0000","last_undegraded":"2024-09-19T04:20:19.466779+0000","last_fullsized":"2024-09-19T04:20:19.466779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:35.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.236131+0000","last_change":"2024-09-19T04:18:12.503510+0000","last_active":"2024-09-19T04:20:19.236131+0000","last_peered":"2024-09-19T04:20:19.236131+0000","last_clean":"2024-09-19T04:20:19.236131+0000","last_became_active":"2024-09-19T04:18:12.495165+0000","last_became_peered":"2024-09-19T04:18:12.495165+0000","last_unstale":"2024-09-19T04:20:19.236131+0000","last_undegraded":"2024-09-19T04:20:19.236131+0000","last_fullsized":"2024-09-19T04:20:19.236131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:13.934328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260678+0000","last_change":"2024-09-19T04:18:10.364783+0000","last_active":"2024-09-19T04:20:19.260678+0000","last_peered":"2024-09-19T04:20:19.260678+0000","last_clean":"2024-09-19T04:20:19.260678+0000","last_became_active":"2024-09-19T04:18:10.364202+0000","last_became_peered":"2024-09-19T04:18:10.364202+0000","last_unstale":"2024-09-19T04:20:19.260678+0000","last_undegraded":"2024-09-19T04:20:19.260678+0000","last_fullsized":"2024-09-19T04:20:19.260678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:18:55.770637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260163+0000","last_change":"2024-09-19T04:18:12.495825+0000","last_active":"2024-09-19T04:20:19.260163+0000","last_peered":"2024-09-19T04:20:19.260163+0000","last_clean":"2024-09-19T04:20:19.260163+0000","last_became_active":"2024-09-19T04:18:12.494525+0000","last_became_peered":"2024-09-19T04:18:12.494525+0000","last_unstale":"2024-09-19T04:20:19.260163+0000","last_undegraded":"2024-09-19T04:20:19.260163+0000","last_fullsized":"2024-09-19T04:20:19.260163+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:57.827135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":155,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738811+0000","last_change":"2024-09-19T04:18:10.102145+0000","last_active":"2024-09-19T04:20:19.738811+0000","last_peered":"2024-09-19T04:20:19.738811+0000","last_clean":"2024-09-19T04:20:19.738811+0000","last_became_active":"2024-09-19T04:18:10.101410+0000","last_became_peered":"2024-09-19T04:18:10.101410+0000","last_unstale":"2024-09-19T04:20:19.738811+0000","last_undegraded":"2024-09-19T04:20:19.738811+0000","last_fullsized":"2024-09-19T04:20:19.738811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:20.354296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601458,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1291,"num_read_kb":762,"num_write":381,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1804374,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107517,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":450642,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1089536,"data_stored":1167124,"data_compressed":41814,"data_compressed_allocated":53248,"data_compressed_original":659456,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972402,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":970752,"data_stored":842861,"data_compressed":22034,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201335,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":843239,"data_compressed":21997,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":1044960,"data_compressed":38727,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":449673,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822727,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":1174748,"data_compressed":44883,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":450009,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215184,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":974003,"data_compressed":28190,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":519225,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":517462,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":451628,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135740,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200605,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:20:23.826 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.032+0000 7fdb27600640 1 -- 172.21.15.107:0/1762197068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb20103d30 msgr2=0x7fdb20104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.032+0000 7fdb27600640 1 --2- 172.21.15.107:0/1762197068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb20103d30 0x7fdb20104130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009900 tx=0x7fdb1402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.032+0000 7fdb27600640 1 -- 172.21.15.107:0/1762197068 shutdown_connections 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.032+0000 7fdb27600640 1 --2- 172.21.15.107:0/1762197068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb201058e0 0x7fdb2010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.032+0000 7fdb27600640 1 --2- 172.21.15.107:0/1762197068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20104f20 0x7fdb201053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.032+0000 7fdb27600640 1 --2- 172.21.15.107:0/1762197068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb20103d30 0x7fdb20104130 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.032+0000 7fdb27600640 1 -- 172.21.15.107:0/1762197068 >> 172.21.15.107:0/1762197068 conn(0x7fdb200ff4b0 msgr2=0x7fdb201018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.032+0000 7fdb27600640 1 -- 172.21.15.107:0/1762197068 shutdown_connections 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.033+0000 7fdb27600640 1 -- 172.21.15.107:0/1762197068 wait complete. 2024-09-19T04:20:24.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.033+0000 7fdb27600640 1 Processor -- start 2024-09-19T04:20:24.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.033+0000 7fdb27600640 1 -- start start 2024-09-19T04:20:24.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb27600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb20103d30 0x7fdb2019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb27600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20104f20 0x7fdb2019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb27600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb201058e0 0x7fdb201a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb27600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2010f9e0 con 0x7fdb20103d30 2024-09-19T04:20:24.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb27600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2010f860 con 0x7fdb20104f20 2024-09-19T04:20:24.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb27600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2010fb60 con 0x7fdb201058e0 2024-09-19T04:20:24.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb1fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20104f20 0x7fdb2019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb1fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20104f20 0x7fdb2019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:49530/0 (socket says 172.21.15.107:49530) 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb1fe00640 1 -- 172.21.15.107:0/3777012219 learned_addr learned my addr 172.21.15.107:0/3777012219 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb24c00640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb20103d30 0x7fdb2019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb25600640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb201058e0 0x7fdb201a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb1fe00640 1 -- 172.21.15.107:0/3777012219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb201058e0 msgr2=0x7fdb201a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb1fe00640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb201058e0 0x7fdb201a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb1fe00640 1 -- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb20103d30 msgr2=0x7fdb2019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb1fe00640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb20103d30 0x7fdb2019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb1fe00640 1 -- 172.21.15.107:0/3777012219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb201a1c90 con 0x7fdb20104f20 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.034+0000 7fdb24c00640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb20103d30 0x7fdb2019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.035+0000 7fdb1fe00640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20104f20 0x7fdb2019a5b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00ea60 tx=0x7fdb0c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.035+0000 7fdb1d600640 1 -- 172.21.15.107:0/3777012219 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00cde0 con 0x7fdb20104f20 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.035+0000 7fdb1d600640 1 -- 172.21.15.107:0/3777012219 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb0c0035f0 con 0x7fdb20104f20 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.035+0000 7fdb27600640 1 -- 172.21.15.107:0/3777012219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb201a1f80 con 0x7fdb20104f20 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.035+0000 7fdb1d600640 1 -- 172.21.15.107:0/3777012219 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c010670 con 0x7fdb20104f20 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.035+0000 7fdb27600640 1 -- 172.21.15.107:0/3777012219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb201a2440 con 0x7fdb20104f20 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.036+0000 7fdb27600640 1 -- 172.21.15.107:0/3777012219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2010b0c0 con 0x7fdb20104f20 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.039+0000 7fdb1d600640 1 -- 172.21.15.107:0/3777012219 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 138476+0+0 (secure 0 0 0) 0x7fdb0c0037a0 con 0x7fdb20104f20 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.041+0000 7fdb1d600640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdb080a33c0 0x7fdb080a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.041+0000 7fdb1d600640 1 -- 172.21.15.107:0/3777012219 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fdb0c0d0080 con 0x7fdb20104f20 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.041+0000 7fdb24c00640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdb080a33c0 0x7fdb080a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.042+0000 7fdb24c00640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdb080a33c0 0x7fdb080a5880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdb1402a9e0 tx=0x7fdb14000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.044+0000 7fdb1d600640 1 -- 172.21.15.107:0/3777012219 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdb0c09ca50 con 0x7fdb20104f20 2024-09-19T04:20:24.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.147+0000 7fdb1d600640 1 -- 172.21.15.107:0/3777012219 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7fdb0c0989c0 con 0x7fdb20104f20 2024-09-19T04:20:24.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.248+0000 7fdb27600640 1 -- 172.21.15.107:0/3777012219 --> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb20062630 con 0x7fdb080a33c0 2024-09-19T04:20:24.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.256+0000 7fdb1d600640 1 -- 172.21.15.107:0/3777012219 <== mgr.15114 v2:172.21.15.107:6800/1724794486 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258046 (secure 0 0 0) 0x7fdb20062630 con 0x7fdb080a33c0 2024-09-19T04:20:24.261 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:20:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-19T04:20:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 -- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdb080a33c0 msgr2=0x7fdb080a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdb080a33c0 0x7fdb080a5880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdb1402a9e0 tx=0x7fdb14000bc0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 -- 172.21.15.107:0/3777012219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20104f20 msgr2=0x7fdb2019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20104f20 0x7fdb2019a5b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00ea60 tx=0x7fdb0c00ef20 comp rx=0 tx=0).stop 2024-09-19T04:20:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 -- 172.21.15.107:0/3777012219 shutdown_connections 2024-09-19T04:20:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdb080a33c0 0x7fdb080a5880 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb201058e0 0x7fdb201a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20104f20 0x7fdb2019a5b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 --2- 172.21.15.107:0/3777012219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb20103d30 0x7fdb2019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 -- 172.21.15.107:0/3777012219 >> 172.21.15.107:0/3777012219 conn(0x7fdb200ff4b0 msgr2=0x7fdb20108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 -- 172.21.15.107:0/3777012219 shutdown_connections 2024-09-19T04:20:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.260+0000 7fdb27600640 1 -- 172.21.15.107:0/3777012219 wait complete. 2024-09-19T04:20:24.278 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T04:20:23.171159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192060,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1427,"num_read_kb":881,"num_write":564,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":946,"ondisk_log_size":946,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377448,"kb_used_data":11516,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564376,"statfs":{"total":1151940427776,"available":1151553921024,"internally_reserved":0,"allocated":11792384,"data_stored":8885574,"data_compressed":214973,"data_compressed_allocated":278528,"data_compressed_original":3354624,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.980758"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.467136+0000","last_change":"2024-09-19T04:18:12.499720+0000","last_active":"2024-09-19T04:20:19.467136+0000","last_peered":"2024-09-19T04:20:19.467136+0000","last_clean":"2024-09-19T04:20:19.467136+0000","last_became_active":"2024-09-19T04:18:12.499381+0000","last_became_peered":"2024-09-19T04:18:12.499381+0000","last_unstale":"2024-09-19T04:20:19.467136+0000","last_undegraded":"2024-09-19T04:20:19.467136+0000","last_fullsized":"2024-09-19T04:20:19.467136+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:46.708870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235809+0000","last_change":"2024-09-19T04:18:12.498163+0000","last_active":"2024-09-19T04:20:19.235809+0000","last_peered":"2024-09-19T04:20:19.235809+0000","last_clean":"2024-09-19T04:20:19.235809+0000","last_became_active":"2024-09-19T04:18:12.495706+0000","last_became_peered":"2024-09-19T04:18:12.495706+0000","last_unstale":"2024-09-19T04:20:19.235809+0000","last_undegraded":"2024-09-19T04:20:19.235809+0000","last_fullsized":"2024-09-19T04:20:19.235809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:50.604410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235810+0000","last_change":"2024-09-19T04:18:12.493903+0000","last_active":"2024-09-19T04:20:19.235810+0000","last_peered":"2024-09-19T04:20:19.235810+0000","last_clean":"2024-09-19T04:20:19.235810+0000","last_became_active":"2024-09-19T04:18:12.493443+0000","last_became_peered":"2024-09-19T04:18:12.493443+0000","last_unstale":"2024-09-19T04:20:19.235810+0000","last_undegraded":"2024-09-19T04:20:19.235810+0000","last_fullsized":"2024-09-19T04:20:19.235810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:51.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.458868+0000","last_change":"2024-09-19T04:18:12.500460+0000","last_active":"2024-09-19T04:20:19.458868+0000","last_peered":"2024-09-19T04:20:19.458868+0000","last_clean":"2024-09-19T04:20:19.458868+0000","last_became_active":"2024-09-19T04:18:12.494675+0000","last_became_peered":"2024-09-19T04:18:12.494675+0000","last_unstale":"2024-09-19T04:20:19.458868+0000","last_undegraded":"2024-09-19T04:20:19.458868+0000","last_fullsized":"2024-09-19T04:20:19.458868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:13:32.863156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173788+0000","last_change":"2024-09-19T04:18:12.494936+0000","last_active":"2024-09-19T04:20:19.173788+0000","last_peered":"2024-09-19T04:20:19.173788+0000","last_clean":"2024-09-19T04:20:19.173788+0000","last_became_active":"2024-09-19T04:18:12.494556+0000","last_became_peered":"2024-09-19T04:18:12.494556+0000","last_unstale":"2024-09-19T04:20:19.173788+0000","last_undegraded":"2024-09-19T04:20:19.173788+0000","last_fullsized":"2024-09-19T04:20:19.173788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:06.001529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158827+0000","last_change":"2024-09-19T04:18:12.495880+0000","last_active":"2024-09-19T04:20:19.158827+0000","last_peered":"2024-09-19T04:20:19.158827+0000","last_clean":"2024-09-19T04:20:19.158827+0000","last_became_active":"2024-09-19T04:18:12.495562+0000","last_became_peered":"2024-09-19T04:18:12.495562+0000","last_unstale":"2024-09-19T04:20:19.158827+0000","last_undegraded":"2024-09-19T04:20:19.158827+0000","last_fullsized":"2024-09-19T04:20:19.158827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:20.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454525+0000","last_change":"2024-09-19T04:18:12.497991+0000","last_active":"2024-09-19T04:20:19.454525+0000","last_peered":"2024-09-19T04:20:19.454525+0000","last_clean":"2024-09-19T04:20:19.454525+0000","last_became_active":"2024-09-19T04:18:12.497644+0000","last_became_peered":"2024-09-19T04:18:12.497644+0000","last_unstale":"2024-09-19T04:20:19.454525+0000","last_undegraded":"2024-09-19T04:20:19.454525+0000","last_fullsized":"2024-09-19T04:20:19.454525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.851930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259408+0000","last_change":"2024-09-19T04:18:12.495789+0000","last_active":"2024-09-19T04:20:19.259408+0000","last_peered":"2024-09-19T04:20:19.259408+0000","last_clean":"2024-09-19T04:20:19.259408+0000","last_became_active":"2024-09-19T04:18:12.494637+0000","last_became_peered":"2024-09-19T04:18:12.494637+0000","last_unstale":"2024-09-19T04:20:19.259408+0000","last_undegraded":"2024-09-19T04:20:19.259408+0000","last_fullsized":"2024-09-19T04:20:19.259408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:14.245180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.457103+0000","last_change":"2024-09-19T04:18:12.491433+0000","last_active":"2024-09-19T04:20:19.457103+0000","last_peered":"2024-09-19T04:20:19.457103+0000","last_clean":"2024-09-19T04:20:19.457103+0000","last_became_active":"2024-09-19T04:18:12.490444+0000","last_became_peered":"2024-09-19T04:18:12.490444+0000","last_unstale":"2024-09-19T04:20:19.457103+0000","last_undegraded":"2024-09-19T04:20:19.457103+0000","last_fullsized":"2024-09-19T04:20:19.457103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:43.068355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159407+0000","last_change":"2024-09-19T04:18:12.493188+0000","last_active":"2024-09-19T04:20:19.159407+0000","last_peered":"2024-09-19T04:20:19.159407+0000","last_clean":"2024-09-19T04:20:19.159407+0000","last_became_active":"2024-09-19T04:18:12.492788+0000","last_became_peered":"2024-09-19T04:18:12.492788+0000","last_unstale":"2024-09-19T04:20:19.159407+0000","last_undegraded":"2024-09-19T04:20:19.159407+0000","last_fullsized":"2024-09-19T04:20:19.159407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:17.912820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454786+0000","last_change":"2024-09-19T04:18:12.490616+0000","last_active":"2024-09-19T04:20:19.454786+0000","last_peered":"2024-09-19T04:20:19.454786+0000","last_clean":"2024-09-19T04:20:19.454786+0000","last_became_active":"2024-09-19T04:18:12.490155+0000","last_became_peered":"2024-09-19T04:18:12.490155+0000","last_unstale":"2024-09-19T04:20:19.454786+0000","last_undegraded":"2024-09-19T04:20:19.454786+0000","last_fullsized":"2024-09-19T04:20:19.454786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:47.082082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.458913+0000","last_change":"2024-09-19T04:18:12.494191+0000","last_active":"2024-09-19T04:20:19.458913+0000","last_peered":"2024-09-19T04:20:19.458913+0000","last_clean":"2024-09-19T04:20:19.458913+0000","last_became_active":"2024-09-19T04:18:12.493786+0000","last_became_peered":"2024-09-19T04:18:12.493786+0000","last_unstale":"2024-09-19T04:20:19.458913+0000","last_undegraded":"2024-09-19T04:20:19.458913+0000","last_fullsized":"2024-09-19T04:20:19.458913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:29.691556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157025+0000","last_change":"2024-09-19T04:18:12.482167+0000","last_active":"2024-09-19T04:20:19.157025+0000","last_peered":"2024-09-19T04:20:19.157025+0000","last_clean":"2024-09-19T04:20:19.157025+0000","last_became_active":"2024-09-19T04:18:12.481314+0000","last_became_peered":"2024-09-19T04:18:12.481314+0000","last_unstale":"2024-09-19T04:20:19.157025+0000","last_undegraded":"2024-09-19T04:20:19.157025+0000","last_fullsized":"2024-09-19T04:20:19.157025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:28.081641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235565+0000","last_change":"2024-09-19T04:18:12.491711+0000","last_active":"2024-09-19T04:20:19.235565+0000","last_peered":"2024-09-19T04:20:19.235565+0000","last_clean":"2024-09-19T04:20:19.235565+0000","last_became_active":"2024-09-19T04:18:12.490964+0000","last_became_peered":"2024-09-19T04:18:12.490964+0000","last_unstale":"2024-09-19T04:20:19.235565+0000","last_undegraded":"2024-09-19T04:20:19.235565+0000","last_fullsized":"2024-09-19T04:20:19.235565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:16.607679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235461+0000","last_change":"2024-09-19T04:18:12.504737+0000","last_active":"2024-09-19T04:20:19.235461+0000","last_peered":"2024-09-19T04:20:19.235461+0000","last_clean":"2024-09-19T04:20:19.235461+0000","last_became_active":"2024-09-19T04:18:12.497583+0000","last_became_peered":"2024-09-19T04:18:12.497583+0000","last_unstale":"2024-09-19T04:20:19.235461+0000","last_undegraded":"2024-09-19T04:20:19.235461+0000","last_fullsized":"2024-09-19T04:20:19.235461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:48.456738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456542+0000","last_change":"2024-09-19T04:18:12.499045+0000","last_active":"2024-09-19T04:20:19.456542+0000","last_peered":"2024-09-19T04:20:19.456542+0000","last_clean":"2024-09-19T04:20:19.456542+0000","last_became_active":"2024-09-19T04:18:12.498713+0000","last_became_peered":"2024-09-19T04:18:12.498713+0000","last_unstale":"2024-09-19T04:20:19.456542+0000","last_undegraded":"2024-09-19T04:20:19.456542+0000","last_fullsized":"2024-09-19T04:20:19.456542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:54.337769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.467108+0000","last_change":"2024-09-19T04:18:12.495989+0000","last_active":"2024-09-19T04:20:19.467108+0000","last_peered":"2024-09-19T04:20:19.467108+0000","last_clean":"2024-09-19T04:20:19.467108+0000","last_became_active":"2024-09-19T04:18:12.495633+0000","last_became_peered":"2024-09-19T04:18:12.495633+0000","last_unstale":"2024-09-19T04:20:19.467108+0000","last_undegraded":"2024-09-19T04:20:19.467108+0000","last_fullsized":"2024-09-19T04:20:19.467108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:17.225861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.255995+0000","last_change":"2024-09-19T04:18:12.488532+0000","last_active":"2024-09-19T04:20:19.255995+0000","last_peered":"2024-09-19T04:20:19.255995+0000","last_clean":"2024-09-19T04:20:19.255995+0000","last_became_active":"2024-09-19T04:18:12.488117+0000","last_became_peered":"2024-09-19T04:18:12.488117+0000","last_unstale":"2024-09-19T04:20:19.255995+0000","last_undegraded":"2024-09-19T04:20:19.255995+0000","last_fullsized":"2024-09-19T04:20:19.255995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:45.318852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455765+0000","last_change":"2024-09-19T04:18:12.496784+0000","last_active":"2024-09-19T04:20:19.455765+0000","last_peered":"2024-09-19T04:20:19.455765+0000","last_clean":"2024-09-19T04:20:19.455765+0000","last_became_active":"2024-09-19T04:18:12.496443+0000","last_became_peered":"2024-09-19T04:18:12.496443+0000","last_unstale":"2024-09-19T04:20:19.455765+0000","last_undegraded":"2024-09-19T04:20:19.455765+0000","last_fullsized":"2024-09-19T04:20:19.455765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:25:46.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466725+0000","last_change":"2024-09-19T04:19:00.980557+0000","last_active":"2024-09-19T04:20:19.466725+0000","last_peered":"2024-09-19T04:20:19.466725+0000","last_clean":"2024-09-19T04:20:19.466725+0000","last_became_active":"2024-09-19T04:19:00.980149+0000","last_became_peered":"2024-09-19T04:19:00.980149+0000","last_unstale":"2024-09-19T04:20:19.466725+0000","last_undegraded":"2024-09-19T04:20:19.466725+0000","last_fullsized":"2024-09-19T04:20:19.466725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:06.140085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157083+0000","last_change":"2024-09-19T04:18:12.493731+0000","last_active":"2024-09-19T04:20:19.157083+0000","last_peered":"2024-09-19T04:20:19.157083+0000","last_clean":"2024-09-19T04:20:19.157083+0000","last_became_active":"2024-09-19T04:18:12.493412+0000","last_became_peered":"2024-09-19T04:18:12.493412+0000","last_unstale":"2024-09-19T04:20:19.157083+0000","last_undegraded":"2024-09-19T04:20:19.157083+0000","last_fullsized":"2024-09-19T04:20:19.157083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:41.089637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456785+0000","last_change":"2024-09-19T04:18:12.497033+0000","last_active":"2024-09-19T04:20:19.456785+0000","last_peered":"2024-09-19T04:20:19.456785+0000","last_clean":"2024-09-19T04:20:19.456785+0000","last_became_active":"2024-09-19T04:18:12.496710+0000","last_became_peered":"2024-09-19T04:18:12.496710+0000","last_unstale":"2024-09-19T04:20:19.456785+0000","last_undegraded":"2024-09-19T04:20:19.456785+0000","last_fullsized":"2024-09-19T04:20:19.456785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:19.193869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259529+0000","last_change":"2024-09-19T04:18:12.499213+0000","last_active":"2024-09-19T04:20:19.259529+0000","last_peered":"2024-09-19T04:20:19.259529+0000","last_clean":"2024-09-19T04:20:19.259529+0000","last_became_active":"2024-09-19T04:18:12.498728+0000","last_became_peered":"2024-09-19T04:18:12.498728+0000","last_unstale":"2024-09-19T04:20:19.259529+0000","last_undegraded":"2024-09-19T04:20:19.259529+0000","last_fullsized":"2024-09-19T04:20:19.259529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:37.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173675+0000","last_change":"2024-09-19T04:19:00.982647+0000","last_active":"2024-09-19T04:20:19.173675+0000","last_peered":"2024-09-19T04:20:19.173675+0000","last_clean":"2024-09-19T04:20:19.173675+0000","last_became_active":"2024-09-19T04:19:00.982246+0000","last_became_peered":"2024-09-19T04:19:00.982246+0000","last_unstale":"2024-09-19T04:20:19.173675+0000","last_undegraded":"2024-09-19T04:20:19.173675+0000","last_fullsized":"2024-09-19T04:20:19.173675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:48.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260612+0000","last_change":"2024-09-19T04:18:12.506679+0000","last_active":"2024-09-19T04:20:19.260612+0000","last_peered":"2024-09-19T04:20:19.260612+0000","last_clean":"2024-09-19T04:20:19.260612+0000","last_became_active":"2024-09-19T04:18:12.505953+0000","last_became_peered":"2024-09-19T04:18:12.505953+0000","last_unstale":"2024-09-19T04:20:19.260612+0000","last_undegraded":"2024-09-19T04:20:19.260612+0000","last_fullsized":"2024-09-19T04:20:19.260612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:53.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455913+0000","last_change":"2024-09-19T04:18:12.490511+0000","last_active":"2024-09-19T04:20:19.455913+0000","last_peered":"2024-09-19T04:20:19.455913+0000","last_clean":"2024-09-19T04:20:19.455913+0000","last_became_active":"2024-09-19T04:18:12.490051+0000","last_became_peered":"2024-09-19T04:18:12.490051+0000","last_unstale":"2024-09-19T04:20:19.455913+0000","last_undegraded":"2024-09-19T04:20:19.455913+0000","last_fullsized":"2024-09-19T04:20:19.455913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:32.283312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.467072+0000","last_change":"2024-09-19T04:18:12.491465+0000","last_active":"2024-09-19T04:20:19.467072+0000","last_peered":"2024-09-19T04:20:19.467072+0000","last_clean":"2024-09-19T04:20:19.467072+0000","last_became_active":"2024-09-19T04:18:12.491136+0000","last_became_peered":"2024-09-19T04:18:12.491136+0000","last_unstale":"2024-09-19T04:20:19.467072+0000","last_undegraded":"2024-09-19T04:20:19.467072+0000","last_fullsized":"2024-09-19T04:20:19.467072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:29.498909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157299+0000","last_change":"2024-09-19T04:19:01.599944+0000","last_active":"2024-09-19T04:20:19.157299+0000","last_peered":"2024-09-19T04:20:19.157299+0000","last_clean":"2024-09-19T04:20:19.157299+0000","last_became_active":"2024-09-19T04:19:01.599571+0000","last_became_peered":"2024-09-19T04:19:01.599571+0000","last_unstale":"2024-09-19T04:20:19.157299+0000","last_undegraded":"2024-09-19T04:20:19.157299+0000","last_fullsized":"2024-09-19T04:20:19.157299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:17.929785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235780+0000","last_change":"2024-09-19T04:18:12.504364+0000","last_active":"2024-09-19T04:20:19.235780+0000","last_peered":"2024-09-19T04:20:19.235780+0000","last_clean":"2024-09-19T04:20:19.235780+0000","last_became_active":"2024-09-19T04:18:12.504006+0000","last_became_peered":"2024-09-19T04:18:12.504006+0000","last_unstale":"2024-09-19T04:20:19.235780+0000","last_undegraded":"2024-09-19T04:20:19.235780+0000","last_fullsized":"2024-09-19T04:20:19.235780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:53.309963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455576+0000","last_change":"2024-09-19T04:18:12.498223+0000","last_active":"2024-09-19T04:20:19.455576+0000","last_peered":"2024-09-19T04:20:19.455576+0000","last_clean":"2024-09-19T04:20:19.455576+0000","last_became_active":"2024-09-19T04:18:12.497862+0000","last_became_peered":"2024-09-19T04:18:12.497862+0000","last_unstale":"2024-09-19T04:20:19.455576+0000","last_undegraded":"2024-09-19T04:20:19.455576+0000","last_fullsized":"2024-09-19T04:20:19.455576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:28.858620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455732+0000","last_change":"2024-09-19T04:18:12.502335+0000","last_active":"2024-09-19T04:20:19.455732+0000","last_peered":"2024-09-19T04:20:19.455732+0000","last_clean":"2024-09-19T04:20:19.455732+0000","last_became_active":"2024-09-19T04:18:12.501686+0000","last_became_peered":"2024-09-19T04:18:12.501686+0000","last_unstale":"2024-09-19T04:20:19.455732+0000","last_undegraded":"2024-09-19T04:20:19.455732+0000","last_fullsized":"2024-09-19T04:20:19.455732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:22.348745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466807+0000","last_change":"2024-09-19T04:18:12.490905+0000","last_active":"2024-09-19T04:20:19.466807+0000","last_peered":"2024-09-19T04:20:19.466807+0000","last_clean":"2024-09-19T04:20:19.466807+0000","last_became_active":"2024-09-19T04:18:12.490489+0000","last_became_peered":"2024-09-19T04:18:12.490489+0000","last_unstale":"2024-09-19T04:20:19.466807+0000","last_undegraded":"2024-09-19T04:20:19.466807+0000","last_fullsized":"2024-09-19T04:20:19.466807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.946474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173739+0000","last_change":"2024-09-19T04:18:12.494607+0000","last_active":"2024-09-19T04:20:19.173739+0000","last_peered":"2024-09-19T04:20:19.173739+0000","last_clean":"2024-09-19T04:20:19.173739+0000","last_became_active":"2024-09-19T04:18:12.494288+0000","last_became_peered":"2024-09-19T04:18:12.494288+0000","last_unstale":"2024-09-19T04:20:19.173739+0000","last_undegraded":"2024-09-19T04:20:19.173739+0000","last_fullsized":"2024-09-19T04:20:19.173739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:42.824951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":153,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738608+0000","last_change":"2024-09-19T04:18:10.103108+0000","last_active":"2024-09-19T04:20:19.738608+0000","last_peered":"2024-09-19T04:20:19.738608+0000","last_clean":"2024-09-19T04:20:19.738608+0000","last_became_active":"2024-09-19T04:18:10.102783+0000","last_became_peered":"2024-09-19T04:18:10.102783+0000","last_unstale":"2024-09-19T04:20:19.738608+0000","last_undegraded":"2024-09-19T04:20:19.738608+0000","last_fullsized":"2024-09-19T04:20:19.738608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:25.521953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158921+0000","last_change":"2024-09-19T04:18:12.495453+0000","last_active":"2024-09-19T04:20:19.158921+0000","last_peered":"2024-09-19T04:20:19.158921+0000","last_clean":"2024-09-19T04:20:19.158921+0000","last_became_active":"2024-09-19T04:18:12.494744+0000","last_became_peered":"2024-09-19T04:18:12.494744+0000","last_unstale":"2024-09-19T04:20:19.158921+0000","last_undegraded":"2024-09-19T04:20:19.158921+0000","last_fullsized":"2024-09-19T04:20:19.158921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:41.071723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158885+0000","last_change":"2024-09-19T04:18:10.102675+0000","last_active":"2024-09-19T04:20:19.158885+0000","last_peered":"2024-09-19T04:20:19.158885+0000","last_clean":"2024-09-19T04:20:19.158885+0000","last_became_active":"2024-09-19T04:18:10.102368+0000","last_became_peered":"2024-09-19T04:18:10.102368+0000","last_unstale":"2024-09-19T04:20:19.158885+0000","last_undegraded":"2024-09-19T04:20:19.158885+0000","last_fullsized":"2024-09-19T04:20:19.158885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:45.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.156851+0000","last_change":"2024-09-19T04:18:10.099932+0000","last_active":"2024-09-19T04:20:19.156851+0000","last_peered":"2024-09-19T04:20:19.156851+0000","last_clean":"2024-09-19T04:20:19.156851+0000","last_became_active":"2024-09-19T04:18:10.099637+0000","last_became_peered":"2024-09-19T04:18:10.099637+0000","last_unstale":"2024-09-19T04:20:19.156851+0000","last_undegraded":"2024-09-19T04:20:19.156851+0000","last_fullsized":"2024-09-19T04:20:19.156851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:50.594382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259565+0000","last_change":"2024-09-19T04:18:12.502229+0000","last_active":"2024-09-19T04:20:19.259565+0000","last_peered":"2024-09-19T04:20:19.259565+0000","last_clean":"2024-09-19T04:20:19.259565+0000","last_became_active":"2024-09-19T04:18:12.501921+0000","last_became_peered":"2024-09-19T04:18:12.501921+0000","last_unstale":"2024-09-19T04:20:19.259565+0000","last_undegraded":"2024-09-19T04:20:19.259565+0000","last_fullsized":"2024-09-19T04:20:19.259565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:55.412103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158774+0000","last_change":"2024-09-19T04:18:10.103961+0000","last_active":"2024-09-19T04:20:19.158774+0000","last_peered":"2024-09-19T04:20:19.158774+0000","last_clean":"2024-09-19T04:20:19.158774+0000","last_became_active":"2024-09-19T04:18:10.102737+0000","last_became_peered":"2024-09-19T04:18:10.102737+0000","last_unstale":"2024-09-19T04:20:19.158774+0000","last_undegraded":"2024-09-19T04:20:19.158774+0000","last_fullsized":"2024-09-19T04:20:19.158774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:31:01.806124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.255887+0000","last_change":"2024-09-19T04:18:12.502385+0000","last_active":"2024-09-19T04:20:19.255887+0000","last_peered":"2024-09-19T04:20:19.255887+0000","last_clean":"2024-09-19T04:20:19.255887+0000","last_became_active":"2024-09-19T04:18:12.502025+0000","last_became_peered":"2024-09-19T04:18:12.502025+0000","last_unstale":"2024-09-19T04:20:19.255887+0000","last_undegraded":"2024-09-19T04:20:19.255887+0000","last_fullsized":"2024-09-19T04:20:19.255887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:24.650911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'8","reported_seq":262,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.737999+0000","last_change":"2024-09-19T04:18:10.103526+0000","last_active":"2024-09-19T04:20:19.737999+0000","last_peered":"2024-09-19T04:20:19.737999+0000","last_clean":"2024-09-19T04:20:19.737999+0000","last_became_active":"2024-09-19T04:18:10.103191+0000","last_became_peered":"2024-09-19T04:18:10.103191+0000","last_unstale":"2024-09-19T04:20:19.737999+0000","last_undegraded":"2024-09-19T04:20:19.737999+0000","last_fullsized":"2024-09-19T04:20:19.737999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:07.695715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456912+0000","last_change":"2024-09-19T04:18:12.493169+0000","last_active":"2024-09-19T04:20:19.456912+0000","last_peered":"2024-09-19T04:20:19.456912+0000","last_clean":"2024-09-19T04:20:19.456912+0000","last_became_active":"2024-09-19T04:18:12.492669+0000","last_became_peered":"2024-09-19T04:18:12.492669+0000","last_unstale":"2024-09-19T04:20:19.456912+0000","last_undegraded":"2024-09-19T04:20:19.456912+0000","last_fullsized":"2024-09-19T04:20:19.456912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:04.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454874+0000","last_change":"2024-09-19T04:19:01.007275+0000","last_active":"2024-09-19T04:20:19.454874+0000","last_peered":"2024-09-19T04:20:19.454874+0000","last_clean":"2024-09-19T04:20:19.454874+0000","last_became_active":"2024-09-19T04:19:01.003361+0000","last_became_peered":"2024-09-19T04:19:01.003361+0000","last_unstale":"2024-09-19T04:20:19.454874+0000","last_undegraded":"2024-09-19T04:20:19.454874+0000","last_fullsized":"2024-09-19T04:20:19.454874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:07:11.907462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.236090+0000","last_change":"2024-09-19T04:18:12.504348+0000","last_active":"2024-09-19T04:20:19.236090+0000","last_peered":"2024-09-19T04:20:19.236090+0000","last_clean":"2024-09-19T04:20:19.236090+0000","last_became_active":"2024-09-19T04:18:12.503180+0000","last_became_peered":"2024-09-19T04:18:12.503180+0000","last_unstale":"2024-09-19T04:20:19.236090+0000","last_undegraded":"2024-09-19T04:20:19.236090+0000","last_fullsized":"2024-09-19T04:20:19.236090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:03.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159013+0000","last_change":"2024-09-19T04:18:10.365136+0000","last_active":"2024-09-19T04:20:19.159013+0000","last_peered":"2024-09-19T04:20:19.159013+0000","last_clean":"2024-09-19T04:20:19.159013+0000","last_became_active":"2024-09-19T04:18:10.364594+0000","last_became_peered":"2024-09-19T04:18:10.364594+0000","last_unstale":"2024-09-19T04:20:19.159013+0000","last_undegraded":"2024-09-19T04:20:19.159013+0000","last_fullsized":"2024-09-19T04:20:19.159013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:52.252157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456842+0000","last_change":"2024-09-19T04:18:12.502051+0000","last_active":"2024-09-19T04:20:19.456842+0000","last_peered":"2024-09-19T04:20:19.456842+0000","last_clean":"2024-09-19T04:20:19.456842+0000","last_became_active":"2024-09-19T04:18:12.501578+0000","last_became_peered":"2024-09-19T04:18:12.501578+0000","last_unstale":"2024-09-19T04:20:19.456842+0000","last_undegraded":"2024-09-19T04:20:19.456842+0000","last_fullsized":"2024-09-19T04:20:19.456842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:59.693175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'37","reported_seq":303,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738014+0000","last_change":"2024-09-19T04:18:10.101507+0000","last_active":"2024-09-19T04:20:19.738014+0000","last_peered":"2024-09-19T04:20:19.738014+0000","last_clean":"2024-09-19T04:20:19.738014+0000","last_became_active":"2024-09-19T04:18:10.101152+0000","last_became_peered":"2024-09-19T04:18:10.101152+0000","last_unstale":"2024-09-19T04:20:19.738014+0000","last_undegraded":"2024-09-19T04:20:19.738014+0000","last_fullsized":"2024-09-19T04:20:19.738014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:01.397605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66483,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":186,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174032+0000","last_change":"2024-09-19T04:18:12.491611+0000","last_active":"2024-09-19T04:20:19.174032+0000","last_peered":"2024-09-19T04:20:19.174032+0000","last_clean":"2024-09-19T04:20:19.174032+0000","last_became_active":"2024-09-19T04:18:12.491279+0000","last_became_peered":"2024-09-19T04:18:12.491279+0000","last_unstale":"2024-09-19T04:20:19.174032+0000","last_undegraded":"2024-09-19T04:20:19.174032+0000","last_fullsized":"2024-09-19T04:20:19.174032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:01.414713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455643+0000","last_change":"2024-09-19T04:18:10.105574+0000","last_active":"2024-09-19T04:20:19.455643+0000","last_peered":"2024-09-19T04:20:19.455643+0000","last_clean":"2024-09-19T04:20:19.455643+0000","last_became_active":"2024-09-19T04:18:10.105203+0000","last_became_peered":"2024-09-19T04:18:10.105203+0000","last_unstale":"2024-09-19T04:20:19.455643+0000","last_undegraded":"2024-09-19T04:20:19.455643+0000","last_fullsized":"2024-09-19T04:20:19.455643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:00.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.156878+0000","last_change":"2024-09-19T04:18:12.497242+0000","last_active":"2024-09-19T04:20:19.156878+0000","last_peered":"2024-09-19T04:20:19.156878+0000","last_clean":"2024-09-19T04:20:19.156878+0000","last_became_active":"2024-09-19T04:18:12.496732+0000","last_became_peered":"2024-09-19T04:18:12.496732+0000","last_unstale":"2024-09-19T04:20:19.156878+0000","last_undegraded":"2024-09-19T04:20:19.156878+0000","last_fullsized":"2024-09-19T04:20:19.156878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:56.520341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158716+0000","last_change":"2024-09-19T04:18:10.103905+0000","last_active":"2024-09-19T04:20:19.158716+0000","last_peered":"2024-09-19T04:20:19.158716+0000","last_clean":"2024-09-19T04:20:19.158716+0000","last_became_active":"2024-09-19T04:18:10.103517+0000","last_became_peered":"2024-09-19T04:18:10.103517+0000","last_unstale":"2024-09-19T04:20:19.158716+0000","last_undegraded":"2024-09-19T04:20:19.158716+0000","last_fullsized":"2024-09-19T04:20:19.158716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:48.292878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454822+0000","last_change":"2024-09-19T04:18:12.499380+0000","last_active":"2024-09-19T04:20:19.454822+0000","last_peered":"2024-09-19T04:20:19.454822+0000","last_clean":"2024-09-19T04:20:19.454822+0000","last_became_active":"2024-09-19T04:18:12.498995+0000","last_became_peered":"2024-09-19T04:18:12.498995+0000","last_unstale":"2024-09-19T04:20:19.454822+0000","last_undegraded":"2024-09-19T04:20:19.454822+0000","last_fullsized":"2024-09-19T04:20:19.454822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:20.653923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738043+0000","last_change":"2024-09-19T04:18:10.105619+0000","last_active":"2024-09-19T04:20:19.738043+0000","last_peered":"2024-09-19T04:20:19.738043+0000","last_clean":"2024-09-19T04:20:19.738043+0000","last_became_active":"2024-09-19T04:18:10.105082+0000","last_became_peered":"2024-09-19T04:18:10.105082+0000","last_unstale":"2024-09-19T04:20:19.738043+0000","last_undegraded":"2024-09-19T04:20:19.738043+0000","last_fullsized":"2024-09-19T04:20:19.738043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:56.843068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454480+0000","last_change":"2024-09-19T04:18:12.498473+0000","last_active":"2024-09-19T04:20:19.454480+0000","last_peered":"2024-09-19T04:20:19.454480+0000","last_clean":"2024-09-19T04:20:19.454480+0000","last_became_active":"2024-09-19T04:18:12.496904+0000","last_became_peered":"2024-09-19T04:18:12.496904+0000","last_unstale":"2024-09-19T04:20:19.454480+0000","last_undegraded":"2024-09-19T04:20:19.454480+0000","last_fullsized":"2024-09-19T04:20:19.454480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:03.155899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173775+0000","last_change":"2024-09-19T04:18:10.365182+0000","last_active":"2024-09-19T04:20:19.173775+0000","last_peered":"2024-09-19T04:20:19.173775+0000","last_clean":"2024-09-19T04:20:19.173775+0000","last_became_active":"2024-09-19T04:18:10.364717+0000","last_became_peered":"2024-09-19T04:18:10.364717+0000","last_unstale":"2024-09-19T04:20:19.173775+0000","last_undegraded":"2024-09-19T04:20:19.173775+0000","last_fullsized":"2024-09-19T04:20:19.173775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:32.063066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158550+0000","last_change":"2024-09-19T04:18:12.493585+0000","last_active":"2024-09-19T04:20:19.158550+0000","last_peered":"2024-09-19T04:20:19.158550+0000","last_clean":"2024-09-19T04:20:19.158550+0000","last_became_active":"2024-09-19T04:18:12.493255+0000","last_became_peered":"2024-09-19T04:18:12.493255+0000","last_unstale":"2024-09-19T04:20:19.158550+0000","last_undegraded":"2024-09-19T04:20:19.158550+0000","last_fullsized":"2024-09-19T04:20:19.158550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:33.139727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174154+0000","last_change":"2024-09-19T04:18:10.101537+0000","last_active":"2024-09-19T04:20:19.174154+0000","last_peered":"2024-09-19T04:20:19.174154+0000","last_clean":"2024-09-19T04:20:19.174154+0000","last_became_active":"2024-09-19T04:18:10.100951+0000","last_became_peered":"2024-09-19T04:18:10.100951+0000","last_unstale":"2024-09-19T04:20:19.174154+0000","last_undegraded":"2024-09-19T04:20:19.174154+0000","last_fullsized":"2024-09-19T04:20:19.174154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:08.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466903+0000","last_change":"2024-09-19T04:18:12.495530+0000","last_active":"2024-09-19T04:20:19.466903+0000","last_peered":"2024-09-19T04:20:19.466903+0000","last_clean":"2024-09-19T04:20:19.466903+0000","last_became_active":"2024-09-19T04:18:12.495176+0000","last_became_peered":"2024-09-19T04:18:12.495176+0000","last_unstale":"2024-09-19T04:20:19.466903+0000","last_undegraded":"2024-09-19T04:20:19.466903+0000","last_fullsized":"2024-09-19T04:20:19.466903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:24.308047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157053+0000","last_change":"2024-09-19T04:18:10.102539+0000","last_active":"2024-09-19T04:20:19.157053+0000","last_peered":"2024-09-19T04:20:19.157053+0000","last_clean":"2024-09-19T04:20:19.157053+0000","last_became_active":"2024-09-19T04:18:10.102109+0000","last_became_peered":"2024-09-19T04:18:10.102109+0000","last_unstale":"2024-09-19T04:20:19.157053+0000","last_undegraded":"2024-09-19T04:20:19.157053+0000","last_fullsized":"2024-09-19T04:20:19.157053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:58.852767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455095+0000","last_change":"2024-09-19T04:18:12.491769+0000","last_active":"2024-09-19T04:20:19.455095+0000","last_peered":"2024-09-19T04:20:19.455095+0000","last_clean":"2024-09-19T04:20:19.455095+0000","last_became_active":"2024-09-19T04:18:12.491435+0000","last_became_peered":"2024-09-19T04:18:12.491435+0000","last_unstale":"2024-09-19T04:20:19.455095+0000","last_undegraded":"2024-09-19T04:20:19.455095+0000","last_fullsized":"2024-09-19T04:20:19.455095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:54.880533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'41","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.411367+0000","last_change":"2024-09-19T04:18:10.103559+0000","last_active":"2024-09-19T04:20:19.411367+0000","last_peered":"2024-09-19T04:20:19.411367+0000","last_clean":"2024-09-19T04:20:19.411367+0000","last_became_active":"2024-09-19T04:18:10.103208+0000","last_became_peered":"2024-09-19T04:18:10.103208+0000","last_unstale":"2024-09-19T04:20:19.411367+0000","last_undegraded":"2024-09-19T04:20:19.411367+0000","last_fullsized":"2024-09-19T04:20:19.411367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:16.564916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454709+0000","last_change":"2024-09-19T04:18:12.495616+0000","last_active":"2024-09-19T04:20:19.454709+0000","last_peered":"2024-09-19T04:20:19.454709+0000","last_clean":"2024-09-19T04:20:19.454709+0000","last_became_active":"2024-09-19T04:18:12.495243+0000","last_became_peered":"2024-09-19T04:18:12.495243+0000","last_unstale":"2024-09-19T04:20:19.454709+0000","last_undegraded":"2024-09-19T04:20:19.454709+0000","last_fullsized":"2024-09-19T04:20:19.454709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:12.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'8","reported_seq":152,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738564+0000","last_change":"2024-09-19T04:18:10.626874+0000","last_active":"2024-09-19T04:20:19.738564+0000","last_peered":"2024-09-19T04:20:19.738564+0000","last_clean":"2024-09-19T04:20:19.738564+0000","last_became_active":"2024-09-19T04:18:10.626506+0000","last_became_peered":"2024-09-19T04:18:10.626506+0000","last_unstale":"2024-09-19T04:20:19.738564+0000","last_undegraded":"2024-09-19T04:20:19.738564+0000","last_fullsized":"2024-09-19T04:20:19.738564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:48.725143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454917+0000","last_change":"2024-09-19T04:18:12.496350+0000","last_active":"2024-09-19T04:20:19.454917+0000","last_peered":"2024-09-19T04:20:19.454917+0000","last_clean":"2024-09-19T04:20:19.454917+0000","last_became_active":"2024-09-19T04:18:12.495904+0000","last_became_peered":"2024-09-19T04:18:12.495904+0000","last_unstale":"2024-09-19T04:20:19.454917+0000","last_undegraded":"2024-09-19T04:20:19.454917+0000","last_fullsized":"2024-09-19T04:20:19.454917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:08.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454676+0000","last_change":"2024-09-19T04:16:56.164450+0000","last_active":"2024-09-19T04:20:19.454676+0000","last_peered":"2024-09-19T04:20:19.454676+0000","last_clean":"2024-09-19T04:20:19.454676+0000","last_became_active":"2024-09-19T04:16:56.164008+0000","last_became_peered":"2024-09-19T04:16:56.164008+0000","last_unstale":"2024-09-19T04:20:19.454676+0000","last_undegraded":"2024-09-19T04:20:19.454676+0000","last_fullsized":"2024-09-19T04:20:19.454676+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_clean_scrub_stamp":"2024-09-19T04:13:18.308718+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:14.697725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.459005+0000","last_change":"2024-09-19T04:18:10.101598+0000","last_active":"2024-09-19T04:20:19.459005+0000","last_peered":"2024-09-19T04:20:19.459005+0000","last_clean":"2024-09-19T04:20:19.459005+0000","last_became_active":"2024-09-19T04:18:10.101210+0000","last_became_peered":"2024-09-19T04:18:10.101210+0000","last_unstale":"2024-09-19T04:20:19.459005+0000","last_undegraded":"2024-09-19T04:20:19.459005+0000","last_fullsized":"2024-09-19T04:20:19.459005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:10.653481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260008+0000","last_change":"2024-09-19T04:18:12.495034+0000","last_active":"2024-09-19T04:20:19.260008+0000","last_peered":"2024-09-19T04:20:19.260008+0000","last_clean":"2024-09-19T04:20:19.260008+0000","last_became_active":"2024-09-19T04:18:12.494611+0000","last_became_peered":"2024-09-19T04:18:12.494611+0000","last_unstale":"2024-09-19T04:20:19.260008+0000","last_undegraded":"2024-09-19T04:20:19.260008+0000","last_fullsized":"2024-09-19T04:20:19.260008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:28.448927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454656+0000","last_change":"2024-09-19T04:18:10.105212+0000","last_active":"2024-09-19T04:20:19.454656+0000","last_peered":"2024-09-19T04:20:19.454656+0000","last_clean":"2024-09-19T04:20:19.454656+0000","last_became_active":"2024-09-19T04:18:10.104710+0000","last_became_peered":"2024-09-19T04:18:10.104710+0000","last_unstale":"2024-09-19T04:20:19.454656+0000","last_undegraded":"2024-09-19T04:20:19.454656+0000","last_fullsized":"2024-09-19T04:20:19.454656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:58.007773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.156921+0000","last_change":"2024-09-19T04:19:00.980419+0000","last_active":"2024-09-19T04:20:19.156921+0000","last_peered":"2024-09-19T04:20:19.156921+0000","last_clean":"2024-09-19T04:20:19.156921+0000","last_became_active":"2024-09-19T04:19:00.980008+0000","last_became_peered":"2024-09-19T04:19:00.980008+0000","last_unstale":"2024-09-19T04:20:19.156921+0000","last_undegraded":"2024-09-19T04:20:19.156921+0000","last_fullsized":"2024-09-19T04:20:19.156921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:18.396611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":147,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.737581+0000","last_change":"2024-09-19T04:18:10.364530+0000","last_active":"2024-09-19T04:20:19.737581+0000","last_peered":"2024-09-19T04:20:19.737581+0000","last_clean":"2024-09-19T04:20:19.737581+0000","last_became_active":"2024-09-19T04:18:10.364204+0000","last_became_peered":"2024-09-19T04:18:10.364204+0000","last_unstale":"2024-09-19T04:20:19.737581+0000","last_undegraded":"2024-09-19T04:20:19.737581+0000","last_fullsized":"2024-09-19T04:20:19.737581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:04.229530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158567+0000","last_change":"2024-09-19T04:18:12.493730+0000","last_active":"2024-09-19T04:20:19.158567+0000","last_peered":"2024-09-19T04:20:19.158567+0000","last_clean":"2024-09-19T04:20:19.158567+0000","last_became_active":"2024-09-19T04:18:12.493153+0000","last_became_peered":"2024-09-19T04:18:12.493153+0000","last_unstale":"2024-09-19T04:20:19.158567+0000","last_undegraded":"2024-09-19T04:20:19.158567+0000","last_fullsized":"2024-09-19T04:20:19.158567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.758490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'7","reported_seq":150,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.739821+0000","last_change":"2024-09-19T04:18:10.102073+0000","last_active":"2024-09-19T04:20:19.739821+0000","last_peered":"2024-09-19T04:20:19.739821+0000","last_clean":"2024-09-19T04:20:19.739821+0000","last_became_active":"2024-09-19T04:18:10.101755+0000","last_became_peered":"2024-09-19T04:18:10.101755+0000","last_unstale":"2024-09-19T04:20:19.739821+0000","last_undegraded":"2024-09-19T04:20:19.739821+0000","last_fullsized":"2024-09-19T04:20:19.739821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:21.489146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1292,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159049+0000","last_change":"2024-09-19T04:18:12.494560+0000","last_active":"2024-09-19T04:20:19.159049+0000","last_peered":"2024-09-19T04:20:19.159049+0000","last_clean":"2024-09-19T04:20:19.159049+0000","last_became_active":"2024-09-19T04:18:12.494203+0000","last_became_peered":"2024-09-19T04:18:12.494203+0000","last_unstale":"2024-09-19T04:20:19.159049+0000","last_undegraded":"2024-09-19T04:20:19.159049+0000","last_fullsized":"2024-09-19T04:20:19.159049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:35:17.129649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.256225+0000","last_change":"2024-09-19T04:18:10.365781+0000","last_active":"2024-09-19T04:20:19.256225+0000","last_peered":"2024-09-19T04:20:19.256225+0000","last_clean":"2024-09-19T04:20:19.256225+0000","last_became_active":"2024-09-19T04:18:10.365417+0000","last_became_peered":"2024-09-19T04:18:10.365417+0000","last_unstale":"2024-09-19T04:20:19.256225+0000","last_undegraded":"2024-09-19T04:20:19.256225+0000","last_fullsized":"2024-09-19T04:20:19.256225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:20.097572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159240+0000","last_change":"2024-09-19T04:18:12.495570+0000","last_active":"2024-09-19T04:20:19.159240+0000","last_peered":"2024-09-19T04:20:19.159240+0000","last_clean":"2024-09-19T04:20:19.159240+0000","last_became_active":"2024-09-19T04:18:12.495242+0000","last_became_peered":"2024-09-19T04:18:12.495242+0000","last_unstale":"2024-09-19T04:20:19.159240+0000","last_undegraded":"2024-09-19T04:20:19.159240+0000","last_fullsized":"2024-09-19T04:20:19.159240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:57.448182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174114+0000","last_change":"2024-09-19T04:18:10.365134+0000","last_active":"2024-09-19T04:20:19.174114+0000","last_peered":"2024-09-19T04:20:19.174114+0000","last_clean":"2024-09-19T04:20:19.174114+0000","last_became_active":"2024-09-19T04:18:10.364726+0000","last_became_peered":"2024-09-19T04:18:10.364726+0000","last_unstale":"2024-09-19T04:20:19.174114+0000","last_undegraded":"2024-09-19T04:20:19.174114+0000","last_fullsized":"2024-09-19T04:20:19.174114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:44.207855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157194+0000","last_change":"2024-09-19T04:18:12.488836+0000","last_active":"2024-09-19T04:20:19.157194+0000","last_peered":"2024-09-19T04:20:19.157194+0000","last_clean":"2024-09-19T04:20:19.157194+0000","last_became_active":"2024-09-19T04:18:12.488507+0000","last_became_peered":"2024-09-19T04:18:12.488507+0000","last_unstale":"2024-09-19T04:20:19.157194+0000","last_undegraded":"2024-09-19T04:20:19.157194+0000","last_fullsized":"2024-09-19T04:20:19.157194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:31.308114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'52","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.417279+0000","last_change":"2024-09-19T04:18:10.103623+0000","last_active":"2024-09-19T04:20:19.417279+0000","last_peered":"2024-09-19T04:20:19.417279+0000","last_clean":"2024-09-19T04:20:19.417279+0000","last_became_active":"2024-09-19T04:18:10.103340+0000","last_became_peered":"2024-09-19T04:18:10.103340+0000","last_unstale":"2024-09-19T04:20:19.417279+0000","last_undegraded":"2024-09-19T04:20:19.417279+0000","last_fullsized":"2024-09-19T04:20:19.417279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:11.724013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260722+0000","last_change":"2024-09-19T04:18:12.498167+0000","last_active":"2024-09-19T04:20:19.260722+0000","last_peered":"2024-09-19T04:20:19.260722+0000","last_clean":"2024-09-19T04:20:19.260722+0000","last_became_active":"2024-09-19T04:18:12.496924+0000","last_became_peered":"2024-09-19T04:18:12.496924+0000","last_unstale":"2024-09-19T04:20:19.260722+0000","last_undegraded":"2024-09-19T04:20:19.260722+0000","last_fullsized":"2024-09-19T04:20:19.260722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:13.952030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.459180+0000","last_change":"2024-09-19T04:18:10.627106+0000","last_active":"2024-09-19T04:20:19.459180+0000","last_peered":"2024-09-19T04:20:19.459180+0000","last_clean":"2024-09-19T04:20:19.459180+0000","last_became_active":"2024-09-19T04:18:10.626708+0000","last_became_peered":"2024-09-19T04:18:10.626708+0000","last_unstale":"2024-09-19T04:20:19.459180+0000","last_undegraded":"2024-09-19T04:20:19.459180+0000","last_fullsized":"2024-09-19T04:20:19.459180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:46.049699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235881+0000","last_change":"2024-09-19T04:18:12.506995+0000","last_active":"2024-09-19T04:20:19.235881+0000","last_peered":"2024-09-19T04:20:19.235881+0000","last_clean":"2024-09-19T04:20:19.235881+0000","last_became_active":"2024-09-19T04:18:12.500607+0000","last_became_peered":"2024-09-19T04:18:12.500607+0000","last_unstale":"2024-09-19T04:20:19.235881+0000","last_undegraded":"2024-09-19T04:20:19.235881+0000","last_fullsized":"2024-09-19T04:20:19.235881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:37.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456633+0000","last_change":"2024-09-19T04:18:10.365150+0000","last_active":"2024-09-19T04:20:19.456633+0000","last_peered":"2024-09-19T04:20:19.456633+0000","last_clean":"2024-09-19T04:20:19.456633+0000","last_became_active":"2024-09-19T04:18:10.364755+0000","last_became_peered":"2024-09-19T04:18:10.364755+0000","last_unstale":"2024-09-19T04:20:19.456633+0000","last_undegraded":"2024-09-19T04:20:19.456633+0000","last_fullsized":"2024-09-19T04:20:19.456633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:11.639532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173701+0000","last_change":"2024-09-19T04:18:12.484588+0000","last_active":"2024-09-19T04:20:19.173701+0000","last_peered":"2024-09-19T04:20:19.173701+0000","last_clean":"2024-09-19T04:20:19.173701+0000","last_became_active":"2024-09-19T04:18:12.484218+0000","last_became_peered":"2024-09-19T04:18:12.484218+0000","last_unstale":"2024-09-19T04:20:19.173701+0000","last_undegraded":"2024-09-19T04:20:19.173701+0000","last_fullsized":"2024-09-19T04:20:19.173701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:55.634557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259785+0000","last_change":"2024-09-19T04:18:10.100832+0000","last_active":"2024-09-19T04:20:19.259785+0000","last_peered":"2024-09-19T04:20:19.259785+0000","last_clean":"2024-09-19T04:20:19.259785+0000","last_became_active":"2024-09-19T04:18:10.100518+0000","last_became_peered":"2024-09-19T04:18:10.100518+0000","last_unstale":"2024-09-19T04:20:19.259785+0000","last_undegraded":"2024-09-19T04:20:19.259785+0000","last_fullsized":"2024-09-19T04:20:19.259785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:37.028120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466793+0000","last_change":"2024-09-19T04:18:12.496914+0000","last_active":"2024-09-19T04:20:19.466793+0000","last_peered":"2024-09-19T04:20:19.466793+0000","last_clean":"2024-09-19T04:20:19.466793+0000","last_became_active":"2024-09-19T04:18:12.496556+0000","last_became_peered":"2024-09-19T04:18:12.496556+0000","last_unstale":"2024-09-19T04:20:19.466793+0000","last_undegraded":"2024-09-19T04:20:19.466793+0000","last_fullsized":"2024-09-19T04:20:19.466793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:52.391412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466731+0000","last_change":"2024-09-19T04:19:00.993461+0000","last_active":"2024-09-19T04:20:19.466731+0000","last_peered":"2024-09-19T04:20:19.466731+0000","last_clean":"2024-09-19T04:20:19.466731+0000","last_became_active":"2024-09-19T04:19:00.980704+0000","last_became_peered":"2024-09-19T04:19:00.980704+0000","last_unstale":"2024-09-19T04:20:19.466731+0000","last_undegraded":"2024-09-19T04:20:19.466731+0000","last_fullsized":"2024-09-19T04:20:19.466731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:56.090092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260090+0000","last_change":"2024-09-19T04:18:12.496852+0000","last_active":"2024-09-19T04:20:19.260090+0000","last_peered":"2024-09-19T04:20:19.260090+0000","last_clean":"2024-09-19T04:20:19.260090+0000","last_became_active":"2024-09-19T04:18:12.495365+0000","last_became_peered":"2024-09-19T04:18:12.495365+0000","last_unstale":"2024-09-19T04:20:19.260090+0000","last_undegraded":"2024-09-19T04:20:19.260090+0000","last_fullsized":"2024-09-19T04:20:19.260090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:14:57.012618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173834+0000","last_change":"2024-09-19T04:18:10.104641+0000","last_active":"2024-09-19T04:20:19.173834+0000","last_peered":"2024-09-19T04:20:19.173834+0000","last_clean":"2024-09-19T04:20:19.173834+0000","last_became_active":"2024-09-19T04:18:10.104295+0000","last_became_peered":"2024-09-19T04:18:10.104295+0000","last_unstale":"2024-09-19T04:20:19.173834+0000","last_undegraded":"2024-09-19T04:20:19.173834+0000","last_fullsized":"2024-09-19T04:20:19.173834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:58.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454750+0000","last_change":"2024-09-19T04:18:12.488396+0000","last_active":"2024-09-19T04:20:19.454750+0000","last_peered":"2024-09-19T04:20:19.454750+0000","last_clean":"2024-09-19T04:20:19.454750+0000","last_became_active":"2024-09-19T04:18:12.488048+0000","last_became_peered":"2024-09-19T04:18:12.488048+0000","last_unstale":"2024-09-19T04:20:19.454750+0000","last_undegraded":"2024-09-19T04:20:19.454750+0000","last_fullsized":"2024-09-19T04:20:19.454750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:27.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158938+0000","last_change":"2024-09-19T04:18:12.493722+0000","last_active":"2024-09-19T04:20:19.158938+0000","last_peered":"2024-09-19T04:20:19.158938+0000","last_clean":"2024-09-19T04:20:19.158938+0000","last_became_active":"2024-09-19T04:18:12.492483+0000","last_became_peered":"2024-09-19T04:18:12.492483+0000","last_unstale":"2024-09-19T04:20:19.158938+0000","last_undegraded":"2024-09-19T04:20:19.158938+0000","last_fullsized":"2024-09-19T04:20:19.158938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:33.492623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174205+0000","last_change":"2024-09-19T04:18:10.101326+0000","last_active":"2024-09-19T04:20:19.174205+0000","last_peered":"2024-09-19T04:20:19.174205+0000","last_clean":"2024-09-19T04:20:19.174205+0000","last_became_active":"2024-09-19T04:18:10.100955+0000","last_became_peered":"2024-09-19T04:18:10.100955+0000","last_unstale":"2024-09-19T04:20:19.174205+0000","last_undegraded":"2024-09-19T04:20:19.174205+0000","last_fullsized":"2024-09-19T04:20:19.174205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:41.939741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456660+0000","last_change":"2024-09-19T04:18:12.492750+0000","last_active":"2024-09-19T04:20:19.456660+0000","last_peered":"2024-09-19T04:20:19.456660+0000","last_clean":"2024-09-19T04:20:19.456660+0000","last_became_active":"2024-09-19T04:18:12.492417+0000","last_became_peered":"2024-09-19T04:18:12.492417+0000","last_unstale":"2024-09-19T04:20:19.456660+0000","last_undegraded":"2024-09-19T04:20:19.456660+0000","last_fullsized":"2024-09-19T04:20:19.456660+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:25.279452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466779+0000","last_change":"2024-09-19T04:19:00.989741+0000","last_active":"2024-09-19T04:20:19.466779+0000","last_peered":"2024-09-19T04:20:19.466779+0000","last_clean":"2024-09-19T04:20:19.466779+0000","last_became_active":"2024-09-19T04:19:00.977530+0000","last_became_peered":"2024-09-19T04:19:00.977530+0000","last_unstale":"2024-09-19T04:20:19.466779+0000","last_undegraded":"2024-09-19T04:20:19.466779+0000","last_fullsized":"2024-09-19T04:20:19.466779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:35.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.236131+0000","last_change":"2024-09-19T04:18:12.503510+0000","last_active":"2024-09-19T04:20:19.236131+0000","last_peered":"2024-09-19T04:20:19.236131+0000","last_clean":"2024-09-19T04:20:19.236131+0000","last_became_active":"2024-09-19T04:18:12.495165+0000","last_became_peered":"2024-09-19T04:18:12.495165+0000","last_unstale":"2024-09-19T04:20:19.236131+0000","last_undegraded":"2024-09-19T04:20:19.236131+0000","last_fullsized":"2024-09-19T04:20:19.236131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:13.934328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260678+0000","last_change":"2024-09-19T04:18:10.364783+0000","last_active":"2024-09-19T04:20:19.260678+0000","last_peered":"2024-09-19T04:20:19.260678+0000","last_clean":"2024-09-19T04:20:19.260678+0000","last_became_active":"2024-09-19T04:18:10.364202+0000","last_became_peered":"2024-09-19T04:18:10.364202+0000","last_unstale":"2024-09-19T04:20:19.260678+0000","last_undegraded":"2024-09-19T04:20:19.260678+0000","last_fullsized":"2024-09-19T04:20:19.260678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:18:55.770637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260163+0000","last_change":"2024-09-19T04:18:12.495825+0000","last_active":"2024-09-19T04:20:19.260163+0000","last_peered":"2024-09-19T04:20:19.260163+0000","last_clean":"2024-09-19T04:20:19.260163+0000","last_became_active":"2024-09-19T04:18:12.494525+0000","last_became_peered":"2024-09-19T04:18:12.494525+0000","last_unstale":"2024-09-19T04:20:19.260163+0000","last_undegraded":"2024-09-19T04:20:19.260163+0000","last_fullsized":"2024-09-19T04:20:19.260163+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:57.827135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":155,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738811+0000","last_change":"2024-09-19T04:18:10.102145+0000","last_active":"2024-09-19T04:20:19.738811+0000","last_peered":"2024-09-19T04:20:19.738811+0000","last_clean":"2024-09-19T04:20:19.738811+0000","last_became_active":"2024-09-19T04:18:10.101410+0000","last_became_peered":"2024-09-19T04:18:10.101410+0000","last_unstale":"2024-09-19T04:20:19.738811+0000","last_undegraded":"2024-09-19T04:20:19.738811+0000","last_fullsized":"2024-09-19T04:20:19.738811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:20.354296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601458,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1291,"num_read_kb":762,"num_write":381,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1804374,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107517,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":450642,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1089536,"data_stored":1167124,"data_compressed":41814,"data_compressed_allocated":53248,"data_compressed_original":659456,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972402,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":970752,"data_stored":842861,"data_compressed":22034,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201335,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":843239,"data_compressed":21997,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":1044960,"data_compressed":38727,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":449673,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822727,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":1174748,"data_compressed":44883,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":450009,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215184,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":974003,"data_compressed":28190,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":519225,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":517462,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":451628,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135740,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200605,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:20:24.282 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T04:20:24.282 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:20:24.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.449 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi107 to 176.4M 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Unable to set osd_memory_target on smithi107 to 184978124: error parsing value: Value '184978124' is below minimum 939524096 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:20:24.450 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:20:24.451 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:20:24.451 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:20:24.451 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:20:24.451 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.451 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.451 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.451 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:24 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi107 to 176.4M 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Unable to set osd_memory_target on smithi107 to 184978124: error parsing value: Value '184978124' is below minimum 939524096 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:20:24.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:20:24.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:20:24.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:20:24.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:20:24.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:20:24.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:20:24.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:20:24.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:24 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.477+0000 7f2bd7200640 1 -- 172.21.15.107:0/1741950771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8101f40 msgr2=0x7f2bd8109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.477+0000 7f2bd7200640 1 --2- 172.21.15.107:0/1741950771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8101f40 0x7f2bd8109db0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009ca0 tx=0x7f2bd002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.477+0000 7f2bd7200640 1 -- 172.21.15.107:0/1741950771 shutdown_connections 2024-09-19T04:20:24.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.477+0000 7f2bd7200640 1 --2- 172.21.15.107:0/1741950771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8101f40 0x7f2bd8109db0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.477+0000 7f2bd7200640 1 --2- 172.21.15.107:0/1741950771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd8101580 0x7f2bd8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.477+0000 7f2bd7200640 1 --2- 172.21.15.107:0/1741950771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd8103050 0x7f2bd8103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.477+0000 7f2bd7200640 1 -- 172.21.15.107:0/1741950771 >> 172.21.15.107:0/1741950771 conn(0x7f2bd80fd110 msgr2=0x7f2bd80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.477+0000 7f2bd7200640 1 -- 172.21.15.107:0/1741950771 shutdown_connections 2024-09-19T04:20:24.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.477+0000 7f2bd7200640 1 -- 172.21.15.107:0/1741950771 wait complete. 2024-09-19T04:20:24.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.478+0000 7f2bd7200640 1 Processor -- start 2024-09-19T04:20:24.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.479+0000 7f2bd7200640 1 -- start start 2024-09-19T04:20:24.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.479+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd8101580 0x7f2bd8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.479+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd8101f40 0x7f2bd80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.479+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8103050 0x7f2bd8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.479+0000 7f2bd7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd810d6f0 con 0x7f2bd8101580 2024-09-19T04:20:24.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.479+0000 7f2bd7200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd810d570 con 0x7f2bd8101f40 2024-09-19T04:20:24.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.479+0000 7f2bd7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd810d870 con 0x7f2bd8103050 2024-09-19T04:20:24.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi107 to 176.4M 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Unable to set osd_memory_target on smithi107 to 184978124: error parsing value: Value '184978124' is below minimum 939524096 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-19T04:20:24.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Updating smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: Updating smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:24 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:24.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.479+0000 7f2bd5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd8101f40 0x7f2bd80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.479+0000 7f2bd5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd8101f40 0x7f2bd80745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:49538/0 (socket says 172.21.15.107:49538) 2024-09-19T04:20:24.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.479+0000 7f2bd5400640 1 -- 172.21.15.107:0/2652704324 learned_addr learned my addr 172.21.15.107:0/2652704324 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:24.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.480+0000 7f2bd5e00640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd8101580 0x7f2bd8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.480+0000 7f2bd6800640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8103050 0x7f2bd8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.480+0000 7f2bd5400640 1 -- 172.21.15.107:0/2652704324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8103050 msgr2=0x7f2bd8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.480+0000 7f2bd5400640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8103050 0x7f2bd8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.480+0000 7f2bd5400640 1 -- 172.21.15.107:0/2652704324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd8101580 msgr2=0x7f2bd8075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.480+0000 7f2bd5400640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd8101580 0x7f2bd8075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.480+0000 7f2bd5400640 1 -- 172.21.15.107:0/2652704324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd8075060 con 0x7f2bd8101f40 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.480+0000 7f2bd5400640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd8101f40 0x7f2bd80745e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400e9f0 tx=0x7f2bc400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.481+0000 7f2bcea00640 1 -- 172.21.15.107:0/2652704324 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400cd80 con 0x7f2bd8101f40 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.481+0000 7f2bcea00640 1 -- 172.21.15.107:0/2652704324 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc4010910 con 0x7f2bd8101f40 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.481+0000 7f2bcea00640 1 -- 172.21.15.107:0/2652704324 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4018990 con 0x7f2bd8101f40 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.481+0000 7f2bd7200640 1 -- 172.21.15.107:0/2652704324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd8075350 con 0x7f2bd8101f40 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.481+0000 7f2bd7200640 1 -- 172.21.15.107:0/2652704324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81a6820 con 0x7f2bd8101f40 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.482+0000 7f2bd7200640 1 -- 172.21.15.107:0/2652704324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd8101a00 con 0x7f2bd8101f40 2024-09-19T04:20:24.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.484+0000 7f2bcea00640 1 -- 172.21.15.107:0/2652704324 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7f2bc4010ac0 con 0x7f2bd8101f40 2024-09-19T04:20:24.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.485+0000 7f2bcea00640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2bbc0a34d0 0x7f2bbc0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.485+0000 7f2bcea00640 1 -- 172.21.15.107:0/2652704324 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f2bc40d0370 con 0x7f2bd8101f40 2024-09-19T04:20:24.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.485+0000 7f2bd5e00640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2bbc0a34d0 0x7f2bbc0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.486+0000 7f2bd5e00640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2bbc0a34d0 0x7f2bbc0a5990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2bd81029c0 tx=0x7f2bc8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.487+0000 7f2bcea00640 1 -- 172.21.15.107:0/2652704324 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2bc409c640 con 0x7f2bd8101f40 2024-09-19T04:20:24.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.670+0000 7f2bd7200640 1 -- 172.21.15.107:0/2652704324 --> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bd8062630 con 0x7f2bbc0a34d0 2024-09-19T04:20:24.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.677+0000 7f2bcea00640 1 -- 172.21.15.107:0/2652704324 <== mgr.15114 v2:172.21.15.107:6800/1724794486 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258046 (secure 0 0 0) 0x7f2bd8062630 con 0x7f2bbc0a34d0 2024-09-19T04:20:24.682 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:20:24.683 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-19T04:20:24.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 -- 172.21.15.107:0/2652704324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2bbc0a34d0 msgr2=0x7f2bbc0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2bbc0a34d0 0x7f2bbc0a5990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2bd81029c0 tx=0x7f2bc8005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:24.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 -- 172.21.15.107:0/2652704324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd8101f40 msgr2=0x7f2bd80745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd8101f40 0x7f2bd80745e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400e9f0 tx=0x7f2bc400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 -- 172.21.15.107:0/2652704324 shutdown_connections 2024-09-19T04:20:24.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2bbc0a34d0 0x7f2bbc0a5990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8103050 0x7f2bd8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd8101f40 0x7f2bd80745e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 --2- 172.21.15.107:0/2652704324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd8101580 0x7f2bd8075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 -- 172.21.15.107:0/2652704324 >> 172.21.15.107:0/2652704324 conn(0x7f2bd80fd110 msgr2=0x7f2bd8105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 -- 172.21.15.107:0/2652704324 shutdown_connections 2024-09-19T04:20:24.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:24.681+0000 7f2bd7200640 1 -- 172.21.15.107:0/2652704324 wait complete. 2024-09-19T04:20:24.699 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T04:20:23.171159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192060,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1427,"num_read_kb":881,"num_write":564,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":946,"ondisk_log_size":946,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377448,"kb_used_data":11516,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564376,"statfs":{"total":1151940427776,"available":1151553921024,"internally_reserved":0,"allocated":11792384,"data_stored":8885574,"data_compressed":214973,"data_compressed_allocated":278528,"data_compressed_original":3354624,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.980758"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.467136+0000","last_change":"2024-09-19T04:18:12.499720+0000","last_active":"2024-09-19T04:20:19.467136+0000","last_peered":"2024-09-19T04:20:19.467136+0000","last_clean":"2024-09-19T04:20:19.467136+0000","last_became_active":"2024-09-19T04:18:12.499381+0000","last_became_peered":"2024-09-19T04:18:12.499381+0000","last_unstale":"2024-09-19T04:20:19.467136+0000","last_undegraded":"2024-09-19T04:20:19.467136+0000","last_fullsized":"2024-09-19T04:20:19.467136+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:46.708870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235809+0000","last_change":"2024-09-19T04:18:12.498163+0000","last_active":"2024-09-19T04:20:19.235809+0000","last_peered":"2024-09-19T04:20:19.235809+0000","last_clean":"2024-09-19T04:20:19.235809+0000","last_became_active":"2024-09-19T04:18:12.495706+0000","last_became_peered":"2024-09-19T04:18:12.495706+0000","last_unstale":"2024-09-19T04:20:19.235809+0000","last_undegraded":"2024-09-19T04:20:19.235809+0000","last_fullsized":"2024-09-19T04:20:19.235809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:50.604410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235810+0000","last_change":"2024-09-19T04:18:12.493903+0000","last_active":"2024-09-19T04:20:19.235810+0000","last_peered":"2024-09-19T04:20:19.235810+0000","last_clean":"2024-09-19T04:20:19.235810+0000","last_became_active":"2024-09-19T04:18:12.493443+0000","last_became_peered":"2024-09-19T04:18:12.493443+0000","last_unstale":"2024-09-19T04:20:19.235810+0000","last_undegraded":"2024-09-19T04:20:19.235810+0000","last_fullsized":"2024-09-19T04:20:19.235810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:51.898337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.458868+0000","last_change":"2024-09-19T04:18:12.500460+0000","last_active":"2024-09-19T04:20:19.458868+0000","last_peered":"2024-09-19T04:20:19.458868+0000","last_clean":"2024-09-19T04:20:19.458868+0000","last_became_active":"2024-09-19T04:18:12.494675+0000","last_became_peered":"2024-09-19T04:18:12.494675+0000","last_unstale":"2024-09-19T04:20:19.458868+0000","last_undegraded":"2024-09-19T04:20:19.458868+0000","last_fullsized":"2024-09-19T04:20:19.458868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:13:32.863156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173788+0000","last_change":"2024-09-19T04:18:12.494936+0000","last_active":"2024-09-19T04:20:19.173788+0000","last_peered":"2024-09-19T04:20:19.173788+0000","last_clean":"2024-09-19T04:20:19.173788+0000","last_became_active":"2024-09-19T04:18:12.494556+0000","last_became_peered":"2024-09-19T04:18:12.494556+0000","last_unstale":"2024-09-19T04:20:19.173788+0000","last_undegraded":"2024-09-19T04:20:19.173788+0000","last_fullsized":"2024-09-19T04:20:19.173788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:06.001529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158827+0000","last_change":"2024-09-19T04:18:12.495880+0000","last_active":"2024-09-19T04:20:19.158827+0000","last_peered":"2024-09-19T04:20:19.158827+0000","last_clean":"2024-09-19T04:20:19.158827+0000","last_became_active":"2024-09-19T04:18:12.495562+0000","last_became_peered":"2024-09-19T04:18:12.495562+0000","last_unstale":"2024-09-19T04:20:19.158827+0000","last_undegraded":"2024-09-19T04:20:19.158827+0000","last_fullsized":"2024-09-19T04:20:19.158827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:20.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454525+0000","last_change":"2024-09-19T04:18:12.497991+0000","last_active":"2024-09-19T04:20:19.454525+0000","last_peered":"2024-09-19T04:20:19.454525+0000","last_clean":"2024-09-19T04:20:19.454525+0000","last_became_active":"2024-09-19T04:18:12.497644+0000","last_became_peered":"2024-09-19T04:18:12.497644+0000","last_unstale":"2024-09-19T04:20:19.454525+0000","last_undegraded":"2024-09-19T04:20:19.454525+0000","last_fullsized":"2024-09-19T04:20:19.454525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.851930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259408+0000","last_change":"2024-09-19T04:18:12.495789+0000","last_active":"2024-09-19T04:20:19.259408+0000","last_peered":"2024-09-19T04:20:19.259408+0000","last_clean":"2024-09-19T04:20:19.259408+0000","last_became_active":"2024-09-19T04:18:12.494637+0000","last_became_peered":"2024-09-19T04:18:12.494637+0000","last_unstale":"2024-09-19T04:20:19.259408+0000","last_undegraded":"2024-09-19T04:20:19.259408+0000","last_fullsized":"2024-09-19T04:20:19.259408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:14.245180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.457103+0000","last_change":"2024-09-19T04:18:12.491433+0000","last_active":"2024-09-19T04:20:19.457103+0000","last_peered":"2024-09-19T04:20:19.457103+0000","last_clean":"2024-09-19T04:20:19.457103+0000","last_became_active":"2024-09-19T04:18:12.490444+0000","last_became_peered":"2024-09-19T04:18:12.490444+0000","last_unstale":"2024-09-19T04:20:19.457103+0000","last_undegraded":"2024-09-19T04:20:19.457103+0000","last_fullsized":"2024-09-19T04:20:19.457103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:43.068355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159407+0000","last_change":"2024-09-19T04:18:12.493188+0000","last_active":"2024-09-19T04:20:19.159407+0000","last_peered":"2024-09-19T04:20:19.159407+0000","last_clean":"2024-09-19T04:20:19.159407+0000","last_became_active":"2024-09-19T04:18:12.492788+0000","last_became_peered":"2024-09-19T04:18:12.492788+0000","last_unstale":"2024-09-19T04:20:19.159407+0000","last_undegraded":"2024-09-19T04:20:19.159407+0000","last_fullsized":"2024-09-19T04:20:19.159407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:17.912820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454786+0000","last_change":"2024-09-19T04:18:12.490616+0000","last_active":"2024-09-19T04:20:19.454786+0000","last_peered":"2024-09-19T04:20:19.454786+0000","last_clean":"2024-09-19T04:20:19.454786+0000","last_became_active":"2024-09-19T04:18:12.490155+0000","last_became_peered":"2024-09-19T04:18:12.490155+0000","last_unstale":"2024-09-19T04:20:19.454786+0000","last_undegraded":"2024-09-19T04:20:19.454786+0000","last_fullsized":"2024-09-19T04:20:19.454786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:47.082082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.458913+0000","last_change":"2024-09-19T04:18:12.494191+0000","last_active":"2024-09-19T04:20:19.458913+0000","last_peered":"2024-09-19T04:20:19.458913+0000","last_clean":"2024-09-19T04:20:19.458913+0000","last_became_active":"2024-09-19T04:18:12.493786+0000","last_became_peered":"2024-09-19T04:18:12.493786+0000","last_unstale":"2024-09-19T04:20:19.458913+0000","last_undegraded":"2024-09-19T04:20:19.458913+0000","last_fullsized":"2024-09-19T04:20:19.458913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:29.691556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157025+0000","last_change":"2024-09-19T04:18:12.482167+0000","last_active":"2024-09-19T04:20:19.157025+0000","last_peered":"2024-09-19T04:20:19.157025+0000","last_clean":"2024-09-19T04:20:19.157025+0000","last_became_active":"2024-09-19T04:18:12.481314+0000","last_became_peered":"2024-09-19T04:18:12.481314+0000","last_unstale":"2024-09-19T04:20:19.157025+0000","last_undegraded":"2024-09-19T04:20:19.157025+0000","last_fullsized":"2024-09-19T04:20:19.157025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:29:28.081641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235565+0000","last_change":"2024-09-19T04:18:12.491711+0000","last_active":"2024-09-19T04:20:19.235565+0000","last_peered":"2024-09-19T04:20:19.235565+0000","last_clean":"2024-09-19T04:20:19.235565+0000","last_became_active":"2024-09-19T04:18:12.490964+0000","last_became_peered":"2024-09-19T04:18:12.490964+0000","last_unstale":"2024-09-19T04:20:19.235565+0000","last_undegraded":"2024-09-19T04:20:19.235565+0000","last_fullsized":"2024-09-19T04:20:19.235565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:16.607679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235461+0000","last_change":"2024-09-19T04:18:12.504737+0000","last_active":"2024-09-19T04:20:19.235461+0000","last_peered":"2024-09-19T04:20:19.235461+0000","last_clean":"2024-09-19T04:20:19.235461+0000","last_became_active":"2024-09-19T04:18:12.497583+0000","last_became_peered":"2024-09-19T04:18:12.497583+0000","last_unstale":"2024-09-19T04:20:19.235461+0000","last_undegraded":"2024-09-19T04:20:19.235461+0000","last_fullsized":"2024-09-19T04:20:19.235461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:48.456738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456542+0000","last_change":"2024-09-19T04:18:12.499045+0000","last_active":"2024-09-19T04:20:19.456542+0000","last_peered":"2024-09-19T04:20:19.456542+0000","last_clean":"2024-09-19T04:20:19.456542+0000","last_became_active":"2024-09-19T04:18:12.498713+0000","last_became_peered":"2024-09-19T04:18:12.498713+0000","last_unstale":"2024-09-19T04:20:19.456542+0000","last_undegraded":"2024-09-19T04:20:19.456542+0000","last_fullsized":"2024-09-19T04:20:19.456542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:54.337769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.467108+0000","last_change":"2024-09-19T04:18:12.495989+0000","last_active":"2024-09-19T04:20:19.467108+0000","last_peered":"2024-09-19T04:20:19.467108+0000","last_clean":"2024-09-19T04:20:19.467108+0000","last_became_active":"2024-09-19T04:18:12.495633+0000","last_became_peered":"2024-09-19T04:18:12.495633+0000","last_unstale":"2024-09-19T04:20:19.467108+0000","last_undegraded":"2024-09-19T04:20:19.467108+0000","last_fullsized":"2024-09-19T04:20:19.467108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:17.225861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.255995+0000","last_change":"2024-09-19T04:18:12.488532+0000","last_active":"2024-09-19T04:20:19.255995+0000","last_peered":"2024-09-19T04:20:19.255995+0000","last_clean":"2024-09-19T04:20:19.255995+0000","last_became_active":"2024-09-19T04:18:12.488117+0000","last_became_peered":"2024-09-19T04:18:12.488117+0000","last_unstale":"2024-09-19T04:20:19.255995+0000","last_undegraded":"2024-09-19T04:20:19.255995+0000","last_fullsized":"2024-09-19T04:20:19.255995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:45.318852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455765+0000","last_change":"2024-09-19T04:18:12.496784+0000","last_active":"2024-09-19T04:20:19.455765+0000","last_peered":"2024-09-19T04:20:19.455765+0000","last_clean":"2024-09-19T04:20:19.455765+0000","last_became_active":"2024-09-19T04:18:12.496443+0000","last_became_peered":"2024-09-19T04:18:12.496443+0000","last_unstale":"2024-09-19T04:20:19.455765+0000","last_undegraded":"2024-09-19T04:20:19.455765+0000","last_fullsized":"2024-09-19T04:20:19.455765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:25:46.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466725+0000","last_change":"2024-09-19T04:19:00.980557+0000","last_active":"2024-09-19T04:20:19.466725+0000","last_peered":"2024-09-19T04:20:19.466725+0000","last_clean":"2024-09-19T04:20:19.466725+0000","last_became_active":"2024-09-19T04:19:00.980149+0000","last_became_peered":"2024-09-19T04:19:00.980149+0000","last_unstale":"2024-09-19T04:20:19.466725+0000","last_undegraded":"2024-09-19T04:20:19.466725+0000","last_fullsized":"2024-09-19T04:20:19.466725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:06.140085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157083+0000","last_change":"2024-09-19T04:18:12.493731+0000","last_active":"2024-09-19T04:20:19.157083+0000","last_peered":"2024-09-19T04:20:19.157083+0000","last_clean":"2024-09-19T04:20:19.157083+0000","last_became_active":"2024-09-19T04:18:12.493412+0000","last_became_peered":"2024-09-19T04:18:12.493412+0000","last_unstale":"2024-09-19T04:20:19.157083+0000","last_undegraded":"2024-09-19T04:20:19.157083+0000","last_fullsized":"2024-09-19T04:20:19.157083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:30:41.089637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456785+0000","last_change":"2024-09-19T04:18:12.497033+0000","last_active":"2024-09-19T04:20:19.456785+0000","last_peered":"2024-09-19T04:20:19.456785+0000","last_clean":"2024-09-19T04:20:19.456785+0000","last_became_active":"2024-09-19T04:18:12.496710+0000","last_became_peered":"2024-09-19T04:18:12.496710+0000","last_unstale":"2024-09-19T04:20:19.456785+0000","last_undegraded":"2024-09-19T04:20:19.456785+0000","last_fullsized":"2024-09-19T04:20:19.456785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:19.193869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259529+0000","last_change":"2024-09-19T04:18:12.499213+0000","last_active":"2024-09-19T04:20:19.259529+0000","last_peered":"2024-09-19T04:20:19.259529+0000","last_clean":"2024-09-19T04:20:19.259529+0000","last_became_active":"2024-09-19T04:18:12.498728+0000","last_became_peered":"2024-09-19T04:18:12.498728+0000","last_unstale":"2024-09-19T04:20:19.259529+0000","last_undegraded":"2024-09-19T04:20:19.259529+0000","last_fullsized":"2024-09-19T04:20:19.259529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:37.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173675+0000","last_change":"2024-09-19T04:19:00.982647+0000","last_active":"2024-09-19T04:20:19.173675+0000","last_peered":"2024-09-19T04:20:19.173675+0000","last_clean":"2024-09-19T04:20:19.173675+0000","last_became_active":"2024-09-19T04:19:00.982246+0000","last_became_peered":"2024-09-19T04:19:00.982246+0000","last_unstale":"2024-09-19T04:20:19.173675+0000","last_undegraded":"2024-09-19T04:20:19.173675+0000","last_fullsized":"2024-09-19T04:20:19.173675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:48.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260612+0000","last_change":"2024-09-19T04:18:12.506679+0000","last_active":"2024-09-19T04:20:19.260612+0000","last_peered":"2024-09-19T04:20:19.260612+0000","last_clean":"2024-09-19T04:20:19.260612+0000","last_became_active":"2024-09-19T04:18:12.505953+0000","last_became_peered":"2024-09-19T04:18:12.505953+0000","last_unstale":"2024-09-19T04:20:19.260612+0000","last_undegraded":"2024-09-19T04:20:19.260612+0000","last_fullsized":"2024-09-19T04:20:19.260612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:53.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455913+0000","last_change":"2024-09-19T04:18:12.490511+0000","last_active":"2024-09-19T04:20:19.455913+0000","last_peered":"2024-09-19T04:20:19.455913+0000","last_clean":"2024-09-19T04:20:19.455913+0000","last_became_active":"2024-09-19T04:18:12.490051+0000","last_became_peered":"2024-09-19T04:18:12.490051+0000","last_unstale":"2024-09-19T04:20:19.455913+0000","last_undegraded":"2024-09-19T04:20:19.455913+0000","last_fullsized":"2024-09-19T04:20:19.455913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:32.283312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.467072+0000","last_change":"2024-09-19T04:18:12.491465+0000","last_active":"2024-09-19T04:20:19.467072+0000","last_peered":"2024-09-19T04:20:19.467072+0000","last_clean":"2024-09-19T04:20:19.467072+0000","last_became_active":"2024-09-19T04:18:12.491136+0000","last_became_peered":"2024-09-19T04:18:12.491136+0000","last_unstale":"2024-09-19T04:20:19.467072+0000","last_undegraded":"2024-09-19T04:20:19.467072+0000","last_fullsized":"2024-09-19T04:20:19.467072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:29.498909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157299+0000","last_change":"2024-09-19T04:19:01.599944+0000","last_active":"2024-09-19T04:20:19.157299+0000","last_peered":"2024-09-19T04:20:19.157299+0000","last_clean":"2024-09-19T04:20:19.157299+0000","last_became_active":"2024-09-19T04:19:01.599571+0000","last_became_peered":"2024-09-19T04:19:01.599571+0000","last_unstale":"2024-09-19T04:20:19.157299+0000","last_undegraded":"2024-09-19T04:20:19.157299+0000","last_fullsized":"2024-09-19T04:20:19.157299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:17.929785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235780+0000","last_change":"2024-09-19T04:18:12.504364+0000","last_active":"2024-09-19T04:20:19.235780+0000","last_peered":"2024-09-19T04:20:19.235780+0000","last_clean":"2024-09-19T04:20:19.235780+0000","last_became_active":"2024-09-19T04:18:12.504006+0000","last_became_peered":"2024-09-19T04:18:12.504006+0000","last_unstale":"2024-09-19T04:20:19.235780+0000","last_undegraded":"2024-09-19T04:20:19.235780+0000","last_fullsized":"2024-09-19T04:20:19.235780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:53.309963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455576+0000","last_change":"2024-09-19T04:18:12.498223+0000","last_active":"2024-09-19T04:20:19.455576+0000","last_peered":"2024-09-19T04:20:19.455576+0000","last_clean":"2024-09-19T04:20:19.455576+0000","last_became_active":"2024-09-19T04:18:12.497862+0000","last_became_peered":"2024-09-19T04:18:12.497862+0000","last_unstale":"2024-09-19T04:20:19.455576+0000","last_undegraded":"2024-09-19T04:20:19.455576+0000","last_fullsized":"2024-09-19T04:20:19.455576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:28.858620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455732+0000","last_change":"2024-09-19T04:18:12.502335+0000","last_active":"2024-09-19T04:20:19.455732+0000","last_peered":"2024-09-19T04:20:19.455732+0000","last_clean":"2024-09-19T04:20:19.455732+0000","last_became_active":"2024-09-19T04:18:12.501686+0000","last_became_peered":"2024-09-19T04:18:12.501686+0000","last_unstale":"2024-09-19T04:20:19.455732+0000","last_undegraded":"2024-09-19T04:20:19.455732+0000","last_fullsized":"2024-09-19T04:20:19.455732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:22.348745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466807+0000","last_change":"2024-09-19T04:18:12.490905+0000","last_active":"2024-09-19T04:20:19.466807+0000","last_peered":"2024-09-19T04:20:19.466807+0000","last_clean":"2024-09-19T04:20:19.466807+0000","last_became_active":"2024-09-19T04:18:12.490489+0000","last_became_peered":"2024-09-19T04:18:12.490489+0000","last_unstale":"2024-09-19T04:20:19.466807+0000","last_undegraded":"2024-09-19T04:20:19.466807+0000","last_fullsized":"2024-09-19T04:20:19.466807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.946474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173739+0000","last_change":"2024-09-19T04:18:12.494607+0000","last_active":"2024-09-19T04:20:19.173739+0000","last_peered":"2024-09-19T04:20:19.173739+0000","last_clean":"2024-09-19T04:20:19.173739+0000","last_became_active":"2024-09-19T04:18:12.494288+0000","last_became_peered":"2024-09-19T04:18:12.494288+0000","last_unstale":"2024-09-19T04:20:19.173739+0000","last_undegraded":"2024-09-19T04:20:19.173739+0000","last_fullsized":"2024-09-19T04:20:19.173739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:42.824951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":153,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738608+0000","last_change":"2024-09-19T04:18:10.103108+0000","last_active":"2024-09-19T04:20:19.738608+0000","last_peered":"2024-09-19T04:20:19.738608+0000","last_clean":"2024-09-19T04:20:19.738608+0000","last_became_active":"2024-09-19T04:18:10.102783+0000","last_became_peered":"2024-09-19T04:18:10.102783+0000","last_unstale":"2024-09-19T04:20:19.738608+0000","last_undegraded":"2024-09-19T04:20:19.738608+0000","last_fullsized":"2024-09-19T04:20:19.738608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:25.521953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158921+0000","last_change":"2024-09-19T04:18:12.495453+0000","last_active":"2024-09-19T04:20:19.158921+0000","last_peered":"2024-09-19T04:20:19.158921+0000","last_clean":"2024-09-19T04:20:19.158921+0000","last_became_active":"2024-09-19T04:18:12.494744+0000","last_became_peered":"2024-09-19T04:18:12.494744+0000","last_unstale":"2024-09-19T04:20:19.158921+0000","last_undegraded":"2024-09-19T04:20:19.158921+0000","last_fullsized":"2024-09-19T04:20:19.158921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:41.071723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158885+0000","last_change":"2024-09-19T04:18:10.102675+0000","last_active":"2024-09-19T04:20:19.158885+0000","last_peered":"2024-09-19T04:20:19.158885+0000","last_clean":"2024-09-19T04:20:19.158885+0000","last_became_active":"2024-09-19T04:18:10.102368+0000","last_became_peered":"2024-09-19T04:18:10.102368+0000","last_unstale":"2024-09-19T04:20:19.158885+0000","last_undegraded":"2024-09-19T04:20:19.158885+0000","last_fullsized":"2024-09-19T04:20:19.158885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:45.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.156851+0000","last_change":"2024-09-19T04:18:10.099932+0000","last_active":"2024-09-19T04:20:19.156851+0000","last_peered":"2024-09-19T04:20:19.156851+0000","last_clean":"2024-09-19T04:20:19.156851+0000","last_became_active":"2024-09-19T04:18:10.099637+0000","last_became_peered":"2024-09-19T04:18:10.099637+0000","last_unstale":"2024-09-19T04:20:19.156851+0000","last_undegraded":"2024-09-19T04:20:19.156851+0000","last_fullsized":"2024-09-19T04:20:19.156851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:50.594382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259565+0000","last_change":"2024-09-19T04:18:12.502229+0000","last_active":"2024-09-19T04:20:19.259565+0000","last_peered":"2024-09-19T04:20:19.259565+0000","last_clean":"2024-09-19T04:20:19.259565+0000","last_became_active":"2024-09-19T04:18:12.501921+0000","last_became_peered":"2024-09-19T04:18:12.501921+0000","last_unstale":"2024-09-19T04:20:19.259565+0000","last_undegraded":"2024-09-19T04:20:19.259565+0000","last_fullsized":"2024-09-19T04:20:19.259565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:55.412103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158774+0000","last_change":"2024-09-19T04:18:10.103961+0000","last_active":"2024-09-19T04:20:19.158774+0000","last_peered":"2024-09-19T04:20:19.158774+0000","last_clean":"2024-09-19T04:20:19.158774+0000","last_became_active":"2024-09-19T04:18:10.102737+0000","last_became_peered":"2024-09-19T04:18:10.102737+0000","last_unstale":"2024-09-19T04:20:19.158774+0000","last_undegraded":"2024-09-19T04:20:19.158774+0000","last_fullsized":"2024-09-19T04:20:19.158774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:31:01.806124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.255887+0000","last_change":"2024-09-19T04:18:12.502385+0000","last_active":"2024-09-19T04:20:19.255887+0000","last_peered":"2024-09-19T04:20:19.255887+0000","last_clean":"2024-09-19T04:20:19.255887+0000","last_became_active":"2024-09-19T04:18:12.502025+0000","last_became_peered":"2024-09-19T04:18:12.502025+0000","last_unstale":"2024-09-19T04:20:19.255887+0000","last_undegraded":"2024-09-19T04:20:19.255887+0000","last_fullsized":"2024-09-19T04:20:19.255887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:24.650911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'8","reported_seq":262,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.737999+0000","last_change":"2024-09-19T04:18:10.103526+0000","last_active":"2024-09-19T04:20:19.737999+0000","last_peered":"2024-09-19T04:20:19.737999+0000","last_clean":"2024-09-19T04:20:19.737999+0000","last_became_active":"2024-09-19T04:18:10.103191+0000","last_became_peered":"2024-09-19T04:18:10.103191+0000","last_unstale":"2024-09-19T04:20:19.737999+0000","last_undegraded":"2024-09-19T04:20:19.737999+0000","last_fullsized":"2024-09-19T04:20:19.737999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:07.695715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456912+0000","last_change":"2024-09-19T04:18:12.493169+0000","last_active":"2024-09-19T04:20:19.456912+0000","last_peered":"2024-09-19T04:20:19.456912+0000","last_clean":"2024-09-19T04:20:19.456912+0000","last_became_active":"2024-09-19T04:18:12.492669+0000","last_became_peered":"2024-09-19T04:18:12.492669+0000","last_unstale":"2024-09-19T04:20:19.456912+0000","last_undegraded":"2024-09-19T04:20:19.456912+0000","last_fullsized":"2024-09-19T04:20:19.456912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:04.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454874+0000","last_change":"2024-09-19T04:19:01.007275+0000","last_active":"2024-09-19T04:20:19.454874+0000","last_peered":"2024-09-19T04:20:19.454874+0000","last_clean":"2024-09-19T04:20:19.454874+0000","last_became_active":"2024-09-19T04:19:01.003361+0000","last_became_peered":"2024-09-19T04:19:01.003361+0000","last_unstale":"2024-09-19T04:20:19.454874+0000","last_undegraded":"2024-09-19T04:20:19.454874+0000","last_fullsized":"2024-09-19T04:20:19.454874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:07:11.907462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.236090+0000","last_change":"2024-09-19T04:18:12.504348+0000","last_active":"2024-09-19T04:20:19.236090+0000","last_peered":"2024-09-19T04:20:19.236090+0000","last_clean":"2024-09-19T04:20:19.236090+0000","last_became_active":"2024-09-19T04:18:12.503180+0000","last_became_peered":"2024-09-19T04:18:12.503180+0000","last_unstale":"2024-09-19T04:20:19.236090+0000","last_undegraded":"2024-09-19T04:20:19.236090+0000","last_fullsized":"2024-09-19T04:20:19.236090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:03.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159013+0000","last_change":"2024-09-19T04:18:10.365136+0000","last_active":"2024-09-19T04:20:19.159013+0000","last_peered":"2024-09-19T04:20:19.159013+0000","last_clean":"2024-09-19T04:20:19.159013+0000","last_became_active":"2024-09-19T04:18:10.364594+0000","last_became_peered":"2024-09-19T04:18:10.364594+0000","last_unstale":"2024-09-19T04:20:19.159013+0000","last_undegraded":"2024-09-19T04:20:19.159013+0000","last_fullsized":"2024-09-19T04:20:19.159013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:52.252157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456842+0000","last_change":"2024-09-19T04:18:12.502051+0000","last_active":"2024-09-19T04:20:19.456842+0000","last_peered":"2024-09-19T04:20:19.456842+0000","last_clean":"2024-09-19T04:20:19.456842+0000","last_became_active":"2024-09-19T04:18:12.501578+0000","last_became_peered":"2024-09-19T04:18:12.501578+0000","last_unstale":"2024-09-19T04:20:19.456842+0000","last_undegraded":"2024-09-19T04:20:19.456842+0000","last_fullsized":"2024-09-19T04:20:19.456842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:59.693175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'37","reported_seq":303,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738014+0000","last_change":"2024-09-19T04:18:10.101507+0000","last_active":"2024-09-19T04:20:19.738014+0000","last_peered":"2024-09-19T04:20:19.738014+0000","last_clean":"2024-09-19T04:20:19.738014+0000","last_became_active":"2024-09-19T04:18:10.101152+0000","last_became_peered":"2024-09-19T04:18:10.101152+0000","last_unstale":"2024-09-19T04:20:19.738014+0000","last_undegraded":"2024-09-19T04:20:19.738014+0000","last_fullsized":"2024-09-19T04:20:19.738014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:01.397605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66483,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":186,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174032+0000","last_change":"2024-09-19T04:18:12.491611+0000","last_active":"2024-09-19T04:20:19.174032+0000","last_peered":"2024-09-19T04:20:19.174032+0000","last_clean":"2024-09-19T04:20:19.174032+0000","last_became_active":"2024-09-19T04:18:12.491279+0000","last_became_peered":"2024-09-19T04:18:12.491279+0000","last_unstale":"2024-09-19T04:20:19.174032+0000","last_undegraded":"2024-09-19T04:20:19.174032+0000","last_fullsized":"2024-09-19T04:20:19.174032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:01.414713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455643+0000","last_change":"2024-09-19T04:18:10.105574+0000","last_active":"2024-09-19T04:20:19.455643+0000","last_peered":"2024-09-19T04:20:19.455643+0000","last_clean":"2024-09-19T04:20:19.455643+0000","last_became_active":"2024-09-19T04:18:10.105203+0000","last_became_peered":"2024-09-19T04:18:10.105203+0000","last_unstale":"2024-09-19T04:20:19.455643+0000","last_undegraded":"2024-09-19T04:20:19.455643+0000","last_fullsized":"2024-09-19T04:20:19.455643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:00.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.156878+0000","last_change":"2024-09-19T04:18:12.497242+0000","last_active":"2024-09-19T04:20:19.156878+0000","last_peered":"2024-09-19T04:20:19.156878+0000","last_clean":"2024-09-19T04:20:19.156878+0000","last_became_active":"2024-09-19T04:18:12.496732+0000","last_became_peered":"2024-09-19T04:18:12.496732+0000","last_unstale":"2024-09-19T04:20:19.156878+0000","last_undegraded":"2024-09-19T04:20:19.156878+0000","last_fullsized":"2024-09-19T04:20:19.156878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:56.520341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158716+0000","last_change":"2024-09-19T04:18:10.103905+0000","last_active":"2024-09-19T04:20:19.158716+0000","last_peered":"2024-09-19T04:20:19.158716+0000","last_clean":"2024-09-19T04:20:19.158716+0000","last_became_active":"2024-09-19T04:18:10.103517+0000","last_became_peered":"2024-09-19T04:18:10.103517+0000","last_unstale":"2024-09-19T04:20:19.158716+0000","last_undegraded":"2024-09-19T04:20:19.158716+0000","last_fullsized":"2024-09-19T04:20:19.158716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:48.292878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454822+0000","last_change":"2024-09-19T04:18:12.499380+0000","last_active":"2024-09-19T04:20:19.454822+0000","last_peered":"2024-09-19T04:20:19.454822+0000","last_clean":"2024-09-19T04:20:19.454822+0000","last_became_active":"2024-09-19T04:18:12.498995+0000","last_became_peered":"2024-09-19T04:18:12.498995+0000","last_unstale":"2024-09-19T04:20:19.454822+0000","last_undegraded":"2024-09-19T04:20:19.454822+0000","last_fullsized":"2024-09-19T04:20:19.454822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:20.653923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738043+0000","last_change":"2024-09-19T04:18:10.105619+0000","last_active":"2024-09-19T04:20:19.738043+0000","last_peered":"2024-09-19T04:20:19.738043+0000","last_clean":"2024-09-19T04:20:19.738043+0000","last_became_active":"2024-09-19T04:18:10.105082+0000","last_became_peered":"2024-09-19T04:18:10.105082+0000","last_unstale":"2024-09-19T04:20:19.738043+0000","last_undegraded":"2024-09-19T04:20:19.738043+0000","last_fullsized":"2024-09-19T04:20:19.738043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:56.843068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454480+0000","last_change":"2024-09-19T04:18:12.498473+0000","last_active":"2024-09-19T04:20:19.454480+0000","last_peered":"2024-09-19T04:20:19.454480+0000","last_clean":"2024-09-19T04:20:19.454480+0000","last_became_active":"2024-09-19T04:18:12.496904+0000","last_became_peered":"2024-09-19T04:18:12.496904+0000","last_unstale":"2024-09-19T04:20:19.454480+0000","last_undegraded":"2024-09-19T04:20:19.454480+0000","last_fullsized":"2024-09-19T04:20:19.454480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:03.155899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173775+0000","last_change":"2024-09-19T04:18:10.365182+0000","last_active":"2024-09-19T04:20:19.173775+0000","last_peered":"2024-09-19T04:20:19.173775+0000","last_clean":"2024-09-19T04:20:19.173775+0000","last_became_active":"2024-09-19T04:18:10.364717+0000","last_became_peered":"2024-09-19T04:18:10.364717+0000","last_unstale":"2024-09-19T04:20:19.173775+0000","last_undegraded":"2024-09-19T04:20:19.173775+0000","last_fullsized":"2024-09-19T04:20:19.173775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:32.063066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158550+0000","last_change":"2024-09-19T04:18:12.493585+0000","last_active":"2024-09-19T04:20:19.158550+0000","last_peered":"2024-09-19T04:20:19.158550+0000","last_clean":"2024-09-19T04:20:19.158550+0000","last_became_active":"2024-09-19T04:18:12.493255+0000","last_became_peered":"2024-09-19T04:18:12.493255+0000","last_unstale":"2024-09-19T04:20:19.158550+0000","last_undegraded":"2024-09-19T04:20:19.158550+0000","last_fullsized":"2024-09-19T04:20:19.158550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:33.139727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174154+0000","last_change":"2024-09-19T04:18:10.101537+0000","last_active":"2024-09-19T04:20:19.174154+0000","last_peered":"2024-09-19T04:20:19.174154+0000","last_clean":"2024-09-19T04:20:19.174154+0000","last_became_active":"2024-09-19T04:18:10.100951+0000","last_became_peered":"2024-09-19T04:18:10.100951+0000","last_unstale":"2024-09-19T04:20:19.174154+0000","last_undegraded":"2024-09-19T04:20:19.174154+0000","last_fullsized":"2024-09-19T04:20:19.174154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:08.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466903+0000","last_change":"2024-09-19T04:18:12.495530+0000","last_active":"2024-09-19T04:20:19.466903+0000","last_peered":"2024-09-19T04:20:19.466903+0000","last_clean":"2024-09-19T04:20:19.466903+0000","last_became_active":"2024-09-19T04:18:12.495176+0000","last_became_peered":"2024-09-19T04:18:12.495176+0000","last_unstale":"2024-09-19T04:20:19.466903+0000","last_undegraded":"2024-09-19T04:20:19.466903+0000","last_fullsized":"2024-09-19T04:20:19.466903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:24.308047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157053+0000","last_change":"2024-09-19T04:18:10.102539+0000","last_active":"2024-09-19T04:20:19.157053+0000","last_peered":"2024-09-19T04:20:19.157053+0000","last_clean":"2024-09-19T04:20:19.157053+0000","last_became_active":"2024-09-19T04:18:10.102109+0000","last_became_peered":"2024-09-19T04:18:10.102109+0000","last_unstale":"2024-09-19T04:20:19.157053+0000","last_undegraded":"2024-09-19T04:20:19.157053+0000","last_fullsized":"2024-09-19T04:20:19.157053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:23:58.852767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.455095+0000","last_change":"2024-09-19T04:18:12.491769+0000","last_active":"2024-09-19T04:20:19.455095+0000","last_peered":"2024-09-19T04:20:19.455095+0000","last_clean":"2024-09-19T04:20:19.455095+0000","last_became_active":"2024-09-19T04:18:12.491435+0000","last_became_peered":"2024-09-19T04:18:12.491435+0000","last_unstale":"2024-09-19T04:20:19.455095+0000","last_undegraded":"2024-09-19T04:20:19.455095+0000","last_fullsized":"2024-09-19T04:20:19.455095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:54.880533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'41","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.411367+0000","last_change":"2024-09-19T04:18:10.103559+0000","last_active":"2024-09-19T04:20:19.411367+0000","last_peered":"2024-09-19T04:20:19.411367+0000","last_clean":"2024-09-19T04:20:19.411367+0000","last_became_active":"2024-09-19T04:18:10.103208+0000","last_became_peered":"2024-09-19T04:18:10.103208+0000","last_unstale":"2024-09-19T04:20:19.411367+0000","last_undegraded":"2024-09-19T04:20:19.411367+0000","last_fullsized":"2024-09-19T04:20:19.411367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:16.564916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454709+0000","last_change":"2024-09-19T04:18:12.495616+0000","last_active":"2024-09-19T04:20:19.454709+0000","last_peered":"2024-09-19T04:20:19.454709+0000","last_clean":"2024-09-19T04:20:19.454709+0000","last_became_active":"2024-09-19T04:18:12.495243+0000","last_became_peered":"2024-09-19T04:18:12.495243+0000","last_unstale":"2024-09-19T04:20:19.454709+0000","last_undegraded":"2024-09-19T04:20:19.454709+0000","last_fullsized":"2024-09-19T04:20:19.454709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:12.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'8","reported_seq":152,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738564+0000","last_change":"2024-09-19T04:18:10.626874+0000","last_active":"2024-09-19T04:20:19.738564+0000","last_peered":"2024-09-19T04:20:19.738564+0000","last_clean":"2024-09-19T04:20:19.738564+0000","last_became_active":"2024-09-19T04:18:10.626506+0000","last_became_peered":"2024-09-19T04:18:10.626506+0000","last_unstale":"2024-09-19T04:20:19.738564+0000","last_undegraded":"2024-09-19T04:20:19.738564+0000","last_fullsized":"2024-09-19T04:20:19.738564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:48.725143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454917+0000","last_change":"2024-09-19T04:18:12.496350+0000","last_active":"2024-09-19T04:20:19.454917+0000","last_peered":"2024-09-19T04:20:19.454917+0000","last_clean":"2024-09-19T04:20:19.454917+0000","last_became_active":"2024-09-19T04:18:12.495904+0000","last_became_peered":"2024-09-19T04:18:12.495904+0000","last_unstale":"2024-09-19T04:20:19.454917+0000","last_undegraded":"2024-09-19T04:20:19.454917+0000","last_fullsized":"2024-09-19T04:20:19.454917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:08.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454676+0000","last_change":"2024-09-19T04:16:56.164450+0000","last_active":"2024-09-19T04:20:19.454676+0000","last_peered":"2024-09-19T04:20:19.454676+0000","last_clean":"2024-09-19T04:20:19.454676+0000","last_became_active":"2024-09-19T04:16:56.164008+0000","last_became_peered":"2024-09-19T04:16:56.164008+0000","last_unstale":"2024-09-19T04:20:19.454676+0000","last_undegraded":"2024-09-19T04:20:19.454676+0000","last_fullsized":"2024-09-19T04:20:19.454676+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:18.308718+0000","last_clean_scrub_stamp":"2024-09-19T04:13:18.308718+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:14.697725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.459005+0000","last_change":"2024-09-19T04:18:10.101598+0000","last_active":"2024-09-19T04:20:19.459005+0000","last_peered":"2024-09-19T04:20:19.459005+0000","last_clean":"2024-09-19T04:20:19.459005+0000","last_became_active":"2024-09-19T04:18:10.101210+0000","last_became_peered":"2024-09-19T04:18:10.101210+0000","last_unstale":"2024-09-19T04:20:19.459005+0000","last_undegraded":"2024-09-19T04:20:19.459005+0000","last_fullsized":"2024-09-19T04:20:19.459005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:10.653481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260008+0000","last_change":"2024-09-19T04:18:12.495034+0000","last_active":"2024-09-19T04:20:19.260008+0000","last_peered":"2024-09-19T04:20:19.260008+0000","last_clean":"2024-09-19T04:20:19.260008+0000","last_became_active":"2024-09-19T04:18:12.494611+0000","last_became_peered":"2024-09-19T04:18:12.494611+0000","last_unstale":"2024-09-19T04:20:19.260008+0000","last_undegraded":"2024-09-19T04:20:19.260008+0000","last_fullsized":"2024-09-19T04:20:19.260008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:28.448927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454656+0000","last_change":"2024-09-19T04:18:10.105212+0000","last_active":"2024-09-19T04:20:19.454656+0000","last_peered":"2024-09-19T04:20:19.454656+0000","last_clean":"2024-09-19T04:20:19.454656+0000","last_became_active":"2024-09-19T04:18:10.104710+0000","last_became_peered":"2024-09-19T04:18:10.104710+0000","last_unstale":"2024-09-19T04:20:19.454656+0000","last_undegraded":"2024-09-19T04:20:19.454656+0000","last_fullsized":"2024-09-19T04:20:19.454656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:58.007773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.156921+0000","last_change":"2024-09-19T04:19:00.980419+0000","last_active":"2024-09-19T04:20:19.156921+0000","last_peered":"2024-09-19T04:20:19.156921+0000","last_clean":"2024-09-19T04:20:19.156921+0000","last_became_active":"2024-09-19T04:19:00.980008+0000","last_became_peered":"2024-09-19T04:19:00.980008+0000","last_unstale":"2024-09-19T04:20:19.156921+0000","last_undegraded":"2024-09-19T04:20:19.156921+0000","last_fullsized":"2024-09-19T04:20:19.156921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:18.396611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":147,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.737581+0000","last_change":"2024-09-19T04:18:10.364530+0000","last_active":"2024-09-19T04:20:19.737581+0000","last_peered":"2024-09-19T04:20:19.737581+0000","last_clean":"2024-09-19T04:20:19.737581+0000","last_became_active":"2024-09-19T04:18:10.364204+0000","last_became_peered":"2024-09-19T04:18:10.364204+0000","last_unstale":"2024-09-19T04:20:19.737581+0000","last_undegraded":"2024-09-19T04:20:19.737581+0000","last_fullsized":"2024-09-19T04:20:19.737581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:04.229530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158567+0000","last_change":"2024-09-19T04:18:12.493730+0000","last_active":"2024-09-19T04:20:19.158567+0000","last_peered":"2024-09-19T04:20:19.158567+0000","last_clean":"2024-09-19T04:20:19.158567+0000","last_became_active":"2024-09-19T04:18:12.493153+0000","last_became_peered":"2024-09-19T04:18:12.493153+0000","last_unstale":"2024-09-19T04:20:19.158567+0000","last_undegraded":"2024-09-19T04:20:19.158567+0000","last_fullsized":"2024-09-19T04:20:19.158567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.758490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'7","reported_seq":150,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.739821+0000","last_change":"2024-09-19T04:18:10.102073+0000","last_active":"2024-09-19T04:20:19.739821+0000","last_peered":"2024-09-19T04:20:19.739821+0000","last_clean":"2024-09-19T04:20:19.739821+0000","last_became_active":"2024-09-19T04:18:10.101755+0000","last_became_peered":"2024-09-19T04:18:10.101755+0000","last_unstale":"2024-09-19T04:20:19.739821+0000","last_undegraded":"2024-09-19T04:20:19.739821+0000","last_fullsized":"2024-09-19T04:20:19.739821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:21.489146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1292,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159049+0000","last_change":"2024-09-19T04:18:12.494560+0000","last_active":"2024-09-19T04:20:19.159049+0000","last_peered":"2024-09-19T04:20:19.159049+0000","last_clean":"2024-09-19T04:20:19.159049+0000","last_became_active":"2024-09-19T04:18:12.494203+0000","last_became_peered":"2024-09-19T04:18:12.494203+0000","last_unstale":"2024-09-19T04:20:19.159049+0000","last_undegraded":"2024-09-19T04:20:19.159049+0000","last_fullsized":"2024-09-19T04:20:19.159049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:35:17.129649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.256225+0000","last_change":"2024-09-19T04:18:10.365781+0000","last_active":"2024-09-19T04:20:19.256225+0000","last_peered":"2024-09-19T04:20:19.256225+0000","last_clean":"2024-09-19T04:20:19.256225+0000","last_became_active":"2024-09-19T04:18:10.365417+0000","last_became_peered":"2024-09-19T04:18:10.365417+0000","last_unstale":"2024-09-19T04:20:19.256225+0000","last_undegraded":"2024-09-19T04:20:19.256225+0000","last_fullsized":"2024-09-19T04:20:19.256225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:20.097572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.159240+0000","last_change":"2024-09-19T04:18:12.495570+0000","last_active":"2024-09-19T04:20:19.159240+0000","last_peered":"2024-09-19T04:20:19.159240+0000","last_clean":"2024-09-19T04:20:19.159240+0000","last_became_active":"2024-09-19T04:18:12.495242+0000","last_became_peered":"2024-09-19T04:18:12.495242+0000","last_unstale":"2024-09-19T04:20:19.159240+0000","last_undegraded":"2024-09-19T04:20:19.159240+0000","last_fullsized":"2024-09-19T04:20:19.159240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:57.448182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174114+0000","last_change":"2024-09-19T04:18:10.365134+0000","last_active":"2024-09-19T04:20:19.174114+0000","last_peered":"2024-09-19T04:20:19.174114+0000","last_clean":"2024-09-19T04:20:19.174114+0000","last_became_active":"2024-09-19T04:18:10.364726+0000","last_became_peered":"2024-09-19T04:18:10.364726+0000","last_unstale":"2024-09-19T04:20:19.174114+0000","last_undegraded":"2024-09-19T04:20:19.174114+0000","last_fullsized":"2024-09-19T04:20:19.174114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:44.207855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.157194+0000","last_change":"2024-09-19T04:18:12.488836+0000","last_active":"2024-09-19T04:20:19.157194+0000","last_peered":"2024-09-19T04:20:19.157194+0000","last_clean":"2024-09-19T04:20:19.157194+0000","last_became_active":"2024-09-19T04:18:12.488507+0000","last_became_peered":"2024-09-19T04:18:12.488507+0000","last_unstale":"2024-09-19T04:20:19.157194+0000","last_undegraded":"2024-09-19T04:20:19.157194+0000","last_fullsized":"2024-09-19T04:20:19.157194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:31.308114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'52","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.417279+0000","last_change":"2024-09-19T04:18:10.103623+0000","last_active":"2024-09-19T04:20:19.417279+0000","last_peered":"2024-09-19T04:20:19.417279+0000","last_clean":"2024-09-19T04:20:19.417279+0000","last_became_active":"2024-09-19T04:18:10.103340+0000","last_became_peered":"2024-09-19T04:18:10.103340+0000","last_unstale":"2024-09-19T04:20:19.417279+0000","last_undegraded":"2024-09-19T04:20:19.417279+0000","last_fullsized":"2024-09-19T04:20:19.417279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:11.724013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260722+0000","last_change":"2024-09-19T04:18:12.498167+0000","last_active":"2024-09-19T04:20:19.260722+0000","last_peered":"2024-09-19T04:20:19.260722+0000","last_clean":"2024-09-19T04:20:19.260722+0000","last_became_active":"2024-09-19T04:18:12.496924+0000","last_became_peered":"2024-09-19T04:18:12.496924+0000","last_unstale":"2024-09-19T04:20:19.260722+0000","last_undegraded":"2024-09-19T04:20:19.260722+0000","last_fullsized":"2024-09-19T04:20:19.260722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:13.952030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.459180+0000","last_change":"2024-09-19T04:18:10.627106+0000","last_active":"2024-09-19T04:20:19.459180+0000","last_peered":"2024-09-19T04:20:19.459180+0000","last_clean":"2024-09-19T04:20:19.459180+0000","last_became_active":"2024-09-19T04:18:10.626708+0000","last_became_peered":"2024-09-19T04:18:10.626708+0000","last_unstale":"2024-09-19T04:20:19.459180+0000","last_undegraded":"2024-09-19T04:20:19.459180+0000","last_fullsized":"2024-09-19T04:20:19.459180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:46.049699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.235881+0000","last_change":"2024-09-19T04:18:12.506995+0000","last_active":"2024-09-19T04:20:19.235881+0000","last_peered":"2024-09-19T04:20:19.235881+0000","last_clean":"2024-09-19T04:20:19.235881+0000","last_became_active":"2024-09-19T04:18:12.500607+0000","last_became_peered":"2024-09-19T04:18:12.500607+0000","last_unstale":"2024-09-19T04:20:19.235881+0000","last_undegraded":"2024-09-19T04:20:19.235881+0000","last_fullsized":"2024-09-19T04:20:19.235881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:45:37.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456633+0000","last_change":"2024-09-19T04:18:10.365150+0000","last_active":"2024-09-19T04:20:19.456633+0000","last_peered":"2024-09-19T04:20:19.456633+0000","last_clean":"2024-09-19T04:20:19.456633+0000","last_became_active":"2024-09-19T04:18:10.364755+0000","last_became_peered":"2024-09-19T04:18:10.364755+0000","last_unstale":"2024-09-19T04:20:19.456633+0000","last_undegraded":"2024-09-19T04:20:19.456633+0000","last_fullsized":"2024-09-19T04:20:19.456633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:11.639532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173701+0000","last_change":"2024-09-19T04:18:12.484588+0000","last_active":"2024-09-19T04:20:19.173701+0000","last_peered":"2024-09-19T04:20:19.173701+0000","last_clean":"2024-09-19T04:20:19.173701+0000","last_became_active":"2024-09-19T04:18:12.484218+0000","last_became_peered":"2024-09-19T04:18:12.484218+0000","last_unstale":"2024-09-19T04:20:19.173701+0000","last_undegraded":"2024-09-19T04:20:19.173701+0000","last_fullsized":"2024-09-19T04:20:19.173701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:55.634557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.259785+0000","last_change":"2024-09-19T04:18:10.100832+0000","last_active":"2024-09-19T04:20:19.259785+0000","last_peered":"2024-09-19T04:20:19.259785+0000","last_clean":"2024-09-19T04:20:19.259785+0000","last_became_active":"2024-09-19T04:18:10.100518+0000","last_became_peered":"2024-09-19T04:18:10.100518+0000","last_unstale":"2024-09-19T04:20:19.259785+0000","last_undegraded":"2024-09-19T04:20:19.259785+0000","last_fullsized":"2024-09-19T04:20:19.259785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:37.028120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466793+0000","last_change":"2024-09-19T04:18:12.496914+0000","last_active":"2024-09-19T04:20:19.466793+0000","last_peered":"2024-09-19T04:20:19.466793+0000","last_clean":"2024-09-19T04:20:19.466793+0000","last_became_active":"2024-09-19T04:18:12.496556+0000","last_became_peered":"2024-09-19T04:18:12.496556+0000","last_unstale":"2024-09-19T04:20:19.466793+0000","last_undegraded":"2024-09-19T04:20:19.466793+0000","last_fullsized":"2024-09-19T04:20:19.466793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:52.391412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466731+0000","last_change":"2024-09-19T04:19:00.993461+0000","last_active":"2024-09-19T04:20:19.466731+0000","last_peered":"2024-09-19T04:20:19.466731+0000","last_clean":"2024-09-19T04:20:19.466731+0000","last_became_active":"2024-09-19T04:19:00.980704+0000","last_became_peered":"2024-09-19T04:19:00.980704+0000","last_unstale":"2024-09-19T04:20:19.466731+0000","last_undegraded":"2024-09-19T04:20:19.466731+0000","last_fullsized":"2024-09-19T04:20:19.466731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:56.090092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260090+0000","last_change":"2024-09-19T04:18:12.496852+0000","last_active":"2024-09-19T04:20:19.260090+0000","last_peered":"2024-09-19T04:20:19.260090+0000","last_clean":"2024-09-19T04:20:19.260090+0000","last_became_active":"2024-09-19T04:18:12.495365+0000","last_became_peered":"2024-09-19T04:18:12.495365+0000","last_unstale":"2024-09-19T04:20:19.260090+0000","last_undegraded":"2024-09-19T04:20:19.260090+0000","last_fullsized":"2024-09-19T04:20:19.260090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:14:57.012618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.173834+0000","last_change":"2024-09-19T04:18:10.104641+0000","last_active":"2024-09-19T04:20:19.173834+0000","last_peered":"2024-09-19T04:20:19.173834+0000","last_clean":"2024-09-19T04:20:19.173834+0000","last_became_active":"2024-09-19T04:18:10.104295+0000","last_became_peered":"2024-09-19T04:18:10.104295+0000","last_unstale":"2024-09-19T04:20:19.173834+0000","last_undegraded":"2024-09-19T04:20:19.173834+0000","last_fullsized":"2024-09-19T04:20:19.173834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:58.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.454750+0000","last_change":"2024-09-19T04:18:12.488396+0000","last_active":"2024-09-19T04:20:19.454750+0000","last_peered":"2024-09-19T04:20:19.454750+0000","last_clean":"2024-09-19T04:20:19.454750+0000","last_became_active":"2024-09-19T04:18:12.488048+0000","last_became_peered":"2024-09-19T04:18:12.488048+0000","last_unstale":"2024-09-19T04:20:19.454750+0000","last_undegraded":"2024-09-19T04:20:19.454750+0000","last_fullsized":"2024-09-19T04:20:19.454750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:27.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.158938+0000","last_change":"2024-09-19T04:18:12.493722+0000","last_active":"2024-09-19T04:20:19.158938+0000","last_peered":"2024-09-19T04:20:19.158938+0000","last_clean":"2024-09-19T04:20:19.158938+0000","last_became_active":"2024-09-19T04:18:12.492483+0000","last_became_peered":"2024-09-19T04:18:12.492483+0000","last_unstale":"2024-09-19T04:20:19.158938+0000","last_undegraded":"2024-09-19T04:20:19.158938+0000","last_fullsized":"2024-09-19T04:20:19.158938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:33.492623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.174205+0000","last_change":"2024-09-19T04:18:10.101326+0000","last_active":"2024-09-19T04:20:19.174205+0000","last_peered":"2024-09-19T04:20:19.174205+0000","last_clean":"2024-09-19T04:20:19.174205+0000","last_became_active":"2024-09-19T04:18:10.100955+0000","last_became_peered":"2024-09-19T04:18:10.100955+0000","last_unstale":"2024-09-19T04:20:19.174205+0000","last_undegraded":"2024-09-19T04:20:19.174205+0000","last_fullsized":"2024-09-19T04:20:19.174205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:41.939741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.456660+0000","last_change":"2024-09-19T04:18:12.492750+0000","last_active":"2024-09-19T04:20:19.456660+0000","last_peered":"2024-09-19T04:20:19.456660+0000","last_clean":"2024-09-19T04:20:19.456660+0000","last_became_active":"2024-09-19T04:18:12.492417+0000","last_became_peered":"2024-09-19T04:18:12.492417+0000","last_unstale":"2024-09-19T04:20:19.456660+0000","last_undegraded":"2024-09-19T04:20:19.456660+0000","last_fullsized":"2024-09-19T04:20:19.456660+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:25.279452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.466779+0000","last_change":"2024-09-19T04:19:00.989741+0000","last_active":"2024-09-19T04:20:19.466779+0000","last_peered":"2024-09-19T04:20:19.466779+0000","last_clean":"2024-09-19T04:20:19.466779+0000","last_became_active":"2024-09-19T04:19:00.977530+0000","last_became_peered":"2024-09-19T04:19:00.977530+0000","last_unstale":"2024-09-19T04:20:19.466779+0000","last_undegraded":"2024-09-19T04:20:19.466779+0000","last_fullsized":"2024-09-19T04:20:19.466779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:35.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.236131+0000","last_change":"2024-09-19T04:18:12.503510+0000","last_active":"2024-09-19T04:20:19.236131+0000","last_peered":"2024-09-19T04:20:19.236131+0000","last_clean":"2024-09-19T04:20:19.236131+0000","last_became_active":"2024-09-19T04:18:12.495165+0000","last_became_peered":"2024-09-19T04:18:12.495165+0000","last_unstale":"2024-09-19T04:20:19.236131+0000","last_undegraded":"2024-09-19T04:20:19.236131+0000","last_fullsized":"2024-09-19T04:20:19.236131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:13.934328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260678+0000","last_change":"2024-09-19T04:18:10.364783+0000","last_active":"2024-09-19T04:20:19.260678+0000","last_peered":"2024-09-19T04:20:19.260678+0000","last_clean":"2024-09-19T04:20:19.260678+0000","last_became_active":"2024-09-19T04:18:10.364202+0000","last_became_peered":"2024-09-19T04:18:10.364202+0000","last_unstale":"2024-09-19T04:20:19.260678+0000","last_undegraded":"2024-09-19T04:20:19.260678+0000","last_fullsized":"2024-09-19T04:20:19.260678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:18:55.770637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.260163+0000","last_change":"2024-09-19T04:18:12.495825+0000","last_active":"2024-09-19T04:20:19.260163+0000","last_peered":"2024-09-19T04:20:19.260163+0000","last_clean":"2024-09-19T04:20:19.260163+0000","last_became_active":"2024-09-19T04:18:12.494525+0000","last_became_peered":"2024-09-19T04:18:12.494525+0000","last_unstale":"2024-09-19T04:20:19.260163+0000","last_undegraded":"2024-09-19T04:20:19.260163+0000","last_fullsized":"2024-09-19T04:20:19.260163+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:11.455904+0000","last_clean_scrub_stamp":"2024-09-19T04:18:11.455904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:57.827135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":155,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T04:20:19.738811+0000","last_change":"2024-09-19T04:18:10.102145+0000","last_active":"2024-09-19T04:20:19.738811+0000","last_peered":"2024-09-19T04:20:19.738811+0000","last_clean":"2024-09-19T04:20:19.738811+0000","last_became_active":"2024-09-19T04:18:10.101410+0000","last_became_peered":"2024-09-19T04:18:10.101410+0000","last_unstale":"2024-09-19T04:20:19.738811+0000","last_undegraded":"2024-09-19T04:20:19.738811+0000","last_fullsized":"2024-09-19T04:20:19.738811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:09.081552+0000","last_clean_scrub_stamp":"2024-09-19T04:18:09.081552+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:20.354296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601458,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1291,"num_read_kb":762,"num_write":381,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1804374,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107517,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":450642,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1089536,"data_stored":1167124,"data_compressed":41814,"data_compressed_allocated":53248,"data_compressed_original":659456,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972402,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":970752,"data_stored":842861,"data_compressed":22034,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201335,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":843239,"data_compressed":21997,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":1044960,"data_compressed":38727,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":449673,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822727,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":1174748,"data_compressed":44883,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":450009,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215184,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":974003,"data_compressed":28190,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":519225,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":517462,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":451628,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35839,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135740,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200605,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:20:24.703 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:20:25.125 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:25.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:25 smithi115 ceph-mon[35477]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-09-19T04:20:25.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:25 smithi115 ceph-mon[35477]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:20:25.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:25 smithi115 ceph-mon[35477]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:20:25.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:25 smithi115 ceph-mon[35477]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:20:25.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:25 smithi115 ceph-mon[35477]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:20:25.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:25 smithi107 ceph-mon[30659]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-09-19T04:20:25.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:25 smithi107 ceph-mon[30659]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:20:25.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:25 smithi107 ceph-mon[30659]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:20:25.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:25 smithi107 ceph-mon[30659]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:20:25.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:25 smithi107 ceph-mon[30659]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:20:25.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:25 smithi145 ceph-mon[35096]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-09-19T04:20:25.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:25 smithi145 ceph-mon[35096]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:20:25.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:25 smithi145 ceph-mon[35096]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:20:25.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:25 smithi145 ceph-mon[35096]: Updating smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:20:25.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:25 smithi145 ceph-mon[35096]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:20:25.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.717+0000 7f3d4b000640 1 -- 172.21.15.107:0/213113373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d441049b0 msgr2=0x7f3d44108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:25.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.717+0000 7f3d4b000640 1 --2- 172.21.15.107:0/213113373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d441049b0 0x7f3d44108a00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009900 tx=0x7f3d2c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:20:25.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.717+0000 7f3d4b000640 1 -- 172.21.15.107:0/213113373 shutdown_connections 2024-09-19T04:20:25.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.717+0000 7f3d4b000640 1 --2- 172.21.15.107:0/213113373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d44109130 0x7f3d44110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:25.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.717+0000 7f3d4b000640 1 --2- 172.21.15.107:0/213113373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d441049b0 0x7f3d44108a00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:25.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.717+0000 7f3d4b000640 1 --2- 172.21.15.107:0/213113373 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d44104000 0x7f3d441043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:25.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.717+0000 7f3d4b000640 1 -- 172.21.15.107:0/213113373 >> 172.21.15.107:0/213113373 conn(0x7f3d440ffa20 msgr2=0x7f3d44101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:25.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.717+0000 7f3d4b000640 1 -- 172.21.15.107:0/213113373 shutdown_connections 2024-09-19T04:20:25.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.717+0000 7f3d4b000640 1 -- 172.21.15.107:0/213113373 wait complete. 2024-09-19T04:20:25.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.718+0000 7f3d4b000640 1 Processor -- start 2024-09-19T04:20:25.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.718+0000 7f3d4b000640 1 -- start start 2024-09-19T04:20:25.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d4b000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d44104000 0x7f3d4419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:25.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d4b000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d441049b0 0x7f3d4419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:25.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d4b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109130 0x7f3d4419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:25.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d4b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3d44114180 con 0x7f3d44109130 2024-09-19T04:20:25.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d4b000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3d44114000 con 0x7f3d441049b0 2024-09-19T04:20:25.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d4b000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d44114300 con 0x7f3d44104000 2024-09-19T04:20:25.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d3b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d441049b0 0x7f3d4419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:25.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d49000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109130 0x7f3d4419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:25.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d3b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d441049b0 0x7f3d4419aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:49558/0 (socket says 172.21.15.107:49558) 2024-09-19T04:20:25.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d3b400640 1 -- 172.21.15.107:0/2344426556 learned_addr learned my addr 172.21.15.107:0/2344426556 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:25.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d43e00640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d44104000 0x7f3d4419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:25.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d3b400640 1 -- 172.21.15.107:0/2344426556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d44104000 msgr2=0x7f3d4419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d3b400640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d44104000 0x7f3d4419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d3b400640 1 -- 172.21.15.107:0/2344426556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109130 msgr2=0x7f3d4419df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d3b400640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109130 0x7f3d4419df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d3b400640 1 -- 172.21.15.107:0/2344426556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4419e6b0 con 0x7f3d441049b0 2024-09-19T04:20:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.719+0000 7f3d49000640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109130 0x7f3d4419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.720+0000 7f3d3b400640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d441049b0 0x7f3d4419aca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c004770 tx=0x7f3d2c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.720+0000 7f3d41600640 1 -- 172.21.15.107:0/2344426556 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c03f070 con 0x7f3d441049b0 2024-09-19T04:20:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.720+0000 7f3d4b000640 1 -- 172.21.15.107:0/2344426556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4419e940 con 0x7f3d441049b0 2024-09-19T04:20:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.720+0000 7f3d4b000640 1 -- 172.21.15.107:0/2344426556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d441a7310 con 0x7f3d441049b0 2024-09-19T04:20:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.720+0000 7f3d41600640 1 -- 172.21.15.107:0/2344426556 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2c031ca0 con 0x7f3d441049b0 2024-09-19T04:20:25.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.720+0000 7f3d41600640 1 -- 172.21.15.107:0/2344426556 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c049060 con 0x7f3d441049b0 2024-09-19T04:20:25.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.721+0000 7f3d4b000640 1 -- 172.21.15.107:0/2344426556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d441043e0 con 0x7f3d441049b0 2024-09-19T04:20:25.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.723+0000 7f3d41600640 1 -- 172.21.15.107:0/2344426556 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7f3d2c042070 con 0x7f3d441049b0 2024-09-19T04:20:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.724+0000 7f3d41600640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3d280a3440 0x7f3d280a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.724+0000 7f3d41600640 1 -- 172.21.15.107:0/2344426556 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f3d2c0f0700 con 0x7f3d441049b0 2024-09-19T04:20:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.725+0000 7f3d43e00640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3d280a3440 0x7f3d280a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.725+0000 7f3d43e00640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3d280a3440 0x7f3d280a5900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400bbe0 tx=0x7f3d3400b390 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.726+0000 7f3d41600640 1 -- 172.21.15.107:0/2344426556 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3d2c0bcf60 con 0x7f3d441049b0 2024-09-19T04:20:25.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.937+0000 7f3d4b000640 1 -- 172.21.15.107:0/2344426556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d4410c370 con 0x7f3d441049b0 2024-09-19T04:20:25.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.938+0000 7f3d41600640 1 -- 172.21.15.107:0/2344426556 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11992 (secure 0 0 0) 0x7f3d2c0bc860 con 0x7f3d441049b0 2024-09-19T04:20:25.940 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:20:25.940 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"btime":"2024-09-19T04:20:19:828247+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:20:18.993568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24490,24496,24500,24539,24517]},"id":1}]} 2024-09-19T04:20:25.940 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 -- 172.21.15.107:0/2344426556 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3d280a3440 msgr2=0x7f3d280a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3d280a3440 0x7f3d280a5900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400bbe0 tx=0x7f3d3400b390 comp rx=0 tx=0).stop 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 -- 172.21.15.107:0/2344426556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d441049b0 msgr2=0x7f3d4419aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d441049b0 0x7f3d4419aca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c004770 tx=0x7f3d2c003c10 comp rx=0 tx=0).stop 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 -- 172.21.15.107:0/2344426556 shutdown_connections 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3d280a3440 0x7f3d280a5900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109130 0x7f3d4419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d441049b0 0x7f3d4419aca0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 --2- 172.21.15.107:0/2344426556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d44104000 0x7f3d4419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 -- 172.21.15.107:0/2344426556 >> 172.21.15.107:0/2344426556 conn(0x7f3d440ffa20 msgr2=0x7f3d44100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:25.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 -- 172.21.15.107:0/2344426556 shutdown_connections 2024-09-19T04:20:25.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:25.940+0000 7f3d4b000640 1 -- 172.21.15.107:0/2344426556 wait complete. 2024-09-19T04:20:26.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:26 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:26 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:26 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:26 smithi145 ceph-mon[35096]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:20:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:26 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:26 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:26 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:26 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:26 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2344426556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:26.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:26 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:26 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:26 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:26 smithi107 ceph-mon[30659]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:20:26.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:26 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:26.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:26 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:26.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:26 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:26.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:26 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.337 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:26 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2344426556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:26.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:26 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:26 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:26 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:26 smithi115 ceph-mon[35477]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:20:26.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:26 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:26.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:26 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:26.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:26 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:26.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:26 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:26.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:26 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2344426556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:20:26.706 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:20:27.078 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:27.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.689+0000 7f6e3a600640 1 -- 172.21.15.107:0/2168288608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e34101de0 msgr2=0x7f6e341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.689+0000 7f6e3a600640 1 --2- 172.21.15.107:0/2168288608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e34101de0 0x7f6e341021c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009900 tx=0x7f6e2402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.690+0000 7f6e3a600640 1 -- 172.21.15.107:0/2168288608 shutdown_connections 2024-09-19T04:20:27.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.690+0000 7f6e3a600640 1 --2- 172.21.15.107:0/2168288608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e34106f10 0x7f6e3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.690+0000 7f6e3a600640 1 --2- 172.21.15.107:0/2168288608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e34102790 0x7f6e341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.690+0000 7f6e3a600640 1 --2- 172.21.15.107:0/2168288608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e34101de0 0x7f6e341021c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.690+0000 7f6e3a600640 1 -- 172.21.15.107:0/2168288608 >> 172.21.15.107:0/2168288608 conn(0x7f6e340fd8c0 msgr2=0x7f6e340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:27.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.690+0000 7f6e3a600640 1 -- 172.21.15.107:0/2168288608 shutdown_connections 2024-09-19T04:20:27.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.690+0000 7f6e3a600640 1 -- 172.21.15.107:0/2168288608 wait complete. 2024-09-19T04:20:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.691+0000 7f6e3a600640 1 Processor -- start 2024-09-19T04:20:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.691+0000 7f6e3a600640 1 -- start start 2024-09-19T04:20:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.691+0000 7f6e3a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e34101de0 0x7f6e3419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.691+0000 7f6e3a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e34102790 0x7f6e3419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.691+0000 7f6e3a600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e34106f10 0x7f6e3419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.691+0000 7f6e3a600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34111f80 con 0x7f6e34101de0 2024-09-19T04:20:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.691+0000 7f6e3a600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34111e00 con 0x7f6e34106f10 2024-09-19T04:20:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.691+0000 7f6e3a600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34112100 con 0x7f6e34102790 2024-09-19T04:20:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e33400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e34101de0 0x7f6e3419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e2aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e34102790 0x7f6e3419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e2aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e34102790 0x7f6e3419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46616/0 (socket says 172.21.15.107:46616) 2024-09-19T04:20:27.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e2aa00640 1 -- 172.21.15.107:0/2035817854 learned_addr learned my addr 172.21.15.107:0/2035817854 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:27.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e33e00640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e34106f10 0x7f6e3419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e2aa00640 1 -- 172.21.15.107:0/2035817854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e34106f10 msgr2=0x7f6e3419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e2aa00640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e34106f10 0x7f6e3419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e2aa00640 1 -- 172.21.15.107:0/2035817854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e34101de0 msgr2=0x7f6e3419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e2aa00640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e34101de0 0x7f6e3419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e2aa00640 1 -- 172.21.15.107:0/2035817854 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3419f630 con 0x7f6e34102790 2024-09-19T04:20:27.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.692+0000 7f6e2aa00640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e34102790 0x7f6e3419abc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00e9f0 tx=0x7f6e1c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.693+0000 7f6e30c00640 1 -- 172.21.15.107:0/2035817854 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00cd80 con 0x7f6e34102790 2024-09-19T04:20:27.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.693+0000 7f6e30c00640 1 -- 172.21.15.107:0/2035817854 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e1c010910 con 0x7f6e34102790 2024-09-19T04:20:27.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.693+0000 7f6e30c00640 1 -- 172.21.15.107:0/2035817854 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c018990 con 0x7f6e34102790 2024-09-19T04:20:27.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.693+0000 7f6e3a600640 1 -- 172.21.15.107:0/2035817854 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3419f920 con 0x7f6e34102790 2024-09-19T04:20:27.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.693+0000 7f6e3a600640 1 -- 172.21.15.107:0/2035817854 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e341a7200 con 0x7f6e34102790 2024-09-19T04:20:27.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.694+0000 7f6e3a600640 1 -- 172.21.15.107:0/2035817854 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e341021c0 con 0x7f6e34102790 2024-09-19T04:20:27.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.696+0000 7f6e30c00640 1 -- 172.21.15.107:0/2035817854 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7f6e1c010ac0 con 0x7f6e34102790 2024-09-19T04:20:27.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.697+0000 7f6e30c00640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6e180a34c0 0x7f6e180a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.697+0000 7f6e30c00640 1 -- 172.21.15.107:0/2035817854 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f6e1c0d0580 con 0x7f6e34102790 2024-09-19T04:20:27.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.698+0000 7f6e33400640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6e180a34c0 0x7f6e180a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.698+0000 7f6e33400640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6e180a34c0 0x7f6e180a5980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6e24000c00 tx=0x7f6e24005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.699+0000 7f6e30c00640 1 -- 172.21.15.107:0/2035817854 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6e1c09c790 con 0x7f6e34102790 2024-09-19T04:20:27.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.879+0000 7f6e3a600640 1 -- 172.21.15.107:0/2035817854 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e3419b9e0 con 0x7f6e34102790 2024-09-19T04:20:27.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.881+0000 7f6e30c00640 1 -- 172.21.15.107:0/2035817854 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+23119 (secure 0 0 0) 0x7f6e1c0a0a90 con 0x7f6e34102790 2024-09-19T04:20:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:20:27.883 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":84,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:20:19.150785+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6800/3991722475":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/3866113757":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/1307215535":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/3489252098":"2024-09-20T04:20:19.150743+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/3991722475":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/3889998535":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:20:27.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.883+0000 7f6e3a600640 1 -- 172.21.15.107:0/2035817854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6e180a34c0 msgr2=0x7f6e180a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.883+0000 7f6e3a600640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6e180a34c0 0x7f6e180a5980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6e24000c00 tx=0x7f6e24005b50 comp rx=0 tx=0).stop 2024-09-19T04:20:27.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.883+0000 7f6e3a600640 1 -- 172.21.15.107:0/2035817854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e34102790 msgr2=0x7f6e3419abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.883+0000 7f6e3a600640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e34102790 0x7f6e3419abc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00e9f0 tx=0x7f6e1c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.884+0000 7f6e3a600640 1 -- 172.21.15.107:0/2035817854 shutdown_connections 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.884+0000 7f6e3a600640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6e180a34c0 0x7f6e180a5980 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.884+0000 7f6e3a600640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e34106f10 0x7f6e3419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.884+0000 7f6e3a600640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e34102790 0x7f6e3419abc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.884+0000 7f6e3a600640 1 --2- 172.21.15.107:0/2035817854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e34101de0 0x7f6e3419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.884+0000 7f6e3a600640 1 -- 172.21.15.107:0/2035817854 >> 172.21.15.107:0/2035817854 conn(0x7f6e340fd8c0 msgr2=0x7f6e340ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.884+0000 7f6e3a600640 1 -- 172.21.15.107:0/2035817854 shutdown_connections 2024-09-19T04:20:27.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:27.884+0000 7f6e3a600640 1 -- 172.21.15.107:0/2035817854 wait complete. 2024-09-19T04:20:28.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:28 smithi107 ceph-mon[30659]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-19T04:20:28.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:28 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2035817854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:20:28.616 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T04:20:28.617 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T04:20:28.626 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:20:28.626 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-09-19T04:20:28.626 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T04:20:28.627 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T04:20:28.627 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:20:28.627 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:20:28.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:28 smithi115 ceph-mon[35477]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-19T04:20:28.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:28 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2035817854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:20:28.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:28 smithi145 ceph-mon[35096]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-19T04:20:28.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:28 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2035817854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:20:28.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.784+0000 7fbccdc00640 1 -- 172.21.15.107:0/755746702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc810e170 msgr2=0x7fbcc8110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.784+0000 7fbccdc00640 1 --2- 172.21.15.107:0/755746702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc810e170 0x7fbcc8110560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009ca0 tx=0x7fbcc002a950 comp rx=0 tx=0).stop 2024-09-19T04:20:28.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.784+0000 7fbccdc00640 1 -- 172.21.15.107:0/755746702 shutdown_connections 2024-09-19T04:20:28.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.784+0000 7fbccdc00640 1 --2- 172.21.15.107:0/755746702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc810e170 0x7fbcc8110560 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.784+0000 7fbccdc00640 1 --2- 172.21.15.107:0/755746702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc8074ec0 0x7fbcc810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.784+0000 7fbccdc00640 1 --2- 172.21.15.107:0/755746702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc80745a0 0x7fbcc8074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.784+0000 7fbccdc00640 1 -- 172.21.15.107:0/755746702 >> 172.21.15.107:0/755746702 conn(0x7fbcc80fd5d0 msgr2=0x7fbcc80ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:28.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.784+0000 7fbccdc00640 1 -- 172.21.15.107:0/755746702 shutdown_connections 2024-09-19T04:20:28.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.785+0000 7fbccdc00640 1 -- 172.21.15.107:0/755746702 wait complete. 2024-09-19T04:20:28.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.785+0000 7fbccdc00640 1 Processor -- start 2024-09-19T04:20:28.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbccdc00640 1 -- start start 2024-09-19T04:20:28.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbccdc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc80745a0 0x7fbcc819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbccdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc8074ec0 0x7fbcc819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbccdc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc810e170 0x7fbcc819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbccdc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8113eb0 con 0x7fbcc8074ec0 2024-09-19T04:20:28.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbccdc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8113d30 con 0x7fbcc810e170 2024-09-19T04:20:28.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbccdc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8114030 con 0x7fbcc80745a0 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc8074ec0 0x7fbcc819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc8074ec0 0x7fbcc819a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54312/0 (socket says 172.21.15.107:54312) 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbcbfe00640 1 -- 172.21.15.107:0/2219344555 learned_addr learned my addr 172.21.15.107:0/2219344555 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbcc7e00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc80745a0 0x7fbcc819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.786+0000 7fbccd200640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc810e170 0x7fbcc819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.787+0000 7fbcbfe00640 1 -- 172.21.15.107:0/2219344555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc80745a0 msgr2=0x7fbcc819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.787+0000 7fbcbfe00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc80745a0 0x7fbcc819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.787+0000 7fbcbfe00640 1 -- 172.21.15.107:0/2219344555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc810e170 msgr2=0x7fbcc819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.787+0000 7fbcbfe00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc810e170 0x7fbcc819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.787+0000 7fbcbfe00640 1 -- 172.21.15.107:0/2219344555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc819f380 con 0x7fbcc8074ec0 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.787+0000 7fbcbfe00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc8074ec0 0x7fbcc819a8d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d950 tx=0x7fbcb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.787+0000 7fbcc5600640 1 -- 172.21.15.107:0/2219344555 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0017030 con 0x7fbcc8074ec0 2024-09-19T04:20:28.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.787+0000 7fbccdc00640 1 -- 172.21.15.107:0/2219344555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc819f670 con 0x7fbcc8074ec0 2024-09-19T04:20:28.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.787+0000 7fbcc5600640 1 -- 172.21.15.107:0/2219344555 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb0004020 con 0x7fbcc8074ec0 2024-09-19T04:20:28.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.788+0000 7fbccdc00640 1 -- 172.21.15.107:0/2219344555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc81a6f00 con 0x7fbcc8074ec0 2024-09-19T04:20:28.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.787+0000 7fbcc5600640 1 -- 172.21.15.107:0/2219344555 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00047a0 con 0x7fbcc8074ec0 2024-09-19T04:20:28.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.789+0000 7fbccdc00640 1 -- 172.21.15.107:0/2219344555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc8074980 con 0x7fbcc8074ec0 2024-09-19T04:20:28.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.792+0000 7fbcc5600640 1 -- 172.21.15.107:0/2219344555 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7fbcb00041d0 con 0x7fbcc8074ec0 2024-09-19T04:20:28.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.793+0000 7fbcc5600640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbcac0a3440 0x7fbcac0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.793+0000 7fbcc5600640 1 -- 172.21.15.107:0/2219344555 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fbcb00cf660 con 0x7fbcc8074ec0 2024-09-19T04:20:28.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.794+0000 7fbcc7e00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbcac0a3440 0x7fbcac0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.794+0000 7fbcc7e00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbcac0a3440 0x7fbcac0a5900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80095f0 tx=0x7fbcb8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:28.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.798+0000 7fbcc5600640 1 -- 172.21.15.107:0/2219344555 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbcb0022060 con 0x7fbcc8074ec0 2024-09-19T04:20:28.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.982+0000 7fbccdc00640 1 -- 172.21.15.107:0/2219344555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fbcc802d7c0 con 0x7fbcc8074ec0 2024-09-19T04:20:28.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.987+0000 7fbcc5600640 1 -- 172.21.15.107:0/2219344555 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v32)=0 v32) ==== 129+0+0 (secure 0 0 0) 0x7fbcb009bec0 con 0x7fbcc8074ec0 2024-09-19T04:20:28.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.989+0000 7fbccdc00640 1 -- 172.21.15.107:0/2219344555 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbcac0a3440 msgr2=0x7fbcac0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.989+0000 7fbccdc00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbcac0a3440 0x7fbcac0a5900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80095f0 tx=0x7fbcb8005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:28.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.989+0000 7fbccdc00640 1 -- 172.21.15.107:0/2219344555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc8074ec0 msgr2=0x7fbcc819a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.989+0000 7fbccdc00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc8074ec0 0x7fbcc819a8d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d950 tx=0x7fbcb000de10 comp rx=0 tx=0).stop 2024-09-19T04:20:28.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.990+0000 7fbccdc00640 1 -- 172.21.15.107:0/2219344555 shutdown_connections 2024-09-19T04:20:28.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.990+0000 7fbccdc00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbcac0a3440 0x7fbcac0a5900 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.990+0000 7fbccdc00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc810e170 0x7fbcc819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.990+0000 7fbccdc00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc8074ec0 0x7fbcc819a8d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.990+0000 7fbccdc00640 1 --2- 172.21.15.107:0/2219344555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc80745a0 0x7fbcc819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.990+0000 7fbccdc00640 1 -- 172.21.15.107:0/2219344555 >> 172.21.15.107:0/2219344555 conn(0x7fbcc80fd5d0 msgr2=0x7fbcc80fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:28.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.990+0000 7fbccdc00640 1 -- 172.21.15.107:0/2219344555 shutdown_connections 2024-09-19T04:20:28.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:28.990+0000 7fbccdc00640 1 -- 172.21.15.107:0/2219344555 wait complete. 2024-09-19T04:20:29.012 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T04:20:29.019 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:29.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.193+0000 7f0973a00640 1 -- 172.21.15.107:0/1274909263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f096c103f30 msgr2=0x7f096c104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.193+0000 7f0973a00640 1 --2- 172.21.15.107:0/1274909263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f096c103f30 0x7f096c104310 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0960009900 tx=0x7f096002a410 comp rx=0 tx=0).stop 2024-09-19T04:20:29.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.193+0000 7f0973a00640 1 -- 172.21.15.107:0/1274909263 shutdown_connections 2024-09-19T04:20:29.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.193+0000 7f0973a00640 1 --2- 172.21.15.107:0/1274909263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c109080 0x7f096c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.193+0000 7f0973a00640 1 --2- 172.21.15.107:0/1274909263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f096c1048e0 0x7f096c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.193+0000 7f0973a00640 1 --2- 172.21.15.107:0/1274909263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f096c103f30 0x7f096c104310 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.193+0000 7f0973a00640 1 -- 172.21.15.107:0/1274909263 >> 172.21.15.107:0/1274909263 conn(0x7f096c0ff980 msgr2=0x7f096c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.193+0000 7f0973a00640 1 -- 172.21.15.107:0/1274909263 shutdown_connections 2024-09-19T04:20:29.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.193+0000 7f0973a00640 1 -- 172.21.15.107:0/1274909263 wait complete. 2024-09-19T04:20:29.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0973a00640 1 Processor -- start 2024-09-19T04:20:29.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0973a00640 1 -- start start 2024-09-19T04:20:29.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0973a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f096c103f30 0x7f096c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0973a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c109080 0x7f096c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0973a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f096c19ee10 0x7f096c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0973a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f096c1140a0 con 0x7f096c19ee10 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0973a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f096c113f20 con 0x7f096c109080 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0973a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f096c114220 con 0x7f096c103f30 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0971a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f096c19ee10 0x7f096c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f096be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c109080 0x7f096c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0971a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f096c19ee10 0x7f096c19f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39974/0 (socket says 172.21.15.107:39974) 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.194+0000 7f0971a00640 1 -- 172.21.15.107:0/1161688518 learned_addr learned my addr 172.21.15.107:0/1161688518 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0971000640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f096c103f30 0x7f096c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0971a00640 1 -- 172.21.15.107:0/1161688518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f096c103f30 msgr2=0x7f096c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0971a00640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f096c103f30 0x7f096c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0971a00640 1 -- 172.21.15.107:0/1161688518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c109080 msgr2=0x7f096c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0971a00640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c109080 0x7f096c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0971a00640 1 -- 172.21.15.107:0/1161688518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c19f900 con 0x7f096c19ee10 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0971000640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f096c103f30 0x7f096c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:29.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0971a00640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f096c19ee10 0x7f096c19f2c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f096400eaa0 tx=0x7f096400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0969600640 1 -- 172.21.15.107:0/1161688518 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096400cdb0 con 0x7f096c19ee10 2024-09-19T04:20:29.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0969600640 1 -- 172.21.15.107:0/1161688518 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0964005020 con 0x7f096c19ee10 2024-09-19T04:20:29.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0969600640 1 -- 172.21.15.107:0/1161688518 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09640116b0 con 0x7f096c19ee10 2024-09-19T04:20:29.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0973a00640 1 -- 172.21.15.107:0/1161688518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096c19fbf0 con 0x7f096c19ee10 2024-09-19T04:20:29.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.195+0000 7f0973a00640 1 -- 172.21.15.107:0/1161688518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096c1a7390 con 0x7f096c19ee10 2024-09-19T04:20:29.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.196+0000 7f0973a00640 1 -- 172.21.15.107:0/1161688518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096c104310 con 0x7f096c19ee10 2024-09-19T04:20:29.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.198+0000 7f0969600640 1 -- 172.21.15.107:0/1161688518 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7f0964005560 con 0x7f096c19ee10 2024-09-19T04:20:29.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.199+0000 7f0969600640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09540a3500 0x7f09540a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.200+0000 7f0969600640 1 -- 172.21.15.107:0/1161688518 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f09640d00c0 con 0x7f096c19ee10 2024-09-19T04:20:29.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.200+0000 7f0971000640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09540a3500 0x7f09540a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.200+0000 7f0971000640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09540a3500 0x7f09540a59c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0960005ab0 tx=0x7f0960005a20 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.203+0000 7f0969600640 1 -- 172.21.15.107:0/1161688518 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0964016030 con 0x7f096c19ee10 2024-09-19T04:20:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.387+0000 7f0973a00640 1 -- 172.21.15.107:0/1161688518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f096c1048e0 con 0x7f096c19ee10 2024-09-19T04:20:29.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.392+0000 7f0969600640 1 -- 172.21.15.107:0/1161688518 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) ==== 138+0+0 (secure 0 0 0) 0x7f096409c920 con 0x7f096c19ee10 2024-09-19T04:20:29.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.394+0000 7f0973a00640 1 -- 172.21.15.107:0/1161688518 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09540a3500 msgr2=0x7f09540a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.394+0000 7f0973a00640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09540a3500 0x7f09540a59c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0960005ab0 tx=0x7f0960005a20 comp rx=0 tx=0).stop 2024-09-19T04:20:29.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.394+0000 7f0973a00640 1 -- 172.21.15.107:0/1161688518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f096c19ee10 msgr2=0x7f096c19f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.394+0000 7f0973a00640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f096c19ee10 0x7f096c19f2c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f096400eaa0 tx=0x7f096400ef60 comp rx=0 tx=0).stop 2024-09-19T04:20:29.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.394+0000 7f0973a00640 1 -- 172.21.15.107:0/1161688518 shutdown_connections 2024-09-19T04:20:29.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.394+0000 7f0973a00640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09540a3500 0x7f09540a59c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.394+0000 7f0973a00640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f096c19ee10 0x7f096c19f2c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.394+0000 7f0973a00640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c109080 0x7f096c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.394+0000 7f0973a00640 1 --2- 172.21.15.107:0/1161688518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f096c103f30 0x7f096c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.394+0000 7f0973a00640 1 -- 172.21.15.107:0/1161688518 >> 172.21.15.107:0/1161688518 conn(0x7f096c0ff980 msgr2=0x7f096c100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.395+0000 7f0973a00640 1 -- 172.21.15.107:0/1161688518 shutdown_connections 2024-09-19T04:20:29.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.395+0000 7f0973a00640 1 -- 172.21.15.107:0/1161688518 wait complete. 2024-09-19T04:20:29.415 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T04:20:29.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.616+0000 7f243b200640 1 -- 172.21.15.107:0/3382048340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f243c074f40 msgr2=0x7f243c10dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.616+0000 7f243b200640 1 --2- 172.21.15.107:0/3382048340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f243c074f40 0x7f243c10dbb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2428009900 tx=0x7f242802a480 comp rx=0 tx=0).stop 2024-09-19T04:20:29.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.616+0000 7f243b200640 1 -- 172.21.15.107:0/3382048340 shutdown_connections 2024-09-19T04:20:29.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.616+0000 7f243b200640 1 --2- 172.21.15.107:0/3382048340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f243c10e0f0 0x7f243c110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.616+0000 7f243b200640 1 --2- 172.21.15.107:0/3382048340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f243c074f40 0x7f243c10dbb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.616+0000 7f243b200640 1 --2- 172.21.15.107:0/3382048340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f243c074620 0x7f243c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.616+0000 7f243b200640 1 -- 172.21.15.107:0/3382048340 >> 172.21.15.107:0/3382048340 conn(0x7f243c0fd5d0 msgr2=0x7f243c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.616+0000 7f243b200640 1 -- 172.21.15.107:0/3382048340 shutdown_connections 2024-09-19T04:20:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.616+0000 7f243b200640 1 -- 172.21.15.107:0/3382048340 wait complete. 2024-09-19T04:20:29.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.617+0000 7f243b200640 1 Processor -- start 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.617+0000 7f243b200640 1 -- start start 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.617+0000 7f243b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f243c074620 0x7f243c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.617+0000 7f243b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f243c074f40 0x7f243c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.617+0000 7f243b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f243c10e0f0 0x7f243c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.617+0000 7f243b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f243c113ef0 con 0x7f243c10e0f0 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.617+0000 7f243b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f243c113d70 con 0x7f243c074620 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.617+0000 7f243b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f243c114070 con 0x7f243c074f40 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f243a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f243c10e0f0 0x7f243c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f2439e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f243c074620 0x7f243c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f243a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f243c10e0f0 0x7f243c19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39988/0 (socket says 172.21.15.107:39988) 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f243a800640 1 -- 172.21.15.107:0/2916414478 learned_addr learned my addr 172.21.15.107:0/2916414478 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:29.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f2439400640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f243c074f40 0x7f243c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f2439e00640 1 -- 172.21.15.107:0/2916414478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f243c074f40 msgr2=0x7f243c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f2439e00640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f243c074f40 0x7f243c19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f2439e00640 1 -- 172.21.15.107:0/2916414478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f243c10e0f0 msgr2=0x7f243c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f2439e00640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f243c10e0f0 0x7f243c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f2439e00640 1 -- 172.21.15.107:0/2916414478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c19f410 con 0x7f243c074620 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f243a800640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f243c10e0f0 0x7f243c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.618+0000 7f2439e00640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f243c074620 0x7f243c19a420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f242c00e9f0 tx=0x7f242c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.619+0000 7f2432a00640 1 -- 172.21.15.107:0/2916414478 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c00ccc0 con 0x7f243c074620 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.619+0000 7f243b200640 1 -- 172.21.15.107:0/2916414478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c19f700 con 0x7f243c074620 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.619+0000 7f2432a00640 1 -- 172.21.15.107:0/2916414478 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f242c010910 con 0x7f243c074620 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.619+0000 7f2432a00640 1 -- 172.21.15.107:0/2916414478 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c018960 con 0x7f243c074620 2024-09-19T04:20:29.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.619+0000 7f243b200640 1 -- 172.21.15.107:0/2916414478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c1a6f90 con 0x7f243c074620 2024-09-19T04:20:29.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.620+0000 7f243b200640 1 -- 172.21.15.107:0/2916414478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243c074a00 con 0x7f243c074620 2024-09-19T04:20:29.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.622+0000 7f2432a00640 1 -- 172.21.15.107:0/2916414478 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7f242c010ac0 con 0x7f243c074620 2024-09-19T04:20:29.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.623+0000 7f2432a00640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24200a3440 0x7f24200a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.623+0000 7f2432a00640 1 -- 172.21.15.107:0/2916414478 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f242c0d0790 con 0x7f243c074620 2024-09-19T04:20:29.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.623+0000 7f2439400640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24200a3440 0x7f24200a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.624+0000 7f2439400640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24200a3440 0x7f24200a5900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2428000c00 tx=0x7f2428005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.625+0000 7f2432a00640 1 -- 172.21.15.107:0/2916414478 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f242c09c9a0 con 0x7f243c074620 2024-09-19T04:20:29.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.653+0000 7faf9be00640 1 -- 172.21.15.107:0/2154444100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c1079d0 msgr2=0x7faf9c107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.653+0000 7faf9be00640 1 --2- 172.21.15.107:0/2154444100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c1079d0 0x7faf9c107db0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009930 tx=0x7faf8c02a440 comp rx=0 tx=0).stop 2024-09-19T04:20:29.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.654+0000 7faf9be00640 1 -- 172.21.15.107:0/2154444100 shutdown_connections 2024-09-19T04:20:29.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.654+0000 7faf9be00640 1 --2- 172.21.15.107:0/2154444100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf9c102370 0x7faf9c10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.654+0000 7faf9be00640 1 --2- 172.21.15.107:0/2154444100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9c1019d0 0x7faf9c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.654+0000 7faf9be00640 1 --2- 172.21.15.107:0/2154444100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c1079d0 0x7faf9c107db0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.654+0000 7faf9be00640 1 -- 172.21.15.107:0/2154444100 >> 172.21.15.107:0/2154444100 conn(0x7faf9c0fd710 msgr2=0x7faf9c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.654+0000 7faf9be00640 1 -- 172.21.15.107:0/2154444100 shutdown_connections 2024-09-19T04:20:29.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.654+0000 7faf9be00640 1 -- 172.21.15.107:0/2154444100 wait complete. 2024-09-19T04:20:29.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.655+0000 7faf9be00640 1 Processor -- start 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.655+0000 7faf9be00640 1 -- start start 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf9be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf9c1019d0 0x7faf9c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf9be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c102370 0x7faf9c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf9be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9c1079d0 0x7faf9c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf9be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c111e20 con 0x7faf9c1079d0 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf9be00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c111ca0 con 0x7faf9c1019d0 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf9be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c111fa0 con 0x7faf9c102370 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf9b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9c1079d0 0x7faf9c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf93e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c102370 0x7faf9c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf9aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf9c1019d0 0x7faf9c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf93e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c102370 0x7faf9c19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:58254/0 (socket says 172.21.15.107:58254) 2024-09-19T04:20:29.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf93e00640 1 -- 172.21.15.107:0/3861296835 learned_addr learned my addr 172.21.15.107:0/3861296835 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:29.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf93e00640 1 -- 172.21.15.107:0/3861296835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf9c1019d0 msgr2=0x7faf9c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf93e00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf9c1019d0 0x7faf9c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf93e00640 1 -- 172.21.15.107:0/3861296835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9c1079d0 msgr2=0x7faf9c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.656+0000 7faf93e00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9c1079d0 0x7faf9c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.657+0000 7faf93e00640 1 -- 172.21.15.107:0/3861296835 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9c19f440 con 0x7faf9c102370 2024-09-19T04:20:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.657+0000 7faf93e00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c102370 0x7faf9c19a9d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf8400e9f0 tx=0x7faf8400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.657+0000 7faf93400640 1 -- 172.21.15.107:0/3861296835 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8400cd80 con 0x7faf9c102370 2024-09-19T04:20:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.657+0000 7faf93400640 1 -- 172.21.15.107:0/3861296835 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf84010910 con 0x7faf9c102370 2024-09-19T04:20:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.657+0000 7faf9be00640 1 -- 172.21.15.107:0/3861296835 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9c19f730 con 0x7faf9c102370 2024-09-19T04:20:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.657+0000 7faf93400640 1 -- 172.21.15.107:0/3861296835 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84018990 con 0x7faf9c102370 2024-09-19T04:20:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.657+0000 7faf9be00640 1 -- 172.21.15.107:0/3861296835 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9c108830 con 0x7faf9c102370 2024-09-19T04:20:29.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.661+0000 7fafa4619740 1 Processor -- start 2024-09-19T04:20:29.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.661+0000 7fafa4619740 1 -- start start 2024-09-19T04:20:29.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.662+0000 7fafa4619740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 0x55a6d230d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.662+0000 7fafa4619740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d2317950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.662+0000 7fafa4619740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d2317e90 0x55a6d231a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.662+0000 7fafa4619740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a6d2273290 con 0x55a6d230d5d0 2024-09-19T04:20:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.662+0000 7fafa4619740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a6d221ab40 con 0x55a6d230ccb0 2024-09-19T04:20:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.662+0000 7fafa4619740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a6d226f250 con 0x55a6d2317e90 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.662+0000 7faf9aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d2317950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.662+0000 7faf9b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 0x55a6d230d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.662+0000 7faf9aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d2317950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40030/0 (socket says 172.21.15.107:40030) 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.662+0000 7faf9aa00640 1 -- 172.21.15.107:0/1633465840 learned_addr learned my addr 172.21.15.107:0/1633465840 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.663+0000 7faf9aa00640 1 -- 172.21.15.107:0/1633465840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d2317e90 msgr2=0x55a6d231a280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.663+0000 7faf9aa00640 1 --2- 172.21.15.107:0/1633465840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d2317e90 0x55a6d231a280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.663+0000 7faf9aa00640 1 -- 172.21.15.107:0/1633465840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 msgr2=0x55a6d230d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.663+0000 7faf9aa00640 1 --2- 172.21.15.107:0/1633465840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 0x55a6d230d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.663+0000 7faf9aa00640 1 -- 172.21.15.107:0/1633465840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d1fe46a0 con 0x55a6d230d5d0 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.663+0000 7faf93400640 1 -- 172.21.15.107:0/3861296835 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7faf84010ac0 con 0x7faf9c102370 2024-09-19T04:20:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.663+0000 7faf9aa00640 1 --2- 172.21.15.107:0/1633465840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d2317950 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009930 tx=0x7faf8c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc0a685a4e9838a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.664+0000 7faf8be00640 1 -- 172.21.15.107:0/1633465840 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c04d070 con 0x55a6d230d5d0 2024-09-19T04:20:29.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.664+0000 7faf8be00640 1 -- 172.21.15.107:0/1633465840 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf8c03ed30 con 0x55a6d230d5d0 2024-09-19T04:20:29.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.664+0000 7faf8be00640 1 -- 172.21.15.107:0/1633465840 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c035070 con 0x55a6d230d5d0 2024-09-19T04:20:29.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.664+0000 7fafa4619740 1 -- 172.21.15.107:0/1633465840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 msgr2=0x55a6d2317950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.664+0000 7fafa4619740 1 --2- 172.21.15.107:0/1633465840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d2317950 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009930 tx=0x7faf8c005d50 comp rx=0 tx=0).stop 2024-09-19T04:20:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.665+0000 7fafa4619740 1 -- 172.21.15.107:0/1633465840 shutdown_connections 2024-09-19T04:20:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.665+0000 7fafa4619740 1 --2- 172.21.15.107:0/1633465840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d2317e90 0x55a6d231a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.665+0000 7fafa4619740 1 --2- 172.21.15.107:0/1633465840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d2317950 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.665+0000 7fafa4619740 1 --2- 172.21.15.107:0/1633465840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 0x55a6d230d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.665+0000 7fafa4619740 1 -- 172.21.15.107:0/1633465840 >> 172.21.15.107:0/1633465840 conn(0x55a6d2309240 msgr2=0x55a6d2309620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.665+0000 7faf93400640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7faf800a3400 0x7faf800a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.665+0000 7fafa4619740 1 -- 172.21.15.107:0/1633465840 shutdown_connections 2024-09-19T04:20:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.665+0000 7faf93400640 1 -- 172.21.15.107:0/3861296835 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7faf840d0140 con 0x7faf9c102370 2024-09-19T04:20:29.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.665+0000 7fafa4619740 1 -- 172.21.15.107:0/1633465840 wait complete. 2024-09-19T04:20:29.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.666+0000 7faf9aa00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7faf800a3400 0x7faf800a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.666+0000 7fafa4619740 1 -- start start 2024-09-19T04:20:29.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.666+0000 7fafa4619740 1 Processor -- start 2024-09-19T04:20:29.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.666+0000 7faf9aa00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7faf800a3400 0x7faf800a58c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009900 tx=0x7faf8c02a440 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7fafa4619740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d230ccb0 0x55a6d2317730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7fafa4619740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d23149e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7fafa4619740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d2317e90 0x55a6d2314f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7fafa4619740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a6d231ab00 con 0x55a6d230d5d0 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7fafa4619740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a6d222dfa0 con 0x55a6d2317e90 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7fafa4619740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a6d231ac80 con 0x55a6d230ccb0 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7fafa4619740 10 client.0 _ll_register_callbacks cb 0x55a6d1f81ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7faf93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d23149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7faf9aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d230ccb0 0x55a6d2317730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7faf93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d23149e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40042/0 (socket says 172.21.15.107:40042) 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7faf93e00640 1 -- 172.21.15.107:0/828170013 learned_addr learned my addr 172.21.15.107:0/828170013 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:29.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.667+0000 7faf9b400640 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d2317e90 0x55a6d2314f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.668+0000 7faf93e00640 1 -- 172.21.15.107:0/828170013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d230ccb0 msgr2=0x55a6d2317730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.668+0000 7faf93e00640 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d230ccb0 0x55a6d2317730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.668+0000 7faf93e00640 1 -- 172.21.15.107:0/828170013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d2317e90 msgr2=0x55a6d2314f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.668+0000 7faf93e00640 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d2317e90 0x55a6d2314f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.668+0000 7faf93e00640 1 -- 172.21.15.107:0/828170013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d2315720 con 0x55a6d230d5d0 2024-09-19T04:20:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.668+0000 7faf93e00640 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d23149e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faf8400e9c0 tx=0x7faf8409bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50fc89e6a3be03b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.668+0000 7faf88c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:20:29.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.668+0000 7faf88c00640 1 -- 172.21.15.107:0/828170013 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840bc0e0 con 0x55a6d230d5d0 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.668+0000 7faf88c00640 1 -- 172.21.15.107:0/828170013 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7faf840bc280 con 0x55a6d230d5d0 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.668+0000 7fafa4619740 1 -- 172.21.15.107:0/828170013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf8c03ed30 con 0x55a6d230d5d0 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7faf88c00640 1 -- 172.21.15.107:0/828170013 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf840b5150 con 0x55a6d230d5d0 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7fafa4619740 10 client.15258 fetch_fsmap learned FSMap version 28 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7fafa4619740 1 -- 172.21.15.107:0/828170013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6d2326a20 con 0x55a6d230d5d0 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7faf88c00640 1 -- 172.21.15.107:0/828170013 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf8c03ed30 con 0x55a6d230d5d0 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7faf88c00640 1 -- 172.21.15.107:0/828170013 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faf84018c50 con 0x55a6d230d5d0 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7fafa4619740 10 client.15258 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7fafa4619740 10 client.15258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7fafa4619740 20 client.15258 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7fafa4619740 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a6d2327e30 0x55a6d232a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7fafa4619740 4 client.15258 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.669+0000 7fafa4619740 1 -- 172.21.15.107:0/828170013 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6d2280800 con 0x55a6d2327e30 2024-09-19T04:20:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.670+0000 7faf9aa00640 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a6d2327e30 0x55a6d232a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.670+0000 7faf9aa00640 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a6d2327e30 0x55a6d232a220 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2954632e214d60e server_cookie=9384f99bb02d1c4f in_seq=0 out_seq=0 2024-09-19T04:20:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.670+0000 7faf88c00640 10 client.15258 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7faf88c00640 1 -- 172.21.15.107:0/828170013 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6d2280800 con 0x55a6d2327e30 2024-09-19T04:20:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7faf88c00640 10 client.15258 handle_command_reply: tid=0 2024-09-19T04:20:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 client.15258 shutdown 2024-09-19T04:20:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 -- 172.21.15.107:0/828170013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 msgr2=0x55a6d23149e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d23149e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faf8400e9c0 tx=0x7faf8409bda0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 -- 172.21.15.107:0/828170013 shutdown_connections 2024-09-19T04:20:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a6d2327e30 0x55a6d232a220 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d2317e90 0x55a6d2314f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7faf88c00640 0 client.15258 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d23149e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 --2- 172.21.15.107:0/828170013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d230ccb0 0x55a6d2317730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 -- 172.21.15.107:0/828170013 >> 172.21.15.107:0/828170013 conn(0x55a6d2309240 msgr2=0x55a6d230ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 -- 172.21.15.107:0/828170013 shutdown_connections 2024-09-19T04:20:29.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 1 -- 172.21.15.107:0/828170013 wait complete. 2024-09-19T04:20:29.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.671+0000 7fafa4619740 20 client.15258 trim_cache size 0 max 16384 2024-09-19T04:20:29.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.689+0000 7fafa4619740 1 Processor -- start 2024-09-19T04:20:29.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.689+0000 7fafa4619740 1 -- start start 2024-09-19T04:20:29.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7fafa4619740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d230ccb0 0x55a6d23139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7fafa4619740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230d5d0 0x55a6d232fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7fafa4619740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d2317e90 0x55a6d232ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7fafa4619740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1ab400 con 0x55a6d2317e90 2024-09-19T04:20:29.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7fafa4619740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1acf40 con 0x55a6d230d5d0 2024-09-19T04:20:29.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7fafa4619740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1abad0 con 0x55a6d230ccb0 2024-09-19T04:20:29.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7faf9b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d2317e90 0x55a6d232ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7faf9b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d2317e90 0x55a6d232ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40054/0 (socket says 172.21.15.107:40054) 2024-09-19T04:20:29.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7faf9b400640 1 -- 172.21.15.107:0/4185610215 learned_addr learned my addr 172.21.15.107:0/4185610215 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:29.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7faf9aa00640 1 --2- 172.21.15.107:0/4185610215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d230ccb0 0x55a6d23139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7faf93e00640 1 --2- 172.21.15.107:0/4185610215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230d5d0 0x55a6d232fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7faf9b400640 1 -- 172.21.15.107:0/4185610215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d230ccb0 msgr2=0x55a6d23139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7faf9b400640 1 --2- 172.21.15.107:0/4185610215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d230ccb0 0x55a6d23139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7faf9b400640 1 -- 172.21.15.107:0/4185610215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230d5d0 msgr2=0x55a6d232fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7faf9b400640 1 --2- 172.21.15.107:0/4185610215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230d5d0 0x55a6d232fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.690+0000 7faf9b400640 1 -- 172.21.15.107:0/4185610215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d232a850 con 0x55a6d2317e90 2024-09-19T04:20:29.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.691+0000 7faf9b400640 1 --2- 172.21.15.107:0/4185610215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d2317e90 0x55a6d232ffc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faf94003b60 tx=0x7faf94004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a58e59edb4841b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.691+0000 7faf8b400640 1 -- 172.21.15.107:0/4185610215 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94021070 con 0x55a6d2317e90 2024-09-19T04:20:29.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.691+0000 7faf8b400640 1 -- 172.21.15.107:0/4185610215 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf9401cb40 con 0x55a6d2317e90 2024-09-19T04:20:29.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.691+0000 7faf8b400640 1 -- 172.21.15.107:0/4185610215 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94010b00 con 0x55a6d2317e90 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.692+0000 7fafa4619740 1 -- 172.21.15.107:0/4185610215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d2317e90 msgr2=0x55a6d232ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.692+0000 7fafa4619740 1 --2- 172.21.15.107:0/4185610215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d2317e90 0x55a6d232ffc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faf94003b60 tx=0x7faf94004410 comp rx=0 tx=0).stop 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.692+0000 7fafa4619740 1 -- 172.21.15.107:0/4185610215 shutdown_connections 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.692+0000 7fafa4619740 1 --2- 172.21.15.107:0/4185610215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d2317e90 0x55a6d232ffc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.692+0000 7fafa4619740 1 --2- 172.21.15.107:0/4185610215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230d5d0 0x55a6d232fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.692+0000 7fafa4619740 1 --2- 172.21.15.107:0/4185610215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d230ccb0 0x55a6d23139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.692+0000 7fafa4619740 1 -- 172.21.15.107:0/4185610215 >> 172.21.15.107:0/4185610215 conn(0x55a6d2309240 msgr2=0x55a6d23132d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.692+0000 7fafa4619740 1 -- 172.21.15.107:0/4185610215 shutdown_connections 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.692+0000 7fafa4619740 1 -- 172.21.15.107:0/4185610215 wait complete. 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.693+0000 7fafa4619740 1 -- start start 2024-09-19T04:20:29.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.693+0000 7fafa4619740 1 Processor -- start 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.693+0000 7fafa4619740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 0x55a6d231be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.693+0000 7fafa4619740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d231c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.693+0000 7fafa4619740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d2317e90 0x55a6d231c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.693+0000 7fafa4619740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1ac870 con 0x55a6d230d5d0 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.693+0000 7fafa4619740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1ac1a0 con 0x55a6d230ccb0 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.693+0000 7fafa4619740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1a9440 con 0x55a6d2317e90 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.693+0000 7fafa4619740 10 client.0 _ll_register_callbacks cb 0x7faf840cd650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d231c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf9aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 0x55a6d231be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d231c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40064/0 (socket says 172.21.15.107:40064) 2024-09-19T04:20:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf93e00640 1 -- 172.21.15.107:0/2143644267 learned_addr learned my addr 172.21.15.107:0/2143644267 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf9b400640 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d2317e90 0x55a6d231c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf9aa00640 1 -- 172.21.15.107:0/2143644267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d2317e90 msgr2=0x55a6d231c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf9aa00640 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d2317e90 0x55a6d231c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf9aa00640 1 -- 172.21.15.107:0/2143644267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 msgr2=0x55a6d231c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf9aa00640 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d231c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf9aa00640 1 -- 172.21.15.107:0/2143644267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d2391c50 con 0x55a6d230ccb0 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf93e00640 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d231c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf9aa00640 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 0x55a6d231be60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf8c005c40 tx=0x7faf8c005650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=982210576837e763 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.694+0000 7faf88c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7faf88c00640 1 -- 172.21.15.107:0/2143644267 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c035030 con 0x55a6d230ccb0 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7faf88c00640 1 -- 172.21.15.107:0/2143644267 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7faf8c031a80 con 0x55a6d230ccb0 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7fafa4619740 1 -- 172.21.15.107:0/2143644267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf9401cb40 con 0x55a6d230ccb0 2024-09-19T04:20:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7faf88c00640 1 -- 172.21.15.107:0/2143644267 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf8c04d040 con 0x55a6d230ccb0 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7fafa4619740 10 client.25280 fetch_fsmap learned FSMap version 28 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7fafa4619740 1 -- 172.21.15.107:0/2143644267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6d2392f50 con 0x55a6d230ccb0 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7faf88c00640 1 -- 172.21.15.107:0/2143644267 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf9401cb40 con 0x55a6d230ccb0 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7faf88c00640 1 -- 172.21.15.107:0/2143644267 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faf8c047070 con 0x55a6d230ccb0 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7fafa4619740 10 client.25280 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7fafa4619740 10 client.25280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7fafa4619740 20 client.25280 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7fafa4619740 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a6d2393290 0x55a6d2395680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7fafa4619740 4 client.25280 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.695+0000 7fafa4619740 1 -- 172.21.15.107:0/2143644267 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a6d227ad10 con 0x55a6d2393290 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.696+0000 7faf93e00640 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a6d2393290 0x55a6d2395680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.696+0000 7faf93e00640 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a6d2393290 0x55a6d2395680 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe669dd30c70b0ba server_cookie=e38ed649def5494c in_seq=0 out_seq=0 2024-09-19T04:20:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.696+0000 7faf88c00640 10 client.25280 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:29.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.697+0000 7faf88c00640 1 -- 172.21.15.107:0/2143644267 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a6d227ad10 con 0x55a6d2393290 2024-09-19T04:20:29.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.697+0000 7faf88c00640 10 client.25280 handle_command_reply: tid=0 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.697+0000 7fafa4619740 1 client.25280 shutdown 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 1 -- 172.21.15.107:0/2143644267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 msgr2=0x55a6d231be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 0x55a6d231be60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf8c005c40 tx=0x7faf8c005650 comp rx=0 tx=0).stop 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 1 -- 172.21.15.107:0/2143644267 shutdown_connections 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a6d2393290 0x55a6d2395680 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a6d2317e90 0x55a6d231c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6d230d5d0 0x55a6d231c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 1 --2- 172.21.15.107:0/2143644267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d230ccb0 0x55a6d231be60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 1 -- 172.21.15.107:0/2143644267 >> 172.21.15.107:0/2143644267 conn(0x55a6d2309240 msgr2=0x55a6d2313af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7faf88c00640 0 client.25280 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 1 -- 172.21.15.107:0/2143644267 shutdown_connections 2024-09-19T04:20:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 1 -- 172.21.15.107:0/2143644267 wait complete. 2024-09-19T04:20:29.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.698+0000 7fafa4619740 20 client.25280 trim_cache size 0 max 16384 2024-09-19T04:20:29.703 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:20:29.703 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:20:29.703 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "3b41de20-1b6e-448f-9d84-5e5c2c25eae0", 2024-09-19T04:20:29.703 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:20:29.703 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:20:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 -- 172.21.15.107:0/3861296835 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7faf800a3400 msgr2=0x7faf800a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7faf800a3400 0x7faf800a58c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009900 tx=0x7faf8c02a440 comp rx=0 tx=0).stop 2024-09-19T04:20:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 -- 172.21.15.107:0/3861296835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c102370 msgr2=0x7faf9c19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c102370 0x7faf9c19a9d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf8400e9f0 tx=0x7faf8400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 -- 172.21.15.107:0/3861296835 shutdown_connections 2024-09-19T04:20:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7faf800a3400 0x7faf800a58c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9c1079d0 0x7faf9c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c102370 0x7faf9c19a9d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 --2- 172.21.15.107:0/3861296835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf9c1019d0 0x7faf9c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 -- 172.21.15.107:0/3861296835 >> 172.21.15.107:0/3861296835 conn(0x7faf9c0fd710 msgr2=0x7faf9c10cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 -- 172.21.15.107:0/3861296835 shutdown_connections 2024-09-19T04:20:29.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.699+0000 7faf9be00640 1 -- 172.21.15.107:0/3861296835 wait complete. 2024-09-19T04:20:29.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.832+0000 7f243b200640 1 -- 172.21.15.107:0/2916414478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f243c19b770 con 0x7f243c074620 2024-09-19T04:20:29.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.839+0000 7f2432a00640 1 -- 172.21.15.107:0/2916414478 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7f242c0a0ca0 con 0x7f243c074620 2024-09-19T04:20:29.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.841+0000 7f243b200640 1 -- 172.21.15.107:0/2916414478 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24200a3440 msgr2=0x7f24200a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.841+0000 7f243b200640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24200a3440 0x7f24200a5900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2428000c00 tx=0x7f2428005b50 comp rx=0 tx=0).stop 2024-09-19T04:20:29.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.841+0000 7f243b200640 1 -- 172.21.15.107:0/2916414478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f243c074620 msgr2=0x7f243c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.841+0000 7f243b200640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f243c074620 0x7f243c19a420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f242c00e9f0 tx=0x7f242c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.841+0000 7f243b200640 1 -- 172.21.15.107:0/2916414478 shutdown_connections 2024-09-19T04:20:29.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.841+0000 7f243b200640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24200a3440 0x7f24200a5900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.841+0000 7f243b200640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f243c10e0f0 0x7f243c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.841+0000 7f243b200640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f243c074f40 0x7f243c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.841+0000 7f243b200640 1 --2- 172.21.15.107:0/2916414478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f243c074620 0x7f243c19a420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.842+0000 7f243b200640 1 -- 172.21.15.107:0/2916414478 >> 172.21.15.107:0/2916414478 conn(0x7f243c0fd5d0 msgr2=0x7f243c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.842+0000 7f243b200640 1 -- 172.21.15.107:0/2916414478 shutdown_connections 2024-09-19T04:20:29.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:29.842+0000 7f243b200640 1 -- 172.21.15.107:0/2916414478 wait complete. 2024-09-19T04:20:29.861 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T04:20:30.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.019+0000 7f7735c00640 1 -- 172.21.15.107:0/2161884956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77301078c0 msgr2=0x7f7730107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.019+0000 7f7735c00640 1 --2- 172.21.15.107:0/2161884956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77301078c0 0x7f7730107ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7720009930 tx=0x7f772002a420 comp rx=0 tx=0).stop 2024-09-19T04:20:30.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.019+0000 7f7735c00640 1 -- 172.21.15.107:0/2161884956 shutdown_connections 2024-09-19T04:20:30.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.019+0000 7f7735c00640 1 --2- 172.21.15.107:0/2161884956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730102250 0x7f773010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.019+0000 7f7735c00640 1 --2- 172.21.15.107:0/2161884956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77301018b0 0x7f7730101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.019+0000 7f7735c00640 1 --2- 172.21.15.107:0/2161884956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77301078c0 0x7f7730107ca0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.019+0000 7f7735c00640 1 -- 172.21.15.107:0/2161884956 >> 172.21.15.107:0/2161884956 conn(0x7f77300fd5d0 msgr2=0x7f77300ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:30.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.019+0000 7f7735c00640 1 -- 172.21.15.107:0/2161884956 shutdown_connections 2024-09-19T04:20:30.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.019+0000 7f7735c00640 1 -- 172.21.15.107:0/2161884956 wait complete. 2024-09-19T04:20:30.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.020+0000 7f7735c00640 1 Processor -- start 2024-09-19T04:20:30.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.020+0000 7f7735c00640 1 -- start start 2024-09-19T04:20:30.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.020+0000 7f7735c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77301018b0 0x7f773019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.020+0000 7f7735c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730102250 0x7f773019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.020+0000 7f7735c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77301078c0 0x7f773019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.020+0000 7f7735c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7730111d00 con 0x7f77301018b0 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.020+0000 7f7735c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7730111b80 con 0x7f7730102250 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.020+0000 7f7735c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7730111e80 con 0x7f77301078c0 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.020+0000 7f772fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77301018b0 0x7f773019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77301018b0 0x7f773019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40084/0 (socket says 172.21.15.107:40084) 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772fe00640 1 -- 172.21.15.107:0/327078274 learned_addr learned my addr 172.21.15.107:0/327078274 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f7735200640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77301078c0 0x7f773019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772f400640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730102250 0x7f773019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772fe00640 1 -- 172.21.15.107:0/327078274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77301078c0 msgr2=0x7f773019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772fe00640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77301078c0 0x7f773019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772fe00640 1 -- 172.21.15.107:0/327078274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730102250 msgr2=0x7f773019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772fe00640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730102250 0x7f773019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772fe00640 1 -- 172.21.15.107:0/327078274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773019f440 con 0x7f77301018b0 2024-09-19T04:20:30.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772fe00640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77301018b0 0x7f773019a450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7720009230 tx=0x7f7720004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:30.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772cc00640 1 -- 172.21.15.107:0/327078274 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772003f030 con 0x7f77301018b0 2024-09-19T04:20:30.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772cc00640 1 -- 172.21.15.107:0/327078274 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7720004440 con 0x7f77301018b0 2024-09-19T04:20:30.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f772cc00640 1 -- 172.21.15.107:0/327078274 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7720035070 con 0x7f77301018b0 2024-09-19T04:20:30.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.021+0000 7f7735c00640 1 -- 172.21.15.107:0/327078274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773019f6d0 con 0x7f77301018b0 2024-09-19T04:20:30.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.022+0000 7f7735c00640 1 -- 172.21.15.107:0/327078274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77301a6fc0 con 0x7f77301018b0 2024-09-19T04:20:30.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.023+0000 7f7735c00640 1 -- 172.21.15.107:0/327078274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7730102ff0 con 0x7f77301018b0 2024-09-19T04:20:30.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.025+0000 7f772cc00640 1 -- 172.21.15.107:0/327078274 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7f7720042030 con 0x7f77301018b0 2024-09-19T04:20:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.027+0000 7f772cc00640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77140a34c0 0x7f77140a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.027+0000 7f772cc00640 1 -- 172.21.15.107:0/327078274 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f77200f0110 con 0x7f77301018b0 2024-09-19T04:20:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.027+0000 7f772f400640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77140a34c0 0x7f77140a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.027+0000 7f772f400640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77140a34c0 0x7f77140a5980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f773010eb80 tx=0x7f771c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.031+0000 7f772cc00640 1 -- 172.21.15.107:0/327078274 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f77200bc320 con 0x7f77301018b0 2024-09-19T04:20:30.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2219344555' entity='client.admin' 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1161688518' entity='client.admin' 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:20:30.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:30.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:20:30.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:30.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' 2024-09-19T04:20:30.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:30.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:30.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:29 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:30.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.232+0000 7f7735c00640 1 -- 172.21.15.107:0/327078274 --> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f77300fe3b0 con 0x7f77140a34c0 2024-09-19T04:20:30.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.302+0000 7f772cc00640 1 -- 172.21.15.107:0/327078274 <== mgr.15114 v2:172.21.15.107:6800/1724794486 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f77300fe3b0 con 0x7f77140a34c0 2024-09-19T04:20:30.303 INFO:teuthology.orchestra.run.smithi107.stdout:Schedule set for path / 2024-09-19T04:20:30.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 -- 172.21.15.107:0/327078274 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77140a34c0 msgr2=0x7f77140a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77140a34c0 0x7f77140a5980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f773010eb80 tx=0x7f771c005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 -- 172.21.15.107:0/327078274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77301018b0 msgr2=0x7f773019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77301018b0 0x7f773019a450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7720009230 tx=0x7f7720004040 comp rx=0 tx=0).stop 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 -- 172.21.15.107:0/327078274 shutdown_connections 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77140a34c0 0x7f77140a5980 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77301078c0 0x7f773019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730102250 0x7f773019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 --2- 172.21.15.107:0/327078274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77301018b0 0x7f773019a450 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 -- 172.21.15.107:0/327078274 >> 172.21.15.107:0/327078274 conn(0x7f77300fd5d0 msgr2=0x7f77300fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 -- 172.21.15.107:0/327078274 shutdown_connections 2024-09-19T04:20:30.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.304+0000 7f7735c00640 1 -- 172.21.15.107:0/327078274 wait complete. 2024-09-19T04:20:30.321 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T04:20:30.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:20:30.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2219344555' entity='client.admin' 2024-09-19T04:20:30.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:30.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1161688518' entity='client.admin' 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' 2024-09-19T04:20:30.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:30.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:30.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:29 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:30.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2219344555' entity='client.admin' 2024-09-19T04:20:30.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:30.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:30.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:30.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1161688518' entity='client.admin' 2024-09-19T04:20:30.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:30.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:30.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:30.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:20:30.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:30.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:20:30.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:20:30.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' 2024-09-19T04:20:30.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:20:30.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:20:30.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:20:30.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:29 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:20:30.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.481+0000 7f4daea00640 1 -- 172.21.15.107:0/3113941375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da8101ae0 msgr2=0x7f4da8101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.481+0000 7f4daea00640 1 --2- 172.21.15.107:0/3113941375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da8101ae0 0x7f4da8101f40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009930 tx=0x7f4d9002a420 comp rx=0 tx=0).stop 2024-09-19T04:20:30.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.481+0000 7f4daea00640 1 -- 172.21.15.107:0/3113941375 shutdown_connections 2024-09-19T04:20:30.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.481+0000 7f4daea00640 1 --2- 172.21.15.107:0/3113941375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4da8102480 0x7f4da810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.481+0000 7f4daea00640 1 --2- 172.21.15.107:0/3113941375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da8101ae0 0x7f4da8101f40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.481+0000 7f4daea00640 1 --2- 172.21.15.107:0/3113941375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da8107b80 0x7f4da8107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.481+0000 7f4daea00640 1 -- 172.21.15.107:0/3113941375 >> 172.21.15.107:0/3113941375 conn(0x7f4da80fd7e0 msgr2=0x7f4da80ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:30.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.481+0000 7f4daea00640 1 -- 172.21.15.107:0/3113941375 shutdown_connections 2024-09-19T04:20:30.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.481+0000 7f4daea00640 1 -- 172.21.15.107:0/3113941375 wait complete. 2024-09-19T04:20:30.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.482+0000 7f4daea00640 1 Processor -- start 2024-09-19T04:20:30.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.482+0000 7f4daea00640 1 -- start start 2024-09-19T04:20:30.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.482+0000 7f4daea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da8101ae0 0x7f4da80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.482+0000 7f4daea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da8102480 0x7f4da8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.482+0000 7f4daea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4da8107b80 0x7f4da8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.482+0000 7f4daea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8111f30 con 0x7f4da8102480 2024-09-19T04:20:30.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.482+0000 7f4daea00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8111db0 con 0x7f4da8107b80 2024-09-19T04:20:30.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.482+0000 7f4daea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81120b0 con 0x7f4da8101ae0 2024-09-19T04:20:30.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4da7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da8101ae0 0x7f4da80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4da7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da8101ae0 0x7f4da80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:58328/0 (socket says 172.21.15.107:58328) 2024-09-19T04:20:30.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4d9fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da8102480 0x7f4da8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4da7e00640 1 -- 172.21.15.107:0/3477291326 learned_addr learned my addr 172.21.15.107:0/3477291326 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:30.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4daca00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4da8107b80 0x7f4da8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4da7e00640 1 -- 172.21.15.107:0/3477291326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4da8107b80 msgr2=0x7f4da8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4da7e00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4da8107b80 0x7f4da8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4da7e00640 1 -- 172.21.15.107:0/3477291326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da8102480 msgr2=0x7f4da8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4da7e00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da8102480 0x7f4da8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4da7e00640 1 -- 172.21.15.107:0/3477291326 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da81a6920 con 0x7f4da8101ae0 2024-09-19T04:20:30.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.483+0000 7f4da7e00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da8101ae0 0x7f4da80744f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800e9f0 tx=0x7f4d9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:30.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.484+0000 7f4da5600640 1 -- 172.21.15.107:0/3477291326 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800cd80 con 0x7f4da8101ae0 2024-09-19T04:20:30.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.484+0000 7f4daea00640 1 -- 172.21.15.107:0/3477291326 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da81a6c10 con 0x7f4da8101ae0 2024-09-19T04:20:30.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.484+0000 7f4da5600640 1 -- 172.21.15.107:0/3477291326 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d98010910 con 0x7f4da8101ae0 2024-09-19T04:20:30.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.484+0000 7f4da5600640 1 -- 172.21.15.107:0/3477291326 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d980188f0 con 0x7f4da8101ae0 2024-09-19T04:20:30.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.484+0000 7f4daea00640 1 -- 172.21.15.107:0/3477291326 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da81a7180 con 0x7f4da8101ae0 2024-09-19T04:20:30.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.485+0000 7f4daea00640 1 -- 172.21.15.107:0/3477291326 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da8101f40 con 0x7f4da8101ae0 2024-09-19T04:20:30.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.488+0000 7f4da5600640 1 -- 172.21.15.107:0/3477291326 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7f4d98010ac0 con 0x7f4da8101ae0 2024-09-19T04:20:30.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.489+0000 7f4da5600640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d8c0a3580 0x7f4d8c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.489+0000 7f4da5600640 1 -- 172.21.15.107:0/3477291326 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4d980d02d0 con 0x7f4da8101ae0 2024-09-19T04:20:30.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.490+0000 7f4d9fe00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d8c0a3580 0x7f4d8c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.490+0000 7f4d9fe00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d8c0a3580 0x7f4d8c0a5a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009900 tx=0x7f4d900058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:30.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.490+0000 7f4da5600640 1 -- 172.21.15.107:0/3477291326 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4d9809c600 con 0x7f4da8101ae0 2024-09-19T04:20:30.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.698+0000 7f4daea00640 1 -- 172.21.15.107:0/3477291326 --> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f4da8062630 con 0x7f4d8c0a3580 2024-09-19T04:20:30.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.768+0000 7f4da5600640 1 -- 172.21.15.107:0/3477291326 <== mgr.15114 v2:172.21.15.107:6800/1724794486 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f4da8062630 con 0x7f4d8c0a3580 2024-09-19T04:20:30.769 INFO:teuthology.orchestra.run.smithi107.stdout:Retention added to path / 2024-09-19T04:20:30.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.770+0000 7f4daea00640 1 -- 172.21.15.107:0/3477291326 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d8c0a3580 msgr2=0x7f4d8c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.770+0000 7f4daea00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d8c0a3580 0x7f4d8c0a5a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009900 tx=0x7f4d900058d0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.771+0000 7f4daea00640 1 -- 172.21.15.107:0/3477291326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da8101ae0 msgr2=0x7f4da80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.771+0000 7f4daea00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da8101ae0 0x7f4da80744f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800e9f0 tx=0x7f4d9800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.771+0000 7f4daea00640 1 -- 172.21.15.107:0/3477291326 shutdown_connections 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.771+0000 7f4daea00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d8c0a3580 0x7f4d8c0a5a40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.771+0000 7f4daea00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4da8107b80 0x7f4da8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.771+0000 7f4daea00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da8102480 0x7f4da8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.771+0000 7f4daea00640 1 --2- 172.21.15.107:0/3477291326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da8101ae0 0x7f4da80744f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.771+0000 7f4daea00640 1 -- 172.21.15.107:0/3477291326 >> 172.21.15.107:0/3477291326 conn(0x7f4da80fd7e0 msgr2=0x7f4da80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.771+0000 7f4daea00640 1 -- 172.21.15.107:0/3477291326 shutdown_connections 2024-09-19T04:20:30.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.771+0000 7f4daea00640 1 -- 172.21.15.107:0/3477291326 wait complete. 2024-09-19T04:20:30.788 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T04:20:30.828 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:20:30.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.985+0000 7efe8b600640 1 -- 172.21.15.107:0/2738650258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe84102460 msgr2=0x7efe8410e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.985+0000 7efe8b600640 1 --2- 172.21.15.107:0/2738650258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe84102460 0x7efe8410e990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009c10 tx=0x7efe7c02a6e0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.985+0000 7efe8b600640 1 -- 172.21.15.107:0/2738650258 shutdown_connections 2024-09-19T04:20:30.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.985+0000 7efe8b600640 1 --2- 172.21.15.107:0/2738650258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe84102460 0x7efe8410e990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.985+0000 7efe8b600640 1 --2- 172.21.15.107:0/2738650258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe84101ac0 0x7efe84101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.985+0000 7efe8b600640 1 --2- 172.21.15.107:0/2738650258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe84107ad0 0x7efe84107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.985+0000 7efe8b600640 1 -- 172.21.15.107:0/2738650258 >> 172.21.15.107:0/2738650258 conn(0x7efe840fd7e0 msgr2=0x7efe840ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:30.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.985+0000 7efe8b600640 1 -- 172.21.15.107:0/2738650258 shutdown_connections 2024-09-19T04:20:30.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.985+0000 7efe8b600640 1 -- 172.21.15.107:0/2738650258 wait complete. 2024-09-19T04:20:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.987+0000 7efe8b600640 1 Processor -- start 2024-09-19T04:20:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.987+0000 7efe8b600640 1 -- start start 2024-09-19T04:20:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.987+0000 7efe8b600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe84101ac0 0x7efe8419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.987+0000 7efe8b600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe84102460 0x7efe8419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.987+0000 7efe8b600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe84107ad0 0x7efe8419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.987+0000 7efe8b600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe84111f10 con 0x7efe84101ac0 2024-09-19T04:20:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.987+0000 7efe8b600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe84111d90 con 0x7efe84107ad0 2024-09-19T04:20:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.987+0000 7efe8b600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efe84112090 con 0x7efe84102460 2024-09-19T04:20:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.987+0000 7efe88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe84101ac0 0x7efe8419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe88c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe84101ac0 0x7efe8419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40112/0 (socket says 172.21.15.107:40112) 2024-09-19T04:20:30.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe88c00640 1 -- 172.21.15.107:0/2141312562 learned_addr learned my addr 172.21.15.107:0/2141312562 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:30.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe83e00640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe84102460 0x7efe8419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe88c00640 1 -- 172.21.15.107:0/2141312562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe84102460 msgr2=0x7efe8419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe88c00640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe84102460 0x7efe8419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe88c00640 1 -- 172.21.15.107:0/2141312562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe84107ad0 msgr2=0x7efe8419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe89600640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe84107ad0 0x7efe8419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe88c00640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe84107ad0 0x7efe8419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe88c00640 1 -- 172.21.15.107:0/2141312562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8419f4f0 con 0x7efe84101ac0 2024-09-19T04:20:30.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe89600640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe84107ad0 0x7efe8419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:30.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.988+0000 7efe88c00640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe84101ac0 0x7efe8419a500 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efe7800e9c0 tx=0x7efe7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:30.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.989+0000 7efe81600640 1 -- 172.21.15.107:0/2141312562 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7800cde0 con 0x7efe84101ac0 2024-09-19T04:20:30.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.989+0000 7efe81600640 1 -- 172.21.15.107:0/2141312562 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe780035f0 con 0x7efe84101ac0 2024-09-19T04:20:30.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.989+0000 7efe8b600640 1 -- 172.21.15.107:0/2141312562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8419f7e0 con 0x7efe84101ac0 2024-09-19T04:20:30.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.989+0000 7efe81600640 1 -- 172.21.15.107:0/2141312562 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78010690 con 0x7efe84101ac0 2024-09-19T04:20:30.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.989+0000 7efe8b600640 1 -- 172.21.15.107:0/2141312562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe841a7070 con 0x7efe84101ac0 2024-09-19T04:20:30.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.990+0000 7efe8b600640 1 -- 172.21.15.107:0/2141312562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe84103200 con 0x7efe84101ac0 2024-09-19T04:20:30.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.992+0000 7efe81600640 1 -- 172.21.15.107:0/2141312562 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7efe780037a0 con 0x7efe84101ac0 2024-09-19T04:20:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.994+0000 7efe81600640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efe6c0a3500 0x7efe6c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.994+0000 7efe81600640 1 -- 172.21.15.107:0/2141312562 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7efe780d01f0 con 0x7efe84101ac0 2024-09-19T04:20:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.994+0000 7efe83e00640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efe6c0a3500 0x7efe6c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.995+0000 7efe83e00640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efe6c0a3500 0x7efe6c0a59c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe8410ebf0 tx=0x7efe70005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:30.997+0000 7efe81600640 1 -- 172.21.15.107:0/2141312562 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efe7809ca50 con 0x7efe84101ac0 2024-09-19T04:20:31.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:31 smithi107 ceph-mon[30659]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-19T04:20:31.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.202+0000 7efe8b600640 1 -- 172.21.15.107:0/2141312562 --> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7efe840fed10 con 0x7efe6c0a3500 2024-09-19T04:20:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.209+0000 7efe81600640 1 -- 172.21.15.107:0/2141312562 <== mgr.15114 v2:172.21.15.107:6800/1724794486 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7efe840fed10 con 0x7efe6c0a3500 2024-09-19T04:20:31.210 INFO:teuthology.orchestra.run.smithi107.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T04:20:30", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T04:20:31.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 -- 172.21.15.107:0/2141312562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efe6c0a3500 msgr2=0x7efe6c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efe6c0a3500 0x7efe6c0a59c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe8410ebf0 tx=0x7efe70005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:31.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 -- 172.21.15.107:0/2141312562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe84101ac0 msgr2=0x7efe8419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe84101ac0 0x7efe8419a500 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efe7800e9c0 tx=0x7efe7800ee80 comp rx=0 tx=0).stop 2024-09-19T04:20:31.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 -- 172.21.15.107:0/2141312562 shutdown_connections 2024-09-19T04:20:31.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efe6c0a3500 0x7efe6c0a59c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe84107ad0 0x7efe8419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe84102460 0x7efe8419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 --2- 172.21.15.107:0/2141312562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe84101ac0 0x7efe8419a500 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 -- 172.21.15.107:0/2141312562 >> 172.21.15.107:0/2141312562 conn(0x7efe840fd7e0 msgr2=0x7efe8410d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 -- 172.21.15.107:0/2141312562 shutdown_connections 2024-09-19T04:20:31.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.212+0000 7efe8b600640 1 -- 172.21.15.107:0/2141312562 wait complete. 2024-09-19T04:20:31.230 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T04:20:31.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.375+0000 7fd6eb000640 1 -- 172.21.15.107:0/3625233637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6e41027a0 msgr2=0x7fd6e41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.375+0000 7fd6eb000640 1 --2- 172.21.15.107:0/3625233637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6e41027a0 0x7fd6e41067f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009900 tx=0x7fd6cc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.375+0000 7fd6eb000640 1 -- 172.21.15.107:0/3625233637 shutdown_connections 2024-09-19T04:20:31.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.375+0000 7fd6eb000640 1 --2- 172.21.15.107:0/3625233637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6e4106f20 0x7fd6e410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.375+0000 7fd6eb000640 1 --2- 172.21.15.107:0/3625233637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6e41027a0 0x7fd6e41067f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.375+0000 7fd6eb000640 1 --2- 172.21.15.107:0/3625233637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e4101df0 0x7fd6e41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.375+0000 7fd6eb000640 1 -- 172.21.15.107:0/3625233637 >> 172.21.15.107:0/3625233637 conn(0x7fd6e40fd8b0 msgr2=0x7fd6e40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.375+0000 7fd6eb000640 1 -- 172.21.15.107:0/3625233637 shutdown_connections 2024-09-19T04:20:31.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.375+0000 7fd6eb000640 1 -- 172.21.15.107:0/3625233637 wait complete. 2024-09-19T04:20:31.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.376+0000 7fd6eb000640 1 Processor -- start 2024-09-19T04:20:31.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.376+0000 7fd6eb000640 1 -- start start 2024-09-19T04:20:31.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.376+0000 7fd6eb000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e4101df0 0x7fd6e40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.376+0000 7fd6eb000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6e41027a0 0x7fd6e4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.376+0000 7fd6eb000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6e4106f20 0x7fd6e4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.376+0000 7fd6eb000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4111f80 con 0x7fd6e4106f20 2024-09-19T04:20:31.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.376+0000 7fd6eb000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4111e00 con 0x7fd6e4101df0 2024-09-19T04:20:31.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.376+0000 7fd6eb000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4112100 con 0x7fd6e41027a0 2024-09-19T04:20:31.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6e41027a0 0x7fd6e4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6e41027a0 0x7fd6e4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:58344/0 (socket says 172.21.15.107:58344) 2024-09-19T04:20:31.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6dbe00640 1 -- 172.21.15.107:0/2637049745 learned_addr learned my addr 172.21.15.107:0/2637049745 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:31.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e4101df0 0x7fd6e40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6e9000640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6e4106f20 0x7fd6e4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6dbe00640 1 -- 172.21.15.107:0/2637049745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e4101df0 msgr2=0x7fd6e40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6dbe00640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e4101df0 0x7fd6e40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6dbe00640 1 -- 172.21.15.107:0/2637049745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6e4106f20 msgr2=0x7fd6e4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6dbe00640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6e4106f20 0x7fd6e4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6dbe00640 1 -- 172.21.15.107:0/2637049745 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e41a68a0 con 0x7fd6e41027a0 2024-09-19T04:20:31.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.377+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e4101df0 0x7fd6e40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:31.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.378+0000 7fd6dbe00640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6e41027a0 0x7fd6e4074a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc006010 tx=0x7fd6cc004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.378+0000 7fd6e1600640 1 -- 172.21.15.107:0/2637049745 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc03f030 con 0x7fd6e41027a0 2024-09-19T04:20:31.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.378+0000 7fd6eb000640 1 -- 172.21.15.107:0/2637049745 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e41a6b30 con 0x7fd6e41027a0 2024-09-19T04:20:31.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.378+0000 7fd6e1600640 1 -- 172.21.15.107:0/2637049745 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6cc004440 con 0x7fd6e41027a0 2024-09-19T04:20:31.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.378+0000 7fd6e1600640 1 -- 172.21.15.107:0/2637049745 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc035070 con 0x7fd6e41027a0 2024-09-19T04:20:31.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.378+0000 7fd6eb000640 1 -- 172.21.15.107:0/2637049745 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e41a7060 con 0x7fd6e41027a0 2024-09-19T04:20:31.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.381+0000 7fd6eda9c740 1 Processor -- start 2024-09-19T04:20:31.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.381+0000 7fd6eda9c740 1 -- start start 2024-09-19T04:20:31.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.381+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 0x5593b6f6bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.381+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f76830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f76d70 0x5593b6f79160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5593b6ed23e0 con 0x5593b6f6c4b0 2024-09-19T04:20:31.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5593b6e79d80 con 0x5593b6f6bb90 2024-09-19T04:20:31.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5593b6ece3a0 con 0x5593b6f76d70 2024-09-19T04:20:31.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f76830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f76830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40144/0 (socket says 172.21.15.107:40144) 2024-09-19T04:20:31.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6e3e00640 1 -- 172.21.15.107:0/1744276744 learned_addr learned my addr 172.21.15.107:0/1744276744 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:31.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6e9000640 1 --2- 172.21.15.107:0/1744276744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 0x5593b6f6bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6e3e00640 1 -- 172.21.15.107:0/1744276744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f76d70 msgr2=0x5593b6f79160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:20:31.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/1744276744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f76d70 0x5593b6f79160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6e3e00640 1 -- 172.21.15.107:0/1744276744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 msgr2=0x5593b6f6bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/1744276744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 0x5593b6f6bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.382+0000 7fd6e3e00640 1 -- 172.21.15.107:0/1744276744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b6e98ae0 con 0x5593b6f6c4b0 2024-09-19T04:20:31.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.383+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/1744276744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f76830 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40115f0 tx=0x7fd6d4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af3fb6b643075f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.383+0000 7fd6e1600640 1 -- 172.21.15.107:0/2637049745 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7fd6cc042030 con 0x7fd6e41027a0 2024-09-19T04:20:31.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.383+0000 7fd6d9600640 1 -- 172.21.15.107:0/1744276744 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d401e070 con 0x5593b6f6c4b0 2024-09-19T04:20:31.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.383+0000 7fd6d9600640 1 -- 172.21.15.107:0/1744276744 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6d400fb20 con 0x5593b6f6c4b0 2024-09-19T04:20:31.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.383+0000 7fd6d9600640 1 -- 172.21.15.107:0/1744276744 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4019cd0 con 0x5593b6f6c4b0 2024-09-19T04:20:31.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.384+0000 7fd6eda9c740 1 -- 172.21.15.107:0/1744276744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 msgr2=0x5593b6f76830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.384+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/1744276744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f76830 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40115f0 tx=0x7fd6d4011ab0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.384+0000 7fd6eda9c740 1 -- 172.21.15.107:0/1744276744 shutdown_connections 2024-09-19T04:20:31.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.384+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/1744276744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f76d70 0x5593b6f79160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.384+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/1744276744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f76830 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.384+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/1744276744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 0x5593b6f6bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.384+0000 7fd6eda9c740 1 -- 172.21.15.107:0/1744276744 >> 172.21.15.107:0/1744276744 conn(0x5593b6f68120 msgr2=0x5593b6f68500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.384+0000 7fd6eda9c740 1 -- 172.21.15.107:0/1744276744 shutdown_connections 2024-09-19T04:20:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.384+0000 7fd6eda9c740 1 -- 172.21.15.107:0/1744276744 wait complete. 2024-09-19T04:20:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.384+0000 7fd6e1600640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6c80a33d0 0x7fd6c80a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.385+0000 7fd6e1600640 1 -- 172.21.15.107:0/2637049745 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fd6cc0efed0 con 0x7fd6e41027a0 2024-09-19T04:20:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.385+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6c80a33d0 0x7fd6c80a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.385+0000 7fd6eda9c740 1 -- start start 2024-09-19T04:20:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.385+0000 7fd6eda9c740 1 Processor -- start 2024-09-19T04:20:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.385+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6c80a33d0 0x7fd6c80a5890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6d401c040 tx=0x7fd6d4019810 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f6bb90 0x5593b6f73d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f74290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5593b6f799e0 con 0x5593b6f6c4b0 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5593b6e8d0f0 con 0x5593b6f76d70 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5593b6f79b60 con 0x5593b6f6bb90 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6eda9c740 10 client.0 _ll_register_callbacks cb 0x5593b6e7e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f74290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f74290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40160/0 (socket says 172.21.15.107:40160) 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6dbe00640 1 -- 172.21.15.107:0/2400111566 learned_addr learned my addr 172.21.15.107:0/2400111566 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f6bb90 0x5593b6f73d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.386+0000 7fd6e9000640 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6dbe00640 1 -- 172.21.15.107:0/2400111566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f6bb90 msgr2=0x5593b6f73d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6dbe00640 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f6bb90 0x5593b6f73d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6dbe00640 1 -- 172.21.15.107:0/2400111566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 msgr2=0x5593b6f747d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6dbe00640 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f747d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6dbe00640 1 -- 172.21.15.107:0/2400111566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b6c433c0 con 0x5593b6f6c4b0 2024-09-19T04:20:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6e9000640 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f747d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6dbe00640 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f74290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0b8600 tx=0x7fd6cc0bbff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcd0afbd5e78262 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6d2000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:20:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6d2000640 1 -- 172.21.15.107:0/2400111566 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc049040 con 0x5593b6f6c4b0 2024-09-19T04:20:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6d2000640 1 -- 172.21.15.107:0/2400111566 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fd6cc0c0120 con 0x5593b6f6c4b0 2024-09-19T04:20:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6eda9c740 1 -- 172.21.15.107:0/2400111566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d400fb20 con 0x5593b6f6c4b0 2024-09-19T04:20:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6d2000640 1 -- 172.21.15.107:0/2400111566 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6cc0bc1a0 con 0x5593b6f6c4b0 2024-09-19T04:20:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6eda9c740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-09-19T04:20:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.387+0000 7fd6eda9c740 1 -- 172.21.15.107:0/2400111566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5593b6f85770 con 0x5593b6f6c4b0 2024-09-19T04:20:31.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.388+0000 7fd6d2000640 1 -- 172.21.15.107:0/2400111566 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6d400fb20 con 0x5593b6f6c4b0 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.388+0000 7fd6d2000640 1 -- 172.21.15.107:0/2400111566 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6cc04c060 con 0x5593b6f6c4b0 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.388+0000 7fd6eda9c740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.388+0000 7fd6eda9c740 10 client.15294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.388+0000 7fd6eda9c740 20 client.15294 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.388+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5593b6f86b80 0x5593b6f88f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.388+0000 7fd6eda9c740 4 client.15294 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.388+0000 7fd6eda9c740 1 -- 172.21.15.107:0/2400111566 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593b6edf950 con 0x5593b6f86b80 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.389+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5593b6f86b80 0x5593b6f88f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.389+0000 7fd6d2000640 10 client.15294 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.389+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5593b6f86b80 0x5593b6f88f70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da380e80c3a3a49e server_cookie=1a7b6732cd7b4a8f in_seq=0 out_seq=0 2024-09-19T04:20:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6d2000640 1 -- 172.21.15.107:0/2400111566 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593b6edf950 con 0x5593b6f86b80 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6d2000640 10 client.15294 handle_command_reply: tid=0 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6eda9c740 1 client.15294 shutdown 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6eda9c740 1 -- 172.21.15.107:0/2400111566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 msgr2=0x5593b6f74290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f74290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0b8600 tx=0x7fd6cc0bbff0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6eda9c740 1 -- 172.21.15.107:0/2400111566 shutdown_connections 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5593b6f86b80 0x5593b6f88f70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f747d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f74290 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/2400111566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f6bb90 0x5593b6f73d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6eda9c740 1 -- 172.21.15.107:0/2400111566 >> 172.21.15.107:0/2400111566 conn(0x5593b6f68120 msgr2=0x5593b6f6de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.390+0000 7fd6d2000640 0 client.15294 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:31.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.391+0000 7fd6eda9c740 1 -- 172.21.15.107:0/2400111566 shutdown_connections 2024-09-19T04:20:31.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.391+0000 7fd6eda9c740 1 -- 172.21.15.107:0/2400111566 wait complete. 2024-09-19T04:20:31.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.391+0000 7fd6eda9c740 20 client.15294 trim_cache size 0 max 16384 2024-09-19T04:20:31.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.394+0000 7fe01b200640 1 -- 172.21.15.107:0/640941592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe014107b80 msgr2=0x7fe014107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.394+0000 7fe01b200640 1 --2- 172.21.15.107:0/640941592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe014107b80 0x7fe014107f60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe004009930 tx=0x7fe00402a420 comp rx=0 tx=0).stop 2024-09-19T04:20:31.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.394+0000 7fe01b200640 1 -- 172.21.15.107:0/640941592 shutdown_connections 2024-09-19T04:20:31.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.394+0000 7fe01b200640 1 --2- 172.21.15.107:0/640941592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe014102480 0x7fe01410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.394+0000 7fe01b200640 1 --2- 172.21.15.107:0/640941592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe014101ae0 0x7fe014101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.394+0000 7fe01b200640 1 --2- 172.21.15.107:0/640941592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe014107b80 0x7fe014107f60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.394+0000 7fe01b200640 1 -- 172.21.15.107:0/640941592 >> 172.21.15.107:0/640941592 conn(0x7fe0140fd7e0 msgr2=0x7fe0140ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.394+0000 7fe01b200640 1 -- 172.21.15.107:0/640941592 shutdown_connections 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.394+0000 7fe01b200640 1 -- 172.21.15.107:0/640941592 wait complete. 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.395+0000 7fe01b200640 1 Processor -- start 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.395+0000 7fe01b200640 1 -- start start 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe01b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe014101ae0 0x7fe014108a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe01b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe014102480 0x7fe014108f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe01b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe014107b80 0x7fe014109490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe01b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe014111f40 con 0x7fe014102480 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe01b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe014111dc0 con 0x7fe014107b80 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe01b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141120c0 con 0x7fe014101ae0 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe00be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe014102480 0x7fe014108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe00be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe014102480 0x7fe014108f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40188/0 (socket says 172.21.15.107:40188) 2024-09-19T04:20:31.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe00be00640 1 -- 172.21.15.107:0/2809957329 learned_addr learned my addr 172.21.15.107:0/2809957329 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe019200640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe014107b80 0x7fe014109490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe013e00640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe014101ae0 0x7fe014108a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe00be00640 1 -- 172.21.15.107:0/2809957329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe014101ae0 msgr2=0x7fe014108a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe00be00640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe014101ae0 0x7fe014108a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe00be00640 1 -- 172.21.15.107:0/2809957329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe014107b80 msgr2=0x7fe014109490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe00be00640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe014107b80 0x7fe014109490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe00be00640 1 -- 172.21.15.107:0/2809957329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0141aabe0 con 0x7fe014102480 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.396+0000 7fe013e00640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe014101ae0 0x7fe014108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.397+0000 7fe00be00640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe014102480 0x7fe014108f50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00ea90 tx=0x7fdffc00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.397+0000 7fe011600640 1 -- 172.21.15.107:0/2809957329 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc00ce40 con 0x7fe014102480 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.397+0000 7fe011600640 1 -- 172.21.15.107:0/2809957329 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdffc0035f0 con 0x7fe014102480 2024-09-19T04:20:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.397+0000 7fe011600640 1 -- 172.21.15.107:0/2809957329 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc010690 con 0x7fe014102480 2024-09-19T04:20:31.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.397+0000 7fe01b200640 1 -- 172.21.15.107:0/2809957329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0141aaed0 con 0x7fe014102480 2024-09-19T04:20:31.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.398+0000 7fe01b200640 1 -- 172.21.15.107:0/2809957329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0141ab2e0 con 0x7fe014102480 2024-09-19T04:20:31.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.399+0000 7fe01b200640 1 -- 172.21.15.107:0/2809957329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0141031e0 con 0x7fe014102480 2024-09-19T04:20:31.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.401+0000 7fe011600640 1 -- 172.21.15.107:0/2809957329 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 138532+0+0 (secure 0 0 0) 0x7fdffc0037a0 con 0x7fe014102480 2024-09-19T04:20:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.403+0000 7fe011600640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdff80a3580 0x7fdff80a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.403+0000 7fe011600640 1 -- 172.21.15.107:0/2809957329 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fdffc0d0530 con 0x7fe014102480 2024-09-19T04:20:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.403+0000 7fe013e00640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdff80a3580 0x7fdff80a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.404+0000 7fe013e00640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdff80a3580 0x7fdff80a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe004006010 tx=0x7fe004005a20 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.407+0000 7fe011600640 1 -- 172.21.15.107:0/2809957329 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdffc01d060 con 0x7fe014102480 2024-09-19T04:20:31.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.410+0000 7fd6eda9c740 1 Processor -- start 2024-09-19T04:20:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.410+0000 7fd6eda9c740 1 -- start start 2024-09-19T04:20:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.410+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6bb90 0x5593b6f74660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.410+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f6c4b0 0x5593b6f6dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f8e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41af8a0 con 0x5593b6f6bb90 2024-09-19T04:20:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41b13e0 con 0x5593b6f76d70 2024-09-19T04:20:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41aff70 con 0x5593b6f6c4b0 2024-09-19T04:20:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6e9000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f8e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6bb90 0x5593b6f74660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6e9000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f8e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57062/0 (socket says 172.21.15.107:57062) 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6e9000640 1 -- 172.21.15.107:0/3431300543 learned_addr learned my addr 172.21.15.107:0/3431300543 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6dbe00640 1 --2- 172.21.15.107:0/3431300543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f6c4b0 0x5593b6f6dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6e9000640 1 -- 172.21.15.107:0/3431300543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f6c4b0 msgr2=0x5593b6f6dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6e9000640 1 --2- 172.21.15.107:0/3431300543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f6c4b0 0x5593b6f6dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6e9000640 1 -- 172.21.15.107:0/3431300543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6bb90 msgr2=0x5593b6f74660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6e9000640 1 --2- 172.21.15.107:0/3431300543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6bb90 0x5593b6f74660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.411+0000 7fd6e9000640 1 -- 172.21.15.107:0/3431300543 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b6f895d0 con 0x5593b6f76d70 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.412+0000 7fd6e9000640 1 --2- 172.21.15.107:0/3431300543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f8e800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc003b00 tx=0x7fd6dc004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5657c4f25f2864c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.412+0000 7fd6d8c00640 1 -- 172.21.15.107:0/3431300543 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc020070 con 0x5593b6f76d70 2024-09-19T04:20:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.412+0000 7fd6d8c00640 1 -- 172.21.15.107:0/3431300543 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6dc010910 con 0x5593b6f76d70 2024-09-19T04:20:31.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.412+0000 7fd6d8c00640 1 -- 172.21.15.107:0/3431300543 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc01bb60 con 0x5593b6f76d70 2024-09-19T04:20:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 -- 172.21.15.107:0/3431300543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 msgr2=0x5593b6f8e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/3431300543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f8e800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc003b00 tx=0x7fd6dc004040 comp rx=0 tx=0).stop 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 -- 172.21.15.107:0/3431300543 shutdown_connections 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/3431300543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f76d70 0x5593b6f8e800 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/3431300543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f6c4b0 0x5593b6f6dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/3431300543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6bb90 0x5593b6f74660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 -- 172.21.15.107:0/3431300543 >> 172.21.15.107:0/3431300543 conn(0x5593b6f68120 msgr2=0x5593b6f6d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 -- 172.21.15.107:0/3431300543 shutdown_connections 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 -- 172.21.15.107:0/3431300543 wait complete. 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 -- start start 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.413+0000 7fd6eda9c740 1 Processor -- start 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 0x5593b6f7edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f7f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6eda9c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f76d70 0x5593b6f7bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41b0d10 con 0x5593b6f6c4b0 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41b0640 con 0x5593b6f6bb90 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6eda9c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41ad880 con 0x5593b6f76d70 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6eda9c740 10 client.0 _ll_register_callbacks cb 0x7fd6cc0b8440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 0x5593b6f7edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f7f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 0x5593b6f7edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57078/0 (socket says 172.21.15.107:57078) 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f7f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40214/0 (socket says 172.21.15.107:40214) 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6e3e00640 1 -- 172.21.15.107:0/588933656 learned_addr learned my addr 172.21.15.107:0/588933656 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.414+0000 7fd6e9000640 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f76d70 0x5593b6f7bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.415+0000 7fd6e3e00640 1 -- 172.21.15.107:0/588933656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f76d70 msgr2=0x5593b6f7bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.415+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f76d70 0x5593b6f7bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.415+0000 7fd6e3e00640 1 -- 172.21.15.107:0/588933656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 msgr2=0x5593b6f7f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.415+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f7f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.415+0000 7fd6e3e00640 1 -- 172.21.15.107:0/588933656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b6f68db0 con 0x5593b6f6bb90 2024-09-19T04:20:31.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.415+0000 7fd6e9000640 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f76d70 0x5593b6f7bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:31.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.415+0000 7fd6e3e00640 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 0x5593b6f7edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4019920 tx=0x7fd6d4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d31c2ae1701932b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.415+0000 7fd6d2000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.415+0000 7fd6d2000640 1 -- 172.21.15.107:0/588933656 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d401e040 con 0x5593b6f6bb90 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6eda9c740 1 -- 172.21.15.107:0/588933656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6dc010910 con 0x5593b6f6bb90 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6d2000640 1 -- 172.21.15.107:0/588933656 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fd6d4019410 con 0x5593b6f6bb90 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6d2000640 1 -- 172.21.15.107:0/588933656 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6d4004410 con 0x5593b6f6bb90 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6eda9c740 10 client.25319 fetch_fsmap learned FSMap version 28 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6eda9c740 1 -- 172.21.15.107:0/588933656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5593b6ff1d30 con 0x5593b6f6bb90 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6d2000640 1 -- 172.21.15.107:0/588933656 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6dc010910 con 0x5593b6f6bb90 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6d2000640 1 -- 172.21.15.107:0/588933656 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6d4003b90 con 0x5593b6f6bb90 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6eda9c740 10 client.25319 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6eda9c740 10 client.25319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:20:31.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6eda9c740 20 client.25319 populate_metadata read hostname 'smithi107' 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5593b6ff2250 0x5593b6ff4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6eda9c740 4 client.25319 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.416+0000 7fd6eda9c740 1 -- 172.21.15.107:0/588933656 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593b6ed9e60 con 0x5593b6ff2250 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.417+0000 7fd6dbe00640 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5593b6ff2250 0x5593b6ff4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.417+0000 7fd6dbe00640 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5593b6ff2250 0x5593b6ff4670 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e35371ff699bd13 server_cookie=85c4223a0837e785 in_seq=0 out_seq=0 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.417+0000 7fd6d2000640 10 client.25319 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6d2000640 1 -- 172.21.15.107:0/588933656 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5593b6ed9e60 con 0x5593b6ff2250 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6d2000640 10 client.25319 handle_command_reply: tid=0 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 client.25319 shutdown 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 -- 172.21.15.107:0/588933656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 msgr2=0x5593b6f7edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 0x5593b6f7edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4019920 tx=0x7fd6d4005f90 comp rx=0 tx=0).stop 2024-09-19T04:20:31.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 -- 172.21.15.107:0/588933656 shutdown_connections 2024-09-19T04:20:31.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5593b6ff2250 0x5593b6ff4670 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5593b6f76d70 0x5593b6f7bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593b6f6c4b0 0x5593b6f7f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 --2- 172.21.15.107:0/588933656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593b6f6bb90 0x5593b6f7edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 -- 172.21.15.107:0/588933656 >> 172.21.15.107:0/588933656 conn(0x5593b6f68120 msgr2=0x5593b6f75dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6d2000640 0 client.25319 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:20:31.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 -- 172.21.15.107:0/588933656 shutdown_connections 2024-09-19T04:20:31.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 1 -- 172.21.15.107:0/588933656 wait complete. 2024-09-19T04:20:31.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.418+0000 7fd6eda9c740 20 client.25319 trim_cache size 0 max 16384 2024-09-19T04:20:31.422 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:20:31.423 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:20:31.423 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:20:31.423 INFO:teuthology.orchestra.run.smithi107.stdout: "3b41de20-1b6e-448f-9d84-5e5c2c25eae0": { 2024-09-19T04:20:31.423 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:20:31.423 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "3b41de20-1b6e-448f-9d84-5e5c2c25eae0", 2024-09-19T04:20:31.423 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:20:31.423 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:20:31.423 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:20:31.423 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:20:31.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 -- 172.21.15.107:0/2637049745 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6c80a33d0 msgr2=0x7fd6c80a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6c80a33d0 0x7fd6c80a5890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6d401c040 tx=0x7fd6d4019810 comp rx=0 tx=0).stop 2024-09-19T04:20:31.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 -- 172.21.15.107:0/2637049745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6e41027a0 msgr2=0x7fd6e4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6e41027a0 0x7fd6e4074a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc006010 tx=0x7fd6cc004040 comp rx=0 tx=0).stop 2024-09-19T04:20:31.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 -- 172.21.15.107:0/2637049745 shutdown_connections 2024-09-19T04:20:31.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6c80a33d0 0x7fd6c80a5890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6e4106f20 0x7fd6e4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6e41027a0 0x7fd6e4074a30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 --2- 172.21.15.107:0/2637049745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e4101df0 0x7fd6e40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 -- 172.21.15.107:0/2637049745 >> 172.21.15.107:0/2637049745 conn(0x7fd6e40fd8b0 msgr2=0x7fd6e40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 -- 172.21.15.107:0/2637049745 shutdown_connections 2024-09-19T04:20:31.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.419+0000 7fd6eb000640 1 -- 172.21.15.107:0/2637049745 wait complete. 2024-09-19T04:20:31.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:31 smithi115 ceph-mon[35477]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-19T04:20:31.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:31 smithi145 ceph-mon[35096]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-19T04:20:31.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.619+0000 7fe01b200640 1 -- 172.21.15.107:0/2809957329 --> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe0140fed80 con 0x7fdff80a3580 2024-09-19T04:20:31.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.627+0000 7fe011600640 1 -- 172.21.15.107:0/2809957329 <== mgr.15114 v2:172.21.15.107:6800/1724794486 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe0140fed80 con 0x7fdff80a3580 2024-09-19T04:20:31.628 INFO:teuthology.orchestra.run.smithi107.stdout:/ 1m 6m3h 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 -- 172.21.15.107:0/2809957329 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdff80a3580 msgr2=0x7fdff80a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdff80a3580 0x7fdff80a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe004006010 tx=0x7fe004005a20 comp rx=0 tx=0).stop 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 -- 172.21.15.107:0/2809957329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe014102480 msgr2=0x7fe014108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe014102480 0x7fe014108f50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00ea90 tx=0x7fdffc00ef50 comp rx=0 tx=0).stop 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 -- 172.21.15.107:0/2809957329 shutdown_connections 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdff80a3580 0x7fdff80a5a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe014107b80 0x7fe014109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe014102480 0x7fe014108f50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 --2- 172.21.15.107:0/2809957329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe014101ae0 0x7fe014108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 -- 172.21.15.107:0/2809957329 >> 172.21.15.107:0/2809957329 conn(0x7fe0140fd7e0 msgr2=0x7fe01410d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 -- 172.21.15.107:0/2809957329 shutdown_connections 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:20:31.629+0000 7fe01b200640 1 -- 172.21.15.107:0/2809957329 wait complete. 2024-09-19T04:20:31.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T04:20:31.705 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T04:20:31.716 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T04:20:31.838 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T04:20:31.839 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T04:20:31.839 INFO:tasks.workunit:timeout=3h 2024-09-19T04:20:31.839 INFO:tasks.workunit:cleanup=True 2024-09-19T04:20:31.839 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T04:20:31.875 INFO:teuthology.orchestra.run.smithi107.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T04:20:31.875 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T04:20:31.875 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T04:20:31.875 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:20:31.875 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T04:20:31.875 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-19 04:18:33.597958744 +0000 2024-09-19T04:20:31.875 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-19 04:18:33.597958744 +0000 2024-09-19T04:20:31.875 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-19 04:20:18.086272965 +0000 2024-09-19T04:20:31.875 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-19 04:18:33.597958744 +0000 2024-09-19T04:20:31.875 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T04:20:31.876 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T04:20:31.983 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T04:20:32.021 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T04:20:32.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:32 smithi107 ceph-mon[30659]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:32.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:32 smithi107 ceph-mon[30659]: from='client.25279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:32.241 INFO:tasks.cephfs.filesystem:scrub status for tag:3b41de20-1b6e-448f-9d84-5e5c2c25eae0 - {'path': '/', 'tag': '3b41de20-1b6e-448f-9d84-5e5c2c25eae0', 'options': 'recursive,force'} 2024-09-19T04:20:32.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:32 smithi115 ceph-mon[35477]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:32.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:32 smithi115 ceph-mon[35477]: from='client.25279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:32.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:32 smithi145 ceph-mon[35096]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:32.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:32 smithi145 ceph-mon[35096]: from='client.25279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:33.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:33 smithi115 ceph-mon[35477]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 20 op/s 2024-09-19T04:20:33.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:33 smithi115 ceph-mon[35477]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:33.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:33 smithi115 ceph-mon[35477]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:33.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:33 smithi145 ceph-mon[35096]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 20 op/s 2024-09-19T04:20:33.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:33 smithi145 ceph-mon[35096]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:33.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:33 smithi145 ceph-mon[35096]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:33.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:33 smithi107 ceph-mon[30659]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 20 op/s 2024-09-19T04:20:33.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:33 smithi107 ceph-mon[30659]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:33.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:33 smithi107 ceph-mon[30659]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:20:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:35 smithi115 ceph-mon[35477]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-19T04:20:35.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:35 smithi145 ceph-mon[35096]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-19T04:20:35.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:35 smithi107 ceph-mon[30659]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-19T04:20:37.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:37 smithi115 ceph-mon[35477]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-19T04:20:37.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:37 smithi145 ceph-mon[35096]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-19T04:20:37.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:37 smithi107 ceph-mon[30659]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-19T04:20:39.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:39 smithi115 ceph-mon[35477]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-19T04:20:39.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:39 smithi145 ceph-mon[35096]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-19T04:20:39.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:39 smithi107 ceph-mon[30659]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-19T04:20:41.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:41 smithi115 ceph-mon[35477]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:20:41.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:41 smithi145 ceph-mon[35096]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:20:41.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:41 smithi107 ceph-mon[30659]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:20:42.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:42 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:20:42.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:42 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:20:42.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:42 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:20:43.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:43 smithi115 ceph-mon[35477]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-19T04:20:43.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:43 smithi145 ceph-mon[35096]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-19T04:20:43.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:43 smithi107 ceph-mon[30659]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-19T04:20:45.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:45 smithi115 ceph-mon[35477]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:20:45.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:45 smithi145 ceph-mon[35096]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:20:45.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:45 smithi107 ceph-mon[30659]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:20:47.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:47 smithi115 ceph-mon[35477]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:20:47.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:47 smithi145 ceph-mon[35096]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:20:47.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:47 smithi107 ceph-mon[30659]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T04:20:49.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:49 smithi115 ceph-mon[35477]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-19T04:20:49.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:49 smithi145 ceph-mon[35096]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-19T04:20:49.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:49 smithi107 ceph-mon[30659]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-19T04:20:51.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:51 smithi115 ceph-mon[35477]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:20:51.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:51 smithi107 ceph-mon[30659]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:20:51.616 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:51 smithi145 ceph-mon[35096]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:20:53.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:53 smithi115 ceph-mon[35477]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:20:53.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:53 smithi145 ceph-mon[35096]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:20:53.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:53 smithi107 ceph-mon[30659]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:20:55.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:55 smithi115 ceph-mon[35477]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:20:55.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:55 smithi145 ceph-mon[35096]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:20:55.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:55 smithi107 ceph-mon[30659]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:20:57.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:57 smithi107 ceph-mon[30659]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:20:57.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:57 smithi115 ceph-mon[35477]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:20:57.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:57 smithi145 ceph-mon[35096]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:20:59.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:20:59 smithi115 ceph-mon[35477]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:20:59.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:20:59 smithi145 ceph-mon[35096]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:20:59.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:20:59 smithi107 ceph-mon[30659]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:21:01.074 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:21:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:21:00.778+0000 7fd909c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:21:01.075 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:21:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:21:00.778+0000 7fd909c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:21:01.075 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:21:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:21:00.778+0000 7fd909c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:21:01.075 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:21:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:21:00.778+0000 7fd909c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:21:01.075 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:21:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:21:00.778+0000 7fd909c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:21:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:01 smithi115 ceph-mon[35477]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:01 smithi145 ceph-mon[35096]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:01.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:01 smithi107 ceph-mon[30659]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:02.242 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:21:02.630 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.269+0000 7fe917e00640 1 -- 172.21.15.107:0/1569632140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe918102bf0 msgr2=0x7fe918102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.269+0000 7fe917e00640 1 --2- 172.21.15.107:0/1569632140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe918102bf0 0x7fe918102fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe908009860 tx=0x7fe90802a3d0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.269+0000 7fe917e00640 1 -- 172.21.15.107:0/1569632140 shutdown_connections 2024-09-19T04:21:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.269+0000 7fe917e00640 1 --2- 172.21.15.107:0/1569632140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe91810bfd0 0x7fe91810e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.269+0000 7fe917e00640 1 --2- 172.21.15.107:0/1569632140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe918103510 0x7fe91810ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.269+0000 7fe917e00640 1 --2- 172.21.15.107:0/1569632140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe918102bf0 0x7fe918102fd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.269+0000 7fe917e00640 1 -- 172.21.15.107:0/1569632140 >> 172.21.15.107:0/1569632140 conn(0x7fe9180fc610 msgr2=0x7fe9180fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.269+0000 7fe917e00640 1 -- 172.21.15.107:0/1569632140 shutdown_connections 2024-09-19T04:21:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.269+0000 7fe917e00640 1 -- 172.21.15.107:0/1569632140 wait complete. 2024-09-19T04:21:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.270+0000 7fe917e00640 1 Processor -- start 2024-09-19T04:21:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.270+0000 7fe917e00640 1 -- start start 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.270+0000 7fe917e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe918102bf0 0x7fe91819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.270+0000 7fe917e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe918103510 0x7fe91819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.270+0000 7fe917e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe91810bfd0 0x7fe91819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.270+0000 7fe917e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe918111db0 con 0x7fe918103510 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.270+0000 7fe917e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe918111c30 con 0x7fe918102bf0 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.270+0000 7fe917e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe918111f30 con 0x7fe91810bfd0 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.271+0000 7fe916000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe918103510 0x7fe91819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.271+0000 7fe916a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe918102bf0 0x7fe91819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.271+0000 7fe916000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe918103510 0x7fe91819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46044/0 (socket says 172.21.15.107:46044) 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.271+0000 7fe916000640 1 -- 172.21.15.107:0/202621299 learned_addr learned my addr 172.21.15.107:0/202621299 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.271+0000 7fe917400640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe91810bfd0 0x7fe91819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.271+0000 7fe916000640 1 -- 172.21.15.107:0/202621299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe91810bfd0 msgr2=0x7fe91819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.271+0000 7fe916000640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe91810bfd0 0x7fe91819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.271+0000 7fe916000640 1 -- 172.21.15.107:0/202621299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe918102bf0 msgr2=0x7fe91819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.271+0000 7fe916000640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe918102bf0 0x7fe91819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.271+0000 7fe916000640 1 -- 172.21.15.107:0/202621299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91819f450 con 0x7fe918103510 2024-09-19T04:21:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.272+0000 7fe916000640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe918103510 0x7fe91819a9e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe90400e9c0 tx=0x7fe90400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.272+0000 7fe90f400640 1 -- 172.21.15.107:0/202621299 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90400ccc0 con 0x7fe918103510 2024-09-19T04:21:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.272+0000 7fe90f400640 1 -- 172.21.15.107:0/202621299 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe904005500 con 0x7fe918103510 2024-09-19T04:21:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.272+0000 7fe917e00640 1 -- 172.21.15.107:0/202621299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91819f740 con 0x7fe918103510 2024-09-19T04:21:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.272+0000 7fe917e00640 1 -- 172.21.15.107:0/202621299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9181a7020 con 0x7fe918103510 2024-09-19T04:21:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.272+0000 7fe90f400640 1 -- 172.21.15.107:0/202621299 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904010430 con 0x7fe918103510 2024-09-19T04:21:03.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.276+0000 7fe90f400640 1 -- 172.21.15.107:0/202621299 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 138589+0+0 (secure 0 0 0) 0x7fe9040046c0 con 0x7fe918103510 2024-09-19T04:21:03.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.277+0000 7fe9201bc740 1 Processor -- start 2024-09-19T04:21:03.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.277+0000 7fe9201bc740 1 -- start start 2024-09-19T04:21:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.277+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c8406130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.277+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c84109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8410f30 0x5630c8413320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe9201bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5630c836c3e0 con 0x5630c8405d50 2024-09-19T04:21:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe9201bc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5630c8313d80 con 0x5630c8410f30 2024-09-19T04:21:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe9201bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5630c83683a0 con 0x5630c8406670 2024-09-19T04:21:03.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe917400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c8406130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe916a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c84109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe916a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c84109f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33500/0 (socket says 172.21.15.107:33500) 2024-09-19T04:21:03.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe916a00640 1 -- 172.21.15.107:0/3189591178 learned_addr learned my addr 172.21.15.107:0/3189591178 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:03.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe917400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c8406130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46050/0 (socket says 172.21.15.107:46050) 2024-09-19T04:21:03.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe916000640 1 --2- 172.21.15.107:0/3189591178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8410f30 0x5630c8413320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.278+0000 7fe90f400640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8fc0a35d0 0x7fe8fc0a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe916a00640 1 -- 172.21.15.107:0/3189591178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8410f30 msgr2=0x5630c8413320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:03.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe916a00640 1 --2- 172.21.15.107:0/3189591178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8410f30 0x5630c8413320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe916a00640 1 -- 172.21.15.107:0/3189591178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 msgr2=0x5630c8406130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:03.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe916a00640 1 --2- 172.21.15.107:0/3189591178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c8406130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe916a00640 1 -- 172.21.15.107:0/3189591178 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c8332ae0 con 0x5630c8406670 2024-09-19T04:21:03.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe90f400640 1 -- 172.21.15.107:0/202621299 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fe9040d0110 con 0x7fe918103510 2024-09-19T04:21:03.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe917400640 1 --2- 172.21.15.107:0/3189591178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c8406130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:03.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe917400640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8fc0a35d0 0x7fe8fc0a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe916a00640 1 --2- 172.21.15.107:0/3189591178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c84109f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe908034000 tx=0x7fe908032f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37680420be2ed0ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:03.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe903e00640 1 -- 172.21.15.107:0/3189591178 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908030cb0 con 0x5630c8406670 2024-09-19T04:21:03.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe903e00640 1 -- 172.21.15.107:0/3189591178 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe908030e50 con 0x5630c8406670 2024-09-19T04:21:03.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.279+0000 7fe903e00640 1 -- 172.21.15.107:0/3189591178 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908004920 con 0x5630c8406670 2024-09-19T04:21:03.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.280+0000 7fe917400640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8fc0a35d0 0x7fe8fc0a5a90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe910009fd0 tx=0x7fe910004a40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:03.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.280+0000 7fe9201bc740 1 -- 172.21.15.107:0/3189591178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 msgr2=0x5630c84109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:03.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.280+0000 7fe9201bc740 1 --2- 172.21.15.107:0/3189591178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c84109f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe908034000 tx=0x7fe908032f60 comp rx=0 tx=0).stop 2024-09-19T04:21:03.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.280+0000 7fe9201bc740 1 -- 172.21.15.107:0/3189591178 shutdown_connections 2024-09-19T04:21:03.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.280+0000 7fe9201bc740 1 --2- 172.21.15.107:0/3189591178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8410f30 0x5630c8413320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.280+0000 7fe9201bc740 1 --2- 172.21.15.107:0/3189591178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c84109f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.281+0000 7fe9201bc740 1 --2- 172.21.15.107:0/3189591178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c8406130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.281+0000 7fe9201bc740 1 -- 172.21.15.107:0/3189591178 >> 172.21.15.107:0/3189591178 conn(0x5630c84022e0 msgr2=0x5630c84026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:03.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.281+0000 7fe9201bc740 1 -- 172.21.15.107:0/3189591178 shutdown_connections 2024-09-19T04:21:03.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.281+0000 7fe9201bc740 1 -- 172.21.15.107:0/3189591178 wait complete. 2024-09-19T04:21:03.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.281+0000 7fe9201bc740 1 -- start start 2024-09-19T04:21:03.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.282+0000 7fe9201bc740 1 Processor -- start 2024-09-19T04:21:03.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.282+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 0x5630c840ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.282+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c840e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.282+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8410f30 0x5630c840e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.282+0000 7fe9201bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5630c8413ba0 con 0x5630c8410f30 2024-09-19T04:21:03.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.282+0000 7fe9201bc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5630c83270f0 con 0x5630c8405d50 2024-09-19T04:21:03.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.282+0000 7fe9201bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5630c8413d20 con 0x5630c8406670 2024-09-19T04:21:03.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.282+0000 7fe9201bc740 10 client.0 _ll_register_callbacks cb 0x5630c83189e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:03.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe916000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c840e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe916000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c840e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33514/0 (socket says 172.21.15.107:33514) 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe916000640 1 -- 172.21.15.107:0/358785160 learned_addr learned my addr 172.21.15.107:0/358785160 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe916a00640 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 0x5630c840ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe917400640 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8410f30 0x5630c840e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe916a00640 1 -- 172.21.15.107:0/358785160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 msgr2=0x5630c840e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe916a00640 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c840e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe916a00640 1 -- 172.21.15.107:0/358785160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8410f30 msgr2=0x5630c840e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe916a00640 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8410f30 0x5630c840e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe916a00640 1 -- 172.21.15.107:0/358785160 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c80dd3c0 con 0x5630c8405d50 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.283+0000 7fe916000640 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c840e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.284+0000 7fe916a00640 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 0x5630c840ddb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe908032ff0 tx=0x7fe90802a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b5e703c3cfa56f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.284+0000 7fe900c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.284+0000 7fe900c00640 1 -- 172.21.15.107:0/358785160 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908055060 con 0x5630c8405d50 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.284+0000 7fe9201bc740 1 -- 172.21.15.107:0/358785160 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe908030e50 con 0x5630c8405d50 2024-09-19T04:21:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.284+0000 7fe900c00640 1 -- 172.21.15.107:0/358785160 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fe9080305b0 con 0x5630c8405d50 2024-09-19T04:21:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.284+0000 7fe900c00640 1 -- 172.21.15.107:0/358785160 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe908009450 con 0x5630c8405d50 2024-09-19T04:21:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.284+0000 7fe9201bc740 10 client.25337 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.284+0000 7fe9201bc740 1 -- 172.21.15.107:0/358785160 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630c8413ba0 con 0x5630c8405d50 2024-09-19T04:21:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.285+0000 7fe900c00640 1 -- 172.21.15.107:0/358785160 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe908030e50 con 0x5630c8405d50 2024-09-19T04:21:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.285+0000 7fe900c00640 1 -- 172.21.15.107:0/358785160 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe90802f570 con 0x5630c8405d50 2024-09-19T04:21:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.285+0000 7fe9201bc740 10 client.25337 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.285+0000 7fe9201bc740 10 client.25337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.285+0000 7fe9201bc740 20 client.25337 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.285+0000 7fe9201bc740 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5630c84209c0 0x5630c8422db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.285+0000 7fe9201bc740 4 client.25337 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:03.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.285+0000 7fe9201bc740 1 -- 172.21.15.107:0/358785160 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630c8379950 con 0x5630c84209c0 2024-09-19T04:21:03.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.285+0000 7fe916000640 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5630c84209c0 0x5630c8422db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.286+0000 7fe916000640 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5630c84209c0 0x5630c8422db0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c913bfa53aca3370 server_cookie=f9e87939b33f8b3f in_seq=0 out_seq=0 2024-09-19T04:21:03.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.286+0000 7fe900c00640 10 client.25337 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:03.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe900c00640 1 -- 172.21.15.107:0/358785160 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630c8379950 con 0x5630c84209c0 2024-09-19T04:21:03.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe900c00640 10 client.25337 handle_command_reply: tid=0 2024-09-19T04:21:03.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 client.25337 shutdown 2024-09-19T04:21:03.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 -- 172.21.15.107:0/358785160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 msgr2=0x5630c840ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:03.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 0x5630c840ddb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe908032ff0 tx=0x7fe90802a3d0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 -- 172.21.15.107:0/358785160 shutdown_connections 2024-09-19T04:21:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5630c84209c0 0x5630c8422db0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8410f30 0x5630c840e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c840e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 --2- 172.21.15.107:0/358785160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 0x5630c840ddb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 -- 172.21.15.107:0/358785160 >> 172.21.15.107:0/358785160 conn(0x5630c84022e0 msgr2=0x5630c8408010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe900c00640 0 client.25337 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 -- 172.21.15.107:0/358785160 shutdown_connections 2024-09-19T04:21:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 1 -- 172.21.15.107:0/358785160 wait complete. 2024-09-19T04:21:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.287+0000 7fe9201bc740 20 client.25337 trim_cache size 0 max 16384 2024-09-19T04:21:03.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.313+0000 7fe9201bc740 1 Processor -- start 2024-09-19T04:21:03.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.314+0000 7fe9201bc740 1 -- start start 2024-09-19T04:21:03.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.314+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 0x5630c8428520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.314+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c8428a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.314+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8410f30 0x5630c8400b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.314+0000 7fe9201bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f800e650 con 0x5630c8410f30 2024-09-19T04:21:03.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.314+0000 7fe9201bc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8010190 con 0x5630c8405d50 2024-09-19T04:21:03.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.314+0000 7fe9201bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f800ed20 con 0x5630c8406670 2024-09-19T04:21:03.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.314+0000 7fe916000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c8428a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.314+0000 7fe916000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c8428a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33522/0 (socket says 172.21.15.107:33522) 2024-09-19T04:21:03.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.314+0000 7fe916000640 1 -- 172.21.15.107:0/4004076304 learned_addr learned my addr 172.21.15.107:0/4004076304 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:03.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.315+0000 7fe916a00640 1 --2- 172.21.15.107:0/4004076304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 0x5630c8428520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.315+0000 7fe917400640 1 --2- 172.21.15.107:0/4004076304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8410f30 0x5630c8400b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.315+0000 7fe916000640 1 -- 172.21.15.107:0/4004076304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 msgr2=0x5630c8428520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:03.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.315+0000 7fe916000640 1 --2- 172.21.15.107:0/4004076304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 0x5630c8428520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.315+0000 7fe916000640 1 -- 172.21.15.107:0/4004076304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8410f30 msgr2=0x5630c8400b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:03.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.315+0000 7fe916000640 1 --2- 172.21.15.107:0/4004076304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8410f30 0x5630c8400b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.315+0000 7fe916000640 1 -- 172.21.15.107:0/4004076304 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c8402110 con 0x5630c8406670 2024-09-19T04:21:03.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.315+0000 7fe916000640 1 --2- 172.21.15.107:0/4004076304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c8428a60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe90400c6a0 tx=0x7fe9040bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae83675fb007a23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:03.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.315+0000 7fe903400640 1 -- 172.21.15.107:0/4004076304 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90409bcd0 con 0x5630c8406670 2024-09-19T04:21:03.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.316+0000 7fe903400640 1 -- 172.21.15.107:0/4004076304 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe90409c890 con 0x5630c8406670 2024-09-19T04:21:03.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.316+0000 7fe903400640 1 -- 172.21.15.107:0/4004076304 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90409baf0 con 0x5630c8406670 2024-09-19T04:21:03.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.316+0000 7fe9201bc740 1 -- 172.21.15.107:0/4004076304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 msgr2=0x5630c8428a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:03.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.316+0000 7fe9201bc740 1 --2- 172.21.15.107:0/4004076304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c8428a60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe90400c6a0 tx=0x7fe9040bc040 comp rx=0 tx=0).stop 2024-09-19T04:21:03.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.316+0000 7fe9201bc740 1 -- 172.21.15.107:0/4004076304 shutdown_connections 2024-09-19T04:21:03.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.316+0000 7fe9201bc740 1 --2- 172.21.15.107:0/4004076304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8410f30 0x5630c8400b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.316+0000 7fe9201bc740 1 --2- 172.21.15.107:0/4004076304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8406670 0x5630c8428a60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.316+0000 7fe9201bc740 1 --2- 172.21.15.107:0/4004076304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8405d50 0x5630c8428520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.316+0000 7fe9201bc740 1 -- 172.21.15.107:0/4004076304 >> 172.21.15.107:0/4004076304 conn(0x5630c84022e0 msgr2=0x5630c840fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:03.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.317+0000 7fe9201bc740 1 -- 172.21.15.107:0/4004076304 shutdown_connections 2024-09-19T04:21:03.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.317+0000 7fe9201bc740 1 -- 172.21.15.107:0/4004076304 wait complete. 2024-09-19T04:21:03.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.317+0000 7fe9201bc740 1 -- start start 2024-09-19T04:21:03.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.318+0000 7fe9201bc740 1 Processor -- start 2024-09-19T04:21:03.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.318+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c84192a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.318+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8406670 0x5630c84197e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.318+0000 7fe9201bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8410f30 0x5630c8414630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.318+0000 7fe9201bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f800fac0 con 0x5630c8405d50 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.318+0000 7fe9201bc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f800f3f0 con 0x5630c8406670 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.318+0000 7fe9201bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f800c630 con 0x5630c8410f30 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.318+0000 7fe9201bc740 10 client.0 _ll_register_callbacks cb 0x7fe9080096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.318+0000 7fe916a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c84192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe916a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c84192a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46070/0 (socket says 172.21.15.107:46070) 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe916a00640 1 -- 172.21.15.107:0/3243992769 learned_addr learned my addr 172.21.15.107:0/3243992769 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe917400640 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8410f30 0x5630c8414630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe916000640 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8406670 0x5630c84197e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe916a00640 1 -- 172.21.15.107:0/3243992769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8410f30 msgr2=0x5630c8414630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe916a00640 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8410f30 0x5630c8414630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe916a00640 1 -- 172.21.15.107:0/3243992769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8406670 msgr2=0x5630c84197e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe916a00640 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8406670 0x5630c84197e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe916a00640 1 -- 172.21.15.107:0/3243992769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c8414e60 con 0x5630c8405d50 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe917400640 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8410f30 0x5630c8414630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe916000640 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8406670 0x5630c84197e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe916a00640 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c84192a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe90802fab0 tx=0x7fe908005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ab58b8eff21146 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.319+0000 7fe900c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.320+0000 7fe900c00640 1 -- 172.21.15.107:0/3243992769 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908055040 con 0x5630c8405d50 2024-09-19T04:21:03.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.320+0000 7fe900c00640 1 -- 172.21.15.107:0/3243992769 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fe9080305b0 con 0x5630c8405d50 2024-09-19T04:21:03.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.320+0000 7fe9201bc740 1 -- 172.21.15.107:0/3243992769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe90409c890 con 0x5630c8405d50 2024-09-19T04:21:03.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.320+0000 7fe900c00640 1 -- 172.21.15.107:0/3243992769 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe9080307d0 con 0x5630c8405d50 2024-09-19T04:21:03.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.320+0000 7fe9201bc740 10 client.15330 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:03.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.320+0000 7fe9201bc740 1 -- 172.21.15.107:0/3243992769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630c848b080 con 0x5630c8405d50 2024-09-19T04:21:03.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.320+0000 7fe900c00640 1 -- 172.21.15.107:0/3243992769 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe90409c890 con 0x5630c8405d50 2024-09-19T04:21:03.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.320+0000 7fe900c00640 1 -- 172.21.15.107:0/3243992769 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe9080046c0 con 0x5630c8405d50 2024-09-19T04:21:03.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.321+0000 7fe9201bc740 10 client.15330 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:03.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.321+0000 7fe9201bc740 10 client.15330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.321+0000 7fe9201bc740 20 client.15330 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.321+0000 7fe9201bc740 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5630c848b730 0x5630c848db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.321+0000 7fe9201bc740 4 client.15330 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.321+0000 7fe9201bc740 1 -- 172.21.15.107:0/3243992769 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630c8373e60 con 0x5630c848b730 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.321+0000 7fe916000640 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5630c848b730 0x5630c848db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe916000640 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5630c848b730 0x5630c848db50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3fcb67b9887ab9 server_cookie=80bee795e7b2c151 in_seq=0 out_seq=0 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe900c00640 10 client.15330 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe900c00640 1 -- 172.21.15.107:0/3243992769 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5630c8373e60 con 0x5630c848b730 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe900c00640 10 client.15330 handle_command_reply: tid=0 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe9201bc740 1 client.15330 shutdown 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe9201bc740 1 -- 172.21.15.107:0/3243992769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 msgr2=0x5630c84192a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe9201bc740 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c84192a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe90802fab0 tx=0x7fe908005250 comp rx=0 tx=0).stop 2024-09-19T04:21:03.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe9201bc740 1 -- 172.21.15.107:0/3243992769 shutdown_connections 2024-09-19T04:21:03.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe9201bc740 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5630c848b730 0x5630c848db50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe9201bc740 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c8410f30 0x5630c8414630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe9201bc740 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630c8406670 0x5630c84197e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe9201bc740 1 --2- 172.21.15.107:0/3243992769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630c8405d50 0x5630c84192a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe900c00640 0 client.15330 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:03.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.322+0000 7fe9201bc740 1 -- 172.21.15.107:0/3243992769 >> 172.21.15.107:0/3243992769 conn(0x5630c84022e0 msgr2=0x5630c8400b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:03.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.323+0000 7fe9201bc740 1 -- 172.21.15.107:0/3243992769 shutdown_connections 2024-09-19T04:21:03.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.323+0000 7fe9201bc740 1 -- 172.21.15.107:0/3243992769 wait complete. 2024-09-19T04:21:03.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.323+0000 7fe9201bc740 20 client.15330 trim_cache size 0 max 16384 2024-09-19T04:21:03.327 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:21:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:21:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:21:03.328 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:21:03.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.323+0000 7fe917e00640 1 -- 172.21.15.107:0/202621299 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8fc0a35d0 msgr2=0x7fe8fc0a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:03.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.323+0000 7fe917e00640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8fc0a35d0 0x7fe8fc0a5a90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe910009fd0 tx=0x7fe910004a40 comp rx=0 tx=0).stop 2024-09-19T04:21:03.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.323+0000 7fe917e00640 1 -- 172.21.15.107:0/202621299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe918103510 msgr2=0x7fe91819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:03.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.323+0000 7fe917e00640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe918103510 0x7fe91819a9e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe90400e9c0 tx=0x7fe90400ee80 comp rx=0 tx=0).stop 2024-09-19T04:21:03.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.324+0000 7fe917e00640 1 -- 172.21.15.107:0/202621299 shutdown_connections 2024-09-19T04:21:03.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.324+0000 7fe917e00640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8fc0a35d0 0x7fe8fc0a5a90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.324+0000 7fe917e00640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe91810bfd0 0x7fe91819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.324+0000 7fe917e00640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe918103510 0x7fe91819a9e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.324+0000 7fe917e00640 1 --2- 172.21.15.107:0/202621299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe918102bf0 0x7fe91819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:03.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.324+0000 7fe917e00640 1 -- 172.21.15.107:0/202621299 >> 172.21.15.107:0/202621299 conn(0x7fe9180fc610 msgr2=0x7fe91810d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:03.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.324+0000 7fe917e00640 1 -- 172.21.15.107:0/202621299 shutdown_connections 2024-09-19T04:21:03.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:03.324+0000 7fe917e00640 1 -- 172.21.15.107:0/202621299 wait complete. 2024-09-19T04:21:03.339 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:03 smithi107 ceph-mon[30659]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:21:03.339 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:03 smithi107 ceph-mon[30659]: mgrmap e26: x(active, since 42s), standbys: z, y 2024-09-19T04:21:03.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:03 smithi115 ceph-mon[35477]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:21:03.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:03 smithi115 ceph-mon[35477]: mgrmap e26: x(active, since 42s), standbys: z, y 2024-09-19T04:21:03.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:03 smithi145 ceph-mon[35096]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:21:03.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:03 smithi145 ceph-mon[35096]: mgrmap e26: x(active, since 42s), standbys: z, y 2024-09-19T04:21:04.336 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:21:04.337 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:21:04.786 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:05.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:05 smithi107 ceph-mon[30659]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:05.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.426+0000 7ff2a0800640 1 -- 172.21.15.107:0/1644501426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff298077040 msgr2=0x7ff2981124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:05.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.427+0000 7ff2a0800640 1 --2- 172.21.15.107:0/1644501426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff298077040 0x7ff2981124f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff288009860 tx=0x7ff28802a340 comp rx=0 tx=0).stop 2024-09-19T04:21:05.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.427+0000 7ff2a0800640 1 -- 172.21.15.107:0/1644501426 shutdown_connections 2024-09-19T04:21:05.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.427+0000 7ff2a0800640 1 --2- 172.21.15.107:0/1644501426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff298112a30 0x7ff298114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:05.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.427+0000 7ff2a0800640 1 --2- 172.21.15.107:0/1644501426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff298077040 0x7ff2981124f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:05.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.427+0000 7ff2a0800640 1 --2- 172.21.15.107:0/1644501426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff298076720 0x7ff298076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:05.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.427+0000 7ff2a0800640 1 -- 172.21.15.107:0/1644501426 >> 172.21.15.107:0/1644501426 conn(0x7ff2980ffa90 msgr2=0x7ff298101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:05.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.427+0000 7ff2a0800640 1 -- 172.21.15.107:0/1644501426 shutdown_connections 2024-09-19T04:21:05.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.427+0000 7ff2a0800640 1 -- 172.21.15.107:0/1644501426 wait complete. 2024-09-19T04:21:05.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.428+0000 7ff2a0800640 1 Processor -- start 2024-09-19T04:21:05.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.428+0000 7ff2a0800640 1 -- start start 2024-09-19T04:21:05.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff2a0800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff298076720 0x7ff2981a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:05.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff2a0800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff298077040 0x7ff2981a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:05.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff2a0800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff298112a30 0x7ff29819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:05.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff2a0800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981185c0 con 0x7ff298077040 2024-09-19T04:21:05.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff2a0800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff298118440 con 0x7ff298076720 2024-09-19T04:21:05.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff2a0800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff298118740 con 0x7ff298112a30 2024-09-19T04:21:05.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff298076720 0x7ff2981a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff298112a30 0x7ff29819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff298077040 0x7ff2981a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff298112a30 0x7ff29819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33554/0 (socket says 172.21.15.107:33554) 2024-09-19T04:21:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff298076720 0x7ff2981a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:56510/0 (socket says 172.21.15.107:56510) 2024-09-19T04:21:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29e800640 1 -- 172.21.15.107:0/3288362361 learned_addr learned my addr 172.21.15.107:0/3288362361 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29d400640 1 -- 172.21.15.107:0/3288362361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff298112a30 msgr2=0x7ff29819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29d400640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff298112a30 0x7ff29819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29d400640 1 -- 172.21.15.107:0/3288362361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff298076720 msgr2=0x7ff2981a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29d400640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff298076720 0x7ff2981a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.429+0000 7ff29d400640 1 -- 172.21.15.107:0/3288362361 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29819f000 con 0x7ff298077040 2024-09-19T04:21:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.430+0000 7ff29d400640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff298077040 0x7ff2981a4c50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff288035040 tx=0x7ff288033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.430+0000 7ff292a00640 1 -- 172.21.15.107:0/3288362361 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28803f020 con 0x7ff298077040 2024-09-19T04:21:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.430+0000 7ff292a00640 1 -- 172.21.15.107:0/3288362361 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff288030550 con 0x7ff298077040 2024-09-19T04:21:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.431+0000 7ff2a0800640 1 -- 172.21.15.107:0/3288362361 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29819f290 con 0x7ff298077040 2024-09-19T04:21:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.431+0000 7ff292a00640 1 -- 172.21.15.107:0/3288362361 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288037040 con 0x7ff298077040 2024-09-19T04:21:05.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.431+0000 7ff2a0800640 1 -- 172.21.15.107:0/3288362361 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2981ab5e0 con 0x7ff298077040 2024-09-19T04:21:05.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.432+0000 7ff2a0800640 1 -- 172.21.15.107:0/3288362361 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff298076b00 con 0x7ff298077040 2024-09-19T04:21:05.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.435+0000 7ff292a00640 1 -- 172.21.15.107:0/3288362361 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 138589+0+0 (secure 0 0 0) 0x7ff288042020 con 0x7ff298077040 2024-09-19T04:21:05.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.436+0000 7ff292a00640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2800a3620 0x7ff2800a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:05.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.437+0000 7ff292a00640 1 -- 172.21.15.107:0/3288362361 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7ff2880efb10 con 0x7ff298077040 2024-09-19T04:21:05.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.437+0000 7ff29de00640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2800a3620 0x7ff2800a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.438+0000 7ff29de00640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2800a3620 0x7ff2800a5ae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff29819fa10 tx=0x7ff28c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.440+0000 7ff292a00640 1 -- 172.21.15.107:0/3288362361 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff288049070 con 0x7ff298077040 2024-09-19T04:21:05.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:05 smithi115 ceph-mon[35477]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:05.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:05 smithi145 ceph-mon[35096]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:05.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.659+0000 7ff2a0800640 1 -- 172.21.15.107:0/3288362361 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2981ab840 con 0x7ff298077040 2024-09-19T04:21:05.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.661+0000 7ff292a00640 1 -- 172.21.15.107:0/3288362361 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11992 (secure 0 0 0) 0x7ff2880bc500 con 0x7ff298077040 2024-09-19T04:21:05.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:21:05.663 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"btime":"2024-09-19T04:20:19:828247+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:20:18.993568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24490,24496,24500,24539,24517]},"id":1}]} 2024-09-19T04:21:05.663 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-09-19T04:21:05.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 -- 172.21.15.107:0/3288362361 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2800a3620 msgr2=0x7ff2800a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:05.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2800a3620 0x7ff2800a5ae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff29819fa10 tx=0x7ff28c005f90 comp rx=0 tx=0).stop 2024-09-19T04:21:05.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 -- 172.21.15.107:0/3288362361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff298077040 msgr2=0x7ff2981a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:05.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff298077040 0x7ff2981a4c50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff288035040 tx=0x7ff288033b60 comp rx=0 tx=0).stop 2024-09-19T04:21:05.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 -- 172.21.15.107:0/3288362361 shutdown_connections 2024-09-19T04:21:05.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2800a3620 0x7ff2800a5ae0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:05.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff298112a30 0x7ff29819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:05.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff298077040 0x7ff2981a4c50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:05.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 --2- 172.21.15.107:0/3288362361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff298076720 0x7ff2981a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:05.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 -- 172.21.15.107:0/3288362361 >> 172.21.15.107:0/3288362361 conn(0x7ff2980ffa90 msgr2=0x7ff298101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:05.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.664+0000 7ff2a0800640 1 -- 172.21.15.107:0/3288362361 shutdown_connections 2024-09-19T04:21:05.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:05.665+0000 7ff2a0800640 1 -- 172.21.15.107:0/3288362361 wait complete. 2024-09-19T04:21:06.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:06 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3288362361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:21:06.459 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:06 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3288362361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:21:06.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:06 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3288362361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:21:06.507 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:21:06.943 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:07.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:07 smithi107 ceph-mon[30659]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:07 smithi115 ceph-mon[35477]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:07 smithi145 ceph-mon[35096]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:07.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.564+0000 7f4993200640 1 -- 172.21.15.107:0/2922340812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4994102370 msgr2=0x7f499410e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:07.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.564+0000 7f4993200640 1 --2- 172.21.15.107:0/2922340812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4994102370 0x7f499410e8a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f498c009cd0 tx=0x7f498c02a8e0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.564+0000 7f4993200640 1 -- 172.21.15.107:0/2922340812 shutdown_connections 2024-09-19T04:21:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.564+0000 7f4993200640 1 --2- 172.21.15.107:0/2922340812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4994102370 0x7f499410e8a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.564+0000 7f4993200640 1 --2- 172.21.15.107:0/2922340812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49941019d0 0x7f4994101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.564+0000 7f4993200640 1 --2- 172.21.15.107:0/2922340812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49941079d0 0x7f4994107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.564+0000 7f4993200640 1 -- 172.21.15.107:0/2922340812 >> 172.21.15.107:0/2922340812 conn(0x7f49940fd710 msgr2=0x7f49940ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.565+0000 7f4993200640 1 -- 172.21.15.107:0/2922340812 shutdown_connections 2024-09-19T04:21:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.565+0000 7f4993200640 1 -- 172.21.15.107:0/2922340812 wait complete. 2024-09-19T04:21:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.566+0000 7f4993200640 1 Processor -- start 2024-09-19T04:21:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.566+0000 7f4993200640 1 -- start start 2024-09-19T04:21:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4993200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49941019d0 0x7f499419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4993200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4994102370 0x7f499419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4993200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49941079d0 0x7f499419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4993200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4994111dd0 con 0x7f4994102370 2024-09-19T04:21:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4993200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4994111c50 con 0x7f49941079d0 2024-09-19T04:21:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4993200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4994111f50 con 0x7f49941019d0 2024-09-19T04:21:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4991e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49941019d0 0x7f499419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4991400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4994102370 0x7f499419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4991e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49941019d0 0x7f499419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33576/0 (socket says 172.21.15.107:33576) 2024-09-19T04:21:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4991e00640 1 -- 172.21.15.107:0/1080820968 learned_addr learned my addr 172.21.15.107:0/1080820968 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.567+0000 7f4992800640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49941079d0 0x7f499419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.568+0000 7f4991400640 1 -- 172.21.15.107:0/1080820968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49941019d0 msgr2=0x7f499419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.568+0000 7f4991400640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49941019d0 0x7f499419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.568+0000 7f4991400640 1 -- 172.21.15.107:0/1080820968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49941079d0 msgr2=0x7f499419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.568+0000 7f4991400640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49941079d0 0x7f499419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.568+0000 7f4991400640 1 -- 172.21.15.107:0/1080820968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499419f4a0 con 0x7f4994102370 2024-09-19T04:21:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.568+0000 7f4991e00640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49941019d0 0x7f499419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.568+0000 7f4991400640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4994102370 0x7f499419aa30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f498000e9f0 tx=0x7f498000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:07.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.569+0000 7f498aa00640 1 -- 172.21.15.107:0/1080820968 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498000ccc0 con 0x7f4994102370 2024-09-19T04:21:07.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.569+0000 7f498aa00640 1 -- 172.21.15.107:0/1080820968 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4980010910 con 0x7f4994102370 2024-09-19T04:21:07.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.569+0000 7f498aa00640 1 -- 172.21.15.107:0/1080820968 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980018960 con 0x7f4994102370 2024-09-19T04:21:07.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.569+0000 7f4993200640 1 -- 172.21.15.107:0/1080820968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499419f790 con 0x7f4994102370 2024-09-19T04:21:07.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.569+0000 7f4993200640 1 -- 172.21.15.107:0/1080820968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49941a7070 con 0x7f4994102370 2024-09-19T04:21:07.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.573+0000 7f499bc9c740 1 Processor -- start 2024-09-19T04:21:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.573+0000 7f499bc9c740 1 -- start start 2024-09-19T04:21:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b551023140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b5510362f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b551038c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f499bc9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b550f893e0 con 0x55b551022d60 2024-09-19T04:21:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f499bc9c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b550f30d80 con 0x55b551023680 2024-09-19T04:21:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f499bc9c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b550f853a0 con 0x55b551036830 2024-09-19T04:21:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f4991e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b5510362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f4991e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b5510362f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:56542/0 (socket says 172.21.15.107:56542) 2024-09-19T04:21:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f4991e00640 1 -- 172.21.15.107:0/931873648 learned_addr learned my addr 172.21.15.107:0/931873648 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f4992800640 1 --2- 172.21.15.107:0/931873648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b551023140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.574+0000 7f498aa00640 1 -- 172.21.15.107:0/1080820968 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 138589+0+0 (secure 0 0 0) 0x7f4980010ac0 con 0x7f4994102370 2024-09-19T04:21:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.575+0000 7f4991400640 1 --2- 172.21.15.107:0/931873648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b551038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.575+0000 7f4992800640 1 -- 172.21.15.107:0/931873648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 msgr2=0x55b551038c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.575+0000 7f4992800640 1 --2- 172.21.15.107:0/931873648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b551038c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.575+0000 7f4992800640 1 -- 172.21.15.107:0/931873648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 msgr2=0x55b5510362f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.575+0000 7f4992800640 1 --2- 172.21.15.107:0/931873648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b5510362f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.575+0000 7f4992800640 1 -- 172.21.15.107:0/931873648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b550f4fae0 con 0x55b551022d60 2024-09-19T04:21:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.575+0000 7f4991e00640 1 --2- 172.21.15.107:0/931873648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b5510362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.576+0000 7f4992800640 1 --2- 172.21.15.107:0/931873648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b551023140 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f498c009ca0 tx=0x7f498c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467578dd7acf9766 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:07.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.576+0000 7f497f400640 1 -- 172.21.15.107:0/931873648 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c048070 con 0x55b551022d60 2024-09-19T04:21:07.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.576+0000 7f497f400640 1 -- 172.21.15.107:0/931873648 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c036070 con 0x55b551022d60 2024-09-19T04:21:07.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.576+0000 7f497f400640 1 -- 172.21.15.107:0/931873648 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c004390 con 0x55b551022d60 2024-09-19T04:21:07.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.576+0000 7f498aa00640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f49780a3550 0x7f49780a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.576+0000 7f498aa00640 1 -- 172.21.15.107:0/1080820968 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f49800d0370 con 0x7f4994102370 2024-09-19T04:21:07.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.577+0000 7f499bc9c740 1 -- 172.21.15.107:0/931873648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 msgr2=0x55b551023140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:07.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.577+0000 7f499bc9c740 1 --2- 172.21.15.107:0/931873648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b551023140 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f498c009ca0 tx=0x7f498c009a20 comp rx=0 tx=0).stop 2024-09-19T04:21:07.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.577+0000 7f4991e00640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f49780a3550 0x7f49780a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.577+0000 7f499bc9c740 1 -- 172.21.15.107:0/931873648 shutdown_connections 2024-09-19T04:21:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.577+0000 7f499bc9c740 1 --2- 172.21.15.107:0/931873648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b551038c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.577+0000 7f499bc9c740 1 --2- 172.21.15.107:0/931873648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b5510362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.577+0000 7f499bc9c740 1 --2- 172.21.15.107:0/931873648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b551023140 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.577+0000 7f499bc9c740 1 -- 172.21.15.107:0/931873648 >> 172.21.15.107:0/931873648 conn(0x55b55101f3d0 msgr2=0x55b55101f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:07.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.578+0000 7f499bc9c740 1 -- 172.21.15.107:0/931873648 shutdown_connections 2024-09-19T04:21:07.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.578+0000 7f4991e00640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f49780a3550 0x7f49780a5a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4984009f40 tx=0x7f4984005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:07.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.578+0000 7f499bc9c740 1 -- 172.21.15.107:0/931873648 wait complete. 2024-09-19T04:21:07.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.579+0000 7f499bc9c740 1 -- start start 2024-09-19T04:21:07.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.579+0000 7f499bc9c740 1 Processor -- start 2024-09-19T04:21:07.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.580+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b551033820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.580+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551023680 0x55b551033d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.580+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551036830 0x55b5510342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.580+0000 7f499bc9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b551039340 con 0x55b551022d60 2024-09-19T04:21:07.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.580+0000 7f499bc9c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b550f440f0 con 0x55b551036830 2024-09-19T04:21:07.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.580+0000 7f499bc9c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b5510394c0 con 0x55b551023680 2024-09-19T04:21:07.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.580+0000 7f499bc9c740 10 client.0 _ll_register_callbacks cb 0x55b550f2bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:07.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.581+0000 7f4992800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551036830 0x55b5510342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.581+0000 7f4991400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551023680 0x55b551033d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.581+0000 7f4991e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b551033820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.581+0000 7f4992800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551036830 0x55b5510342a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:56546/0 (socket says 172.21.15.107:56546) 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.581+0000 7f4992800640 1 -- 172.21.15.107:0/2714579052 learned_addr learned my addr 172.21.15.107:0/2714579052 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.581+0000 7f4991e00640 1 -- 172.21.15.107:0/2714579052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551023680 msgr2=0x55b551033d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.581+0000 7f4991e00640 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551023680 0x55b551033d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.581+0000 7f4991e00640 1 -- 172.21.15.107:0/2714579052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551036830 msgr2=0x55b5510342a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.581+0000 7f4991e00640 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551036830 0x55b5510342a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.581+0000 7f4991e00640 1 -- 172.21.15.107:0/2714579052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b550cfa3c0 con 0x55b551022d60 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.582+0000 7f4991400640 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551023680 0x55b551033d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.582+0000 7f4991e00640 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b551033820 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4984009fd0 tx=0x7f4984004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d629c5c8c84c7345 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:07.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.582+0000 7f4973e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:21:07.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.582+0000 7f4973e00640 1 -- 172.21.15.107:0/2714579052 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498400fe60 con 0x55b551022d60 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.582+0000 7f499bc9c740 1 -- 172.21.15.107:0/2714579052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498c036070 con 0x55b551022d60 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.582+0000 7f4973e00640 1 -- 172.21.15.107:0/2714579052 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4984011b20 con 0x55b551022d60 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.583+0000 7f4973e00640 1 -- 172.21.15.107:0/2714579052 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4984011d20 con 0x55b551022d60 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.583+0000 7f499bc9c740 10 client.15348 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.583+0000 7f499bc9c740 1 -- 172.21.15.107:0/2714579052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b551045570 con 0x55b551022d60 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.583+0000 7f4973e00640 1 -- 172.21.15.107:0/2714579052 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c036070 con 0x55b551022d60 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.583+0000 7f4973e00640 1 -- 172.21.15.107:0/2714579052 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4984027070 con 0x55b551022d60 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.584+0000 7f499bc9c740 10 client.15348 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.584+0000 7f499bc9c740 10 client.15348 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.584+0000 7f499bc9c740 20 client.15348 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.584+0000 7f499bc9c740 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b551046980 0x55b551048e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.584+0000 7f499bc9c740 4 client.15348 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:07.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.584+0000 7f499bc9c740 1 -- 172.21.15.107:0/2714579052 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b550f96950 con 0x55b551046980 2024-09-19T04:21:07.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.584+0000 7f4991400640 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b551046980 0x55b551048e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.585+0000 7f4973e00640 10 client.15348 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:21:07.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.585+0000 7f4991400640 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b551046980 0x55b551048e40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2cfcfcd5f683494e server_cookie=daf40bf32cac7ff8 in_seq=0 out_seq=0 2024-09-19T04:21:07.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.585+0000 7f4973e00640 1 -- 172.21.15.107:0/2714579052 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b550f96950 con 0x55b551046980 2024-09-19T04:21:07.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.585+0000 7f4973e00640 10 client.15348 handle_command_reply: tid=0 2024-09-19T04:21:07.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.586+0000 7f499bc9c740 1 client.15348 shutdown 2024-09-19T04:21:07.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.586+0000 7f499bc9c740 1 -- 172.21.15.107:0/2714579052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 msgr2=0x55b551033820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:07.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.586+0000 7f499bc9c740 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b551033820 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4984009fd0 tx=0x7f4984004730 comp rx=0 tx=0).stop 2024-09-19T04:21:07.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.586+0000 7f499bc9c740 1 -- 172.21.15.107:0/2714579052 shutdown_connections 2024-09-19T04:21:07.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.586+0000 7f499bc9c740 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b551046980 0x55b551048e40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.586+0000 7f499bc9c740 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551036830 0x55b5510342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.586+0000 7f499bc9c740 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551023680 0x55b551033d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.586+0000 7f499bc9c740 1 --2- 172.21.15.107:0/2714579052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b551033820 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.586+0000 7f499bc9c740 1 -- 172.21.15.107:0/2714579052 >> 172.21.15.107:0/2714579052 conn(0x55b55101f3d0 msgr2=0x55b551025060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:07.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.586+0000 7f4973e00640 0 client.15348 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:21:07.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.587+0000 7f499bc9c740 1 -- 172.21.15.107:0/2714579052 shutdown_connections 2024-09-19T04:21:07.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.587+0000 7f499bc9c740 1 -- 172.21.15.107:0/2714579052 wait complete. 2024-09-19T04:21:07.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.587+0000 7f499bc9c740 20 client.15348 trim_cache size 0 max 16384 2024-09-19T04:21:07.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.612+0000 7f499bc9c740 1 Processor -- start 2024-09-19T04:21:07.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.612+0000 7f499bc9c740 1 -- start start 2024-09-19T04:21:07.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551022d60 0x55b551033ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551023680 0x55b55104e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b55104eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f499bc9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49941ab360 con 0x55b551023680 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f499bc9c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49941acea0 con 0x55b551022d60 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f499bc9c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49941aba30 con 0x55b551036830 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f4991e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551022d60 0x55b551033ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f4991e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551022d60 0x55b551033ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:56562/0 (socket says 172.21.15.107:56562) 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f4991e00640 1 -- 172.21.15.107:0/1356937693 learned_addr learned my addr 172.21.15.107:0/1356937693 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f4992800640 1 --2- 172.21.15.107:0/1356937693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b55104eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f4991e00640 1 -- 172.21.15.107:0/1356937693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 msgr2=0x55b55104eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f4991e00640 1 --2- 172.21.15.107:0/1356937693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b55104eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f4991e00640 1 -- 172.21.15.107:0/1356937693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551023680 msgr2=0x55b55104e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f4991e00640 1 --2- 172.21.15.107:0/1356937693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551023680 0x55b55104e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.613+0000 7f4991e00640 1 -- 172.21.15.107:0/1356937693 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b55101ec80 con 0x55b551022d60 2024-09-19T04:21:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.614+0000 7f4991e00640 1 --2- 172.21.15.107:0/1356937693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551022d60 0x55b551033ef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f49840288a0 tx=0x7f498400f7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bccc9032c0ee9698 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.614+0000 7f497ea00640 1 -- 172.21.15.107:0/1356937693 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984035020 con 0x55b551022d60 2024-09-19T04:21:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.614+0000 7f497ea00640 1 -- 172.21.15.107:0/1356937693 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498400fe60 con 0x55b551022d60 2024-09-19T04:21:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.614+0000 7f497ea00640 1 -- 172.21.15.107:0/1356937693 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49840039f0 con 0x55b551022d60 2024-09-19T04:21:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.615+0000 7f499bc9c740 1 -- 172.21.15.107:0/1356937693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551022d60 msgr2=0x55b551033ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.615+0000 7f499bc9c740 1 --2- 172.21.15.107:0/1356937693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551022d60 0x55b551033ef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f49840288a0 tx=0x7f498400f7e0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.615+0000 7f499bc9c740 1 -- 172.21.15.107:0/1356937693 shutdown_connections 2024-09-19T04:21:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.615+0000 7f499bc9c740 1 --2- 172.21.15.107:0/1356937693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b55104eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.615+0000 7f499bc9c740 1 --2- 172.21.15.107:0/1356937693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551023680 0x55b55104e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.615+0000 7f499bc9c740 1 --2- 172.21.15.107:0/1356937693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551022d60 0x55b551033ef0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.615+0000 7f499bc9c740 1 -- 172.21.15.107:0/1356937693 >> 172.21.15.107:0/1356937693 conn(0x55b55101f3d0 msgr2=0x55b5510337e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.615+0000 7f499bc9c740 1 -- 172.21.15.107:0/1356937693 shutdown_connections 2024-09-19T04:21:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.615+0000 7f499bc9c740 1 -- 172.21.15.107:0/1356937693 wait complete. 2024-09-19T04:21:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.616+0000 7f499bc9c740 1 -- start start 2024-09-19T04:21:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.616+0000 7f499bc9c740 1 Processor -- start 2024-09-19T04:21:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b55103a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b55103adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f499bc9c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b5510b0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f499bc9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49941ac7d0 con 0x55b551022d60 2024-09-19T04:21:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f499bc9c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49941ac100 con 0x55b551023680 2024-09-19T04:21:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f499bc9c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49941a9380 con 0x55b551036830 2024-09-19T04:21:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f499bc9c740 10 client.0 _ll_register_callbacks cb 0x7f4984012eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f4991e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b55103a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f4991e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b55103a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46138/0 (socket says 172.21.15.107:46138) 2024-09-19T04:21:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f4991e00640 1 -- 172.21.15.107:0/824140338 learned_addr learned my addr 172.21.15.107:0/824140338 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.617+0000 7f4991400640 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b55103adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.618+0000 7f4991400640 1 -- 172.21.15.107:0/824140338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 msgr2=0x55b5510b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.618+0000 7f4991400640 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b5510b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.618+0000 7f4991400640 1 -- 172.21.15.107:0/824140338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 msgr2=0x55b55103a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.618+0000 7f4991400640 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b55103a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.618+0000 7f4991400640 1 -- 172.21.15.107:0/824140338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5510b0c70 con 0x55b551023680 2024-09-19T04:21:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.618+0000 7f4991e00640 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b55103a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.618+0000 7f4991400640 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b55103adf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f49800cd930 tx=0x7f498009c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc3e6516f7c96ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.618+0000 7f4973e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:21:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.618+0000 7f4973e00640 1 -- 172.21.15.107:0/824140338 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49800cde10 con 0x55b551023680 2024-09-19T04:21:07.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.619+0000 7f499bc9c740 1 -- 172.21.15.107:0/824140338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498400fe60 con 0x55b551023680 2024-09-19T04:21:07.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.619+0000 7f4973e00640 1 -- 172.21.15.107:0/824140338 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f498009b8f0 con 0x55b551023680 2024-09-19T04:21:07.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.619+0000 7f4973e00640 1 -- 172.21.15.107:0/824140338 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4980085020 con 0x55b551023680 2024-09-19T04:21:07.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.619+0000 7f499bc9c740 10 client.25367 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:07.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.619+0000 7f499bc9c740 1 -- 172.21.15.107:0/824140338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5510b1dc0 con 0x55b551023680 2024-09-19T04:21:07.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.619+0000 7f4973e00640 1 -- 172.21.15.107:0/824140338 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498400fe60 con 0x55b551023680 2024-09-19T04:21:07.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.619+0000 7f4973e00640 1 -- 172.21.15.107:0/824140338 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f49800b5230 con 0x55b551023680 2024-09-19T04:21:07.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.620+0000 7f499bc9c740 10 client.25367 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:07.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.620+0000 7f499bc9c740 10 client.25367 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:21:07.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.620+0000 7f499bc9c740 20 client.25367 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:07.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.620+0000 7f499bc9c740 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b5510b2540 0x55b5510b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:07.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.620+0000 7f499bc9c740 4 client.25367 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.620+0000 7f499bc9c740 1 -- 172.21.15.107:0/824140338 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b550f90e60 con 0x55b5510b2540 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.620+0000 7f4991e00640 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b5510b2540 0x55b5510b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f4991e00640 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b5510b2540 0x55b5510b4960 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff674b272f54c809 server_cookie=74fdd530b579736a in_seq=0 out_seq=0 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f4973e00640 10 client.25367 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f4973e00640 1 -- 172.21.15.107:0/824140338 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b550f90e60 con 0x55b5510b2540 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f4973e00640 10 client.25367 handle_command_reply: tid=0 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f499bc9c740 1 client.25367 shutdown 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f499bc9c740 1 -- 172.21.15.107:0/824140338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 msgr2=0x55b55103adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f499bc9c740 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b55103adf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f49800cd930 tx=0x7f498009c270 comp rx=0 tx=0).stop 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f499bc9c740 1 -- 172.21.15.107:0/824140338 shutdown_connections 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f499bc9c740 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b5510b2540 0x55b5510b4960 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f499bc9c740 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b551036830 0x55b5510b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f499bc9c740 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b551023680 0x55b55103adf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.621+0000 7f499bc9c740 1 --2- 172.21.15.107:0/824140338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b551022d60 0x55b55103a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.622+0000 7f4973e00640 0 client.25367 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:21:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.622+0000 7f499bc9c740 1 -- 172.21.15.107:0/824140338 >> 172.21.15.107:0/824140338 conn(0x55b55101f3d0 msgr2=0x55b55101dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.622+0000 7f499bc9c740 1 -- 172.21.15.107:0/824140338 shutdown_connections 2024-09-19T04:21:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.622+0000 7f499bc9c740 1 -- 172.21.15.107:0/824140338 wait complete. 2024-09-19T04:21:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.622+0000 7f499bc9c740 20 client.25367 trim_cache size 0 max 16384 2024-09-19T04:21:07.625 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:21:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 -- 172.21.15.107:0/1080820968 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f49780a3550 msgr2=0x7f49780a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f49780a3550 0x7f49780a5a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4984009f40 tx=0x7f4984005f90 comp rx=0 tx=0).stop 2024-09-19T04:21:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 -- 172.21.15.107:0/1080820968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4994102370 msgr2=0x7f499419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4994102370 0x7f499419aa30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f498000e9f0 tx=0x7f498000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 -- 172.21.15.107:0/1080820968 shutdown_connections 2024-09-19T04:21:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f49780a3550 0x7f49780a5a10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49941079d0 0x7f499419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4994102370 0x7f499419aa30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 --2- 172.21.15.107:0/1080820968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49941019d0 0x7f499419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 -- 172.21.15.107:0/1080820968 >> 172.21.15.107:0/1080820968 conn(0x7f49940fd710 msgr2=0x7f49940fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 -- 172.21.15.107:0/1080820968 shutdown_connections 2024-09-19T04:21:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:07.623+0000 7f4993200640 1 -- 172.21.15.107:0/1080820968 wait complete. 2024-09-19T04:21:08.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:08 smithi115 ceph-mon[35477]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T04:21:08.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:08 smithi145 ceph-mon[35096]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T04:21:08.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:08 smithi107 ceph-mon[30659]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T04:21:08.592 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:21:09.104 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:09.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.726+0000 7fa790a00640 1 -- 172.21.15.107:0/4140702567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa78c074ee0 msgr2=0x7fa78c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:09.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.726+0000 7fa790a00640 1 --2- 172.21.15.107:0/4140702567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa78c074ee0 0x7fa78c10dd20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa778009860 tx=0x7fa77802a3d0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.727+0000 7fa790a00640 1 -- 172.21.15.107:0/4140702567 shutdown_connections 2024-09-19T04:21:09.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.727+0000 7fa790a00640 1 --2- 172.21.15.107:0/4140702567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa78c10e260 0x7fa78c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.727+0000 7fa790a00640 1 --2- 172.21.15.107:0/4140702567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa78c074ee0 0x7fa78c10dd20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.727+0000 7fa790a00640 1 --2- 172.21.15.107:0/4140702567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa78c0745c0 0x7fa78c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.727+0000 7fa790a00640 1 -- 172.21.15.107:0/4140702567 >> 172.21.15.107:0/4140702567 conn(0x7fa78c0fd740 msgr2=0x7fa78c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.727+0000 7fa790a00640 1 -- 172.21.15.107:0/4140702567 shutdown_connections 2024-09-19T04:21:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.727+0000 7fa790a00640 1 -- 172.21.15.107:0/4140702567 wait complete. 2024-09-19T04:21:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.728+0000 7fa790a00640 1 Processor -- start 2024-09-19T04:21:09.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.728+0000 7fa790a00640 1 -- start start 2024-09-19T04:21:09.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa790a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa78c0745c0 0x7fa78c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa790a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa78c074ee0 0x7fa78c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa790a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa78c10e260 0x7fa78c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa790a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c113fb0 con 0x7fa78c074ee0 2024-09-19T04:21:09.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa790a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c113e30 con 0x7fa78c0745c0 2024-09-19T04:21:09.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa790a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c114130 con 0x7fa78c10e260 2024-09-19T04:21:09.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa78a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa78c0745c0 0x7fa78c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa78a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa78c0745c0 0x7fa78c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54686/0 (socket says 172.21.15.107:54686) 2024-09-19T04:21:09.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa78a800640 1 -- 172.21.15.107:0/1970866708 learned_addr learned my addr 172.21.15.107:0/1970866708 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:09.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa789e00640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa78c074ee0 0x7fa78c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.729+0000 7fa78b200640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa78c10e260 0x7fa78c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.730+0000 7fa78a800640 1 -- 172.21.15.107:0/1970866708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa78c10e260 msgr2=0x7fa78c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.730+0000 7fa78a800640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa78c10e260 0x7fa78c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.730+0000 7fa78a800640 1 -- 172.21.15.107:0/1970866708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa78c074ee0 msgr2=0x7fa78c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.730+0000 7fa78a800640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa78c074ee0 0x7fa78c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.730+0000 7fa78a800640 1 -- 172.21.15.107:0/1970866708 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78c19f440 con 0x7fa78c0745c0 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.730+0000 7fa78a800640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa78c0745c0 0x7fa78c19a490 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00e9c0 tx=0x7fa77c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.730+0000 7fa783400640 1 -- 172.21.15.107:0/1970866708 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00cde0 con 0x7fa78c0745c0 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.731+0000 7fa783400640 1 -- 172.21.15.107:0/1970866708 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77c0035f0 con 0x7fa78c0745c0 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.731+0000 7fa783400640 1 -- 172.21.15.107:0/1970866708 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c010650 con 0x7fa78c0745c0 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.731+0000 7fa790a00640 1 -- 172.21.15.107:0/1970866708 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78c19f730 con 0x7fa78c0745c0 2024-09-19T04:21:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.731+0000 7fa790a00640 1 -- 172.21.15.107:0/1970866708 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78c1a7010 con 0x7fa78c0745c0 2024-09-19T04:21:09.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.735+0000 7fa793e33740 1 Processor -- start 2024-09-19T04:21:09.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.736+0000 7fa793e33740 1 -- start start 2024-09-19T04:21:09.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.736+0000 7fa783400640 1 -- 172.21.15.107:0/1970866708 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) ==== 138589+0+0 (secure 0 0 0) 0x7fa77c0037a0 con 0x7fa78c0745c0 2024-09-19T04:21:09.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.736+0000 7fa793e33740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef95e40 0x56066ef96220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.736+0000 7fa793e33740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef96760 0x56066efa0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.736+0000 7fa793e33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066efa3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.736+0000 7fa793e33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56066eefc3e0 con 0x56066efa1020 2024-09-19T04:21:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.736+0000 7fa793e33740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56066eea3d80 con 0x56066ef96760 2024-09-19T04:21:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.736+0000 7fa793e33740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56066eef83a0 con 0x56066ef95e40 2024-09-19T04:21:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.736+0000 7fa78a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef96760 0x56066efa0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.736+0000 7fa78b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef95e40 0x56066ef96220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.737+0000 7fa78b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef95e40 0x56066ef96220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41640/0 (socket says 172.21.15.107:41640) 2024-09-19T04:21:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.737+0000 7fa78a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef96760 0x56066efa0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54690/0 (socket says 172.21.15.107:54690) 2024-09-19T04:21:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.737+0000 7fa78b200640 1 -- 172.21.15.107:0/777594429 learned_addr learned my addr 172.21.15.107:0/777594429 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.737+0000 7fa789e00640 1 --2- 172.21.15.107:0/777594429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066efa3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.737+0000 7fa789e00640 1 -- 172.21.15.107:0/777594429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef95e40 msgr2=0x56066ef96220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.737+0000 7fa789e00640 1 --2- 172.21.15.107:0/777594429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef95e40 0x56066ef96220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.737+0000 7fa789e00640 1 -- 172.21.15.107:0/777594429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef96760 msgr2=0x56066efa0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.737+0000 7fa789e00640 1 --2- 172.21.15.107:0/777594429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef96760 0x56066efa0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.737+0000 7fa789e00640 1 -- 172.21.15.107:0/777594429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56066eec2ae0 con 0x56066efa1020 2024-09-19T04:21:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.738+0000 7fa789e00640 1 --2- 172.21.15.107:0/777594429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066efa3410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa778005e00 tx=0x7fa77802f300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855fdb599af688b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.738+0000 7fa783400640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa7700a34d0 0x7fa7700a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.738+0000 7fa783400640 1 -- 172.21.15.107:0/1970866708 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fa77c0d02e0 con 0x7fa78c0745c0 2024-09-19T04:21:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.738+0000 7fa78b200640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa7700a34d0 0x7fa7700a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.739+0000 7fa777400640 1 -- 172.21.15.107:0/777594429 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778030cb0 con 0x56066efa1020 2024-09-19T04:21:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.739+0000 7fa777400640 1 -- 172.21.15.107:0/777594429 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa778030e50 con 0x56066efa1020 2024-09-19T04:21:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.739+0000 7fa777400640 1 -- 172.21.15.107:0/777594429 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7780046c0 con 0x56066efa1020 2024-09-19T04:21:09.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.740+0000 7fa793e33740 1 -- 172.21.15.107:0/777594429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 msgr2=0x56066efa3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:09.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.740+0000 7fa793e33740 1 --2- 172.21.15.107:0/777594429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066efa3410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa778005e00 tx=0x7fa77802f300 comp rx=0 tx=0).stop 2024-09-19T04:21:09.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.740+0000 7fa78b200640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa7700a34d0 0x7fa7700a5990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa784013520 tx=0x7fa784009600 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:09.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.740+0000 7fa793e33740 1 -- 172.21.15.107:0/777594429 shutdown_connections 2024-09-19T04:21:09.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.740+0000 7fa793e33740 1 --2- 172.21.15.107:0/777594429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066efa3410 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.740+0000 7fa793e33740 1 --2- 172.21.15.107:0/777594429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef96760 0x56066efa0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.740+0000 7fa793e33740 1 --2- 172.21.15.107:0/777594429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef95e40 0x56066ef96220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.740+0000 7fa793e33740 1 -- 172.21.15.107:0/777594429 >> 172.21.15.107:0/777594429 conn(0x56066ef923d0 msgr2=0x56066ef927b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:09.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.740+0000 7fa793e33740 1 -- 172.21.15.107:0/777594429 shutdown_connections 2024-09-19T04:21:09.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.740+0000 7fa793e33740 1 -- 172.21.15.107:0/777594429 wait complete. 2024-09-19T04:21:09.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.741+0000 7fa793e33740 1 -- start start 2024-09-19T04:21:09.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.741+0000 7fa793e33740 1 Processor -- start 2024-09-19T04:21:09.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa793e33740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa793e33740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066ef9e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa793e33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066ef9ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa793e33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56066efac400 con 0x56066efa1020 2024-09-19T04:21:09.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa793e33740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56066eeb70f0 con 0x56066ef95e40 2024-09-19T04:21:09.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa793e33740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56066efac580 con 0x56066ef96760 2024-09-19T04:21:09.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa793e33740 10 client.0 _ll_register_callbacks cb 0x56066ee9ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:09.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa78a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa78a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54698/0 (socket says 172.21.15.107:54698) 2024-09-19T04:21:09.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa78a800640 1 -- 172.21.15.107:0/2660171118 learned_addr learned my addr 172.21.15.107:0/2660171118 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:09.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.742+0000 7fa789e00640 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066ef9e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.743+0000 7fa78a800640 1 -- 172.21.15.107:0/2660171118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 msgr2=0x56066ef9e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:09.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.743+0000 7fa78a800640 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066ef9e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.743+0000 7fa78a800640 1 -- 172.21.15.107:0/2660171118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 msgr2=0x56066ef9ea60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:21:09.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.743+0000 7fa78b200640 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066ef9ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.743+0000 7fa78a800640 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066ef9ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.743+0000 7fa78a800640 1 -- 172.21.15.107:0/2660171118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56066ec6d3c0 con 0x56066ef95e40 2024-09-19T04:21:09.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.743+0000 7fa78b200640 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066ef9ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:09.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.744+0000 7fa78a800640 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9dfe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa77c09c060 tx=0x7fa77c0d3980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3725ef145a4f554b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:09.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.744+0000 7fa76be00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:21:09.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.744+0000 7fa76be00640 1 -- 172.21.15.107:0/2660171118 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0cf030 con 0x56066ef95e40 2024-09-19T04:21:09.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.744+0000 7fa76be00640 1 -- 172.21.15.107:0/2660171118 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fa77c0cdd80 con 0x56066ef95e40 2024-09-19T04:21:09.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.744+0000 7fa793e33740 1 -- 172.21.15.107:0/2660171118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa778030e50 con 0x56066ef95e40 2024-09-19T04:21:09.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.745+0000 7fa76be00640 1 -- 172.21.15.107:0/2660171118 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa77c09bea0 con 0x56066ef95e40 2024-09-19T04:21:09.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.745+0000 7fa793e33740 10 client.25379 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:09.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.745+0000 7fa793e33740 1 -- 172.21.15.107:0/2660171118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56066efb85a0 con 0x56066ef95e40 2024-09-19T04:21:09.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.746+0000 7fa76be00640 1 -- 172.21.15.107:0/2660171118 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa778030e50 con 0x56066ef95e40 2024-09-19T04:21:09.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.746+0000 7fa76be00640 1 -- 172.21.15.107:0/2660171118 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa77c09b7b0 con 0x56066ef95e40 2024-09-19T04:21:09.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.746+0000 7fa793e33740 10 client.25379 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:09.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.746+0000 7fa793e33740 10 client.25379 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:21:09.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.746+0000 7fa793e33740 20 client.25379 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:09.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.746+0000 7fa793e33740 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56066efb99b0 0x56066efbbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.746+0000 7fa793e33740 4 client.25379 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:09.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.746+0000 7fa793e33740 1 -- 172.21.15.107:0/2660171118 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56066ef09950 con 0x56066efb99b0 2024-09-19T04:21:09.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.747+0000 7fa789e00640 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56066efb99b0 0x56066efbbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.747+0000 7fa789e00640 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56066efb99b0 0x56066efbbda0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c68dfa8a8c42a7b1 server_cookie=517611b8f73424a2 in_seq=0 out_seq=0 2024-09-19T04:21:09.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.747+0000 7fa76be00640 10 client.25379 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:21:09.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.748+0000 7fa76be00640 1 -- 172.21.15.107:0/2660171118 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56066ef09950 con 0x56066efb99b0 2024-09-19T04:21:09.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.748+0000 7fa76be00640 10 client.25379 handle_command_reply: tid=0 2024-09-19T04:21:09.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.748+0000 7fa793e33740 1 client.25379 shutdown 2024-09-19T04:21:09.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 1 -- 172.21.15.107:0/2660171118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 msgr2=0x56066ef9dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9dfe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa77c09c060 tx=0x7fa77c0d3980 comp rx=0 tx=0).stop 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 1 -- 172.21.15.107:0/2660171118 shutdown_connections 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56066efb99b0 0x56066efbbda0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066ef9ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066ef9e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 1 --2- 172.21.15.107:0/2660171118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9dfe0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 1 -- 172.21.15.107:0/2660171118 >> 172.21.15.107:0/2660171118 conn(0x56066ef923d0 msgr2=0x56066ef980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa76be00640 0 client.25379 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 1 -- 172.21.15.107:0/2660171118 shutdown_connections 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 1 -- 172.21.15.107:0/2660171118 wait complete. 2024-09-19T04:21:09.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.749+0000 7fa793e33740 20 client.25379 trim_cache size 0 max 16384 2024-09-19T04:21:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.765+0000 7fa793e33740 1 Processor -- start 2024-09-19T04:21:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.765+0000 7fa793e33740 1 -- start start 2024-09-19T04:21:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa793e33740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa793e33740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066efc1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa793e33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066efc1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa793e33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1ab3c0 con 0x56066efa1020 2024-09-19T04:21:09.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa793e33740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1acf00 con 0x56066ef95e40 2024-09-19T04:21:09.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa793e33740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1aba90 con 0x56066ef96760 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa78a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa789e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066efc1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa78a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54704/0 (socket says 172.21.15.107:54704) 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa78a800640 1 -- 172.21.15.107:0/1332785483 learned_addr learned my addr 172.21.15.107:0/1332785483 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.766+0000 7fa78b200640 1 --2- 172.21.15.107:0/1332785483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066efc1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.767+0000 7fa78a800640 1 -- 172.21.15.107:0/1332785483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 msgr2=0x56066efc1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.767+0000 7fa78a800640 1 --2- 172.21.15.107:0/1332785483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066efc1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.767+0000 7fa78a800640 1 -- 172.21.15.107:0/1332785483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 msgr2=0x56066efc1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.767+0000 7fa78a800640 1 --2- 172.21.15.107:0/1332785483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066efc1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.767+0000 7fa78a800640 1 -- 172.21.15.107:0/1332785483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56066efbc420 con 0x56066ef95e40 2024-09-19T04:21:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.767+0000 7fa789e00640 1 --2- 172.21.15.107:0/1332785483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066efc1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:09.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.767+0000 7fa78a800640 1 --2- 172.21.15.107:0/1332785483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9ff50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa77c09c2f0 tx=0x7fa77c003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b6a3a3448801bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:09.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.767+0000 7fa776a00640 1 -- 172.21.15.107:0/1332785483 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0cf070 con 0x56066ef95e40 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.767+0000 7fa776a00640 1 -- 172.21.15.107:0/1332785483 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77c0cd7e0 con 0x56066ef95e40 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.767+0000 7fa776a00640 1 -- 172.21.15.107:0/1332785483 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c09bcc0 con 0x56066ef95e40 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.768+0000 7fa793e33740 1 -- 172.21.15.107:0/1332785483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 msgr2=0x56066ef9ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.768+0000 7fa793e33740 1 --2- 172.21.15.107:0/1332785483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9ff50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa77c09c2f0 tx=0x7fa77c003d20 comp rx=0 tx=0).stop 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.768+0000 7fa793e33740 1 -- 172.21.15.107:0/1332785483 shutdown_connections 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.768+0000 7fa793e33740 1 --2- 172.21.15.107:0/1332785483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066efa1020 0x56066efc1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.768+0000 7fa793e33740 1 --2- 172.21.15.107:0/1332785483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066efc1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.768+0000 7fa793e33740 1 --2- 172.21.15.107:0/1332785483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066ef95e40 0x56066ef9ff50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.768+0000 7fa793e33740 1 -- 172.21.15.107:0/1332785483 >> 172.21.15.107:0/1332785483 conn(0x56066ef923d0 msgr2=0x56066ef9f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.769+0000 7fa793e33740 1 -- 172.21.15.107:0/1332785483 shutdown_connections 2024-09-19T04:21:09.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.769+0000 7fa793e33740 1 -- 172.21.15.107:0/1332785483 wait complete. 2024-09-19T04:21:09.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.769+0000 7fa793e33740 1 -- start start 2024-09-19T04:21:09.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.769+0000 7fa793e33740 1 Processor -- start 2024-09-19T04:21:09.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.770+0000 7fa793e33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066ef95e40 0x56066ef9af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.770+0000 7fa793e33740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066ef9b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.770+0000 7fa793e33740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066efa1020 0x56066ef9b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.770+0000 7fa793e33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1ac830 con 0x56066ef95e40 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.770+0000 7fa793e33740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1ac160 con 0x56066efa1020 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.770+0000 7fa793e33740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1a93a0 con 0x56066ef96760 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.770+0000 7fa793e33740 10 client.0 _ll_register_callbacks cb 0x7fa77c003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.770+0000 7fa78a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066ef95e40 0x56066ef9af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.770+0000 7fa78a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066ef95e40 0x56066ef9af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39766/0 (socket says 172.21.15.107:39766) 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.770+0000 7fa78a800640 1 -- 172.21.15.107:0/4253121467 learned_addr learned my addr 172.21.15.107:0/4253121467 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.771+0000 7fa789e00640 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066ef9b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.771+0000 7fa78b200640 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066efa1020 0x56066ef9b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.771+0000 7fa78a800640 1 -- 172.21.15.107:0/4253121467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 msgr2=0x56066ef9b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.771+0000 7fa78a800640 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066ef9b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.771+0000 7fa78a800640 1 -- 172.21.15.107:0/4253121467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066efa1020 msgr2=0x56066ef9b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:09.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.771+0000 7fa78a800640 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066efa1020 0x56066ef9b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.771+0000 7fa78a800640 1 -- 172.21.15.107:0/4253121467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56066f023640 con 0x56066ef95e40 2024-09-19T04:21:09.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.771+0000 7fa78b200640 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066efa1020 0x56066ef9b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:09.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.771+0000 7fa78a800640 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066ef95e40 0x56066ef9af20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0a4aa0 tx=0x7fa77c004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fe8bb0b1104e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:09.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.771+0000 7fa76be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:21:09.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.772+0000 7fa76be00640 1 -- 172.21.15.107:0/4253121467 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0cf070 con 0x56066ef95e40 2024-09-19T04:21:09.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.772+0000 7fa76be00640 1 -- 172.21.15.107:0/4253121467 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fa77c0cdd80 con 0x56066ef95e40 2024-09-19T04:21:09.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.772+0000 7fa793e33740 1 -- 172.21.15.107:0/4253121467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa77c0cd7e0 con 0x56066ef95e40 2024-09-19T04:21:09.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.772+0000 7fa76be00640 1 -- 172.21.15.107:0/4253121467 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa77c081050 con 0x56066ef95e40 2024-09-19T04:21:09.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.772+0000 7fa793e33740 10 client.15366 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:09.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.772+0000 7fa793e33740 1 -- 172.21.15.107:0/4253121467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56066f024840 con 0x56066ef95e40 2024-09-19T04:21:09.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.772+0000 7fa76be00640 1 -- 172.21.15.107:0/4253121467 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77c0cd7e0 con 0x56066ef95e40 2024-09-19T04:21:09.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.773+0000 7fa76be00640 1 -- 172.21.15.107:0/4253121467 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa77c084020 con 0x56066ef95e40 2024-09-19T04:21:09.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.773+0000 7fa793e33740 10 client.15366 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:09.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.773+0000 7fa793e33740 10 client.15366 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:21:09.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.773+0000 7fa793e33740 20 client.15366 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:09.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.773+0000 7fa793e33740 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56066f024c20 0x56066f027010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:09.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.773+0000 7fa793e33740 4 client.15366 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.773+0000 7fa793e33740 1 -- 172.21.15.107:0/4253121467 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56066ef03e60 con 0x56066f024c20 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.773+0000 7fa789e00640 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56066f024c20 0x56066f027010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa789e00640 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56066f024c20 0x56066f027010 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4320a3f11b55021e server_cookie=c0d30b57bbd26412 in_seq=0 out_seq=0 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa76be00640 10 client.15366 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa76be00640 1 -- 172.21.15.107:0/4253121467 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56066ef03e60 con 0x56066f024c20 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa76be00640 10 client.15366 handle_command_reply: tid=0 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa793e33740 1 client.15366 shutdown 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa793e33740 1 -- 172.21.15.107:0/4253121467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066ef95e40 msgr2=0x56066ef9af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa793e33740 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066ef95e40 0x56066ef9af20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0a4aa0 tx=0x7fa77c004520 comp rx=0 tx=0).stop 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa793e33740 1 -- 172.21.15.107:0/4253121467 shutdown_connections 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa793e33740 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56066f024c20 0x56066f027010 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa793e33740 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56066efa1020 0x56066ef9b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa793e33740 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56066ef96760 0x56066ef9b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa793e33740 1 --2- 172.21.15.107:0/4253121467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56066ef95e40 0x56066ef9af20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.774+0000 7fa793e33740 1 -- 172.21.15.107:0/4253121467 >> 172.21.15.107:0/4253121467 conn(0x56066ef923d0 msgr2=0x56066ef9f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:09.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.775+0000 7fa76be00640 0 client.15366 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:21:09.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.775+0000 7fa793e33740 1 -- 172.21.15.107:0/4253121467 shutdown_connections 2024-09-19T04:21:09.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.775+0000 7fa793e33740 1 -- 172.21.15.107:0/4253121467 wait complete. 2024-09-19T04:21:09.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.775+0000 7fa793e33740 20 client.15366 trim_cache size 0 max 16384 2024-09-19T04:21:09.779 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:21:09.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 -- 172.21.15.107:0/1970866708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa7700a34d0 msgr2=0x7fa7700a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:09.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa7700a34d0 0x7fa7700a5990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa784013520 tx=0x7fa784009600 comp rx=0 tx=0).stop 2024-09-19T04:21:09.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 -- 172.21.15.107:0/1970866708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa78c0745c0 msgr2=0x7fa78c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:09.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa78c0745c0 0x7fa78c19a490 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00e9c0 tx=0x7fa77c00ee80 comp rx=0 tx=0).stop 2024-09-19T04:21:09.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 -- 172.21.15.107:0/1970866708 shutdown_connections 2024-09-19T04:21:09.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa7700a34d0 0x7fa7700a5990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa78c10e260 0x7fa78c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa78c074ee0 0x7fa78c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 --2- 172.21.15.107:0/1970866708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa78c0745c0 0x7fa78c19a490 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:09.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 -- 172.21.15.107:0/1970866708 >> 172.21.15.107:0/1970866708 conn(0x7fa78c0fd740 msgr2=0x7fa78c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:09.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 -- 172.21.15.107:0/1970866708 shutdown_connections 2024-09-19T04:21:09.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:09.776+0000 7fa790a00640 1 -- 172.21.15.107:0/1970866708 wait complete. 2024-09-19T04:21:10.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:10 smithi107 ceph-mon[30659]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T04:21:10.646 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:21:10.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:10 smithi115 ceph-mon[35477]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T04:21:10.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:10 smithi145 ceph-mon[35096]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T04:21:11.144 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:11.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.686+0000 7fb533200640 1 -- 172.21.15.107:0/1526219892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb53410bfc0 msgr2=0x7fb53410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.686+0000 7fb533200640 1 --2- 172.21.15.107:0/1526219892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb53410bfc0 0x7fb53410e460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009ca0 tx=0x7fb52c02a950 comp rx=0 tx=0).stop 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.686+0000 7fb533200640 1 -- 172.21.15.107:0/1526219892 shutdown_connections 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.686+0000 7fb533200640 1 --2- 172.21.15.107:0/1526219892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb53410bfc0 0x7fb53410e460 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.686+0000 7fb533200640 1 --2- 172.21.15.107:0/1526219892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5341034b0 0x7fb53410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.686+0000 7fb533200640 1 --2- 172.21.15.107:0/1526219892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb534102b90 0x7fb534102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.686+0000 7fb533200640 1 -- 172.21.15.107:0/1526219892 >> 172.21.15.107:0/1526219892 conn(0x7fb5340fc610 msgr2=0x7fb5340fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.686+0000 7fb533200640 1 -- 172.21.15.107:0/1526219892 shutdown_connections 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.686+0000 7fb533200640 1 -- 172.21.15.107:0/1526219892 wait complete. 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.687+0000 7fb533200640 1 Processor -- start 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.687+0000 7fb533200640 1 -- start start 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.687+0000 7fb533200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb534102b90 0x7fb534196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.688+0000 7fb533200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5341034b0 0x7fb534196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.688+0000 7fb533200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb53410bfc0 0x7fb53419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.688+0000 7fb533200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb53410fc20 con 0x7fb534102b90 2024-09-19T04:21:11.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.688+0000 7fb533200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb53410faa0 con 0x7fb5341034b0 2024-09-19T04:21:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.688+0000 7fb533200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb53410fda0 con 0x7fb53410bfc0 2024-09-19T04:21:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.688+0000 7fb531400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5341034b0 0x7fb534196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.688+0000 7fb532800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb53410bfc0 0x7fb53419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.688+0000 7fb531400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5341034b0 0x7fb534196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54726/0 (socket says 172.21.15.107:54726) 2024-09-19T04:21:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.688+0000 7fb531400640 1 -- 172.21.15.107:0/3702086091 learned_addr learned my addr 172.21.15.107:0/3702086091 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.688+0000 7fb531e00640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb534102b90 0x7fb534196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.689+0000 7fb532800640 1 -- 172.21.15.107:0/3702086091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5341034b0 msgr2=0x7fb534196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.689+0000 7fb532800640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5341034b0 0x7fb534196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.689+0000 7fb532800640 1 -- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb534102b90 msgr2=0x7fb534196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.689+0000 7fb532800640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb534102b90 0x7fb534196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.689+0000 7fb532800640 1 -- 172.21.15.107:0/3702086091 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53419aff0 con 0x7fb53410bfc0 2024-09-19T04:21:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.689+0000 7fb532800640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb53410bfc0 0x7fb53419a910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb52c005e30 tx=0x7fb52c003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.689+0000 7fb531e00640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb534102b90 0x7fb534196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.689+0000 7fb52aa00640 1 -- 172.21.15.107:0/3702086091 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c034040 con 0x7fb53410bfc0 2024-09-19T04:21:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.689+0000 7fb533200640 1 -- 172.21.15.107:0/3702086091 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53419b280 con 0x7fb53410bfc0 2024-09-19T04:21:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.690+0000 7fb533200640 1 -- 172.21.15.107:0/3702086091 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5341a2bc0 con 0x7fb53410bfc0 2024-09-19T04:21:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.690+0000 7fb52aa00640 1 -- 172.21.15.107:0/3702086091 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52c02f940 con 0x7fb53410bfc0 2024-09-19T04:21:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.690+0000 7fb52aa00640 1 -- 172.21.15.107:0/3702086091 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c0045e0 con 0x7fb53410bfc0 2024-09-19T04:21:11.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.694+0000 7fb53b419740 1 Processor -- start 2024-09-19T04:21:11.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.695+0000 7fb52aa00640 1 -- 172.21.15.107:0/3702086091 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) ==== 138589+0+0 (secure 0 0 0) 0x7fb52c037020 con 0x7fb53410bfc0 2024-09-19T04:21:11.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.696+0000 7fb53b419740 1 -- start start 2024-09-19T04:21:11.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.696+0000 7fb52aa00640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb5180a3550 0x7fb5180a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.696+0000 7fb52aa00640 1 -- 172.21.15.107:0/3702086091 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fb52c0f0c50 con 0x7fb53410bfc0 2024-09-19T04:21:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.697+0000 7fb53b419740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb4e40 0x55e20fbb5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.697+0000 7fb53b419740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb5760 0x55e20fbbfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.697+0000 7fb53b419740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbc2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.697+0000 7fb53b419740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e20fb1b3e0 con 0x55e20fbb4e40 2024-09-19T04:21:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.697+0000 7fb53b419740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e20fac2d80 con 0x55e20fbc0020 2024-09-19T04:21:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.697+0000 7fb53b419740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e20fb173a0 con 0x55e20fbb5760 2024-09-19T04:21:11.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.697+0000 7fb532800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbc2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.697+0000 7fb532800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbc2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54736/0 (socket says 172.21.15.107:54736) 2024-09-19T04:21:11.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.697+0000 7fb532800640 1 -- 172.21.15.107:0/3574883300 learned_addr learned my addr 172.21.15.107:0/3574883300 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.697+0000 7fb531e00640 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb4e40 0x55e20fbb5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb531400640 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb5760 0x55e20fbbfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb532800640 1 -- 172.21.15.107:0/3574883300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb5760 msgr2=0x55e20fbbfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb532800640 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb5760 0x55e20fbbfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb532800640 1 -- 172.21.15.107:0/3574883300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb4e40 msgr2=0x55e20fbb5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb532800640 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb4e40 0x55e20fbb5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb532800640 1 -- 172.21.15.107:0/3574883300 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20fae1ae0 con 0x55e20fbc0020 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb531e00640 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb4e40 0x55e20fbb5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb531400640 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb5760 0x55e20fbbfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb532800640 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbc2410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009ca0 tx=0x7fb52c0bc740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2d2b4d6e38f278 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb51f000640 1 -- 172.21.15.107:0/3574883300 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c0c1190 con 0x55e20fbc0020 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.698+0000 7fb51f000640 1 -- 172.21.15.107:0/3574883300 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52c0c5a50 con 0x55e20fbc0020 2024-09-19T04:21:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.699+0000 7fb51f000640 1 -- 172.21.15.107:0/3574883300 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c0bd040 con 0x55e20fbc0020 2024-09-19T04:21:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.699+0000 7fb53b419740 1 -- 172.21.15.107:0/3574883300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 msgr2=0x55e20fbc2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.699+0000 7fb53b419740 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbc2410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009ca0 tx=0x7fb52c0bc740 comp rx=0 tx=0).stop 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.699+0000 7fb531400640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb5180a3550 0x7fb5180a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.700+0000 7fb53b419740 1 -- 172.21.15.107:0/3574883300 shutdown_connections 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.700+0000 7fb53b419740 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbc2410 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.700+0000 7fb53b419740 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb5760 0x55e20fbbfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.700+0000 7fb53b419740 1 --2- 172.21.15.107:0/3574883300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb4e40 0x55e20fbb5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.700+0000 7fb53b419740 1 -- 172.21.15.107:0/3574883300 >> 172.21.15.107:0/3574883300 conn(0x55e20fbb13d0 msgr2=0x55e20fbb17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.700+0000 7fb53b419740 1 -- 172.21.15.107:0/3574883300 shutdown_connections 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.700+0000 7fb53b419740 1 -- 172.21.15.107:0/3574883300 wait complete. 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.700+0000 7fb53b419740 1 -- start start 2024-09-19T04:21:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.701+0000 7fb53b419740 1 Processor -- start 2024-09-19T04:21:11.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.701+0000 7fb53b419740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbbcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.701+0000 7fb53b419740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbb5760 0x55e20fbbd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.701+0000 7fb531400640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb5180a3550 0x7fb5180a5a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb520005f60 tx=0x7fb520004730 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.701+0000 7fb53b419740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbc0020 0x55e20fbbd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.701+0000 7fb53b419740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e20fac2d80 con 0x55e20fbc0020 2024-09-19T04:21:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.701+0000 7fb53b419740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e20fad60f0 con 0x55e20fbb5760 2024-09-19T04:21:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.701+0000 7fb53b419740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e20fbcb1c0 con 0x55e20fbb4e40 2024-09-19T04:21:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.702+0000 7fb531400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbb5760 0x55e20fbbd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.702+0000 7fb53b419740 10 client.0 _ll_register_callbacks cb 0x55e20fabdbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.702+0000 7fb531400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbb5760 0x55e20fbbd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54742/0 (socket says 172.21.15.107:54742) 2024-09-19T04:21:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.702+0000 7fb531400640 1 -- 172.21.15.107:0/2566099041 learned_addr learned my addr 172.21.15.107:0/2566099041 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.702+0000 7fb531e00640 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbbcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.702+0000 7fb532800640 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbc0020 0x55e20fbbd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.703+0000 7fb531e00640 1 -- 172.21.15.107:0/2566099041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbb5760 msgr2=0x55e20fbbd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.703+0000 7fb531e00640 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbb5760 0x55e20fbbd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.703+0000 7fb531e00640 1 -- 172.21.15.107:0/2566099041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbc0020 msgr2=0x55e20fbbd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.703+0000 7fb531e00640 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbc0020 0x55e20fbbd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.703+0000 7fb531e00640 1 -- 172.21.15.107:0/2566099041 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20f88c3c0 con 0x55e20fbb4e40 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.703+0000 7fb532800640 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbc0020 0x55e20fbbd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.704+0000 7fb531e00640 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbbcf00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb52400b910 tx=0x7fb52400bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a710e411ef99dbc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.704+0000 7fb513400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.704+0000 7fb513400640 1 -- 172.21.15.107:0/2566099041 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52401b030 con 0x55e20fbb4e40 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.704+0000 7fb513400640 1 -- 172.21.15.107:0/2566099041 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fb524008810 con 0x55e20fbb4e40 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.704+0000 7fb53b419740 1 -- 172.21.15.107:0/2566099041 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52c0c5a50 con 0x55e20fbb4e40 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb513400640 1 -- 172.21.15.107:0/2566099041 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb524008a10 con 0x55e20fbb4e40 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb53b419740 10 client.25378 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb53b419740 1 -- 172.21.15.107:0/2566099041 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e20fbd6f80 con 0x55e20fbb4e40 2024-09-19T04:21:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb513400640 1 -- 172.21.15.107:0/2566099041 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52c0c5a50 con 0x55e20fbb4e40 2024-09-19T04:21:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb513400640 1 -- 172.21.15.107:0/2566099041 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb5240163d0 con 0x55e20fbb4e40 2024-09-19T04:21:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb53b419740 10 client.25378 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb53b419740 10 client.25378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:21:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb53b419740 20 client.25378 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb53b419740 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e20fbd8490 0x55e20fbda8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb53b419740 4 client.25378 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.705+0000 7fb53b419740 1 -- 172.21.15.107:0/2566099041 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e20fb28950 con 0x55e20fbd8490 2024-09-19T04:21:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.706+0000 7fb531400640 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e20fbd8490 0x55e20fbda8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.707+0000 7fb513400640 10 client.25378 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:21:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.707+0000 7fb531400640 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e20fbd8490 0x55e20fbda8f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6c182de79eb0c2e server_cookie=223f5e0953707007 in_seq=0 out_seq=0 2024-09-19T04:21:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb513400640 1 -- 172.21.15.107:0/2566099041 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e20fb28950 con 0x55e20fbd8490 2024-09-19T04:21:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb513400640 10 client.25378 handle_command_reply: tid=0 2024-09-19T04:21:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 client.25378 shutdown 2024-09-19T04:21:11.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 -- 172.21.15.107:0/2566099041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 msgr2=0x55e20fbbcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:11.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbbcf00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb52400b910 tx=0x7fb52400bd10 comp rx=0 tx=0).stop 2024-09-19T04:21:11.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 -- 172.21.15.107:0/2566099041 shutdown_connections 2024-09-19T04:21:11.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e20fbd8490 0x55e20fbda8f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbc0020 0x55e20fbbd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbb5760 0x55e20fbbd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 --2- 172.21.15.107:0/2566099041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbbcf00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb513400640 0 client.25378 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:21:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 -- 172.21.15.107:0/2566099041 >> 172.21.15.107:0/2566099041 conn(0x55e20fbb13d0 msgr2=0x55e20fbb7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 -- 172.21.15.107:0/2566099041 shutdown_connections 2024-09-19T04:21:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.708+0000 7fb53b419740 1 -- 172.21.15.107:0/2566099041 wait complete. 2024-09-19T04:21:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.709+0000 7fb53b419740 20 client.25378 trim_cache size 0 max 16384 2024-09-19T04:21:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.733+0000 7fb53b419740 1 Processor -- start 2024-09-19T04:21:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.733+0000 7fb53b419740 1 -- start start 2024-09-19T04:21:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.734+0000 7fb53b419740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.734+0000 7fb53b419740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 0x55e20fbe01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.734+0000 7fb53b419740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbe0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.734+0000 7fb53b419740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5341ab360 con 0x55e20fbb5760 2024-09-19T04:21:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.734+0000 7fb53b419740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb5341acea0 con 0x55e20fbc0020 2024-09-19T04:21:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.734+0000 7fb53b419740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb5341aba30 con 0x55e20fbb4e40 2024-09-19T04:21:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.734+0000 7fb531e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.734+0000 7fb531e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41728/0 (socket says 172.21.15.107:41728) 2024-09-19T04:21:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.734+0000 7fb531e00640 1 -- 172.21.15.107:0/2465225631 learned_addr learned my addr 172.21.15.107:0/2465225631 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.734+0000 7fb531400640 1 --2- 172.21.15.107:0/2465225631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 0x55e20fbe01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.735+0000 7fb532800640 1 --2- 172.21.15.107:0/2465225631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbe0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.735+0000 7fb531e00640 1 -- 172.21.15.107:0/2465225631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 msgr2=0x55e20fbe0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.735+0000 7fb531e00640 1 --2- 172.21.15.107:0/2465225631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbe0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.735+0000 7fb531e00640 1 -- 172.21.15.107:0/2465225631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 msgr2=0x55e20fbe01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.735+0000 7fb531e00640 1 --2- 172.21.15.107:0/2465225631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 0x55e20fbe01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.735+0000 7fb531e00640 1 -- 172.21.15.107:0/2465225631 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20fbb2540 con 0x55e20fbb4e40 2024-09-19T04:21:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.735+0000 7fb531e00640 1 --2- 172.21.15.107:0/2465225631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb6ff0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb524001eb0 tx=0x7fb52400efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e4b9e53341fd41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.736+0000 7fb51e600640 1 -- 172.21.15.107:0/2465225631 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52402d070 con 0x55e20fbb4e40 2024-09-19T04:21:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.736+0000 7fb51e600640 1 -- 172.21.15.107:0/2465225631 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb524005520 con 0x55e20fbb4e40 2024-09-19T04:21:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.736+0000 7fb51e600640 1 -- 172.21.15.107:0/2465225631 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52401b040 con 0x55e20fbb4e40 2024-09-19T04:21:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.737+0000 7fb53b419740 1 -- 172.21.15.107:0/2465225631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 msgr2=0x55e20fbb6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.737+0000 7fb53b419740 1 --2- 172.21.15.107:0/2465225631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb6ff0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb524001eb0 tx=0x7fb52400efd0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.737+0000 7fb53b419740 1 -- 172.21.15.107:0/2465225631 shutdown_connections 2024-09-19T04:21:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.737+0000 7fb53b419740 1 --2- 172.21.15.107:0/2465225631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbe0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.737+0000 7fb53b419740 1 --2- 172.21.15.107:0/2465225631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 0x55e20fbe01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.737+0000 7fb53b419740 1 --2- 172.21.15.107:0/2465225631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb6ff0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.737+0000 7fb53b419740 1 -- 172.21.15.107:0/2465225631 >> 172.21.15.107:0/2465225631 conn(0x55e20fbb13d0 msgr2=0x55e20fbb68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.737+0000 7fb53b419740 1 -- 172.21.15.107:0/2465225631 shutdown_connections 2024-09-19T04:21:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.738+0000 7fb53b419740 1 -- 172.21.15.107:0/2465225631 wait complete. 2024-09-19T04:21:11.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.738+0000 7fb53b419740 1 -- start start 2024-09-19T04:21:11.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.738+0000 7fb53b419740 1 Processor -- start 2024-09-19T04:21:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.739+0000 7fb53b419740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.739+0000 7fb53b419740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 0x55e20fbba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.739+0000 7fb53b419740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.739+0000 7fb53b419740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5341ac7d0 con 0x55e20fbb5760 2024-09-19T04:21:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.739+0000 7fb53b419740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb5341ac100 con 0x55e20fbc0020 2024-09-19T04:21:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.739+0000 7fb53b419740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb5341a9340 con 0x55e20fbb4e40 2024-09-19T04:21:11.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.739+0000 7fb53b419740 10 client.0 _ll_register_callbacks cb 0x7fb52400d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:11.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.739+0000 7fb531e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.740+0000 7fb531e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41734/0 (socket says 172.21.15.107:41734) 2024-09-19T04:21:11.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.740+0000 7fb531e00640 1 -- 172.21.15.107:0/3471911084 learned_addr learned my addr 172.21.15.107:0/3471911084 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:11.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.740+0000 7fb532800640 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.740+0000 7fb531e00640 1 -- 172.21.15.107:0/3471911084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 msgr2=0x55e20fbba880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.740+0000 7fb531e00640 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbba880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.740+0000 7fb531400640 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 0x55e20fbba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.740+0000 7fb531e00640 1 -- 172.21.15.107:0/3471911084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 msgr2=0x55e20fbba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.740+0000 7fb531e00640 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 0x55e20fbba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.740+0000 7fb531e00640 1 -- 172.21.15.107:0/3471911084 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20fc42190 con 0x55e20fbb4e40 2024-09-19T04:21:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.740+0000 7fb531400640 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 0x55e20fbba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.741+0000 7fb531e00640 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb9e00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb524016960 tx=0x7fb52400bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ccacaa1388b66e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.741+0000 7fb513400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:21:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.741+0000 7fb513400640 1 -- 172.21.15.107:0/3471911084 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52402d070 con 0x55e20fbb4e40 2024-09-19T04:21:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.741+0000 7fb513400640 1 -- 172.21.15.107:0/3471911084 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fb524008dc0 con 0x55e20fbb4e40 2024-09-19T04:21:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.741+0000 7fb53b419740 1 -- 172.21.15.107:0/3471911084 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb524005520 con 0x55e20fbb4e40 2024-09-19T04:21:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.741+0000 7fb513400640 1 -- 172.21.15.107:0/3471911084 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb52403f020 con 0x55e20fbb4e40 2024-09-19T04:21:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.741+0000 7fb53b419740 10 client.25384 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.741+0000 7fb53b419740 1 -- 172.21.15.107:0/3471911084 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e20fc43370 con 0x55e20fbb4e40 2024-09-19T04:21:11.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.742+0000 7fb513400640 1 -- 172.21.15.107:0/3471911084 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5240163d0 con 0x55e20fbb4e40 2024-09-19T04:21:11.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.742+0000 7fb513400640 1 -- 172.21.15.107:0/3471911084 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb52403d4b0 con 0x55e20fbb4e40 2024-09-19T04:21:11.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.742+0000 7fb53b419740 10 client.25384 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:11.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.742+0000 7fb53b419740 10 client.25384 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:21:11.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.742+0000 7fb53b419740 20 client.25384 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:11.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.742+0000 7fb53b419740 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e20fc439d0 0x55e20fc45dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:11.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.742+0000 7fb53b419740 4 client.25384 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:21:11.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.742+0000 7fb53b419740 1 -- 172.21.15.107:0/3471911084 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb524005520 con 0x55e20fc439d0 2024-09-19T04:21:11.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.743+0000 7fb531400640 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e20fc439d0 0x55e20fc45dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:11.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.743+0000 7fb531400640 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e20fc439d0 0x55e20fc45dc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4c7a409eaaa561c server_cookie=adec19d41779b2df in_seq=0 out_seq=0 2024-09-19T04:21:11.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.743+0000 7fb513400640 10 client.25384 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:21:11.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb513400640 1 -- 172.21.15.107:0/3471911084 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb524005520 con 0x55e20fc439d0 2024-09-19T04:21:11.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb513400640 10 client.25384 handle_command_reply: tid=0 2024-09-19T04:21:11.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb53b419740 1 client.25384 shutdown 2024-09-19T04:21:11.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb53b419740 1 -- 172.21.15.107:0/3471911084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 msgr2=0x55e20fbb9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:11.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb53b419740 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb9e00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb524016960 tx=0x7fb52400bd10 comp rx=0 tx=0).stop 2024-09-19T04:21:11.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb53b419740 1 -- 172.21.15.107:0/3471911084 shutdown_connections 2024-09-19T04:21:11.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb53b419740 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e20fc439d0 0x55e20fc45dc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb53b419740 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e20fbc0020 0x55e20fbba880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb53b419740 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20fbb5760 0x55e20fbba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb513400640 0 client.25384 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:21:11.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb53b419740 1 --2- 172.21.15.107:0/3471911084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e20fbb4e40 0x55e20fbb9e00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.744+0000 7fb53b419740 1 -- 172.21.15.107:0/3471911084 >> 172.21.15.107:0/3471911084 conn(0x55e20fbb13d0 msgr2=0x55e20fbb7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:11.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.745+0000 7fb53b419740 1 -- 172.21.15.107:0/3471911084 shutdown_connections 2024-09-19T04:21:11.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.745+0000 7fb53b419740 1 -- 172.21.15.107:0/3471911084 wait complete. 2024-09-19T04:21:11.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.745+0000 7fb53b419740 20 client.25384 trim_cache size 0 max 16384 2024-09-19T04:21:11.748 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:21:11.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.747+0000 7fb533200640 1 -- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb5180a3550 msgr2=0x7fb5180a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.747+0000 7fb533200640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb5180a3550 0x7fb5180a5a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb520005f60 tx=0x7fb520004730 comp rx=0 tx=0).stop 2024-09-19T04:21:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.747+0000 7fb533200640 1 -- 172.21.15.107:0/3702086091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb53410bfc0 msgr2=0x7fb53419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.747+0000 7fb533200640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb53410bfc0 0x7fb53419a910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb52c005e30 tx=0x7fb52c003b80 comp rx=0 tx=0).stop 2024-09-19T04:21:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.747+0000 7fb533200640 1 -- 172.21.15.107:0/3702086091 shutdown_connections 2024-09-19T04:21:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.747+0000 7fb533200640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb5180a3550 0x7fb5180a5a10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.747+0000 7fb533200640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb53410bfc0 0x7fb53419a910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.747+0000 7fb533200640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5341034b0 0x7fb534196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.747+0000 7fb533200640 1 --2- 172.21.15.107:0/3702086091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb534102b90 0x7fb534196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.747+0000 7fb533200640 1 -- 172.21.15.107:0/3702086091 >> 172.21.15.107:0/3702086091 conn(0x7fb5340fc610 msgr2=0x7fb5340fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.748+0000 7fb533200640 1 -- 172.21.15.107:0/3702086091 shutdown_connections 2024-09-19T04:21:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:11.748+0000 7fb533200640 1 -- 172.21.15.107:0/3702086091 wait complete. 2024-09-19T04:21:12.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:12 smithi107 ceph-mon[30659]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-19T04:21:12.653 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:21:12.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:12 smithi115 ceph-mon[35477]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-19T04:21:12.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:12 smithi145 ceph-mon[35096]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-19T04:21:13.237 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:13.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.872+0000 7f4c7d000640 1 -- 172.21.15.107:0/2776923233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7810bfc0 msgr2=0x7f4c7810e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:13.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.872+0000 7f4c7d000640 1 --2- 172.21.15.107:0/2776923233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7810bfc0 0x7f4c7810e460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009ca0 tx=0x7f4c7002a950 comp rx=0 tx=0).stop 2024-09-19T04:21:13.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.872+0000 7f4c7d000640 1 -- 172.21.15.107:0/2776923233 shutdown_connections 2024-09-19T04:21:13.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.872+0000 7f4c7d000640 1 --2- 172.21.15.107:0/2776923233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7810bfc0 0x7f4c7810e460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.872+0000 7f4c7d000640 1 --2- 172.21.15.107:0/2776923233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c781034b0 0x7f4c7810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.872+0000 7f4c7d000640 1 --2- 172.21.15.107:0/2776923233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c78102b90 0x7f4c78102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.872+0000 7f4c7d000640 1 -- 172.21.15.107:0/2776923233 >> 172.21.15.107:0/2776923233 conn(0x7f4c780fc610 msgr2=0x7f4c780fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:13.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.873+0000 7f4c7d000640 1 -- 172.21.15.107:0/2776923233 shutdown_connections 2024-09-19T04:21:13.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.873+0000 7f4c7d000640 1 -- 172.21.15.107:0/2776923233 wait complete. 2024-09-19T04:21:13.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.874+0000 7f4c7d000640 1 Processor -- start 2024-09-19T04:21:13.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.875+0000 7f4c7d000640 1 -- start start 2024-09-19T04:21:13.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.875+0000 7f4c7d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c78102b90 0x7f4c78073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.875+0000 7f4c7d000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c781034b0 0x7f4c7806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.875+0000 7f4c7d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7810bfc0 0x7f4c7806f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.875+0000 7f4c7d000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4c78111dd0 con 0x7f4c78102b90 2024-09-19T04:21:13.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.875+0000 7f4c7d000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4c78111c50 con 0x7f4c781034b0 2024-09-19T04:21:13.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.875+0000 7f4c7d000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4c78111f50 con 0x7f4c7810bfc0 2024-09-19T04:21:13.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c76a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c781034b0 0x7f4c7806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c76a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c781034b0 0x7f4c7806ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54760/0 (socket says 172.21.15.107:54760) 2024-09-19T04:21:13.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c76a00640 1 -- 172.21.15.107:0/3587645630 learned_addr learned my addr 172.21.15.107:0/3587645630 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:13.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c77400640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c78102b90 0x7f4c78073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c77e00640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7810bfc0 0x7f4c7806f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c76a00640 1 -- 172.21.15.107:0/3587645630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7810bfc0 msgr2=0x7f4c7806f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:13.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c76a00640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7810bfc0 0x7f4c7806f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c76a00640 1 -- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c78102b90 msgr2=0x7f4c78073ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:13.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c76a00640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c78102b90 0x7f4c78073ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c76a00640 1 -- 172.21.15.107:0/3587645630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7806fbc0 con 0x7f4c781034b0 2024-09-19T04:21:13.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.876+0000 7f4c77400640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c78102b90 0x7f4c78073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:13.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.877+0000 7f4c76a00640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c781034b0 0x7f4c7806ef50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400e9f0 tx=0x7f4c6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:13.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.877+0000 7f4c6fe00640 1 -- 172.21.15.107:0/3587645630 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6400ccc0 con 0x7f4c781034b0 2024-09-19T04:21:13.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.877+0000 7f4c7d000640 1 -- 172.21.15.107:0/3587645630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c781af2f0 con 0x7f4c781034b0 2024-09-19T04:21:13.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.877+0000 7f4c7d000640 1 -- 172.21.15.107:0/3587645630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c781af7d0 con 0x7f4c781034b0 2024-09-19T04:21:13.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.877+0000 7f4c6fe00640 1 -- 172.21.15.107:0/3587645630 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c64010910 con 0x7f4c781034b0 2024-09-19T04:21:13.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.877+0000 7f4c6fe00640 1 -- 172.21.15.107:0/3587645630 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64018920 con 0x7f4c781034b0 2024-09-19T04:21:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.881+0000 7f4c6fe00640 1 -- 172.21.15.107:0/3587645630 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) ==== 138589+0+0 (secure 0 0 0) 0x7f4c64010430 con 0x7f4c781034b0 2024-09-19T04:21:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.882+0000 7f4c81036740 1 Processor -- start 2024-09-19T04:21:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.883+0000 7f4c81036740 1 -- start start 2024-09-19T04:21:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.883+0000 7f4c81036740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 0x557030424a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.883+0000 7f4c81036740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x55703042f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.883+0000 7f4c81036740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042f830 0x557030431c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.883+0000 7f4c81036740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5570303823e0 con 0x557030424650 2024-09-19T04:21:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.883+0000 7f4c81036740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557030329d80 con 0x557030424f70 2024-09-19T04:21:13.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.883+0000 7f4c81036740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55703037e3a0 con 0x55703042f830 2024-09-19T04:21:13.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.883+0000 7f4c6fe00640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c5c0a3550 0x7f4c5c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.883+0000 7f4c6fe00640 1 -- 172.21.15.107:0/3587645630 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4c640d00f0 con 0x7f4c781034b0 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.884+0000 7f4c76a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042f830 0x557030431c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.884+0000 7f4c76a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042f830 0x557030431c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41760/0 (socket says 172.21.15.107:41760) 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.884+0000 7f4c76a00640 1 -- 172.21.15.107:0/3113476114 learned_addr learned my addr 172.21.15.107:0/3113476114 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.884+0000 7f4c77400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x55703042f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.884+0000 7f4c77e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 0x557030424a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.884+0000 7f4c77e00640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c5c0a3550 0x7f4c5c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.885+0000 7f4c76a00640 1 -- 172.21.15.107:0/3113476114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 msgr2=0x55703042f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.885+0000 7f4c76a00640 1 --2- 172.21.15.107:0/3113476114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x55703042f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.885+0000 7f4c76a00640 1 -- 172.21.15.107:0/3113476114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 msgr2=0x557030424a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.885+0000 7f4c76a00640 1 --2- 172.21.15.107:0/3113476114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 0x557030424a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.885+0000 7f4c76a00640 1 -- 172.21.15.107:0/3113476114 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557030348ae0 con 0x55703042f830 2024-09-19T04:21:13.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.885+0000 7f4c77e00640 1 --2- 172.21.15.107:0/3113476114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 0x557030424a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:13.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.886+0000 7f4c77e00640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c5c0a3550 0x7f4c5c0a5a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c70006010 tx=0x7f4c700095c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:13.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.886+0000 7f4c76a00640 1 --2- 172.21.15.107:0/3113476114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042f830 0x557030431c20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4c6409c100 tx=0x7f4c6409c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551f3fe298babce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:13.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.886+0000 7f4c6cc00640 1 -- 172.21.15.107:0/3113476114 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c640a4de0 con 0x55703042f830 2024-09-19T04:21:13.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.886+0000 7f4c6cc00640 1 -- 172.21.15.107:0/3113476114 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c64010720 con 0x55703042f830 2024-09-19T04:21:13.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.886+0000 7f4c6cc00640 1 -- 172.21.15.107:0/3113476114 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c640cd990 con 0x55703042f830 2024-09-19T04:21:13.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.887+0000 7f4c81036740 1 -- 172.21.15.107:0/3113476114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042f830 msgr2=0x557030431c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:13.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.887+0000 7f4c81036740 1 --2- 172.21.15.107:0/3113476114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042f830 0x557030431c20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4c6409c100 tx=0x7f4c6409c930 comp rx=0 tx=0).stop 2024-09-19T04:21:13.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.887+0000 7f4c81036740 1 -- 172.21.15.107:0/3113476114 shutdown_connections 2024-09-19T04:21:13.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.887+0000 7f4c81036740 1 --2- 172.21.15.107:0/3113476114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042f830 0x557030431c20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.887+0000 7f4c81036740 1 --2- 172.21.15.107:0/3113476114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x55703042f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.887+0000 7f4c81036740 1 --2- 172.21.15.107:0/3113476114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 0x557030424a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.887+0000 7f4c81036740 1 -- 172.21.15.107:0/3113476114 >> 172.21.15.107:0/3113476114 conn(0x557030420be0 msgr2=0x557030420fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:13.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.887+0000 7f4c81036740 1 -- 172.21.15.107:0/3113476114 shutdown_connections 2024-09-19T04:21:13.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.888+0000 7f4c81036740 1 -- 172.21.15.107:0/3113476114 wait complete. 2024-09-19T04:21:13.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.888+0000 7f4c81036740 1 -- start start 2024-09-19T04:21:13.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.889+0000 7f4c81036740 1 Processor -- start 2024-09-19T04:21:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.889+0000 7f4c81036740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 0x55703042c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.889+0000 7f4c81036740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x55703042ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.889+0000 7f4c81036740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042d3a0 0x55703043b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.889+0000 7f4c81036740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557030432400 con 0x557030424650 2024-09-19T04:21:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.889+0000 7f4c81036740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55703033d0f0 con 0x557030424f70 2024-09-19T04:21:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.889+0000 7f4c81036740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557030432580 con 0x55703042d3a0 2024-09-19T04:21:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.889+0000 7f4c81036740 10 client.0 _ll_register_callbacks cb 0x557030324bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.890+0000 7f4c77e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042d3a0 0x55703043b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.890+0000 7f4c77e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042d3a0 0x55703043b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41774/0 (socket says 172.21.15.107:41774) 2024-09-19T04:21:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.890+0000 7f4c77e00640 1 -- 172.21.15.107:0/872424539 learned_addr learned my addr 172.21.15.107:0/872424539 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.890+0000 7f4c76a00640 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x55703042ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.890+0000 7f4c77400640 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 0x55703042c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.890+0000 7f4c77e00640 1 -- 172.21.15.107:0/872424539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 msgr2=0x55703042ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:13.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.891+0000 7f4c77e00640 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x55703042ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.891+0000 7f4c77e00640 1 -- 172.21.15.107:0/872424539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 msgr2=0x55703042c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:13.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.891+0000 7f4c77e00640 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 0x55703042c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.891+0000 7f4c77e00640 1 -- 172.21.15.107:0/872424539 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570300f33c0 con 0x55703042d3a0 2024-09-19T04:21:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.891+0000 7f4c77400640 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 0x55703042c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.891+0000 7f4c76a00640 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x55703042ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.891+0000 7f4c77e00640 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042d3a0 0x55703043b8d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4c7002fbf0 tx=0x7f4c7002a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=674a791f7783d3cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.891+0000 7f4c61600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:21:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.892+0000 7f4c61600640 1 -- 172.21.15.107:0/872424539 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7004c030 con 0x55703042d3a0 2024-09-19T04:21:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.892+0000 7f4c81036740 1 -- 172.21.15.107:0/872424539 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c64010720 con 0x55703042d3a0 2024-09-19T04:21:13.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.892+0000 7f4c61600640 1 -- 172.21.15.107:0/872424539 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4c70004250 con 0x55703042d3a0 2024-09-19T04:21:13.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.892+0000 7f4c61600640 1 -- 172.21.15.107:0/872424539 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c70055030 con 0x55703042d3a0 2024-09-19T04:21:13.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.892+0000 7f4c81036740 10 client.25396 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:13.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.892+0000 7f4c81036740 1 -- 172.21.15.107:0/872424539 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55703043ea70 con 0x55703042d3a0 2024-09-19T04:21:13.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.893+0000 7f4c61600640 1 -- 172.21.15.107:0/872424539 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c64010720 con 0x55703042d3a0 2024-09-19T04:21:13.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.893+0000 7f4c61600640 1 -- 172.21.15.107:0/872424539 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c70047040 con 0x55703042d3a0 2024-09-19T04:21:13.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.893+0000 7f4c81036740 10 client.25396 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:13.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.893+0000 7f4c81036740 10 client.25396 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:21:13.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.893+0000 7f4c81036740 20 client.25396 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:13.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.893+0000 7f4c81036740 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55703043ff80 0x557030442400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.893+0000 7f4c81036740 4 client.25396 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:13.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.893+0000 7f4c81036740 1 -- 172.21.15.107:0/872424539 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55703038f950 con 0x55703043ff80 2024-09-19T04:21:13.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.894+0000 7f4c77400640 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55703043ff80 0x557030442400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.895+0000 7f4c77400640 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55703043ff80 0x557030442400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16590eb708463e79 server_cookie=e8567d6d6b330a9c in_seq=0 out_seq=0 2024-09-19T04:21:13.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.895+0000 7f4c61600640 10 client.25396 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:21:13.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.895+0000 7f4c61600640 1 -- 172.21.15.107:0/872424539 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55703038f950 con 0x55703043ff80 2024-09-19T04:21:13.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.895+0000 7f4c61600640 10 client.25396 handle_command_reply: tid=0 2024-09-19T04:21:13.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c81036740 1 client.25396 shutdown 2024-09-19T04:21:13.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c81036740 1 -- 172.21.15.107:0/872424539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042d3a0 msgr2=0x55703043b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:13.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c81036740 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042d3a0 0x55703043b8d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4c7002fbf0 tx=0x7f4c7002a950 comp rx=0 tx=0).stop 2024-09-19T04:21:13.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c81036740 1 -- 172.21.15.107:0/872424539 shutdown_connections 2024-09-19T04:21:13.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c81036740 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55703043ff80 0x557030442400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c81036740 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55703042d3a0 0x55703043b8d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c81036740 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x55703042ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c81036740 1 --2- 172.21.15.107:0/872424539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557030424650 0x55703042c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c81036740 1 -- 172.21.15.107:0/872424539 >> 172.21.15.107:0/872424539 conn(0x557030420be0 msgr2=0x557030426910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:13.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c61600640 0 client.25396 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:21:13.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.896+0000 7f4c81036740 1 -- 172.21.15.107:0/872424539 shutdown_connections 2024-09-19T04:21:13.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.897+0000 7f4c81036740 1 -- 172.21.15.107:0/872424539 wait complete. 2024-09-19T04:21:13.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.897+0000 7f4c81036740 20 client.25396 trim_cache size 0 max 16384 2024-09-19T04:21:13.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.923+0000 7f4c81036740 1 Processor -- start 2024-09-19T04:21:13.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.923+0000 7f4c81036740 1 -- start start 2024-09-19T04:21:13.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c81036740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 0x55703043b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c81036740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x557030447cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c81036740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 0x557030448210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c81036740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5800e760 con 0x55703042d3a0 2024-09-19T04:21:13.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c81036740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4c580102a0 con 0x557030424f70 2024-09-19T04:21:13.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c81036740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5800ee30 con 0x557030424650 2024-09-19T04:21:13.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c77e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 0x557030448210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c76a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x557030447cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c76a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x557030447cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54784/0 (socket says 172.21.15.107:54784) 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c76a00640 1 -- 172.21.15.107:0/3372171646 learned_addr learned my addr 172.21.15.107:0/3372171646 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.924+0000 7f4c77400640 1 --2- 172.21.15.107:0/3372171646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 0x55703043b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.925+0000 7f4c77e00640 1 -- 172.21.15.107:0/3372171646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 msgr2=0x55703043b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.925+0000 7f4c77e00640 1 --2- 172.21.15.107:0/3372171646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 0x55703043b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.925+0000 7f4c77e00640 1 -- 172.21.15.107:0/3372171646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 msgr2=0x557030447cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.925+0000 7f4c77e00640 1 --2- 172.21.15.107:0/3372171646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x557030447cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.925+0000 7f4c77e00640 1 -- 172.21.15.107:0/3372171646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557030424480 con 0x55703042d3a0 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.925+0000 7f4c76a00640 1 --2- 172.21.15.107:0/3372171646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x557030447cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.926+0000 7f4c77e00640 1 --2- 172.21.15.107:0/3372171646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 0x557030448210 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009ca0 tx=0x7f4c70009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bf69153e1c70ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.926+0000 7f4c63e00640 1 -- 172.21.15.107:0/3372171646 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7004c020 con 0x55703042d3a0 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.926+0000 7f4c63e00640 1 -- 172.21.15.107:0/3372171646 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c70005bf0 con 0x55703042d3a0 2024-09-19T04:21:13.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.926+0000 7f4c63e00640 1 -- 172.21.15.107:0/3372171646 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70034040 con 0x55703042d3a0 2024-09-19T04:21:13.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.927+0000 7f4c81036740 1 -- 172.21.15.107:0/3372171646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 msgr2=0x557030448210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:13.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.927+0000 7f4c81036740 1 --2- 172.21.15.107:0/3372171646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 0x557030448210 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009ca0 tx=0x7f4c70009820 comp rx=0 tx=0).stop 2024-09-19T04:21:13.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.927+0000 7f4c81036740 1 -- 172.21.15.107:0/3372171646 shutdown_connections 2024-09-19T04:21:13.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.927+0000 7f4c81036740 1 --2- 172.21.15.107:0/3372171646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 0x557030448210 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.927+0000 7f4c81036740 1 --2- 172.21.15.107:0/3372171646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x557030447cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.927+0000 7f4c81036740 1 --2- 172.21.15.107:0/3372171646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 0x55703043b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.927+0000 7f4c81036740 1 -- 172.21.15.107:0/3372171646 >> 172.21.15.107:0/3372171646 conn(0x557030420be0 msgr2=0x55703042b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:13.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.927+0000 7f4c81036740 1 -- 172.21.15.107:0/3372171646 shutdown_connections 2024-09-19T04:21:13.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.928+0000 7f4c81036740 1 -- 172.21.15.107:0/3372171646 wait complete. 2024-09-19T04:21:13.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.928+0000 7f4c81036740 1 -- start start 2024-09-19T04:21:13.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.928+0000 7f4c81036740 1 Processor -- start 2024-09-19T04:21:13.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c81036740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 0x557030437b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c81036740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x557030438050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c81036740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 0x557030432ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c81036740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5800fbd0 con 0x55703042d3a0 2024-09-19T04:21:13.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c81036740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5800f500 con 0x557030424f70 2024-09-19T04:21:13.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c81036740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5800c740 con 0x557030424650 2024-09-19T04:21:13.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c81036740 10 client.0 _ll_register_callbacks cb 0x7f4c64005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:13.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c77400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 0x557030437b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c77400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 0x557030437b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41802/0 (socket says 172.21.15.107:41802) 2024-09-19T04:21:13.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c77400640 1 -- 172.21.15.107:0/1682657111 learned_addr learned my addr 172.21.15.107:0/1682657111 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:13.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.929+0000 7f4c76a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x557030438050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.930+0000 7f4c77e00640 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 0x557030432ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.930+0000 7f4c77400640 1 -- 172.21.15.107:0/1682657111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 msgr2=0x557030438050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.930+0000 7f4c77400640 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x557030438050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.930+0000 7f4c77400640 1 -- 172.21.15.107:0/1682657111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 msgr2=0x557030432ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.930+0000 7f4c77400640 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 0x557030432ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.930+0000 7f4c77400640 1 -- 172.21.15.107:0/1682657111 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557030421870 con 0x557030424650 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.930+0000 7f4c77e00640 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 0x557030432ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.930+0000 7f4c77400640 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 0x557030437b10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800bcc0 tx=0x7f4c680149f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d174ba8247321890 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.930+0000 7f4c61600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.931+0000 7f4c61600640 1 -- 172.21.15.107:0/1682657111 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68005230 con 0x557030424650 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.931+0000 7f4c61600640 1 -- 172.21.15.107:0/1682657111 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4c680053d0 con 0x557030424650 2024-09-19T04:21:13.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.931+0000 7f4c81036740 1 -- 172.21.15.107:0/1682657111 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c70005bf0 con 0x557030424650 2024-09-19T04:21:13.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.931+0000 7f4c61600640 1 -- 172.21.15.107:0/1682657111 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c680055d0 con 0x557030424650 2024-09-19T04:21:13.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.931+0000 7f4c81036740 10 client.25402 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:13.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.931+0000 7f4c81036740 1 -- 172.21.15.107:0/1682657111 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570304aa6c0 con 0x557030424650 2024-09-19T04:21:13.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.932+0000 7f4c61600640 1 -- 172.21.15.107:0/1682657111 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c70005bf0 con 0x557030424650 2024-09-19T04:21:13.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.932+0000 7f4c61600640 1 -- 172.21.15.107:0/1682657111 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c68022020 con 0x557030424650 2024-09-19T04:21:13.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.932+0000 7f4c81036740 10 client.25402 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:13.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.932+0000 7f4c81036740 10 client.25402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:21:13.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.932+0000 7f4c81036740 20 client.25402 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:13.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.932+0000 7f4c81036740 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5570304aad40 0x5570304ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:13.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.932+0000 7f4c81036740 4 client.25402 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:21:13.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.932+0000 7f4c81036740 1 -- 172.21.15.107:0/1682657111 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557030389e60 con 0x5570304aad40 2024-09-19T04:21:13.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.933+0000 7f4c76a00640 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5570304aad40 0x5570304ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:13.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.933+0000 7f4c76a00640 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5570304aad40 0x5570304ad160 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a963fccb215cc30 server_cookie=5a641a3cda5e2ff3 in_seq=0 out_seq=0 2024-09-19T04:21:13.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.933+0000 7f4c61600640 10 client.25402 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:21:13.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c61600640 1 -- 172.21.15.107:0/1682657111 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557030389e60 con 0x5570304aad40 2024-09-19T04:21:13.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c61600640 10 client.25402 handle_command_reply: tid=0 2024-09-19T04:21:13.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c81036740 1 client.25402 shutdown 2024-09-19T04:21:13.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c81036740 1 -- 172.21.15.107:0/1682657111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 msgr2=0x557030437b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:13.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c81036740 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 0x557030437b10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800bcc0 tx=0x7f4c680149f0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c81036740 1 -- 172.21.15.107:0/1682657111 shutdown_connections 2024-09-19T04:21:13.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c81036740 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5570304aad40 0x5570304ad160 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c81036740 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55703042d3a0 0x557030432ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c81036740 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557030424f70 0x557030438050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c81036740 1 --2- 172.21.15.107:0/1682657111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557030424650 0x557030437b10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c81036740 1 -- 172.21.15.107:0/1682657111 >> 172.21.15.107:0/1682657111 conn(0x557030420be0 msgr2=0x55703042e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:13.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.934+0000 7f4c61600640 0 client.25402 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:21:13.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.935+0000 7f4c81036740 1 -- 172.21.15.107:0/1682657111 shutdown_connections 2024-09-19T04:21:13.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.935+0000 7f4c81036740 1 -- 172.21.15.107:0/1682657111 wait complete. 2024-09-19T04:21:13.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.935+0000 7f4c81036740 20 client.25402 trim_cache size 0 max 16384 2024-09-19T04:21:13.938 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:21:13.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.936+0000 7f4c7d000640 1 -- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c5c0a3550 msgr2=0x7f4c5c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:13.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.936+0000 7f4c7d000640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c5c0a3550 0x7f4c5c0a5a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c70006010 tx=0x7f4c700095c0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.936+0000 7f4c7d000640 1 -- 172.21.15.107:0/3587645630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c781034b0 msgr2=0x7f4c7806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:13.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.936+0000 7f4c7d000640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c781034b0 0x7f4c7806ef50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400e9f0 tx=0x7f4c6400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.936+0000 7f4c7d000640 1 -- 172.21.15.107:0/3587645630 shutdown_connections 2024-09-19T04:21:13.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.936+0000 7f4c7d000640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c5c0a3550 0x7f4c5c0a5a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.936+0000 7f4c7d000640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7810bfc0 0x7f4c7806f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.936+0000 7f4c7d000640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c781034b0 0x7f4c7806ef50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.936+0000 7f4c7d000640 1 --2- 172.21.15.107:0/3587645630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c78102b90 0x7f4c78073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:13.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.936+0000 7f4c7d000640 1 -- 172.21.15.107:0/3587645630 >> 172.21.15.107:0/3587645630 conn(0x7f4c780fc610 msgr2=0x7f4c78101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:13.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.937+0000 7f4c7d000640 1 -- 172.21.15.107:0/3587645630 shutdown_connections 2024-09-19T04:21:13.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:13.937+0000 7f4c7d000640 1 -- 172.21.15.107:0/3587645630 wait complete. 2024-09-19T04:21:14.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:14 smithi115 ceph-mon[35477]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-19T04:21:14.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:14 smithi145 ceph-mon[35096]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-19T04:21:14.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:14 smithi107 ceph-mon[30659]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-19T04:21:14.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:21:15.397 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:16.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.047+0000 7f6de7200640 1 -- 172.21.15.107:0/2063491850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de0100980 msgr2=0x7f6de0100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.047+0000 7f6de7200640 1 --2- 172.21.15.107:0/2063491850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de0100980 0x7f6de0100d60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009900 tx=0x7f6dd402a470 comp rx=0 tx=0).stop 2024-09-19T04:21:16.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.047+0000 7f6de7200640 1 -- 172.21.15.107:0/2063491850 shutdown_connections 2024-09-19T04:21:16.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.047+0000 7f6de7200640 1 --2- 172.21.15.107:0/2063491850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6de0105d90 0x7f6de0106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.047+0000 7f6de7200640 1 --2- 172.21.15.107:0/2063491850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de01012a0 0x7f6de0105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.047+0000 7f6de7200640 1 --2- 172.21.15.107:0/2063491850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de0100980 0x7f6de0100d60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.047+0000 7f6de7200640 1 -- 172.21.15.107:0/2063491850 >> 172.21.15.107:0/2063491850 conn(0x7f6de00fc610 msgr2=0x7f6de00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.047+0000 7f6de7200640 1 -- 172.21.15.107:0/2063491850 shutdown_connections 2024-09-19T04:21:16.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.047+0000 7f6de7200640 1 -- 172.21.15.107:0/2063491850 wait complete. 2024-09-19T04:21:16.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.049+0000 7f6de7200640 1 Processor -- start 2024-09-19T04:21:16.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.049+0000 7f6de7200640 1 -- start start 2024-09-19T04:21:16.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.049+0000 7f6de7200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de0100980 0x7f6de00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.049+0000 7f6de7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01012a0 0x7f6de006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.049+0000 7f6de7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6de0105d90 0x7f6de006cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.049+0000 7f6de7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01183c0 con 0x7f6de01012a0 2024-09-19T04:21:16.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.049+0000 7f6de7200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0118240 con 0x7f6de0100980 2024-09-19T04:21:16.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.049+0000 7f6de7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0118540 con 0x7f6de0105d90 2024-09-19T04:21:16.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.049+0000 7f6de5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01012a0 0x7f6de006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.050+0000 7f6de5e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de0100980 0x7f6de00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.050+0000 7f6de5e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de0100980 0x7f6de00739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54816/0 (socket says 172.21.15.107:54816) 2024-09-19T04:21:16.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.050+0000 7f6de6800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6de0105d90 0x7f6de006cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.050+0000 7f6de5e00640 1 -- 172.21.15.107:0/323030619 learned_addr learned my addr 172.21.15.107:0/323030619 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:16.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.050+0000 7f6de5e00640 1 -- 172.21.15.107:0/323030619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6de0105d90 msgr2=0x7f6de006cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.050+0000 7f6de5e00640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6de0105d90 0x7f6de006cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.050+0000 7f6de5e00640 1 -- 172.21.15.107:0/323030619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01012a0 msgr2=0x7f6de006ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.050+0000 7f6de5e00640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01012a0 0x7f6de006ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.050+0000 7f6de5e00640 1 -- 172.21.15.107:0/323030619 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de006d550 con 0x7f6de0100980 2024-09-19T04:21:16.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.050+0000 7f6de5e00640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de0100980 0x7f6de00739c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006010 tx=0x7f6dd4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.051+0000 7f6ddaa00640 1 -- 172.21.15.107:0/323030619 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd403f030 con 0x7f6de0100980 2024-09-19T04:21:16.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.051+0000 7f6ddaa00640 1 -- 172.21.15.107:0/323030619 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd4004440 con 0x7f6de0100980 2024-09-19T04:21:16.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.051+0000 7f6de7200640 1 -- 172.21.15.107:0/323030619 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de006d7e0 con 0x7f6de0100980 2024-09-19T04:21:16.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.051+0000 7f6ddaa00640 1 -- 172.21.15.107:0/323030619 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4035070 con 0x7f6de0100980 2024-09-19T04:21:16.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.051+0000 7f6de7200640 1 -- 172.21.15.107:0/323030619 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de0071770 con 0x7f6de0100980 2024-09-19T04:21:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.054+0000 7f6deb293740 1 Processor -- start 2024-09-19T04:21:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.054+0000 7f6deb293740 1 -- start start 2024-09-19T04:21:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.055+0000 7f6deb293740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 0x5581723f6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.055+0000 7f6deb293740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f6760 0x558172400ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.055+0000 7f6deb293740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x558172403410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.055+0000 7f6deb293740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55817235c3e0 con 0x558172401020 2024-09-19T04:21:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.055+0000 7f6deb293740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558172303d80 con 0x5581723f5e40 2024-09-19T04:21:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.055+0000 7f6deb293740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5581723583a0 con 0x5581723f6760 2024-09-19T04:21:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.056+0000 7f6de5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x558172403410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.056+0000 7f6de5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x558172403410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39880/0 (socket says 172.21.15.107:39880) 2024-09-19T04:21:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.056+0000 7f6de5400640 1 -- 172.21.15.107:0/2248025992 learned_addr learned my addr 172.21.15.107:0/2248025992 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.056+0000 7f6de6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 0x5581723f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.056+0000 7f6ddaa00640 1 -- 172.21.15.107:0/323030619 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) ==== 138589+0+0 (secure 0 0 0) 0x7f6dd4042030 con 0x7f6de0100980 2024-09-19T04:21:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.056+0000 7f6de6800640 1 -- 172.21.15.107:0/2248025992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f6760 msgr2=0x558172400ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:21:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.056+0000 7f6de6800640 1 --2- 172.21.15.107:0/2248025992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f6760 0x558172400ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.056+0000 7f6de6800640 1 -- 172.21.15.107:0/2248025992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 msgr2=0x558172403410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.056+0000 7f6de6800640 1 --2- 172.21.15.107:0/2248025992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x558172403410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.056+0000 7f6de6800640 1 -- 172.21.15.107:0/2248025992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558172322ae0 con 0x5581723f5e40 2024-09-19T04:21:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.057+0000 7f6de6800640 1 --2- 172.21.15.107:0/2248025992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 0x5581723f6220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009f00 tx=0x7f6ddc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee225efb6c8ded9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.057+0000 7f6dcf000640 1 -- 172.21.15.107:0/2248025992 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc004d20 con 0x5581723f5e40 2024-09-19T04:21:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.057+0000 7f6dcf000640 1 -- 172.21.15.107:0/2248025992 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ddc004050 con 0x5581723f5e40 2024-09-19T04:21:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.057+0000 7f6dcf000640 1 -- 172.21.15.107:0/2248025992 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc005280 con 0x5581723f5e40 2024-09-19T04:21:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.058+0000 7f6ddaa00640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6dc80a34d0 0x7f6dc80a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.058+0000 7f6deb293740 1 -- 172.21.15.107:0/2248025992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 msgr2=0x5581723f6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.058+0000 7f6ddaa00640 1 -- 172.21.15.107:0/323030619 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f6dd40f02b0 con 0x7f6de0100980 2024-09-19T04:21:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.058+0000 7f6deb293740 1 --2- 172.21.15.107:0/2248025992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 0x5581723f6220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009f00 tx=0x7f6ddc011810 comp rx=0 tx=0).stop 2024-09-19T04:21:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.059+0000 7f6deb293740 1 -- 172.21.15.107:0/2248025992 shutdown_connections 2024-09-19T04:21:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.059+0000 7f6deb293740 1 --2- 172.21.15.107:0/2248025992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x558172403410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.059+0000 7f6deb293740 1 --2- 172.21.15.107:0/2248025992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f6760 0x558172400ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.059+0000 7f6deb293740 1 --2- 172.21.15.107:0/2248025992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 0x5581723f6220 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.059+0000 7f6deb293740 1 -- 172.21.15.107:0/2248025992 >> 172.21.15.107:0/2248025992 conn(0x5581723f23d0 msgr2=0x5581723f27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.059+0000 7f6de5e00640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6dc80a34d0 0x7f6dc80a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.059+0000 7f6deb293740 1 -- 172.21.15.107:0/2248025992 shutdown_connections 2024-09-19T04:21:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.059+0000 7f6deb293740 1 -- 172.21.15.107:0/2248025992 wait complete. 2024-09-19T04:21:16.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.060+0000 7f6deb293740 1 -- start start 2024-09-19T04:21:16.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.060+0000 7f6de5e00640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6dc80a34d0 0x7f6dc80a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40ed930 tx=0x7f6dd4005c90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.060+0000 7f6deb293740 1 Processor -- start 2024-09-19T04:21:16.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6deb293740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 0x5581723fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6deb293740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f6760 0x5581723fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6deb293740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x5581723fe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6deb293740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558172403bf0 con 0x558172401020 2024-09-19T04:21:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6deb293740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5581723170f0 con 0x5581723f5e40 2024-09-19T04:21:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6deb293740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558172403d70 con 0x5581723f6760 2024-09-19T04:21:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6deb293740 10 client.0 _ll_register_callbacks cb 0x5581722febd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6de6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 0x5581723fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6de5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f6760 0x5581723fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6de6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 0x5581723fde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54830/0 (socket says 172.21.15.107:54830) 2024-09-19T04:21:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6de6800640 1 -- 172.21.15.107:0/3281680614 learned_addr learned my addr 172.21.15.107:0/3281680614 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:16.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.061+0000 7f6de6800640 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x5581723fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.062+0000 7f6de5400640 1 -- 172.21.15.107:0/3281680614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 msgr2=0x5581723fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.062+0000 7f6de5400640 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 0x5581723fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.062+0000 7f6de5400640 1 -- 172.21.15.107:0/3281680614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 msgr2=0x5581723fe900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.062+0000 7f6de5400640 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x5581723fe900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.062+0000 7f6de5400640 1 -- 172.21.15.107:0/3281680614 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581720cd3c0 con 0x5581723f6760 2024-09-19T04:21:16.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.063+0000 7f6de5400640 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f6760 0x5581723fe3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009ed0 tx=0x7f6dd00127c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3855309be082282 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.063+0000 7f6dc3400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:21:16.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.063+0000 7f6dc3400640 1 -- 172.21.15.107:0/3281680614 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00108d0 con 0x5581723f6760 2024-09-19T04:21:16.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.063+0000 7f6dc3400640 1 -- 172.21.15.107:0/3281680614 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f6dd0010a70 con 0x5581723f6760 2024-09-19T04:21:16.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.063+0000 7f6deb293740 1 -- 172.21.15.107:0/3281680614 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ddc004050 con 0x5581723f6760 2024-09-19T04:21:16.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.063+0000 7f6dc3400640 1 -- 172.21.15.107:0/3281680614 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6dd0010c70 con 0x5581723f6760 2024-09-19T04:21:16.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.064+0000 7f6deb293740 10 client.25411 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:16.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.064+0000 7f6deb293740 1 -- 172.21.15.107:0/3281680614 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55817240fc10 con 0x5581723f6760 2024-09-19T04:21:16.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.064+0000 7f6dc3400640 1 -- 172.21.15.107:0/3281680614 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ddc004050 con 0x5581723f6760 2024-09-19T04:21:16.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.064+0000 7f6dc3400640 1 -- 172.21.15.107:0/3281680614 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6dd0019070 con 0x5581723f6760 2024-09-19T04:21:16.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.064+0000 7f6deb293740 10 client.25411 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:16.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.064+0000 7f6deb293740 10 client.25411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:16.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.064+0000 7f6deb293740 20 client.25411 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:16.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.064+0000 7f6deb293740 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5581724111c0 0x5581724135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.064+0000 7f6deb293740 4 client.25411 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:16.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.064+0000 7f6deb293740 1 -- 172.21.15.107:0/3281680614 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558172369950 con 0x5581724111c0 2024-09-19T04:21:16.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.065+0000 7f6de6800640 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5581724111c0 0x5581724135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.065+0000 7f6de6800640 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5581724111c0 0x5581724135e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8269140177d1a4b server_cookie=f5140927187d67cc in_seq=0 out_seq=0 2024-09-19T04:21:16.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.065+0000 7f6dc3400640 10 client.25411 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:16.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.066+0000 7f6dc3400640 1 -- 172.21.15.107:0/3281680614 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558172369950 con 0x5581724111c0 2024-09-19T04:21:16.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.066+0000 7f6dc3400640 10 client.25411 handle_command_reply: tid=0 2024-09-19T04:21:16.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.066+0000 7f6deb293740 1 client.25411 shutdown 2024-09-19T04:21:16.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6deb293740 1 -- 172.21.15.107:0/3281680614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f6760 msgr2=0x5581723fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6deb293740 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f6760 0x5581723fe3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009ed0 tx=0x7f6dd00127c0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6deb293740 1 -- 172.21.15.107:0/3281680614 shutdown_connections 2024-09-19T04:21:16.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6deb293740 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5581724111c0 0x5581724135e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6deb293740 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x5581723fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6deb293740 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f6760 0x5581723fe3c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6deb293740 1 --2- 172.21.15.107:0/3281680614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f5e40 0x5581723fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6dc3400640 0 client.25411 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:16.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6deb293740 1 -- 172.21.15.107:0/3281680614 >> 172.21.15.107:0/3281680614 conn(0x5581723f23d0 msgr2=0x5581723f80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6deb293740 1 -- 172.21.15.107:0/3281680614 shutdown_connections 2024-09-19T04:21:16.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.067+0000 7f6deb293740 1 -- 172.21.15.107:0/3281680614 wait complete. 2024-09-19T04:21:16.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.068+0000 7f6deb293740 20 client.25411 trim_cache size 0 max 16384 2024-09-19T04:21:16.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.094+0000 7f6deb293740 1 Processor -- start 2024-09-19T04:21:16.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.094+0000 7f6deb293740 1 -- start start 2024-09-19T04:21:16.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.094+0000 7f6deb293740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581723f5e40 0x5581723fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.094+0000 7f6deb293740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x558172418e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.094+0000 7f6deb293740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558172401020 0x5581724193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.094+0000 7f6deb293740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01bc3a0 con 0x5581723f5e40 2024-09-19T04:21:16.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.094+0000 7f6deb293740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01bdee0 con 0x5581723f6760 2024-09-19T04:21:16.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.094+0000 7f6deb293740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01bca70 con 0x558172401020 2024-09-19T04:21:16.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.095+0000 7f6de6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581723f5e40 0x5581723fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.095+0000 7f6de5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x558172418e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.095+0000 7f6de5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x558172418e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:54838/0 (socket says 172.21.15.107:54838) 2024-09-19T04:21:16.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.095+0000 7f6de5400640 1 -- 172.21.15.107:0/223746241 learned_addr learned my addr 172.21.15.107:0/223746241 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:16.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6de6800640 1 --2- 172.21.15.107:0/223746241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558172401020 0x5581724193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6de5400640 1 -- 172.21.15.107:0/223746241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558172401020 msgr2=0x5581724193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6de5400640 1 --2- 172.21.15.107:0/223746241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558172401020 0x5581724193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6de5400640 1 -- 172.21.15.107:0/223746241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581723f5e40 msgr2=0x5581723fe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6de5400640 1 --2- 172.21.15.107:0/223746241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581723f5e40 0x5581723fe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6de5400640 1 -- 172.21.15.107:0/223746241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581724007f0 con 0x5581723f6760 2024-09-19T04:21:16.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6de5400640 1 --2- 172.21.15.107:0/223746241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x558172418e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0005d30 tx=0x7f6dd0005d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3e0be37c0510e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6de6800640 1 --2- 172.21.15.107:0/223746241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558172401020 0x5581724193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:16.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6dce600640 1 -- 172.21.15.107:0/223746241 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0028070 con 0x5581723f6760 2024-09-19T04:21:16.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6dce600640 1 -- 172.21.15.107:0/223746241 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd0005030 con 0x5581723f6760 2024-09-19T04:21:16.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.096+0000 7f6dce600640 1 -- 172.21.15.107:0/223746241 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd001aa90 con 0x5581723f6760 2024-09-19T04:21:16.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.097+0000 7f6deb293740 1 -- 172.21.15.107:0/223746241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 msgr2=0x558172418e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.097+0000 7f6deb293740 1 --2- 172.21.15.107:0/223746241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x558172418e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0005d30 tx=0x7f6dd0005d60 comp rx=0 tx=0).stop 2024-09-19T04:21:16.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.097+0000 7f6deb293740 1 -- 172.21.15.107:0/223746241 shutdown_connections 2024-09-19T04:21:16.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.097+0000 7f6deb293740 1 --2- 172.21.15.107:0/223746241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558172401020 0x5581724193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.097+0000 7f6deb293740 1 --2- 172.21.15.107:0/223746241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x558172418e90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.097+0000 7f6deb293740 1 --2- 172.21.15.107:0/223746241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581723f5e40 0x5581723fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.097+0000 7f6deb293740 1 -- 172.21.15.107:0/223746241 >> 172.21.15.107:0/223746241 conn(0x5581723f23d0 msgr2=0x5581723fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.098+0000 7f6deb293740 1 -- 172.21.15.107:0/223746241 shutdown_connections 2024-09-19T04:21:16.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.098+0000 7f6deb293740 1 -- 172.21.15.107:0/223746241 wait complete. 2024-09-19T04:21:16.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.098+0000 7f6deb293740 1 -- start start 2024-09-19T04:21:16.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.099+0000 7f6deb293740 1 Processor -- start 2024-09-19T04:21:16.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.099+0000 7f6deb293740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f5e40 0x5581723fade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.099+0000 7f6deb293740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x5581723fb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.099+0000 7f6deb293740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x5581723fb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.099+0000 7f6deb293740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01bd810 con 0x558172401020 2024-09-19T04:21:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.099+0000 7f6deb293740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01bd140 con 0x5581723f6760 2024-09-19T04:21:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.099+0000 7f6deb293740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01ba380 con 0x5581723f5e40 2024-09-19T04:21:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.099+0000 7f6deb293740 10 client.0 _ll_register_callbacks cb 0x7f6de006afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.100+0000 7f6de5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x5581723fb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.100+0000 7f6de6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x5581723fb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.100+0000 7f6de6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x5581723fb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39914/0 (socket says 172.21.15.107:39914) 2024-09-19T04:21:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.100+0000 7f6de6800640 1 -- 172.21.15.107:0/2071633092 learned_addr learned my addr 172.21.15.107:0/2071633092 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.100+0000 7f6de6800640 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f5e40 0x5581723fade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.100+0000 7f6de5400640 1 -- 172.21.15.107:0/2071633092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f5e40 msgr2=0x5581723fade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.100+0000 7f6de5400640 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f5e40 0x5581723fade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.100+0000 7f6de5400640 1 -- 172.21.15.107:0/2071633092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 msgr2=0x5581723fb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:16.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.101+0000 7f6de5400640 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x5581723fb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.101+0000 7f6de5400640 1 -- 172.21.15.107:0/2071633092 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55817247ae60 con 0x5581723f6760 2024-09-19T04:21:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.101+0000 7f6de6800640 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x5581723fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.101+0000 7f6de5400640 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x5581723fb320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6dd001a670 tx=0x7f6dd0012280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=796713f0d701db70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.101+0000 7f6dc3400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:21:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.101+0000 7f6dc3400640 1 -- 172.21.15.107:0/2071633092 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0028070 con 0x5581723f6760 2024-09-19T04:21:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.101+0000 7f6dc3400640 1 -- 172.21.15.107:0/2071633092 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f6dd00053f0 con 0x5581723f6760 2024-09-19T04:21:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.101+0000 7f6deb293740 1 -- 172.21.15.107:0/2071633092 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd0005030 con 0x5581723f6760 2024-09-19T04:21:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.102+0000 7f6dc3400640 1 -- 172.21.15.107:0/2071633092 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6dd0012cc0 con 0x5581723f6760 2024-09-19T04:21:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.102+0000 7f6deb293740 10 client.25439 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.102+0000 7f6deb293740 1 -- 172.21.15.107:0/2071633092 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55817247bfb0 con 0x5581723f6760 2024-09-19T04:21:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.102+0000 7f6dc3400640 1 -- 172.21.15.107:0/2071633092 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd0005030 con 0x5581723f6760 2024-09-19T04:21:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.102+0000 7f6dc3400640 1 -- 172.21.15.107:0/2071633092 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6dd0019070 con 0x5581723f6760 2024-09-19T04:21:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.103+0000 7f6deb293740 10 client.25439 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.103+0000 7f6deb293740 10 client.25439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.103+0000 7f6deb293740 20 client.25439 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.103+0000 7f6deb293740 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55817247c690 0x55817247eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.103+0000 7f6deb293740 4 client.25439 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:21:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.103+0000 7f6deb293740 1 -- 172.21.15.107:0/2071633092 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558172363e60 con 0x55817247c690 2024-09-19T04:21:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.103+0000 7f6de6800640 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55817247c690 0x55817247eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.104+0000 7f6dc3400640 10 client.25439 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.104+0000 7f6de6800640 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55817247c690 0x55817247eab0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9138dcc4936abece server_cookie=6c23e12f9e9c1d66 in_seq=0 out_seq=0 2024-09-19T04:21:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6dc3400640 1 -- 172.21.15.107:0/2071633092 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558172363e60 con 0x55817247c690 2024-09-19T04:21:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6dc3400640 10 client.25439 handle_command_reply: tid=0 2024-09-19T04:21:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6deb293740 1 client.25439 shutdown 2024-09-19T04:21:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6deb293740 1 -- 172.21.15.107:0/2071633092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 msgr2=0x5581723fb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6deb293740 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x5581723fb320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6dd001a670 tx=0x7f6dd0012280 comp rx=0 tx=0).stop 2024-09-19T04:21:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6deb293740 1 -- 172.21.15.107:0/2071633092 shutdown_connections 2024-09-19T04:21:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6deb293740 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55817247c690 0x55817247eab0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6deb293740 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558172401020 0x5581723fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6deb293740 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581723f6760 0x5581723fb320 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6deb293740 1 --2- 172.21.15.107:0/2071633092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5581723f5e40 0x5581723fade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6deb293740 1 -- 172.21.15.107:0/2071633092 >> 172.21.15.107:0/2071633092 conn(0x5581723f23d0 msgr2=0x5581723f0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6dc3400640 0 client.25439 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:16.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.105+0000 7f6deb293740 1 -- 172.21.15.107:0/2071633092 shutdown_connections 2024-09-19T04:21:16.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.106+0000 7f6deb293740 1 -- 172.21.15.107:0/2071633092 wait complete. 2024-09-19T04:21:16.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.106+0000 7f6deb293740 20 client.25439 trim_cache size 0 max 16384 2024-09-19T04:21:16.109 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:21:16.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 -- 172.21.15.107:0/323030619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6dc80a34d0 msgr2=0x7f6dc80a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6dc80a34d0 0x7f6dc80a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40ed930 tx=0x7f6dd4005c90 comp rx=0 tx=0).stop 2024-09-19T04:21:16.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 -- 172.21.15.107:0/323030619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de0100980 msgr2=0x7f6de00739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:16.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de0100980 0x7f6de00739c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006010 tx=0x7f6dd4004040 comp rx=0 tx=0).stop 2024-09-19T04:21:16.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 -- 172.21.15.107:0/323030619 shutdown_connections 2024-09-19T04:21:16.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6dc80a34d0 0x7f6dc80a5990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6de0105d90 0x7f6de006cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01012a0 0x7f6de006ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 --2- 172.21.15.107:0/323030619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de0100980 0x7f6de00739c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:16.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 -- 172.21.15.107:0/323030619 >> 172.21.15.107:0/323030619 conn(0x7f6de00fc610 msgr2=0x7f6de010f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:16.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 -- 172.21.15.107:0/323030619 shutdown_connections 2024-09-19T04:21:16.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:16.107+0000 7f6de7200640 1 -- 172.21.15.107:0/323030619 wait complete. 2024-09-19T04:21:16.526 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:16 smithi145 ceph-mon[35096]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-19T04:21:16.528 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:16 smithi107 ceph-mon[30659]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-19T04:21:16.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:16 smithi115 ceph-mon[35477]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-19T04:21:18.042 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:21:18.042 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:21:18.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:18 smithi107 ceph-mon[30659]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-19T04:21:18.571 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:18.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:18 smithi115 ceph-mon[35477]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-19T04:21:18.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:18 smithi145 ceph-mon[35096]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-19T04:21:19.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.175+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1303875966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1079c0 msgr2=0x7f6aac107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.175+0000 7f6ab1a00640 1 --2- 172.21.15.107:0/1303875966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1079c0 0x7f6aac107da0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009900 tx=0x7f6a9402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.175+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1303875966 shutdown_connections 2024-09-19T04:21:19.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.175+0000 7f6ab1a00640 1 --2- 172.21.15.107:0/1303875966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6aac102360 0x7f6aac10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.175+0000 7f6ab1a00640 1 --2- 172.21.15.107:0/1303875966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6aac1019c0 0x7f6aac101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.175+0000 7f6ab1a00640 1 --2- 172.21.15.107:0/1303875966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1079c0 0x7f6aac107da0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.175+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1303875966 >> 172.21.15.107:0/1303875966 conn(0x7f6aac0fd740 msgr2=0x7f6aac0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:19.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.177+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1303875966 shutdown_connections 2024-09-19T04:21:19.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.177+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1303875966 wait complete. 2024-09-19T04:21:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.178+0000 7f6ab1a00640 1 Processor -- start 2024-09-19T04:21:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.179+0000 7f6ab1a00640 1 -- start start 2024-09-19T04:21:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.179+0000 7f6ab1a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6aac1019c0 0x7f6aac19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.179+0000 7f6ab1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6aac102360 0x7f6aac19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.179+0000 7f6ab1a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1079c0 0x7f6aac19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.179+0000 7f6ab1a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac111e50 con 0x7f6aac102360 2024-09-19T04:21:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.179+0000 7f6ab1a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac111cd0 con 0x7f6aac1079c0 2024-09-19T04:21:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.179+0000 7f6ab1a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac111fd0 con 0x7f6aac1019c0 2024-09-19T04:21:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6aabe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6aac1019c0 0x7f6aac19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6aabe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6aac1019c0 0x7f6aac19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38278/0 (socket says 172.21.15.107:38278) 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6aabe00640 1 -- 172.21.15.107:0/1439044465 learned_addr learned my addr 172.21.15.107:0/1439044465 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6ab1000640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1079c0 0x7f6aac19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6aab400640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6aac102360 0x7f6aac19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6ab1000640 1 -- 172.21.15.107:0/1439044465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6aac1019c0 msgr2=0x7f6aac19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6ab1000640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6aac1019c0 0x7f6aac19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6ab1000640 1 -- 172.21.15.107:0/1439044465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6aac102360 msgr2=0x7f6aac19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6ab1000640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6aac102360 0x7f6aac19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6ab1000640 1 -- 172.21.15.107:0/1439044465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac19f4a0 con 0x7f6aac1079c0 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.180+0000 7f6aabe00640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6aac1019c0 0x7f6aac19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.181+0000 7f6ab1000640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1079c0 0x7f6aac19edc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009f00 tx=0x7f6aa400bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.181+0000 7f6aa8c00640 1 -- 172.21.15.107:0/1439044465 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa402a070 con 0x7f6aac1079c0 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.181+0000 7f6aa8c00640 1 -- 172.21.15.107:0/1439044465 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa4025920 con 0x7f6aac1079c0 2024-09-19T04:21:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.181+0000 7f6aa8c00640 1 -- 172.21.15.107:0/1439044465 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40238e0 con 0x7f6aac1079c0 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.181+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1439044465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac19f790 con 0x7f6aac1079c0 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.181+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1439044465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac108820 con 0x7f6aac1079c0 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.185+0000 7f6ab5bf7740 1 Processor -- start 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.186+0000 7f6aa8c00640 1 -- 172.21.15.107:0/1439044465 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) ==== 138589+0+0 (secure 0 0 0) 0x7f6aa402d070 con 0x7f6aac1079c0 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.187+0000 7f6ab5bf7740 1 -- start start 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.188+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f37cb0 0x55fca0f38090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.189+0000 7f6aa8c00640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6a900a34d0 0x7f6a900a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.189+0000 7f6aa8c00640 1 -- 172.21.15.107:0/1439044465 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f6aa40e0690 con 0x7f6aac1079c0 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.189+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f42950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.189+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f42e90 0x55fca0f45280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.190+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fca0e9e290 con 0x55fca0f37cb0 2024-09-19T04:21:19.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.190+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fca0e45b40 con 0x55fca0f385d0 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.190+0000 7f6aabe00640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6a900a34d0 0x7f6a900a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.191+0000 7f6aab400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f42950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.191+0000 7f6ab1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f42e90 0x55fca0f45280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.191+0000 7f6aab400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f42950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51762/0 (socket says 172.21.15.107:51762) 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.191+0000 7f6aab400640 1 -- 172.21.15.107:0/1820041661 learned_addr learned my addr 172.21.15.107:0/1820041661 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.191+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/1820041661 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fca0e9a250 con 0x55fca0f42e90 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.191+0000 7f6aabe00640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6a900a34d0 0x7f6a900a5990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a94000c00 tx=0x7f6a940046c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.192+0000 7f6aab400640 1 --2- 172.21.15.107:0/1820041661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f37cb0 0x55fca0f38090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.192+0000 7f6ab1000640 1 -- 172.21.15.107:0/1820041661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 msgr2=0x55fca0f42950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.192+0000 7f6ab1000640 1 --2- 172.21.15.107:0/1820041661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f42950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.192+0000 7f6ab1000640 1 -- 172.21.15.107:0/1820041661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f37cb0 msgr2=0x55fca0f38090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.192+0000 7f6ab1000640 1 --2- 172.21.15.107:0/1820041661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f37cb0 0x55fca0f38090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.193+0000 7f6ab1000640 1 -- 172.21.15.107:0/1820041661 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca0c0f6a0 con 0x55fca0f42e90 2024-09-19T04:21:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.193+0000 7f6aab400640 1 --2- 172.21.15.107:0/1820041661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f37cb0 0x55fca0f38090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.193+0000 7f6ab1000640 1 --2- 172.21.15.107:0/1820041661 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f42e90 0x55fca0f45280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40ddd60 tx=0x7f6aa4004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f1acd51f4d113bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.193+0000 7f6aa1600640 1 -- 172.21.15.107:0/1820041661 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40ac8d0 con 0x55fca0f42e90 2024-09-19T04:21:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.194+0000 7f6aa1600640 1 -- 172.21.15.107:0/1820041661 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa40b0bd0 con 0x55fca0f42e90 2024-09-19T04:21:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.194+0000 7f6aa1600640 1 -- 172.21.15.107:0/1820041661 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40b5000 con 0x55fca0f42e90 2024-09-19T04:21:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.194+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/1820041661 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f42e90 msgr2=0x55fca0f45280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.194+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/1820041661 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f42e90 0x55fca0f45280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40ddd60 tx=0x7f6aa4004b50 comp rx=0 tx=0).stop 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.195+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/1820041661 shutdown_connections 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.195+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/1820041661 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f42e90 0x55fca0f45280 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.195+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/1820041661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f42950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.195+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/1820041661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f37cb0 0x55fca0f38090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.195+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/1820041661 >> 172.21.15.107:0/1820041661 conn(0x55fca0f34240 msgr2=0x55fca0f34620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.195+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/1820041661 shutdown_connections 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.195+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/1820041661 wait complete. 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.196+0000 7f6ab5bf7740 1 -- start start 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.196+0000 7f6ab5bf7740 1 Processor -- start 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.197+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.197+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f385d0 0x55fca0f3fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.197+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f42e90 0x55fca0f3ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.197+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fca0f45b00 con 0x55fca0f385d0 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.197+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fca0e58fa0 con 0x55fca0f42e90 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.197+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fca0f45c80 con 0x55fca0f37cb0 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.197+0000 7f6ab5bf7740 10 client.0 _ll_register_callbacks cb 0x55fca0bacba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.198+0000 7f6ab1000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f42e90 0x55fca0f3ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.198+0000 7f6ab1000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f42e90 0x55fca0f3ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51772/0 (socket says 172.21.15.107:51772) 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.198+0000 7f6ab1000640 1 -- 172.21.15.107:0/3963366183 learned_addr learned my addr 172.21.15.107:0/3963366183 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.198+0000 7f6aabe00640 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.198+0000 7f6aab400640 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f385d0 0x55fca0f3fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.198+0000 7f6aab400640 1 -- 172.21.15.107:0/3963366183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 msgr2=0x55fca0f427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.198+0000 7f6aab400640 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.199+0000 7f6aab400640 1 -- 172.21.15.107:0/3963366183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f42e90 msgr2=0x55fca0f3ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.199+0000 7f6aab400640 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f42e90 0x55fca0f3ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.199+0000 7f6aab400640 1 -- 172.21.15.107:0/3963366183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca0f407f0 con 0x55fca0f385d0 2024-09-19T04:21:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.199+0000 7f6ab1000640 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f42e90 0x55fca0f3ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.200+0000 7f6aab400640 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f385d0 0x55fca0f3fa80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55fca0f350a0 tx=0x7f6a9c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb7d267dbd0cb7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.200+0000 7f6a9a000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:21:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.200+0000 7f6a9a000640 1 -- 172.21.15.107:0/3963366183 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c005220 con 0x55fca0f385d0 2024-09-19T04:21:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.200+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/3963366183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa40b0bd0 con 0x55fca0f385d0 2024-09-19T04:21:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.200+0000 7f6a9a000640 1 -- 172.21.15.107:0/3963366183 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f6a9c0053c0 con 0x55fca0f385d0 2024-09-19T04:21:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.201+0000 7f6a9a000640 1 -- 172.21.15.107:0/3963366183 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6a9c0055c0 con 0x55fca0f385d0 2024-09-19T04:21:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.201+0000 7f6ab5bf7740 10 client.15411 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.201+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/3963366183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fca0f51ac0 con 0x55fca0f385d0 2024-09-19T04:21:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.202+0000 7f6a9a000640 1 -- 172.21.15.107:0/3963366183 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa40b0bd0 con 0x55fca0f385d0 2024-09-19T04:21:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.202+0000 7f6a9a000640 1 -- 172.21.15.107:0/3963366183 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6a9c01c030 con 0x55fca0f385d0 2024-09-19T04:21:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.202+0000 7f6ab5bf7740 10 client.15411 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.202+0000 7f6ab5bf7740 10 client.15411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.202+0000 7f6ab5bf7740 20 client.15411 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.202+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fca0f52f80 0x55fca0f55370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.202+0000 7f6ab5bf7740 4 client.15411 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.202+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/3963366183 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca0eab800 con 0x55fca0f52f80 2024-09-19T04:21:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.203+0000 7f6aabe00640 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fca0f52f80 0x55fca0f55370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.204+0000 7f6aabe00640 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fca0f52f80 0x55fca0f55370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75151f3227135acb server_cookie=922b91970c1abc20 in_seq=0 out_seq=0 2024-09-19T04:21:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.204+0000 7f6a9a000640 10 client.15411 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.205+0000 7f6a9a000640 1 -- 172.21.15.107:0/3963366183 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fca0eab800 con 0x55fca0f52f80 2024-09-19T04:21:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.205+0000 7f6a9a000640 10 client.15411 handle_command_reply: tid=0 2024-09-19T04:21:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.205+0000 7f6ab5bf7740 1 client.15411 shutdown 2024-09-19T04:21:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/3963366183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f385d0 msgr2=0x55fca0f3fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f385d0 0x55fca0f3fa80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55fca0f350a0 tx=0x7f6a9c0039c0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/3963366183 shutdown_connections 2024-09-19T04:21:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fca0f52f80 0x55fca0f55370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f42e90 0x55fca0f3ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f385d0 0x55fca0f3fa80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/3963366183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/3963366183 >> 172.21.15.107:0/3963366183 conn(0x55fca0f34240 msgr2=0x55fca0f39f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6a9a000640 0 client.15411 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/3963366183 shutdown_connections 2024-09-19T04:21:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.206+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/3963366183 wait complete. 2024-09-19T04:21:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.207+0000 7f6ab5bf7740 20 client.15411 trim_cache size 0 max 16384 2024-09-19T04:21:19.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.242+0000 7f6ab5bf7740 1 Processor -- start 2024-09-19T04:21:19.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.242+0000 7f6ab5bf7740 1 -- start start 2024-09-19T04:21:19.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.242+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f39e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.242+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f5ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.242+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f42e90 0x55fca0f5b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.242+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1ab470 con 0x55fca0f42e90 2024-09-19T04:21:19.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.242+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1acfb0 con 0x55fca0f385d0 2024-09-19T04:21:19.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.242+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1abb40 con 0x55fca0f37cb0 2024-09-19T04:21:19.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.243+0000 7f6aabe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f39e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.243+0000 7f6aab400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f5ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.243+0000 7f6aab400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f5ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51774/0 (socket says 172.21.15.107:51774) 2024-09-19T04:21:19.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.243+0000 7f6aab400640 1 -- 172.21.15.107:0/892919541 learned_addr learned my addr 172.21.15.107:0/892919541 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:19.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.243+0000 7f6aab400640 1 -- 172.21.15.107:0/892919541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 msgr2=0x55fca0f39e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.243+0000 7f6aab400640 1 --2- 172.21.15.107:0/892919541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f39e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.243+0000 7f6aab400640 1 -- 172.21.15.107:0/892919541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f42e90 msgr2=0x55fca0f5b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.243+0000 7f6aab400640 1 --2- 172.21.15.107:0/892919541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f42e90 0x55fca0f5b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.243+0000 7f6aab400640 1 -- 172.21.15.107:0/892919541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca0f383c0 con 0x55fca0f385d0 2024-09-19T04:21:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.244+0000 7f6aab400640 1 --2- 172.21.15.107:0/892919541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f5ac40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c000c00 tx=0x7f6a9c00bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88ae811b4e847a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.244+0000 7f6aa0c00640 1 -- 172.21.15.107:0/892919541 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c004500 con 0x55fca0f385d0 2024-09-19T04:21:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.244+0000 7f6aa0c00640 1 -- 172.21.15.107:0/892919541 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9c012d10 con 0x55fca0f385d0 2024-09-19T04:21:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.244+0000 7f6aa0c00640 1 -- 172.21.15.107:0/892919541 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c010d40 con 0x55fca0f385d0 2024-09-19T04:21:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.245+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/892919541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 msgr2=0x55fca0f5ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.245+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/892919541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f5ac40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c000c00 tx=0x7f6a9c00bde0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.245+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/892919541 shutdown_connections 2024-09-19T04:21:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.245+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/892919541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f42e90 0x55fca0f5b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.245+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/892919541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f5ac40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.245+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/892919541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f39e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.245+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/892919541 >> 172.21.15.107:0/892919541 conn(0x55fca0f34240 msgr2=0x55fca0f39750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.245+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/892919541 shutdown_connections 2024-09-19T04:21:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.246+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/892919541 wait complete. 2024-09-19T04:21:19.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.246+0000 7f6ab5bf7740 1 -- start start 2024-09-19T04:21:19.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.246+0000 7f6ab5bf7740 1 Processor -- start 2024-09-19T04:21:19.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6ab5bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f42e90 0x55fca0fbcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1ac8e0 con 0x55fca0f42e90 2024-09-19T04:21:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1ac210 con 0x55fca0f385d0 2024-09-19T04:21:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6ab5bf7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1a9450 con 0x55fca0f37cb0 2024-09-19T04:21:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6ab5bf7740 10 client.0 _ll_register_callbacks cb 0x7f6a9c00be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6aab400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6aab400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51786/0 (socket says 172.21.15.107:51786) 2024-09-19T04:21:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6aab400640 1 -- 172.21.15.107:0/4065340460 learned_addr learned my addr 172.21.15.107:0/4065340460 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.247+0000 7f6aabe00640 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.248+0000 7f6aabe00640 1 -- 172.21.15.107:0/4065340460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 msgr2=0x55fca0f478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.248+0000 7f6aabe00640 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.248+0000 7f6aabe00640 1 -- 172.21.15.107:0/4065340460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f42e90 msgr2=0x55fca0fbcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.248+0000 7f6aabe00640 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f42e90 0x55fca0fbcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.248+0000 7f6aabe00640 1 -- 172.21.15.107:0/4065340460 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca0fbd1d0 con 0x55fca0f37cb0 2024-09-19T04:21:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.248+0000 7f6aabe00640 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f473a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55fca0f55bb0 tx=0x7f6a9403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6fb744e91369313 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.248+0000 7f6a9a000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:21:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.249+0000 7f6a9a000640 1 -- 172.21.15.107:0/4065340460 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94035040 con 0x55fca0f37cb0 2024-09-19T04:21:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.249+0000 7f6a9a000640 1 -- 172.21.15.107:0/4065340460 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f6a94031940 con 0x55fca0f37cb0 2024-09-19T04:21:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.249+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/4065340460 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a9c012d10 con 0x55fca0f37cb0 2024-09-19T04:21:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.249+0000 7f6a9a000640 1 -- 172.21.15.107:0/4065340460 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6a94031b40 con 0x55fca0f37cb0 2024-09-19T04:21:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.249+0000 7f6ab5bf7740 10 client.25432 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.249+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/4065340460 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fca0fbe530 con 0x55fca0f37cb0 2024-09-19T04:21:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.250+0000 7f6a9a000640 1 -- 172.21.15.107:0/4065340460 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9c012d10 con 0x55fca0f37cb0 2024-09-19T04:21:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.251+0000 7f6a9a000640 1 -- 172.21.15.107:0/4065340460 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6a9404a430 con 0x55fca0f37cb0 2024-09-19T04:21:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.251+0000 7f6ab5bf7740 10 client.25432 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.251+0000 7f6ab5bf7740 10 client.25432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.251+0000 7f6ab5bf7740 20 client.25432 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.251+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fca0fbec00 0x55fca0fc1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.251+0000 7f6ab5bf7740 4 client.25432 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:21:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.251+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/4065340460 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fca0ea5d10 con 0x55fca0fbec00 2024-09-19T04:21:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.251+0000 7f6aab400640 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fca0fbec00 0x55fca0fc1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:19.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.252+0000 7f6aab400640 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fca0fbec00 0x55fca0fc1010 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ced56ded00f98c9 server_cookie=e0a626d0f5e7a256 in_seq=0 out_seq=0 2024-09-19T04:21:19.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.252+0000 7f6a9a000640 10 client.25432 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:19.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.253+0000 7f6a9a000640 1 -- 172.21.15.107:0/4065340460 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fca0ea5d10 con 0x55fca0fbec00 2024-09-19T04:21:19.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.253+0000 7f6a9a000640 10 client.25432 handle_command_reply: tid=0 2024-09-19T04:21:19.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.253+0000 7f6ab5bf7740 1 client.25432 shutdown 2024-09-19T04:21:19.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.254+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/4065340460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 msgr2=0x55fca0f473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.254+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f473a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55fca0f55bb0 tx=0x7f6a9403d000 comp rx=0 tx=0).stop 2024-09-19T04:21:19.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.254+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/4065340460 shutdown_connections 2024-09-19T04:21:19.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.254+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fca0fbec00 0x55fca0fc1010 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.254+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca0f42e90 0x55fca0fbcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.254+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fca0f385d0 0x55fca0f478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.254+0000 7f6ab5bf7740 1 --2- 172.21.15.107:0/4065340460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fca0f37cb0 0x55fca0f473a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.254+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/4065340460 >> 172.21.15.107:0/4065340460 conn(0x55fca0f34240 msgr2=0x55fca0f33290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:19.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.254+0000 7f6a9a000640 0 client.25432 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:19.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.255+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/4065340460 shutdown_connections 2024-09-19T04:21:19.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.255+0000 7f6ab5bf7740 1 -- 172.21.15.107:0/4065340460 wait complete. 2024-09-19T04:21:19.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.256+0000 7f6ab5bf7740 20 client.25432 trim_cache size 0 max 16384 2024-09-19T04:21:19.261 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:21:19.262 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:21:19.262 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "18ce476f-2f3d-48f9-853c-a4709bb8d1d0", 2024-09-19T04:21:19.262 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:21:19.262 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:21:19.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.257+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1439044465 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6a900a34d0 msgr2=0x7f6a900a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.257+0000 7f6ab1a00640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6a900a34d0 0x7f6a900a5990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a94000c00 tx=0x7f6a940046c0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.257+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1439044465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1079c0 msgr2=0x7f6aac19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.257+0000 7f6ab1a00640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1079c0 0x7f6aac19edc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009f00 tx=0x7f6aa400bdf0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.258+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1439044465 shutdown_connections 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.258+0000 7f6ab1a00640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6a900a34d0 0x7f6a900a5990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.258+0000 7f6ab1a00640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1079c0 0x7f6aac19edc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.258+0000 7f6ab1a00640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6aac102360 0x7f6aac19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.258+0000 7f6ab1a00640 1 --2- 172.21.15.107:0/1439044465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6aac1019c0 0x7f6aac19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.258+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1439044465 >> 172.21.15.107:0/1439044465 conn(0x7f6aac0fd740 msgr2=0x7f6aac10cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.258+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1439044465 shutdown_connections 2024-09-19T04:21:19.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:19.258+0000 7f6ab1a00640 1 -- 172.21.15.107:0/1439044465 wait complete. 2024-09-19T04:21:20.132 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:21:20.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:20 smithi107 ceph-mon[30659]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T04:21:20.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:20 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:21:20.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:20 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:20.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:20 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:21:20.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:20 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:20.525 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:20.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:20 smithi115 ceph-mon[35477]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T04:21:20.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:20 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:21:20.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:20 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:20.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:20 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:21:20.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:20 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:20 smithi145 ceph-mon[35096]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T04:21:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:20 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:21:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:20 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:20 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:21:20.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:20 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:21:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.173+0000 7f91eda00640 1 -- 172.21.15.107:0/222175994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e81007a0 msgr2=0x7f91e8100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.173+0000 7f91eda00640 1 --2- 172.21.15.107:0/222175994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e81007a0 0x7f91e8100b80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009900 tx=0x7f91d802a400 comp rx=0 tx=0).stop 2024-09-19T04:21:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.173+0000 7f91eda00640 1 -- 172.21.15.107:0/222175994 shutdown_connections 2024-09-19T04:21:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.173+0000 7f91eda00640 1 --2- 172.21.15.107:0/222175994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e810e350 0x7f91e8110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.173+0000 7f91eda00640 1 --2- 172.21.15.107:0/222175994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91e8101150 0x7f91e810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.173+0000 7f91eda00640 1 --2- 172.21.15.107:0/222175994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e81007a0 0x7f91e8100b80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.173+0000 7f91eda00640 1 -- 172.21.15.107:0/222175994 >> 172.21.15.107:0/222175994 conn(0x7f91e80fc610 msgr2=0x7f91e80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.173+0000 7f91eda00640 1 -- 172.21.15.107:0/222175994 shutdown_connections 2024-09-19T04:21:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.173+0000 7f91eda00640 1 -- 172.21.15.107:0/222175994 wait complete. 2024-09-19T04:21:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.174+0000 7f91eda00640 1 Processor -- start 2024-09-19T04:21:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.174+0000 7f91eda00640 1 -- start start 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91eda00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e81007a0 0x7f91e819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91eda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91e8101150 0x7f91e819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91eda00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e810e350 0x7f91e819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91eda00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91e8114020 con 0x7f91e810e350 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91eda00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f91e8113ea0 con 0x7f91e81007a0 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91eda00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81141a0 con 0x7f91e8101150 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91ed000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e810e350 0x7f91e819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e81007a0 0x7f91e819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91ed000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e810e350 0x7f91e819edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59310/0 (socket says 172.21.15.107:59310) 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91ed000640 1 -- 172.21.15.107:0/2188631864 learned_addr learned my addr 172.21.15.107:0/2188631864 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.175+0000 7f91e7400640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91e8101150 0x7f91e819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.176+0000 7f91e7e00640 1 -- 172.21.15.107:0/2188631864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91e8101150 msgr2=0x7f91e819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.176+0000 7f91e7e00640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91e8101150 0x7f91e819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.176+0000 7f91e7e00640 1 -- 172.21.15.107:0/2188631864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e810e350 msgr2=0x7f91e819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.176+0000 7f91e7e00640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e810e350 0x7f91e819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.176+0000 7f91e7e00640 1 -- 172.21.15.107:0/2188631864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e819f4d0 con 0x7f91e81007a0 2024-09-19T04:21:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.176+0000 7f91e7400640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91e8101150 0x7f91e819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.176+0000 7f91e7e00640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e81007a0 0x7f91e819a520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91d802a920 tx=0x7f91d8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.177+0000 7f91e4c00640 1 -- 172.21.15.107:0/2188631864 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d803f070 con 0x7f91e81007a0 2024-09-19T04:21:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.177+0000 7f91eda00640 1 -- 172.21.15.107:0/2188631864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e819f760 con 0x7f91e81007a0 2024-09-19T04:21:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.177+0000 7f91e4c00640 1 -- 172.21.15.107:0/2188631864 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d8031c90 con 0x7f91e81007a0 2024-09-19T04:21:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.177+0000 7f91e4c00640 1 -- 172.21.15.107:0/2188631864 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8035030 con 0x7f91e81007a0 2024-09-19T04:21:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.177+0000 7f91eda00640 1 -- 172.21.15.107:0/2188631864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e81a70a0 con 0x7f91e81007a0 2024-09-19T04:21:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.180+0000 7f91e4c00640 1 -- 172.21.15.107:0/2188631864 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) ==== 138589+0+0 (secure 0 0 0) 0x7f91d8042070 con 0x7f91e81007a0 2024-09-19T04:21:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.180+0000 7f91f19e2740 1 Processor -- start 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.180+0000 7f91f19e2740 1 -- start start 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.181+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.181+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 0x5622e41fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.181+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 0x5622e4200320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.181+0000 7f91f19e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5622e41593e0 con 0x5622e41f3670 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.181+0000 7f91f19e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5622e4100d80 con 0x5622e41f2d50 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.181+0000 7f91f19e2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5622e41553a0 con 0x5622e41fdf30 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.181+0000 7f91ed000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 0x5622e41fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91ed000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51808/0 (socket says 172.21.15.107:51808) 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91ed000640 1 -- 172.21.15.107:0/2804012715 learned_addr learned my addr 172.21.15.107:0/2804012715 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91e7400640 1 --2- 172.21.15.107:0/2804012715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 0x5622e4200320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91e4c00640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91cc0a34d0 0x7f91cc0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91e4c00640 1 -- 172.21.15.107:0/2188631864 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f91d80f08f0 con 0x7f91e81007a0 2024-09-19T04:21:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91e7e00640 1 -- 172.21.15.107:0/2804012715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 msgr2=0x5622e4200320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91e7e00640 1 --2- 172.21.15.107:0/2804012715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 0x5622e4200320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91e7e00640 1 -- 172.21.15.107:0/2804012715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 msgr2=0x5622e41f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91e7e00640 1 --2- 172.21.15.107:0/2804012715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.182+0000 7f91e7e00640 1 -- 172.21.15.107:0/2804012715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e411fae0 con 0x5622e41f3670 2024-09-19T04:21:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.183+0000 7f91e7e00640 1 --2- 172.21.15.107:0/2804012715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 0x5622e41fd9f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f91d80edeb0 tx=0x7f91d802a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a34f7272dc90474 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.183+0000 7f91ed000640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91cc0a34d0 0x7f91cc0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.184+0000 7f91dd600640 1 -- 172.21.15.107:0/2804012715 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8049020 con 0x5622e41f3670 2024-09-19T04:21:21.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.184+0000 7f91dd600640 1 -- 172.21.15.107:0/2804012715 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d8031e40 con 0x5622e41f3670 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.184+0000 7f91dd600640 1 -- 172.21.15.107:0/2804012715 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d80c56b0 con 0x5622e41f3670 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.185+0000 7f91f19e2740 1 -- 172.21.15.107:0/2804012715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 msgr2=0x5622e41fd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.185+0000 7f91f19e2740 1 --2- 172.21.15.107:0/2804012715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 0x5622e41fd9f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f91d80edeb0 tx=0x7f91d802a400 comp rx=0 tx=0).stop 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.185+0000 7f91ed000640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91cc0a34d0 0x7f91cc0a5990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009fd0 tx=0x7f91e0004a40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.185+0000 7f91f19e2740 1 -- 172.21.15.107:0/2804012715 shutdown_connections 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.185+0000 7f91f19e2740 1 --2- 172.21.15.107:0/2804012715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 0x5622e4200320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.185+0000 7f91f19e2740 1 --2- 172.21.15.107:0/2804012715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 0x5622e41fd9f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.185+0000 7f91f19e2740 1 --2- 172.21.15.107:0/2804012715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.185+0000 7f91f19e2740 1 -- 172.21.15.107:0/2804012715 >> 172.21.15.107:0/2804012715 conn(0x5622e41ef2e0 msgr2=0x5622e41ef6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.185+0000 7f91f19e2740 1 -- 172.21.15.107:0/2804012715 shutdown_connections 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.185+0000 7f91f19e2740 1 -- 172.21.15.107:0/2804012715 wait complete. 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.186+0000 7f91f19e2740 1 -- start start 2024-09-19T04:21:21.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.186+0000 7f91f19e2740 1 Processor -- start 2024-09-19T04:21:21.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41fae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e41fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41fdf30 0x5622e41fb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91f19e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5622e42090d0 con 0x5622e41fdf30 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91f19e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5622e41140f0 con 0x5622e41f2d50 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91f19e2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5622e4209250 con 0x5622e41f3670 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91f19e2740 10 client.0 _ll_register_callbacks cb 0x5622e41059e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91ed000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41fdf30 0x5622e41fb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91e7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e41fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91e7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e41fb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38336/0 (socket says 172.21.15.107:38336) 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91e7400640 1 -- 172.21.15.107:0/956348000 learned_addr learned my addr 172.21.15.107:0/956348000 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.187+0000 7f91e7e00640 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.188+0000 7f91e7400640 1 -- 172.21.15.107:0/956348000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 msgr2=0x5622e41fae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.188+0000 7f91e7400640 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41fae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.188+0000 7f91e7400640 1 -- 172.21.15.107:0/956348000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41fdf30 msgr2=0x5622e41fb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.188+0000 7f91e7400640 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41fdf30 0x5622e41fb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.188+0000 7f91e7400640 1 -- 172.21.15.107:0/956348000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e3eca3c0 con 0x5622e41f3670 2024-09-19T04:21:21.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.188+0000 7f91e7400640 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e41fb3b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f91d40137c0 tx=0x7f91d4013c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e57d05cd50284b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.188+0000 7f91d2000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:21:21.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.189+0000 7f91d2000640 1 -- 172.21.15.107:0/956348000 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4020060 con 0x5622e41f3670 2024-09-19T04:21:21.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.189+0000 7f91f19e2740 1 -- 172.21.15.107:0/956348000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5622e4166950 con 0x5622e41f3670 2024-09-19T04:21:21.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.189+0000 7f91d2000640 1 -- 172.21.15.107:0/956348000 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f91d4011dd0 con 0x5622e41f3670 2024-09-19T04:21:21.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.189+0000 7f91d2000640 1 -- 172.21.15.107:0/956348000 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91d4025460 con 0x5622e41f3670 2024-09-19T04:21:21.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.189+0000 7f91f19e2740 10 client.25444 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:21.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.189+0000 7f91f19e2740 1 -- 172.21.15.107:0/956348000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622e4214e00 con 0x5622e41f3670 2024-09-19T04:21:21.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.189+0000 7f91d2000640 1 -- 172.21.15.107:0/956348000 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5622e4166950 con 0x5622e41f3670 2024-09-19T04:21:21.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.190+0000 7f91d2000640 1 -- 172.21.15.107:0/956348000 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f91d401a030 con 0x5622e41f3670 2024-09-19T04:21:21.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.190+0000 7f91f19e2740 10 client.25444 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:21.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.190+0000 7f91f19e2740 10 client.25444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:21.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.190+0000 7f91f19e2740 20 client.25444 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:21.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.190+0000 7f91f19e2740 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5622e4216210 0x5622e4218640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.190+0000 7f91f19e2740 4 client.25444 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:21.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.190+0000 7f91f19e2740 1 -- 172.21.15.107:0/956348000 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622e4160e60 con 0x5622e4216210 2024-09-19T04:21:21.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.191+0000 7f91e7e00640 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5622e4216210 0x5622e4218640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.191+0000 7f91e7e00640 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5622e4216210 0x5622e4218640 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4862490f3801efd server_cookie=7ba7ad6c0c382c4d in_seq=0 out_seq=0 2024-09-19T04:21:21.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.192+0000 7f91d2000640 10 client.25444 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:21.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.192+0000 7f91d2000640 1 -- 172.21.15.107:0/956348000 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622e4160e60 con 0x5622e4216210 2024-09-19T04:21:21.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.192+0000 7f91d2000640 10 client.25444 handle_command_reply: tid=0 2024-09-19T04:21:21.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.192+0000 7f91f19e2740 1 client.25444 shutdown 2024-09-19T04:21:21.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91f19e2740 1 -- 172.21.15.107:0/956348000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 msgr2=0x5622e41fb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91f19e2740 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e41fb3b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f91d40137c0 tx=0x7f91d4013c80 comp rx=0 tx=0).stop 2024-09-19T04:21:21.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91f19e2740 1 -- 172.21.15.107:0/956348000 shutdown_connections 2024-09-19T04:21:21.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91f19e2740 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5622e4216210 0x5622e4218640 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91f19e2740 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41fdf30 0x5622e41fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91f19e2740 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e41fb3b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91f19e2740 1 --2- 172.21.15.107:0/956348000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91f19e2740 1 -- 172.21.15.107:0/956348000 >> 172.21.15.107:0/956348000 conn(0x5622e41ef2e0 msgr2=0x5622e41f5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91d2000640 0 client.25444 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:21.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91f19e2740 1 -- 172.21.15.107:0/956348000 shutdown_connections 2024-09-19T04:21:21.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.193+0000 7f91f19e2740 1 -- 172.21.15.107:0/956348000 wait complete. 2024-09-19T04:21:21.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.194+0000 7f91f19e2740 20 client.25444 trim_cache size 0 max 16384 2024-09-19T04:21:21.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.227+0000 7f91f19e2740 1 Processor -- start 2024-09-19T04:21:21.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.227+0000 7f91f19e2740 1 -- start start 2024-09-19T04:21:21.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e421df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41fdf30 0x5622e421e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91f19e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81ab430 con 0x5622e41fdf30 2024-09-19T04:21:21.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91f19e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81acf70 con 0x5622e41f2d50 2024-09-19T04:21:21.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91f19e2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81abb00 con 0x5622e41f3670 2024-09-19T04:21:21.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91ed000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41fdf30 0x5622e421e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51836/0 (socket says 172.21.15.107:51836) 2024-09-19T04:21:21.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91e7e00640 1 -- 172.21.15.107:0/2345269320 learned_addr learned my addr 172.21.15.107:0/2345269320 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:21.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.228+0000 7f91e7400640 1 --2- 172.21.15.107:0/2345269320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e421df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.229+0000 7f91e7e00640 1 -- 172.21.15.107:0/2345269320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 msgr2=0x5622e421df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.229+0000 7f91e7e00640 1 --2- 172.21.15.107:0/2345269320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e421df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.229+0000 7f91e7e00640 1 -- 172.21.15.107:0/2345269320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41fdf30 msgr2=0x5622e421e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.229+0000 7f91e7e00640 1 --2- 172.21.15.107:0/2345269320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41fdf30 0x5622e421e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.229+0000 7f91e7e00640 1 -- 172.21.15.107:0/2345269320 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e41f93c0 con 0x5622e41f2d50 2024-09-19T04:21:21.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.229+0000 7f91e7400640 1 --2- 172.21.15.107:0/2345269320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e421df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:21.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.229+0000 7f91e7e00640 1 --2- 172.21.15.107:0/2345269320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f57b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91d80b9410 tx=0x7f91d803b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdcbf25159f723a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.229+0000 7f91dcc00640 1 -- 172.21.15.107:0/2345269320 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8049040 con 0x5622e41f2d50 2024-09-19T04:21:21.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.230+0000 7f91dcc00640 1 -- 172.21.15.107:0/2345269320 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d80bd140 con 0x5622e41f2d50 2024-09-19T04:21:21.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.230+0000 7f91dcc00640 1 -- 172.21.15.107:0/2345269320 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d80b90b0 con 0x5622e41f2d50 2024-09-19T04:21:21.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.231+0000 7f91f19e2740 1 -- 172.21.15.107:0/2345269320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 msgr2=0x5622e41f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.231+0000 7f91f19e2740 1 --2- 172.21.15.107:0/2345269320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f57b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91d80b9410 tx=0x7f91d803b720 comp rx=0 tx=0).stop 2024-09-19T04:21:21.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.231+0000 7f91f19e2740 1 -- 172.21.15.107:0/2345269320 shutdown_connections 2024-09-19T04:21:21.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.231+0000 7f91f19e2740 1 --2- 172.21.15.107:0/2345269320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41fdf30 0x5622e421e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.231+0000 7f91f19e2740 1 --2- 172.21.15.107:0/2345269320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41f3670 0x5622e421df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.231+0000 7f91f19e2740 1 --2- 172.21.15.107:0/2345269320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e41f57b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.231+0000 7f91f19e2740 1 -- 172.21.15.107:0/2345269320 >> 172.21.15.107:0/2345269320 conn(0x5622e41ef2e0 msgr2=0x5622e41fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.231+0000 7f91f19e2740 1 -- 172.21.15.107:0/2345269320 shutdown_connections 2024-09-19T04:21:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.231+0000 7f91f19e2740 1 -- 172.21.15.107:0/2345269320 wait complete. 2024-09-19T04:21:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.231+0000 7f91f19e2740 1 -- start start 2024-09-19T04:21:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.232+0000 7f91f19e2740 1 Processor -- start 2024-09-19T04:21:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.232+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e420a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.232+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 0x5622e420a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.232+0000 7f91f19e2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 0x5622e420ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.232+0000 7f91f19e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81ac8a0 con 0x5622e41f3670 2024-09-19T04:21:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.232+0000 7f91f19e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81ac1d0 con 0x5622e41f2d50 2024-09-19T04:21:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.232+0000 7f91f19e2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81a9430 con 0x5622e41fdf30 2024-09-19T04:21:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.232+0000 7f91f19e2740 10 client.0 _ll_register_callbacks cb 0x7f91d4009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.233+0000 7f91ed000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 0x5622e420ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.233+0000 7f91ed000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 0x5622e420ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38358/0 (socket says 172.21.15.107:38358) 2024-09-19T04:21:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.233+0000 7f91ed000640 1 -- 172.21.15.107:0/1849129331 learned_addr learned my addr 172.21.15.107:0/1849129331 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.233+0000 7f91e7e00640 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e420a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.233+0000 7f91e7400640 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 0x5622e420a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.233+0000 7f91ed000640 1 -- 172.21.15.107:0/1849129331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 msgr2=0x5622e420a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.234+0000 7f91ed000640 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e420a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.234+0000 7f91ed000640 1 -- 172.21.15.107:0/1849129331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 msgr2=0x5622e420a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.234+0000 7f91ed000640 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 0x5622e420a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.234+0000 7f91ed000640 1 -- 172.21.15.107:0/1849129331 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e427ff60 con 0x5622e41fdf30 2024-09-19T04:21:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.234+0000 7f91e7e00640 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e420a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.234+0000 7f91ed000640 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 0x5622e420ad00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f91e001fb50 tx=0x7f91e0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ae5c53fba5bd647 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.234+0000 7f91d2000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:21:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.234+0000 7f91d2000640 1 -- 172.21.15.107:0/1849129331 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0010430 con 0x5622e41fdf30 2024-09-19T04:21:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.234+0000 7f91f19e2740 1 -- 172.21.15.107:0/1849129331 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d80bd140 con 0x5622e41fdf30 2024-09-19T04:21:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.235+0000 7f91d2000640 1 -- 172.21.15.107:0/1849129331 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f91e001d430 con 0x5622e41fdf30 2024-09-19T04:21:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.235+0000 7f91d2000640 1 -- 172.21.15.107:0/1849129331 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91e001d630 con 0x5622e41fdf30 2024-09-19T04:21:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.235+0000 7f91f19e2740 10 client.25450 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.235+0000 7f91f19e2740 1 -- 172.21.15.107:0/1849129331 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622e4281300 con 0x5622e41fdf30 2024-09-19T04:21:21.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.235+0000 7f91d2000640 1 -- 172.21.15.107:0/1849129331 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d80bd140 con 0x5622e41fdf30 2024-09-19T04:21:21.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.236+0000 7f91d2000640 1 -- 172.21.15.107:0/1849129331 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f91e0035a00 con 0x5622e41fdf30 2024-09-19T04:21:21.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.236+0000 7f91f19e2740 10 client.25450 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:21.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.236+0000 7f91f19e2740 10 client.25450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:21.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.236+0000 7f91f19e2740 20 client.25450 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:21.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.236+0000 7f91f19e2740 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5622e42819d0 0x5622e4283dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:21.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.236+0000 7f91f19e2740 4 client.25450 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:21:21.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.236+0000 7f91f19e2740 1 -- 172.21.15.107:0/1849129331 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622e40dd0c0 con 0x5622e42819d0 2024-09-19T04:21:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.237+0000 7f91e7e00640 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5622e42819d0 0x5622e4283dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.238+0000 7f91d2000640 10 client.25450 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.238+0000 7f91e7e00640 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5622e42819d0 0x5622e4283dc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84e8fc29a65aa server_cookie=4114f37088a23960 in_seq=0 out_seq=0 2024-09-19T04:21:21.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.238+0000 7f91d2000640 1 -- 172.21.15.107:0/1849129331 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5622e40dd0c0 con 0x5622e42819d0 2024-09-19T04:21:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.238+0000 7f91d2000640 10 client.25450 handle_command_reply: tid=0 2024-09-19T04:21:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.238+0000 7f91f19e2740 1 client.25450 shutdown 2024-09-19T04:21:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.239+0000 7f91f19e2740 1 -- 172.21.15.107:0/1849129331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 msgr2=0x5622e420ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.239+0000 7f91f19e2740 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 0x5622e420ad00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f91e001fb50 tx=0x7f91e0004ab0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.239+0000 7f91f19e2740 1 -- 172.21.15.107:0/1849129331 shutdown_connections 2024-09-19T04:21:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.239+0000 7f91f19e2740 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5622e42819d0 0x5622e4283dc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.239+0000 7f91f19e2740 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5622e41fdf30 0x5622e420ad00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.239+0000 7f91f19e2740 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e41f3670 0x5622e420a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.239+0000 7f91f19e2740 1 --2- 172.21.15.107:0/1849129331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622e41f2d50 0x5622e420a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.239+0000 7f91d2000640 0 client.25450 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.239+0000 7f91f19e2740 1 -- 172.21.15.107:0/1849129331 >> 172.21.15.107:0/1849129331 conn(0x5622e41ef2e0 msgr2=0x5622e41edb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.239+0000 7f91f19e2740 1 -- 172.21.15.107:0/1849129331 shutdown_connections 2024-09-19T04:21:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.240+0000 7f91f19e2740 1 -- 172.21.15.107:0/1849129331 wait complete. 2024-09-19T04:21:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.240+0000 7f91f19e2740 20 client.25450 trim_cache size 0 max 16384 2024-09-19T04:21:21.246 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:21:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:21:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:21:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "18ce476f-2f3d-48f9-853c-a4709bb8d1d0": { 2024-09-19T04:21:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:21:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "18ce476f-2f3d-48f9-853c-a4709bb8d1d0", 2024-09-19T04:21:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:21:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:21:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:21:21.246 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:21:21.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 -- 172.21.15.107:0/2188631864 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91cc0a34d0 msgr2=0x7f91cc0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91cc0a34d0 0x7f91cc0a5990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009fd0 tx=0x7f91e0004a40 comp rx=0 tx=0).stop 2024-09-19T04:21:21.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 -- 172.21.15.107:0/2188631864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e81007a0 msgr2=0x7f91e819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:21.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e81007a0 0x7f91e819a520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91d802a920 tx=0x7f91d8003b20 comp rx=0 tx=0).stop 2024-09-19T04:21:21.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 -- 172.21.15.107:0/2188631864 shutdown_connections 2024-09-19T04:21:21.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91cc0a34d0 0x7f91cc0a5990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e810e350 0x7f91e819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91e8101150 0x7f91e819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 --2- 172.21.15.107:0/2188631864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e81007a0 0x7f91e819a520 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 -- 172.21.15.107:0/2188631864 >> 172.21.15.107:0/2188631864 conn(0x7f91e80fc610 msgr2=0x7f91e80fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.241+0000 7f91eda00640 1 -- 172.21.15.107:0/2188631864 shutdown_connections 2024-09-19T04:21:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:21.242+0000 7f91eda00640 1 -- 172.21.15.107:0/2188631864 wait complete. 2024-09-19T04:21:22.144 INFO:tasks.cephfs.filesystem:scrub status for tag:18ce476f-2f3d-48f9-853c-a4709bb8d1d0 - {'path': '/', 'tag': '18ce476f-2f3d-48f9-853c-a4709bb8d1d0', 'options': 'recursive,force'} 2024-09-19T04:21:22.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:22 smithi107 ceph-mon[30659]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T04:21:22.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:22 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:21:22.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:22 smithi115 ceph-mon[35477]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T04:21:22.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:22 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:21:22.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:22 smithi145 ceph-mon[35096]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T04:21:22.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:22 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:21:24.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:24 smithi115 ceph-mon[35477]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:21:24.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:24 smithi145 ceph-mon[35096]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:21:24.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:24 smithi107 ceph-mon[30659]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:21:26.533 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:26 smithi107 ceph-mon[30659]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T04:21:26.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:26 smithi115 ceph-mon[35477]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T04:21:26.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:26 smithi145 ceph-mon[35096]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T04:21:28.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:28 smithi115 ceph-mon[35477]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:21:28.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:28 smithi145 ceph-mon[35096]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:21:28.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:28 smithi107 ceph-mon[30659]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:21:30.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:30 smithi115 ceph-mon[35477]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:21:30.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:30 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:21:30.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:30 smithi145 ceph-mon[35096]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:21:30.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:30 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:21:30.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:30 smithi107 ceph-mon[30659]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T04:21:30.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:30 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:21:31.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:31 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:31 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:31 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:31 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:31 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:31 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:31 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:31 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:31 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:31 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:31 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:31 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:31 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:31 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:31 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:31 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:31 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:31.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:31 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:32.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:32 smithi115 ceph-mon[35477]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:21:32.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:32 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:21:32.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:32 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:21:32.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:32 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:32.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:32 smithi145 ceph-mon[35096]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:21:32.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:32 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:21:32.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:32 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:21:32.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:32 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:32.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:32 smithi107 ceph-mon[30659]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T04:21:32.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:32 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:21:32.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:32 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:21:32.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:32 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:21:34.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:34 smithi115 ceph-mon[35477]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:34.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:34 smithi145 ceph-mon[35096]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:34.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:34 smithi107 ceph-mon[30659]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:36.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:36 smithi115 ceph-mon[35477]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:21:36.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:36 smithi145 ceph-mon[35096]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:21:36.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:36 smithi107 ceph-mon[30659]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:21:38.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:38 smithi115 ceph-mon[35477]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-19T04:21:38.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:38 smithi145 ceph-mon[35096]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-19T04:21:38.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:38 smithi107 ceph-mon[30659]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-19T04:21:40.385 INFO:tasks.workunit.client.0.smithi107.stderr:Updating files: 64% (8281/12896) Updating files: 65% (8383/12896) Updating files: 66% (8512/12896) Updating files: 67% (8641/12896) Updating files: 68% (8770/12896) Updating files: 69% (8899/12896) Updating files: 70% (9028/12896) Updating files: 71% (9157/12896) Updating files: 72% (9286/12896) Updating files: 73% (9415/12896) Updating files: 74% (9544/12896) Updating files: 75% (9672/12896) Updating files: 76% (9801/12896) Updating files: 77% (9930/12896) Updating files: 78% (10059/12896) Updating files: 79% (10188/12896) Updating files: 80% (10317/12896) Updating files: 81% (10446/12896) Updating files: 82% (10575/12896) Updating files: 83% (10704/12896) Updating files: 84% (10833/12896) Updating files: 85% (10962/12896) Updating files: 86% (11091/12896) Updating files: 87% (11220/12896) Updating files: 88% (11349/12896) Updating files: 89% (11478/12896) Updating files: 90% (11607/12896) Updating files: 91% (11736/12896) Updating files: 92% (11865/12896) Updating files: 93% (11994/12896) Updating files: 94% (12123/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T04:21:40.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:40 smithi115 ceph-mon[35477]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:40.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:40 smithi145 ceph-mon[35096]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:40.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:40 smithi107 ceph-mon[30659]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:41.272 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T04:21:41.272 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-19T04:21:41.272 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T04:21:41.272 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T04:21:41.272 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T04:21:41.272 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-19T04:21:41.273 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T04:21:41.286 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T04:21:41.329 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T04:21:41.334 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T04:21:41.334 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T04:21:42.082 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T04:21:42.190 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T04:21:42.254 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T04:21:42.258 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T04:21:42.258 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T04:21:42.323 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T04:21:42.331 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:21:42.331 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T04:21:42.407 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-19T04:21:42.408 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-19T04:21:42.409 DEBUG:teuthology.orchestra.run.smithi107:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-19T04:21:42.496 INFO:tasks.workunit.client.0.smithi107.stderr:+ dbench 1 2024-09-19T04:21:42.506 INFO:tasks.workunit.client.0.smithi107.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T04:21:42.506 INFO:tasks.workunit.client.0.smithi107.stdout: 2024-09-19T04:21:42.506 INFO:tasks.workunit.client.0.smithi107.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T04:21:42.512 INFO:tasks.workunit.client.0.smithi107.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-19T04:21:42.612 INFO:tasks.workunit.client.0.smithi107.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-19T04:21:42.612 INFO:tasks.workunit.client.0.smithi107.stdout:releasing clients 2024-09-19T04:21:42.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:42 smithi115 ceph-mon[35477]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:21:42.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:42 smithi145 ceph-mon[35096]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:21:42.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:42 smithi107 ceph-mon[30659]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T04:21:43.612 INFO:tasks.workunit.client.0.smithi107.stdout: 1 3189 34.25 MB/sec warmup 1 sec latency 6.785 ms 2024-09-19T04:21:44.612 INFO:tasks.workunit.client.0.smithi107.stdout: 1 5897 23.47 MB/sec warmup 2 sec latency 9.822 ms 2024-09-19T04:21:44.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:44 smithi115 ceph-mon[35477]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:44.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:44 smithi145 ceph-mon[35096]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:44.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:44 smithi107 ceph-mon[30659]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:21:45.612 INFO:tasks.workunit.client.0.smithi107.stdout: 1 8280 20.42 MB/sec warmup 3 sec latency 9.709 ms 2024-09-19T04:21:46.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 11268 19.58 MB/sec warmup 4 sec latency 7.450 ms 2024-09-19T04:21:46.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:46 smithi115 ceph-mon[35477]: pgmap v46: 97 pgs: 97 active+clean; 4.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 271 KiB/s wr, 49 op/s 2024-09-19T04:21:46.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:46 smithi145 ceph-mon[35096]: pgmap v46: 97 pgs: 97 active+clean; 4.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 271 KiB/s wr, 49 op/s 2024-09-19T04:21:46.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:46 smithi107 ceph-mon[30659]: pgmap v46: 97 pgs: 97 active+clean; 4.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 271 KiB/s wr, 49 op/s 2024-09-19T04:21:47.612 INFO:tasks.workunit.client.0.smithi107.stdout: 1 14503 19.42 MB/sec warmup 5 sec latency 9.924 ms 2024-09-19T04:21:48.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 17535 18.64 MB/sec warmup 6 sec latency 10.942 ms 2024-09-19T04:21:48.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:48 smithi115 ceph-mon[35477]: pgmap v47: 97 pgs: 97 active+clean; 5.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 357 KiB/s wr, 53 op/s 2024-09-19T04:21:48.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:48 smithi145 ceph-mon[35096]: pgmap v47: 97 pgs: 97 active+clean; 5.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 357 KiB/s wr, 53 op/s 2024-09-19T04:21:48.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:48 smithi107 ceph-mon[30659]: pgmap v47: 97 pgs: 97 active+clean; 5.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 357 KiB/s wr, 53 op/s 2024-09-19T04:21:49.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 20221 17.68 MB/sec warmup 7 sec latency 13.167 ms 2024-09-19T04:21:50.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 22796 17.60 MB/sec warmup 8 sec latency 13.816 ms 2024-09-19T04:21:50.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:50 smithi115 ceph-mon[35477]: pgmap v48: 97 pgs: 97 active+clean; 5.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 357 KiB/s wr, 49 op/s 2024-09-19T04:21:50.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:50 smithi145 ceph-mon[35096]: pgmap v48: 97 pgs: 97 active+clean; 5.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 357 KiB/s wr, 49 op/s 2024-09-19T04:21:50.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:50 smithi107 ceph-mon[30659]: pgmap v48: 97 pgs: 97 active+clean; 5.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 357 KiB/s wr, 49 op/s 2024-09-19T04:21:51.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 25541 17.22 MB/sec warmup 9 sec latency 14.721 ms 2024-09-19T04:21:52.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:21:52.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 28743 17.37 MB/sec warmup 10 sec latency 13.719 ms 2024-09-19T04:21:52.678 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:52.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:52 smithi115 ceph-mon[35477]: pgmap v49: 97 pgs: 97 active+clean; 12 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-09-19T04:21:52.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:52 smithi115 ceph-mon[35477]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-09-19T04:21:52.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:52 smithi145 ceph-mon[35096]: pgmap v49: 97 pgs: 97 active+clean; 12 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-09-19T04:21:52.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:52 smithi145 ceph-mon[35096]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-09-19T04:21:52.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:52 smithi107 ceph-mon[30659]: pgmap v49: 97 pgs: 97 active+clean; 12 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-09-19T04:21:52.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:52 smithi107 ceph-mon[30659]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-09-19T04:21:53.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.248+0000 7f291d400640 1 -- 172.21.15.107:0/4036024707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29181049a0 msgr2=0x7f29181089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:53.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.248+0000 7f291d400640 1 --2- 172.21.15.107:0/4036024707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29181049a0 0x7f29181089f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2904009900 tx=0x7f290402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.248+0000 7f291d400640 1 -- 172.21.15.107:0/4036024707 shutdown_connections 2024-09-19T04:21:53.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.248+0000 7f291d400640 1 --2- 172.21.15.107:0/4036024707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2918109120 0x7f2918110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.248+0000 7f291d400640 1 --2- 172.21.15.107:0/4036024707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29181049a0 0x7f29181089f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.248+0000 7f291d400640 1 --2- 172.21.15.107:0/4036024707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2918103ff0 0x7f29181043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.248+0000 7f291d400640 1 -- 172.21.15.107:0/4036024707 >> 172.21.15.107:0/4036024707 conn(0x7f29180ffa90 msgr2=0x7f2918101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:53.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.248+0000 7f291d400640 1 -- 172.21.15.107:0/4036024707 shutdown_connections 2024-09-19T04:21:53.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.248+0000 7f291d400640 1 -- 172.21.15.107:0/4036024707 wait complete. 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.249+0000 7f291d400640 1 Processor -- start 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.249+0000 7f291d400640 1 -- start start 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.249+0000 7f291d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2918103ff0 0x7f2918076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.249+0000 7f291d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29181049a0 0x7f2918076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.249+0000 7f291d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2918109120 0x7f2918077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.249+0000 7f291d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2918114190 con 0x7f2918109120 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.249+0000 7f291d400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2918114010 con 0x7f2918103ff0 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.249+0000 7f291d400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2918114310 con 0x7f29181049a0 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2916800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2918103ff0 0x7f2918076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2915e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29181049a0 0x7f2918076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2916800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2918103ff0 0x7f2918076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51628/0 (socket says 172.21.15.107:51628) 2024-09-19T04:21:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2915e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29181049a0 0x7f2918076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51618/0 (socket says 172.21.15.107:51618) 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2916800640 1 -- 172.21.15.107:0/3979359381 learned_addr learned my addr 172.21.15.107:0/3979359381 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2917200640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2918109120 0x7f2918077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2916800640 1 -- 172.21.15.107:0/3979359381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29181049a0 msgr2=0x7f2918076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2916800640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29181049a0 0x7f2918076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2916800640 1 -- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2918109120 msgr2=0x7f2918077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2916800640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2918109120 0x7f2918077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2916800640 1 -- 172.21.15.107:0/3979359381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29181aacd0 con 0x7f2918103ff0 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2917200640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2918109120 0x7f2918077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.250+0000 7f2916800640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2918103ff0 0x7f2918076700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f290800eac0 tx=0x7f290800ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.251+0000 7f290f400640 1 -- 172.21.15.107:0/3979359381 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290800ce40 con 0x7f2918103ff0 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.251+0000 7f290f400640 1 -- 172.21.15.107:0/3979359381 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29080035f0 con 0x7f2918103ff0 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.251+0000 7f291d400640 1 -- 172.21.15.107:0/3979359381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29181aafc0 con 0x7f2918103ff0 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.251+0000 7f290f400640 1 -- 172.21.15.107:0/3979359381 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908010690 con 0x7f2918103ff0 2024-09-19T04:21:53.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.251+0000 7f291d400640 1 -- 172.21.15.107:0/3979359381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29181ab420 con 0x7f2918103ff0 2024-09-19T04:21:53.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.254+0000 7f291fea7740 1 Processor -- start 2024-09-19T04:21:53.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.254+0000 7f290f400640 1 -- 172.21.15.107:0/3979359381 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f29080037a0 con 0x7f2918103ff0 2024-09-19T04:21:53.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.254+0000 7f291fea7740 1 -- start start 2024-09-19T04:21:53.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.254+0000 7f291fea7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 0x560abf715f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.254+0000 7f291fea7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf720830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.254+0000 7f291fea7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 0x560abf723160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.254+0000 7f291fea7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560abf67c3e0 con 0x560abf7164b0 2024-09-19T04:21:53.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.254+0000 7f291fea7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560abf623d80 con 0x560abf720d70 2024-09-19T04:21:53.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.254+0000 7f291fea7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560abf6783a0 con 0x560abf715b90 2024-09-19T04:21:53.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2915e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 0x560abf723160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2916800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf720830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2915e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 0x560abf723160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51632/0 (socket says 172.21.15.107:51632) 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2915e00640 1 -- 172.21.15.107:0/2454746420 learned_addr learned my addr 172.21.15.107:0/2454746420 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2917200640 1 --2- 172.21.15.107:0/2454746420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 0x560abf715f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2915e00640 1 -- 172.21.15.107:0/2454746420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 msgr2=0x560abf715f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2915e00640 1 --2- 172.21.15.107:0/2454746420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 0x560abf715f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2915e00640 1 -- 172.21.15.107:0/2454746420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 msgr2=0x560abf720830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2915e00640 1 --2- 172.21.15.107:0/2454746420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf720830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2915e00640 1 -- 172.21.15.107:0/2454746420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abf642ae0 con 0x560abf720d70 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2917200640 1 --2- 172.21.15.107:0/2454746420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 0x560abf715f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.255+0000 7f2915e00640 1 --2- 172.21.15.107:0/2454746420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 0x560abf723160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2904009900 tx=0x7f2904005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafd67167903c11f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f2903400640 1 -- 172.21.15.107:0/2454746420 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904035040 con 0x560abf720d70 2024-09-19T04:21:53.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f2903400640 1 -- 172.21.15.107:0/2454746420 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2904004400 con 0x560abf720d70 2024-09-19T04:21:53.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f2903400640 1 -- 172.21.15.107:0/2454746420 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29040035f0 con 0x560abf720d70 2024-09-19T04:21:53.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f290f400640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28fc0a33b0 0x7f28fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f290f400640 1 -- 172.21.15.107:0/3979359381 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f29080d0130 con 0x7f2918103ff0 2024-09-19T04:21:53.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f2917200640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28fc0a33b0 0x7f28fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f291fea7740 1 -- 172.21.15.107:0/2454746420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 msgr2=0x560abf723160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:53.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f291fea7740 1 --2- 172.21.15.107:0/2454746420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 0x560abf723160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2904009900 tx=0x7f2904005de0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f291fea7740 1 -- 172.21.15.107:0/2454746420 shutdown_connections 2024-09-19T04:21:53.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f291fea7740 1 --2- 172.21.15.107:0/2454746420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 0x560abf723160 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f291fea7740 1 --2- 172.21.15.107:0/2454746420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf720830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f291fea7740 1 --2- 172.21.15.107:0/2454746420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 0x560abf715f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.256+0000 7f291fea7740 1 -- 172.21.15.107:0/2454746420 >> 172.21.15.107:0/2454746420 conn(0x560abf712120 msgr2=0x560abf712500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.257+0000 7f291fea7740 1 -- 172.21.15.107:0/2454746420 shutdown_connections 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.257+0000 7f291fea7740 1 -- 172.21.15.107:0/2454746420 wait complete. 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.257+0000 7f2917200640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28fc0a33b0 0x7f28fc0a5870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2910005fd0 tx=0x7f2910009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.257+0000 7f291fea7740 1 -- start start 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.257+0000 7f291fea7740 1 Processor -- start 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f291fea7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf71dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f291fea7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf7164b0 0x560abf71e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f291fea7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf720d70 0x560abf71e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f291fea7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560abf7239e0 con 0x560abf720d70 2024-09-19T04:21:53.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f291fea7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560abf6370f0 con 0x560abf715b90 2024-09-19T04:21:53.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f291fea7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560abf723b60 con 0x560abf7164b0 2024-09-19T04:21:53.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f291fea7740 10 client.0 _ll_register_callbacks cb 0x560abf6289e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:53.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f2916800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf71dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f2915e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf7164b0 0x560abf71e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f2916800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf71dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51636/0 (socket says 172.21.15.107:51636) 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f2916800640 1 -- 172.21.15.107:0/648469426 learned_addr learned my addr 172.21.15.107:0/648469426 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f2917200640 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf720d70 0x560abf71e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f2916800640 1 -- 172.21.15.107:0/648469426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf7164b0 msgr2=0x560abf71e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f2916800640 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf7164b0 0x560abf71e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f2916800640 1 -- 172.21.15.107:0/648469426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf720d70 msgr2=0x560abf71e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.258+0000 7f2916800640 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf720d70 0x560abf71e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f2916800640 1 -- 172.21.15.107:0/648469426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abf3ed3c0 con 0x560abf715b90 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f2915e00640 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf7164b0 0x560abf71e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f2916800640 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf71dcd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f290800ea90 tx=0x7f29080986a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59976741df9c0193 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f28f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:21:53.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f28f7e00640 1 -- 172.21.15.107:0/648469426 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290809bd60 con 0x560abf715b90 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f28f7e00640 1 -- 172.21.15.107:0/648469426 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f2908098a70 con 0x560abf715b90 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f291fea7740 1 -- 172.21.15.107:0/648469426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2904004400 con 0x560abf715b90 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f28f7e00640 1 -- 172.21.15.107:0/648469426 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29080b4070 con 0x560abf715b90 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f291fea7740 10 client.25487 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f291fea7740 1 -- 172.21.15.107:0/648469426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560abf72f710 con 0x560abf715b90 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.259+0000 7f28f7e00640 1 -- 172.21.15.107:0/648469426 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2904004400 con 0x560abf715b90 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.260+0000 7f28f7e00640 1 -- 172.21.15.107:0/648469426 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f29080b5260 con 0x560abf715b90 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.260+0000 7f291fea7740 10 client.25487 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.260+0000 7f291fea7740 10 client.25487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.260+0000 7f291fea7740 20 client.25487 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.260+0000 7f291fea7740 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560abf730b20 0x560abf732f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.260+0000 7f291fea7740 4 client.25487 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.260+0000 7f291fea7740 1 -- 172.21.15.107:0/648469426 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560abf689950 con 0x560abf730b20 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.260+0000 7f2915e00640 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560abf730b20 0x560abf732f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.260+0000 7f2915e00640 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560abf730b20 0x560abf732f90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5e21351e35c1c8 server_cookie=6c41624a4033fead in_seq=0 out_seq=0 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.260+0000 7f28f7e00640 10 client.25487 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.262+0000 7f28f7e00640 1 -- 172.21.15.107:0/648469426 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560abf689950 con 0x560abf730b20 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.262+0000 7f28f7e00640 10 client.25487 handle_command_reply: tid=0 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 client.25487 shutdown 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 -- 172.21.15.107:0/648469426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 msgr2=0x560abf71dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf71dcd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f290800ea90 tx=0x7f29080986a0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 -- 172.21.15.107:0/648469426 shutdown_connections 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560abf730b20 0x560abf732f90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf720d70 0x560abf71e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf7164b0 0x560abf71e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 --2- 172.21.15.107:0/648469426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf71dcd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 -- 172.21.15.107:0/648469426 >> 172.21.15.107:0/648469426 conn(0x560abf712120 msgr2=0x560abf717e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:53.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f28f7e00640 0 client.25487 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:53.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 -- 172.21.15.107:0/648469426 shutdown_connections 2024-09-19T04:21:53.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 1 -- 172.21.15.107:0/648469426 wait complete. 2024-09-19T04:21:53.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.263+0000 7f291fea7740 20 client.25487 trim_cache size 0 max 16384 2024-09-19T04:21:53.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.281+0000 7f291fea7740 1 Processor -- start 2024-09-19T04:21:53.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.281+0000 7f291fea7740 1 -- start start 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.281+0000 7f291fea7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf7185f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.281+0000 7f291fea7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf71c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.281+0000 7f291fea7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf720d70 0x560abf7110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.281+0000 7f291fea7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b3d90 con 0x560abf7164b0 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.281+0000 7f291fea7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b58d0 con 0x560abf715b90 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.281+0000 7f291fea7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b4460 con 0x560abf720d70 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2916800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf7185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2915e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf71c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2916800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf7185f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51646/0 (socket says 172.21.15.107:51646) 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2916800640 1 -- 172.21.15.107:0/2646297771 learned_addr learned my addr 172.21.15.107:0/2646297771 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2915e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf71c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50768/0 (socket says 172.21.15.107:50768) 2024-09-19T04:21:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2917200640 1 --2- 172.21.15.107:0/2646297771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf720d70 0x560abf7110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2916800640 1 -- 172.21.15.107:0/2646297771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf720d70 msgr2=0x560abf7110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2916800640 1 --2- 172.21.15.107:0/2646297771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf720d70 0x560abf7110b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2916800640 1 -- 172.21.15.107:0/2646297771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 msgr2=0x560abf71c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2916800640 1 --2- 172.21.15.107:0/2646297771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf71c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.282+0000 7f2916800640 1 -- 172.21.15.107:0/2646297771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abf71cc80 con 0x560abf715b90 2024-09-19T04:21:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.283+0000 7f2916800640 1 --2- 172.21.15.107:0/2646297771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf7185f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f290809f040 tx=0x7f290809a910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6774ae719d0033ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.283+0000 7f2902a00640 1 -- 172.21.15.107:0/2646297771 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290809bd60 con 0x560abf715b90 2024-09-19T04:21:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.283+0000 7f2902a00640 1 -- 172.21.15.107:0/2646297771 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290809c400 con 0x560abf715b90 2024-09-19T04:21:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.283+0000 7f2902a00640 1 -- 172.21.15.107:0/2646297771 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908098a70 con 0x560abf715b90 2024-09-19T04:21:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.283+0000 7f291fea7740 1 -- 172.21.15.107:0/2646297771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 msgr2=0x560abf7185f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:53.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.283+0000 7f291fea7740 1 --2- 172.21.15.107:0/2646297771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf7185f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f290809f040 tx=0x7f290809a910 comp rx=0 tx=0).stop 2024-09-19T04:21:53.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.284+0000 7f291fea7740 1 -- 172.21.15.107:0/2646297771 shutdown_connections 2024-09-19T04:21:53.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.284+0000 7f291fea7740 1 --2- 172.21.15.107:0/2646297771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf720d70 0x560abf7110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.284+0000 7f291fea7740 1 --2- 172.21.15.107:0/2646297771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf71c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.284+0000 7f291fea7740 1 --2- 172.21.15.107:0/2646297771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf715b90 0x560abf7185f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.284+0000 7f291fea7740 1 -- 172.21.15.107:0/2646297771 >> 172.21.15.107:0/2646297771 conn(0x560abf712120 msgr2=0x560abf7109a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:53.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.284+0000 7f291fea7740 1 -- 172.21.15.107:0/2646297771 shutdown_connections 2024-09-19T04:21:53.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.284+0000 7f291fea7740 1 -- 172.21.15.107:0/2646297771 wait complete. 2024-09-19T04:21:53.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.284+0000 7f291fea7740 1 -- start start 2024-09-19T04:21:53.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f291fea7740 1 Processor -- start 2024-09-19T04:21:53.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f291fea7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 0x560abf728db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f291fea7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf7292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f291fea7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 0x560abf725d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f291fea7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b5200 con 0x560abf7164b0 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f291fea7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b4b30 con 0x560abf720d70 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f291fea7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b1d70 con 0x560abf715b90 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f291fea7740 10 client.0 _ll_register_callbacks cb 0x7f2908009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f2916800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 0x560abf728db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f2916800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 0x560abf728db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51664/0 (socket says 172.21.15.107:51664) 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f2916800640 1 -- 172.21.15.107:0/3417110348 learned_addr learned my addr 172.21.15.107:0/3417110348 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.285+0000 7f2915e00640 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf7292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f2917200640 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 0x560abf725d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f2915e00640 1 -- 172.21.15.107:0/3417110348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 msgr2=0x560abf728db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:53.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f2915e00640 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 0x560abf728db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f2915e00640 1 -- 172.21.15.107:0/3417110348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 msgr2=0x560abf725d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f2915e00640 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 0x560abf725d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f2915e00640 1 -- 172.21.15.107:0/3417110348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abf7262f0 con 0x560abf7164b0 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f2915e00640 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf7292f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f290402a910 tx=0x7f290402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a807c8848dff0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f28f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f28f7e00640 1 -- 172.21.15.107:0/3417110348 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904035040 con 0x560abf7164b0 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f28f7e00640 1 -- 172.21.15.107:0/3417110348 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f2904004400 con 0x560abf7164b0 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.286+0000 7f291fea7740 1 -- 172.21.15.107:0/3417110348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290809c400 con 0x560abf7164b0 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f28f7e00640 1 -- 172.21.15.107:0/3417110348 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29040315b0 con 0x560abf7164b0 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f291fea7740 10 client.15447 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f291fea7740 1 -- 172.21.15.107:0/3417110348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560abf79c100 con 0x560abf7164b0 2024-09-19T04:21:53.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f28f7e00640 1 -- 172.21.15.107:0/3417110348 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290809c400 con 0x560abf7164b0 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f28f7e00640 1 -- 172.21.15.107:0/3417110348 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2904003960 con 0x560abf7164b0 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f291fea7740 10 client.15447 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f291fea7740 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f291fea7740 20 client.15447 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f291fea7740 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560abf79c490 0x560abf79e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f291fea7740 4 client.15447 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.287+0000 7f291fea7740 1 -- 172.21.15.107:0/3417110348 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560abf683e60 con 0x560abf79c490 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f2916800640 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560abf79c490 0x560abf79e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f2916800640 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560abf79c490 0x560abf79e880 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe6af1b7c4d1217 server_cookie=92c9e2c83d6a8992 in_seq=0 out_seq=0 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f28f7e00640 10 client.15447 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f28f7e00640 1 -- 172.21.15.107:0/3417110348 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560abf683e60 con 0x560abf79c490 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f28f7e00640 10 client.15447 handle_command_reply: tid=0 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f291fea7740 1 client.15447 shutdown 2024-09-19T04:21:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f291fea7740 1 -- 172.21.15.107:0/3417110348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 msgr2=0x560abf7292f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f291fea7740 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf7292f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f290402a910 tx=0x7f290402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f291fea7740 1 -- 172.21.15.107:0/3417110348 shutdown_connections 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f291fea7740 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560abf79c490 0x560abf79e880 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f291fea7740 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560abf720d70 0x560abf725d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f291fea7740 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560abf7164b0 0x560abf7292f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f291fea7740 1 --2- 172.21.15.107:0/3417110348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560abf715b90 0x560abf728db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f291fea7740 1 -- 172.21.15.107:0/3417110348 >> 172.21.15.107:0/3417110348 conn(0x560abf712120 msgr2=0x560abf7111c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.288+0000 7f28f7e00640 0 client.15447 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.289+0000 7f291fea7740 1 -- 172.21.15.107:0/3417110348 shutdown_connections 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.289+0000 7f291fea7740 1 -- 172.21.15.107:0/3417110348 wait complete. 2024-09-19T04:21:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.289+0000 7f291fea7740 20 client.15447 trim_cache size 0 max 16384 2024-09-19T04:21:53.295 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:21:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:21:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:21:53.295 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.289+0000 7f291d400640 1 -- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28fc0a33b0 msgr2=0x7f28fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.289+0000 7f291d400640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28fc0a33b0 0x7f28fc0a5870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2910005fd0 tx=0x7f2910009500 comp rx=0 tx=0).stop 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.290+0000 7f291d400640 1 -- 172.21.15.107:0/3979359381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2918103ff0 msgr2=0x7f2918076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.290+0000 7f291d400640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2918103ff0 0x7f2918076700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f290800eac0 tx=0x7f290800ef80 comp rx=0 tx=0).stop 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.290+0000 7f291d400640 1 -- 172.21.15.107:0/3979359381 shutdown_connections 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.290+0000 7f291d400640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28fc0a33b0 0x7f28fc0a5870 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.290+0000 7f291d400640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2918109120 0x7f2918077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.290+0000 7f291d400640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29181049a0 0x7f2918076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.290+0000 7f291d400640 1 --2- 172.21.15.107:0/3979359381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2918103ff0 0x7f2918076700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.290+0000 7f291d400640 1 -- 172.21.15.107:0/3979359381 >> 172.21.15.107:0/3979359381 conn(0x7f29180ffa90 msgr2=0x7f2918100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.290+0000 7f291d400640 1 -- 172.21.15.107:0/3979359381 shutdown_connections 2024-09-19T04:21:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:53.290+0000 7f291d400640 1 -- 172.21.15.107:0/3979359381 wait complete. 2024-09-19T04:21:53.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 31766 17.13 MB/sec warmup 11 sec latency 10.090 ms 2024-09-19T04:21:54.100 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:21:54.100 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:21:54.492 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:54.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 34360 16.68 MB/sec warmup 12 sec latency 27.629 ms 2024-09-19T04:21:54.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:54 smithi107 ceph-mon[30659]: pgmap v50: 97 pgs: 97 active+clean; 12 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-09-19T04:21:54.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:54 smithi115 ceph-mon[35477]: pgmap v50: 97 pgs: 97 active+clean; 12 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-09-19T04:21:54.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:54 smithi145 ceph-mon[35096]: pgmap v50: 97 pgs: 97 active+clean; 12 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-09-19T04:21:55.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.092+0000 7f0e41400640 1 -- 172.21.15.107:0/1516280516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e3c104010 msgr2=0x7f0e3c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:55.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.092+0000 7f0e41400640 1 --2- 172.21.15.107:0/1516280516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e3c104010 0x7f0e3c1043f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009900 tx=0x7f0e2c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:21:55.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.092+0000 7f0e41400640 1 -- 172.21.15.107:0/1516280516 shutdown_connections 2024-09-19T04:21:55.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.092+0000 7f0e41400640 1 --2- 172.21.15.107:0/1516280516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e3c109140 0x7f0e3c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:55.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.092+0000 7f0e41400640 1 --2- 172.21.15.107:0/1516280516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c1049c0 0x7f0e3c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:55.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.092+0000 7f0e41400640 1 --2- 172.21.15.107:0/1516280516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e3c104010 0x7f0e3c1043f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:55.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.092+0000 7f0e41400640 1 -- 172.21.15.107:0/1516280516 >> 172.21.15.107:0/1516280516 conn(0x7f0e3c0ffa50 msgr2=0x7f0e3c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:55.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.092+0000 7f0e41400640 1 -- 172.21.15.107:0/1516280516 shutdown_connections 2024-09-19T04:21:55.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.092+0000 7f0e41400640 1 -- 172.21.15.107:0/1516280516 wait complete. 2024-09-19T04:21:55.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.093+0000 7f0e41400640 1 Processor -- start 2024-09-19T04:21:55.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.093+0000 7f0e41400640 1 -- start start 2024-09-19T04:21:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.093+0000 7f0e41400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e3c104010 0x7f0e3c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.093+0000 7f0e41400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c1049c0 0x7f0e3c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.093+0000 7f0e41400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e3c109140 0x7f0e3c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.093+0000 7f0e41400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c1141c0 con 0x7f0e3c1049c0 2024-09-19T04:21:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e41400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c114040 con 0x7f0e3c104010 2024-09-19T04:21:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e41400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c114340 con 0x7f0e3c109140 2024-09-19T04:21:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e3a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e3c104010 0x7f0e3c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e3a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e3c104010 0x7f0e3c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51668/0 (socket says 172.21.15.107:51668) 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e3a800640 1 -- 172.21.15.107:0/2832575150 learned_addr learned my addr 172.21.15.107:0/2832575150 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e3b200640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e3c109140 0x7f0e3c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e33e00640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c1049c0 0x7f0e3c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e3a800640 1 -- 172.21.15.107:0/2832575150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e3c109140 msgr2=0x7f0e3c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e3a800640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e3c109140 0x7f0e3c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e3a800640 1 -- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c1049c0 msgr2=0x7f0e3c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e3a800640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c1049c0 0x7f0e3c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e3a800640 1 -- 172.21.15.107:0/2832575150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c19f5f0 con 0x7f0e3c104010 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e33e00640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c1049c0 0x7f0e3c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:55.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.094+0000 7f0e3a800640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e3c104010 0x7f0e3c19a640 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c004770 tx=0x7f0e2c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.095+0000 7f0e33400640 1 -- 172.21.15.107:0/2832575150 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c03f070 con 0x7f0e3c104010 2024-09-19T04:21:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.095+0000 7f0e41400640 1 -- 172.21.15.107:0/2832575150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c19f880 con 0x7f0e3c104010 2024-09-19T04:21:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.095+0000 7f0e33400640 1 -- 172.21.15.107:0/2832575150 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2c031ca0 con 0x7f0e3c104010 2024-09-19T04:21:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.095+0000 7f0e33400640 1 -- 172.21.15.107:0/2832575150 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c035030 con 0x7f0e3c104010 2024-09-19T04:21:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.095+0000 7f0e41400640 1 -- 172.21.15.107:0/2832575150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3c1a71c0 con 0x7f0e3c104010 2024-09-19T04:21:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.096+0000 7f0e41400640 1 -- 172.21.15.107:0/2832575150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e3c1054f0 con 0x7f0e3c104010 2024-09-19T04:21:55.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.098+0000 7f0e33400640 1 -- 172.21.15.107:0/2832575150 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f0e2c042070 con 0x7f0e3c104010 2024-09-19T04:21:55.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.099+0000 7f0e33400640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0e200a33b0 0x7f0e200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:55.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.099+0000 7f0e33400640 1 -- 172.21.15.107:0/2832575150 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0e2c0f03c0 con 0x7f0e3c104010 2024-09-19T04:21:55.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.099+0000 7f0e33e00640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0e200a33b0 0x7f0e200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:55.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.100+0000 7f0e33e00640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0e200a33b0 0x7f0e200a5870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0ffe70 tx=0x7f0e24005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.103+0000 7f0e33400640 1 -- 172.21.15.107:0/2832575150 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0e2c0bce50 con 0x7f0e3c104010 2024-09-19T04:21:55.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.326+0000 7f0e41400640 1 -- 172.21.15.107:0/2832575150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e3c1043f0 con 0x7f0e3c104010 2024-09-19T04:21:55.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.327+0000 7f0e33400640 1 -- 172.21.15.107:0/2832575150 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11992 (secure 0 0 0) 0x7f0e2c0bc750 con 0x7f0e3c104010 2024-09-19T04:21:55.329 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:21:55.329 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"btime":"2024-09-19T04:20:19:828247+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:20:18.993568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24490,24496,24500,24539,24517]},"id":1}]} 2024-09-19T04:21:55.329 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-09-19T04:21:55.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 -- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0e200a33b0 msgr2=0x7f0e200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:55.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0e200a33b0 0x7f0e200a5870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0ffe70 tx=0x7f0e24005f90 comp rx=0 tx=0).stop 2024-09-19T04:21:55.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 -- 172.21.15.107:0/2832575150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e3c104010 msgr2=0x7f0e3c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:55.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e3c104010 0x7f0e3c19a640 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c004770 tx=0x7f0e2c003c10 comp rx=0 tx=0).stop 2024-09-19T04:21:55.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 -- 172.21.15.107:0/2832575150 shutdown_connections 2024-09-19T04:21:55.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0e200a33b0 0x7f0e200a5870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:55.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e3c109140 0x7f0e3c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:55.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c1049c0 0x7f0e3c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:55.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 --2- 172.21.15.107:0/2832575150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e3c104010 0x7f0e3c19a640 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:55.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 -- 172.21.15.107:0/2832575150 >> 172.21.15.107:0/2832575150 conn(0x7f0e3c0ffa50 msgr2=0x7f0e3c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:55.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 -- 172.21.15.107:0/2832575150 shutdown_connections 2024-09-19T04:21:55.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:55.330+0000 7f0e41400640 1 -- 172.21.15.107:0/2832575150 wait complete. 2024-09-19T04:21:55.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 36675 16.48 MB/sec warmup 13 sec latency 12.200 ms 2024-09-19T04:21:55.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:55 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2832575150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:21:55.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:55 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2832575150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:21:55.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:55 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2832575150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:21:56.178 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:21:56.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:56 smithi107 ceph-mon[30659]: pgmap v51: 97 pgs: 97 active+clean; 16 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-09-19T04:21:56.607 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:56.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 39625 16.52 MB/sec warmup 14 sec latency 30.078 ms 2024-09-19T04:21:56.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:56 smithi115 ceph-mon[35477]: pgmap v51: 97 pgs: 97 active+clean; 16 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-09-19T04:21:56.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:56 smithi145 ceph-mon[35096]: pgmap v51: 97 pgs: 97 active+clean; 16 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-09-19T04:21:57.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.200+0000 7f40ff000640 1 -- 172.21.15.107:0/2478931886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f8106f10 msgr2=0x7f40f810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:57.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.200+0000 7f40ff000640 1 --2- 172.21.15.107:0/2478931886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f8106f10 0x7f40f810ea90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009ca0 tx=0x7f40f002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.201+0000 7f40ff000640 1 -- 172.21.15.107:0/2478931886 shutdown_connections 2024-09-19T04:21:57.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.201+0000 7f40ff000640 1 --2- 172.21.15.107:0/2478931886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f8106f10 0x7f40f810ea90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.201+0000 7f40ff000640 1 --2- 172.21.15.107:0/2478931886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8102790 0x7f40f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.201+0000 7f40ff000640 1 --2- 172.21.15.107:0/2478931886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8101de0 0x7f40f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.201+0000 7f40ff000640 1 -- 172.21.15.107:0/2478931886 >> 172.21.15.107:0/2478931886 conn(0x7f40f80fd8c0 msgr2=0x7f40f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:57.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.201+0000 7f40ff000640 1 -- 172.21.15.107:0/2478931886 shutdown_connections 2024-09-19T04:21:57.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.201+0000 7f40ff000640 1 -- 172.21.15.107:0/2478931886 wait complete. 2024-09-19T04:21:57.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.202+0000 7f40ff000640 1 Processor -- start 2024-09-19T04:21:57.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.202+0000 7f40ff000640 1 -- start start 2024-09-19T04:21:57.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40ff000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f8101de0 0x7f40f819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40ff000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8102790 0x7f40f819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40ff000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8106f10 0x7f40f819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40ff000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8111f70 con 0x7f40f8106f10 2024-09-19T04:21:57.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40ff000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8111df0 con 0x7f40f8101de0 2024-09-19T04:21:57.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40ff000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81120f0 con 0x7f40f8102790 2024-09-19T04:21:57.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f8101de0 0x7f40f819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f8101de0 0x7f40f819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51694/0 (socket says 172.21.15.107:51694) 2024-09-19T04:21:57.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40f7e00640 1 -- 172.21.15.107:0/4096942984 learned_addr learned my addr 172.21.15.107:0/4096942984 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:57.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40fd000640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8106f10 0x7f40f819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40f7400640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8102790 0x7f40f819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40f7e00640 1 -- 172.21.15.107:0/4096942984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8102790 msgr2=0x7f40f819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40f7e00640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8102790 0x7f40f819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40f7e00640 1 -- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8106f10 msgr2=0x7f40f819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40f7e00640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8106f10 0x7f40f819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.203+0000 7f40f7e00640 1 -- 172.21.15.107:0/4096942984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f819f5c0 con 0x7f40f8101de0 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.204+0000 7f40fd000640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8106f10 0x7f40f819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.204+0000 7f40f7e00640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f8101de0 0x7f40f819a610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e9f0 tx=0x7f40e800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.204+0000 7f40f4c00640 1 -- 172.21.15.107:0/4096942984 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e800cd80 con 0x7f40f8101de0 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.204+0000 7f40f4c00640 1 -- 172.21.15.107:0/4096942984 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8005500 con 0x7f40f8101de0 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.204+0000 7f40f4c00640 1 -- 172.21.15.107:0/4096942984 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8010690 con 0x7f40f8101de0 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.204+0000 7f40ff000640 1 -- 172.21.15.107:0/4096942984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f819f8b0 con 0x7f40f8101de0 2024-09-19T04:21:57.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.204+0000 7f40ff000640 1 -- 172.21.15.107:0/4096942984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f81a7190 con 0x7f40f8101de0 2024-09-19T04:21:57.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.208+0000 7f4101a92740 1 Processor -- start 2024-09-19T04:21:57.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.208+0000 7f4101a92740 1 -- start start 2024-09-19T04:21:57.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f40f4c00640 1 -- 172.21.15.107:0/4096942984 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f40e80046c0 con 0x7f40f8101de0 2024-09-19T04:21:57.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f4101a92740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028d95050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f4101a92740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 0x563028d9f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f4101a92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 0x563028da2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f4101a92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563028cfb3e0 con 0x563028d9fe50 2024-09-19T04:21:57.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f4101a92740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563028ca2d80 con 0x563028d94c70 2024-09-19T04:21:57.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f4101a92740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563028cf73a0 con 0x563028d95590 2024-09-19T04:21:57.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 0x563028d9f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f40fd000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028d95050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 0x563028d9f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51710/0 (socket says 172.21.15.107:51710) 2024-09-19T04:21:57.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.209+0000 7f40f7e00640 1 -- 172.21.15.107:0/1621448174 learned_addr learned my addr 172.21.15.107:0/1621448174 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:57.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40f7400640 1 --2- 172.21.15.107:0/1621448174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 0x563028da2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40f7e00640 1 -- 172.21.15.107:0/1621448174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 msgr2=0x563028d95050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:57.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1621448174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028d95050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40f7e00640 1 -- 172.21.15.107:0/1621448174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 msgr2=0x563028da2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:57.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1621448174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 0x563028da2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40f7e00640 1 -- 172.21.15.107:0/1621448174 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563028cc1ae0 con 0x563028d95590 2024-09-19T04:21:57.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40fd000640 1 --2- 172.21.15.107:0/1621448174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028d95050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:57.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40f7400640 1 --2- 172.21.15.107:0/1621448174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 0x563028da2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:57.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1621448174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 0x563028d9f910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40e80cd8b0 tx=0x7f40e80d8330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8304b29846c1b9df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:57.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40ed600640 1 -- 172.21.15.107:0/1621448174 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80e3070 con 0x563028d95590 2024-09-19T04:21:57.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40ed600640 1 -- 172.21.15.107:0/1621448174 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e80d0650 con 0x563028d95590 2024-09-19T04:21:57.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40ed600640 1 -- 172.21.15.107:0/1621448174 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80de640 con 0x563028d95590 2024-09-19T04:21:57.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.210+0000 7f40f4c00640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40dc0a3430 0x7f40dc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.211+0000 7f40f4c00640 1 -- 172.21.15.107:0/4096942984 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f40e80d01a0 con 0x7f40f8101de0 2024-09-19T04:21:57.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.211+0000 7f40fd000640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40dc0a3430 0x7f40dc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.211+0000 7f4101a92740 1 -- 172.21.15.107:0/1621448174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 msgr2=0x563028d9f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:57.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.211+0000 7f4101a92740 1 --2- 172.21.15.107:0/1621448174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 0x563028d9f910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40e80cd8b0 tx=0x7f40e80d8330 comp rx=0 tx=0).stop 2024-09-19T04:21:57.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.211+0000 7f4101a92740 1 -- 172.21.15.107:0/1621448174 shutdown_connections 2024-09-19T04:21:57.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.211+0000 7f4101a92740 1 --2- 172.21.15.107:0/1621448174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 0x563028da2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.211+0000 7f4101a92740 1 --2- 172.21.15.107:0/1621448174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 0x563028d9f910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.211+0000 7f4101a92740 1 --2- 172.21.15.107:0/1621448174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028d95050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.211+0000 7f4101a92740 1 -- 172.21.15.107:0/1621448174 >> 172.21.15.107:0/1621448174 conn(0x563028d91200 msgr2=0x563028d915e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:57.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.211+0000 7f40fd000640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40dc0a3430 0x7f40dc0a58f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005cc0 tx=0x7f40f0005f80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:57.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.212+0000 7f4101a92740 1 -- 172.21.15.107:0/1621448174 shutdown_connections 2024-09-19T04:21:57.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.212+0000 7f4101a92740 1 -- 172.21.15.107:0/1621448174 wait complete. 2024-09-19T04:21:57.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.212+0000 7f4101a92740 1 -- start start 2024-09-19T04:21:57.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.212+0000 7f4101a92740 1 Processor -- start 2024-09-19T04:21:57.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f4101a92740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d94c70 0x563028d9cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f4101a92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d95590 0x563028d9d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f4101a92740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d9fe50 0x563028d9d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f4101a92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563028da2a20 con 0x563028d95590 2024-09-19T04:21:57.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f4101a92740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563028cb60f0 con 0x563028d9fe50 2024-09-19T04:21:57.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f4101a92740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563028da2ba0 con 0x563028d94c70 2024-09-19T04:21:57.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f4101a92740 10 client.0 _ll_register_callbacks cb 0x563028c9dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:57.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40fd000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d9fe50 0x563028d9d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d94c70 0x563028d9cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40fd000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d9fe50 0x563028d9d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51714/0 (socket says 172.21.15.107:51714) 2024-09-19T04:21:57.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40fd000640 1 -- 172.21.15.107:0/1392817185 learned_addr learned my addr 172.21.15.107:0/1392817185 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:57.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40f7400640 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d95590 0x563028d9d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40f7e00640 1 -- 172.21.15.107:0/1392817185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d9fe50 msgr2=0x563028d9d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:57.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d9fe50 0x563028d9d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40f7e00640 1 -- 172.21.15.107:0/1392817185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d95590 msgr2=0x563028d9d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:57.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d95590 0x563028d9d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40f7e00640 1 -- 172.21.15.107:0/1392817185 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563028a6c3c0 con 0x563028d94c70 2024-09-19T04:21:57.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.213+0000 7f40f7400640 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d95590 0x563028d9d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:57.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.214+0000 7f40f7e00640 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d94c70 0x563028d9cc10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f40e800c6a0 tx=0x7f40e80d3ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29c8673da0c2fe6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:57.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.214+0000 7f40e2000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:21:57.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.214+0000 7f40e2000640 1 -- 172.21.15.107:0/1392817185 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8098770 con 0x563028d94c70 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.214+0000 7f40e2000640 1 -- 172.21.15.107:0/1392817185 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f40e8016df0 con 0x563028d94c70 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.214+0000 7f4101a92740 1 -- 172.21.15.107:0/1392817185 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e80d0650 con 0x563028d94c70 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.214+0000 7f40e2000640 1 -- 172.21.15.107:0/1392817185 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f40e809be30 con 0x563028d94c70 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.214+0000 7f4101a92740 10 client.25486 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.214+0000 7f4101a92740 1 -- 172.21.15.107:0/1392817185 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563028da2a20 con 0x563028d94c70 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.214+0000 7f40e2000640 1 -- 172.21.15.107:0/1392817185 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e80d0650 con 0x563028d94c70 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.215+0000 7f40e2000640 1 -- 172.21.15.107:0/1392817185 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f40e8098b40 con 0x563028d94c70 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.215+0000 7f4101a92740 10 client.25486 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.215+0000 7f4101a92740 10 client.25486 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.215+0000 7f4101a92740 20 client.25486 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.215+0000 7f4101a92740 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x563028dafba0 0x563028db1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.215+0000 7f4101a92740 4 client.25486 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.215+0000 7f4101a92740 1 -- 172.21.15.107:0/1392817185 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563028d08950 con 0x563028dafba0 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.215+0000 7f40f7400640 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x563028dafba0 0x563028db1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.216+0000 7f40f7400640 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x563028dafba0 0x563028db1f90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21d82112e166acdc server_cookie=869f136cd0ee787e in_seq=0 out_seq=0 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.216+0000 7f40e2000640 10 client.25486 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.216+0000 7f40e2000640 1 -- 172.21.15.107:0/1392817185 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563028d08950 con 0x563028dafba0 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.216+0000 7f40e2000640 10 client.25486 handle_command_reply: tid=0 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.216+0000 7f4101a92740 1 client.25486 shutdown 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 1 -- 172.21.15.107:0/1392817185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d94c70 msgr2=0x563028d9cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d94c70 0x563028d9cc10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f40e800c6a0 tx=0x7f40e80d3ea0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 1 -- 172.21.15.107:0/1392817185 shutdown_connections 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x563028dafba0 0x563028db1f90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d9fe50 0x563028d9d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d95590 0x563028d9d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 1 --2- 172.21.15.107:0/1392817185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d94c70 0x563028d9cc10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 1 -- 172.21.15.107:0/1392817185 >> 172.21.15.107:0/1392817185 conn(0x563028d91200 msgr2=0x563028d90380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:57.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f40e2000640 0 client.25486 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:21:57.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 1 -- 172.21.15.107:0/1392817185 shutdown_connections 2024-09-19T04:21:57.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 1 -- 172.21.15.107:0/1392817185 wait complete. 2024-09-19T04:21:57.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.217+0000 7f4101a92740 20 client.25486 trim_cache size 0 max 16384 2024-09-19T04:21:57.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.231+0000 7f4101a92740 1 Processor -- start 2024-09-19T04:21:57.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f4101a92740 1 -- start start 2024-09-19T04:21:57.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f4101a92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d94c70 0x563028d976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f4101a92740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d95590 0x563028db7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f4101a92740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d9fe50 0x563028db7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f4101a92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81af990 con 0x563028d94c70 2024-09-19T04:21:57.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f4101a92740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b14d0 con 0x563028d95590 2024-09-19T04:21:57.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f4101a92740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b0060 con 0x563028d9fe50 2024-09-19T04:21:57.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d94c70 0x563028d976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f40f7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d95590 0x563028db7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f40f7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d95590 0x563028db7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51726/0 (socket says 172.21.15.107:51726) 2024-09-19T04:21:57.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f40f7400640 1 -- 172.21.15.107:0/686107192 learned_addr learned my addr 172.21.15.107:0/686107192 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:57.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d94c70 0x563028d976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50838/0 (socket says 172.21.15.107:50838) 2024-09-19T04:21:57.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.232+0000 7f40fd000640 1 --2- 172.21.15.107:0/686107192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d9fe50 0x563028db7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40f7400640 1 -- 172.21.15.107:0/686107192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d9fe50 msgr2=0x563028db7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:57.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40f7400640 1 --2- 172.21.15.107:0/686107192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d9fe50 0x563028db7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40f7400640 1 -- 172.21.15.107:0/686107192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d94c70 msgr2=0x563028d976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:57.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40f7400640 1 --2- 172.21.15.107:0/686107192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d94c70 0x563028d976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40f7400640 1 -- 172.21.15.107:0/686107192 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563028d91030 con 0x563028d95590 2024-09-19T04:21:57.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40fd000640 1 --2- 172.21.15.107:0/686107192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d9fe50 0x563028db7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:21:57.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40f7e00640 1 --2- 172.21.15.107:0/686107192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d94c70 0x563028d976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:57.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40f7400640 1 --2- 172.21.15.107:0/686107192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d95590 0x563028db7560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x563028d92580 tx=0x7f40e400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2805c660862f65bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:57.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40ecc00640 1 -- 172.21.15.107:0/686107192 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e401c470 con 0x563028d95590 2024-09-19T04:21:57.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40ecc00640 1 -- 172.21.15.107:0/686107192 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e401caf0 con 0x563028d95590 2024-09-19T04:21:57.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.233+0000 7f40ecc00640 1 -- 172.21.15.107:0/686107192 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e401bcd0 con 0x563028d95590 2024-09-19T04:21:57.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.234+0000 7f4101a92740 1 -- 172.21.15.107:0/686107192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d95590 msgr2=0x563028db7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:57.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.234+0000 7f4101a92740 1 --2- 172.21.15.107:0/686107192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d95590 0x563028db7560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x563028d92580 tx=0x7f40e400f7f0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.234+0000 7f4101a92740 1 -- 172.21.15.107:0/686107192 shutdown_connections 2024-09-19T04:21:57.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.234+0000 7f4101a92740 1 --2- 172.21.15.107:0/686107192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d9fe50 0x563028db7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.234+0000 7f4101a92740 1 --2- 172.21.15.107:0/686107192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d95590 0x563028db7560 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.234+0000 7f4101a92740 1 --2- 172.21.15.107:0/686107192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d94c70 0x563028d976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.234+0000 7f4101a92740 1 -- 172.21.15.107:0/686107192 >> 172.21.15.107:0/686107192 conn(0x563028d91200 msgr2=0x563028d9ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.234+0000 7f4101a92740 1 -- 172.21.15.107:0/686107192 shutdown_connections 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.234+0000 7f4101a92740 1 -- 172.21.15.107:0/686107192 wait complete. 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.234+0000 7f4101a92740 1 -- start start 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.235+0000 7f4101a92740 1 Processor -- start 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.235+0000 7f4101a92740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028da8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.235+0000 7f4101a92740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 0x563028d995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.235+0000 7f4101a92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 0x563028d99b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.235+0000 7f4101a92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b0e00 con 0x563028d9fe50 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.235+0000 7f4101a92740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b0730 con 0x563028d94c70 2024-09-19T04:21:57.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.235+0000 7f4101a92740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81ad970 con 0x563028d95590 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.235+0000 7f4101a92740 10 client.0 _ll_register_callbacks cb 0x7f40e80e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028da8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028da8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51730/0 (socket says 172.21.15.107:51730) 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40f7e00640 1 -- 172.21.15.107:0/200710082 learned_addr learned my addr 172.21.15.107:0/200710082 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40f7400640 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 0x563028d995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40f7e00640 1 -- 172.21.15.107:0/200710082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 msgr2=0x563028d995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40f7e00640 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 0x563028d995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40f7e00640 1 -- 172.21.15.107:0/200710082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 msgr2=0x563028d99b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40fd000640 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 0x563028d99b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40f7e00640 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 0x563028d99b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40f7e00640 1 -- 172.21.15.107:0/200710082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563028d8c770 con 0x563028d94c70 2024-09-19T04:21:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40fd000640 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 0x563028d99b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40f7e00640 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028da8220 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f40e809c520 tx=0x7f40e800c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3647b25560fc0831 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.236+0000 7f40e2000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:21:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f40e2000640 1 -- 172.21.15.107:0/200710082 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e809c010 con 0x563028d94c70 2024-09-19T04:21:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f40e2000640 1 -- 172.21.15.107:0/200710082 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f40e80d3ac0 con 0x563028d94c70 2024-09-19T04:21:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f4101a92740 1 -- 172.21.15.107:0/200710082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e401caf0 con 0x563028d94c70 2024-09-19T04:21:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f40e2000640 1 -- 172.21.15.107:0/200710082 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f40e809c2a0 con 0x563028d94c70 2024-09-19T04:21:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f4101a92740 10 client.25514 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f4101a92740 1 -- 172.21.15.107:0/200710082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563028e19ff0 con 0x563028d94c70 2024-09-19T04:21:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f40e2000640 1 -- 172.21.15.107:0/200710082 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e401caf0 con 0x563028d94c70 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f40e2000640 1 -- 172.21.15.107:0/200710082 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f40e80a4cb0 con 0x563028d94c70 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f4101a92740 10 client.25514 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f4101a92740 10 client.25514 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f4101a92740 20 client.25514 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f4101a92740 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x563028e1a520 0x563028e1c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f4101a92740 4 client.25514 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.237+0000 7f4101a92740 1 -- 172.21.15.107:0/200710082 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563028d02e60 con 0x563028e1a520 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f40f7400640 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x563028e1a520 0x563028e1c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f40f7400640 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x563028e1a520 0x563028e1c910 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c87e6b38a56c45fa server_cookie=48af35ba2b2506eb in_seq=0 out_seq=0 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f40e2000640 10 client.25514 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:21:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f40e2000640 1 -- 172.21.15.107:0/200710082 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563028d02e60 con 0x563028e1a520 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f40e2000640 10 client.25514 handle_command_reply: tid=0 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f4101a92740 1 client.25514 shutdown 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f4101a92740 1 -- 172.21.15.107:0/200710082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 msgr2=0x563028da8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f4101a92740 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028da8220 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f40e809c520 tx=0x7f40e800c840 comp rx=0 tx=0).stop 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f4101a92740 1 -- 172.21.15.107:0/200710082 shutdown_connections 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f4101a92740 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x563028e1a520 0x563028e1c910 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f4101a92740 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563028d9fe50 0x563028d99b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f4101a92740 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563028d95590 0x563028d995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f40e2000640 0 client.25514 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f4101a92740 1 --2- 172.21.15.107:0/200710082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563028d94c70 0x563028da8220 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.238+0000 7f4101a92740 1 -- 172.21.15.107:0/200710082 >> 172.21.15.107:0/200710082 conn(0x563028d91200 msgr2=0x563028d9cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.239+0000 7f4101a92740 1 -- 172.21.15.107:0/200710082 shutdown_connections 2024-09-19T04:21:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.239+0000 7f4101a92740 1 -- 172.21.15.107:0/200710082 wait complete. 2024-09-19T04:21:57.245 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:21:57.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.239+0000 7f4101a92740 20 client.25514 trim_cache size 0 max 16384 2024-09-19T04:21:57.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 -- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40dc0a3430 msgr2=0x7f40dc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:57.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40dc0a3430 0x7f40dc0a58f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005cc0 tx=0x7f40f0005f80 comp rx=0 tx=0).stop 2024-09-19T04:21:57.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 -- 172.21.15.107:0/4096942984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f8101de0 msgr2=0x7f40f819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:57.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f8101de0 0x7f40f819a610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e9f0 tx=0x7f40e800edf0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 -- 172.21.15.107:0/4096942984 shutdown_connections 2024-09-19T04:21:57.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40dc0a3430 0x7f40dc0a58f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8106f10 0x7f40f819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40f8102790 0x7f40f819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 --2- 172.21.15.107:0/4096942984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40f8101de0 0x7f40f819a610 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:57.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 -- 172.21.15.107:0/4096942984 >> 172.21.15.107:0/4096942984 conn(0x7f40f80fd8c0 msgr2=0x7f40f80ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:57.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 -- 172.21.15.107:0/4096942984 shutdown_connections 2024-09-19T04:21:57.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:57.240+0000 7f40ff000640 1 -- 172.21.15.107:0/4096942984 wait complete. 2024-09-19T04:21:57.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 42862 16.68 MB/sec warmup 15 sec latency 10.562 ms 2024-09-19T04:21:58.047 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:21:58.448 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:21:58.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 45388 16.34 MB/sec warmup 16 sec latency 15.595 ms 2024-09-19T04:21:58.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:21:58 smithi115 ceph-mon[35477]: pgmap v52: 97 pgs: 97 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-09-19T04:21:58.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:21:58 smithi107 ceph-mon[30659]: pgmap v52: 97 pgs: 97 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-09-19T04:21:58.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:21:58 smithi145 ceph-mon[35096]: pgmap v52: 97 pgs: 97 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-09-19T04:21:58.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.992+0000 7f3303200640 1 -- 172.21.15.107:0/131566425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3304103e30 msgr2=0x7f3304104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.992+0000 7f3303200640 1 --2- 172.21.15.107:0/131566425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3304103e30 0x7f3304104210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009900 tx=0x7f32f402a410 comp rx=0 tx=0).stop 2024-09-19T04:21:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.992+0000 7f3303200640 1 -- 172.21.15.107:0/131566425 shutdown_connections 2024-09-19T04:21:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.992+0000 7f3303200640 1 --2- 172.21.15.107:0/131566425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3304108f60 0x7f3304110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.992+0000 7f3303200640 1 --2- 172.21.15.107:0/131566425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33041047e0 0x7f3304108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.992+0000 7f3303200640 1 --2- 172.21.15.107:0/131566425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3304103e30 0x7f3304104210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.992+0000 7f3303200640 1 -- 172.21.15.107:0/131566425 >> 172.21.15.107:0/131566425 conn(0x7f33040ff910 msgr2=0x7f3304101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.992+0000 7f3303200640 1 -- 172.21.15.107:0/131566425 shutdown_connections 2024-09-19T04:21:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.992+0000 7f3303200640 1 -- 172.21.15.107:0/131566425 wait complete. 2024-09-19T04:21:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.993+0000 7f3303200640 1 Processor -- start 2024-09-19T04:21:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.993+0000 7f3303200640 1 -- start start 2024-09-19T04:21:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3303200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3304103e30 0x7f330419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3303200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33041047e0 0x7f330419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3303200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3304108f60 0x7f330419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3303200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3304114010 con 0x7f33041047e0 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3303200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3304113e90 con 0x7f3304108f60 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3303200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3304114190 con 0x7f3304103e30 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3301e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3304103e30 0x7f330419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3301e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3304103e30 0x7f330419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42890/0 (socket says 172.21.15.107:42890) 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3301e00640 1 -- 172.21.15.107:0/422127324 learned_addr learned my addr 172.21.15.107:0/422127324 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3302800640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3304108f60 0x7f330419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f32fbe00640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33041047e0 0x7f330419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3301e00640 1 -- 172.21.15.107:0/422127324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3304108f60 msgr2=0x7f330419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3301e00640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3304108f60 0x7f330419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3301e00640 1 -- 172.21.15.107:0/422127324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33041047e0 msgr2=0x7f330419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3301e00640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33041047e0 0x7f330419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.994+0000 7f3301e00640 1 -- 172.21.15.107:0/422127324 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330419f430 con 0x7f3304103e30 2024-09-19T04:21:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.995+0000 7f3301e00640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3304103e30 0x7f330419a480 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f32f4004770 tx=0x7f32f4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.995+0000 7f32faa00640 1 -- 172.21.15.107:0/422127324 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f403f070 con 0x7f3304103e30 2024-09-19T04:21:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.995+0000 7f32faa00640 1 -- 172.21.15.107:0/422127324 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f4031ca0 con 0x7f3304103e30 2024-09-19T04:21:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.995+0000 7f3303200640 1 -- 172.21.15.107:0/422127324 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330419f6c0 con 0x7f3304103e30 2024-09-19T04:21:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.995+0000 7f3303200640 1 -- 172.21.15.107:0/422127324 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33041a7000 con 0x7f3304103e30 2024-09-19T04:21:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.995+0000 7f32faa00640 1 -- 172.21.15.107:0/422127324 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4035030 con 0x7f3304103e30 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.999+0000 7f32faa00640 1 -- 172.21.15.107:0/422127324 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f32f4042070 con 0x7f3304103e30 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:58.999+0000 7f330b6c8740 1 Processor -- start 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f330b6c8740 1 -- start start 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee83220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee8e020 0x5631dee90410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f330b6c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5631dede93e0 con 0x5631dee8e020 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f330b6c8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5631ded90d80 con 0x5631dee83760 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f330b6c8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5631dede53a0 con 0x5631dee82e40 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f3301e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f3301e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51402/0 (socket says 172.21.15.107:51402) 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f3301e00640 1 -- 172.21.15.107:0/2438944551 learned_addr learned my addr 172.21.15.107:0/2438944551 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f32fbe00640 1 --2- 172.21.15.107:0/2438944551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee8e020 0x5631dee90410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.000+0000 7f3302800640 1 --2- 172.21.15.107:0/2438944551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee83220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.001+0000 7f3301e00640 1 -- 172.21.15.107:0/2438944551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 msgr2=0x5631dee83220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.001+0000 7f3301e00640 1 --2- 172.21.15.107:0/2438944551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee83220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.001+0000 7f3301e00640 1 -- 172.21.15.107:0/2438944551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee8e020 msgr2=0x5631dee90410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.001+0000 7f3301e00640 1 --2- 172.21.15.107:0/2438944551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee8e020 0x5631dee90410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.001+0000 7f3301e00640 1 -- 172.21.15.107:0/2438944551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631dedafae0 con 0x5631dee83760 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.001+0000 7f3302800640 1 --2- 172.21.15.107:0/2438944551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee83220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.001+0000 7f32faa00640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32e80a3430 0x7f32e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.001+0000 7f3301e00640 1 --2- 172.21.15.107:0/2438944551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8dae0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f32f40ed4e0 tx=0x7f32f40f80e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce90b4b542503fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:59.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.001+0000 7f3302800640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32e80a3430 0x7f32e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.002+0000 7f32faa00640 1 -- 172.21.15.107:0/422127324 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f32f40efe60 con 0x7f3304103e30 2024-09-19T04:21:59.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.002+0000 7f32f3000640 1 -- 172.21.15.107:0/2438944551 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40b83d0 con 0x5631dee83760 2024-09-19T04:21:59.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.002+0000 7f32f3000640 1 -- 172.21.15.107:0/2438944551 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f40fd070 con 0x5631dee83760 2024-09-19T04:21:59.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.002+0000 7f32f3000640 1 -- 172.21.15.107:0/2438944551 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4005d50 con 0x5631dee83760 2024-09-19T04:21:59.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.003+0000 7f330b6c8740 1 -- 172.21.15.107:0/2438944551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 msgr2=0x5631dee8dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:59.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.003+0000 7f3302800640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32e80a3430 0x7f32e80a58f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32fc005fd0 tx=0x7f32fc009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:59.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.003+0000 7f330b6c8740 1 --2- 172.21.15.107:0/2438944551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8dae0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f32f40ed4e0 tx=0x7f32f40f80e0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.003+0000 7f330b6c8740 1 -- 172.21.15.107:0/2438944551 shutdown_connections 2024-09-19T04:21:59.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.003+0000 7f330b6c8740 1 --2- 172.21.15.107:0/2438944551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee8e020 0x5631dee90410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.003+0000 7f330b6c8740 1 --2- 172.21.15.107:0/2438944551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8dae0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.003+0000 7f330b6c8740 1 --2- 172.21.15.107:0/2438944551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee83220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.003+0000 7f330b6c8740 1 -- 172.21.15.107:0/2438944551 >> 172.21.15.107:0/2438944551 conn(0x5631dee7f3d0 msgr2=0x5631dee7f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:59.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.003+0000 7f330b6c8740 1 -- 172.21.15.107:0/2438944551 shutdown_connections 2024-09-19T04:21:59.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.003+0000 7f330b6c8740 1 -- 172.21.15.107:0/2438944551 wait complete. 2024-09-19T04:21:59.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.004+0000 7f330b6c8740 1 -- start start 2024-09-19T04:21:59.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.004+0000 7f330b6c8740 1 Processor -- start 2024-09-19T04:21:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee82e40 0x5631dee8aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee8e020 0x5631dee8b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f330b6c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5631dee909b0 con 0x5631dee82e40 2024-09-19T04:21:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f330b6c8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5631deda40f0 con 0x5631dee83760 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f330b6c8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5631dee90b30 con 0x5631dee8e020 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f330b6c8740 10 client.0 _ll_register_callbacks cb 0x5631ded8bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f32fbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f3302800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee8e020 0x5631dee8b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f32fbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51404/0 (socket says 172.21.15.107:51404) 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f3301e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee82e40 0x5631dee8aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f32fbe00640 1 -- 172.21.15.107:0/1494006831 learned_addr learned my addr 172.21.15.107:0/1494006831 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f32fbe00640 1 -- 172.21.15.107:0/1494006831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee8e020 msgr2=0x5631dee8b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f32fbe00640 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee8e020 0x5631dee8b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f32fbe00640 1 -- 172.21.15.107:0/1494006831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee82e40 msgr2=0x5631dee8aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f32fbe00640 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee82e40 0x5631dee8aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f32fbe00640 1 -- 172.21.15.107:0/1494006831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631deb5a3c0 con 0x5631dee83760 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f3302800640 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee8e020 0x5631dee8b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.005+0000 7f3301e00640 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee82e40 0x5631dee8aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.006+0000 7f32fbe00640 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8b400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f32ec012730 tx=0x7f32ec012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d52b7e3ae2c2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.006+0000 7f32e7400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.006+0000 7f32e7400640 1 -- 172.21.15.107:0/1494006831 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec004260 con 0x5631dee83760 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.006+0000 7f32e7400640 1 -- 172.21.15.107:0/1494006831 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f32ec004400 con 0x5631dee83760 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.006+0000 7f330b6c8740 1 -- 172.21.15.107:0/1494006831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f40fd070 con 0x5631dee83760 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.006+0000 7f32e7400640 1 -- 172.21.15.107:0/1494006831 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32ec0119a0 con 0x5631dee83760 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.006+0000 7f330b6c8740 10 client.25526 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.006+0000 7f330b6c8740 1 -- 172.21.15.107:0/1494006831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631dee9ca30 con 0x5631dee83760 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.006+0000 7f32e7400640 1 -- 172.21.15.107:0/1494006831 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f40fd070 con 0x5631dee83760 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.007+0000 7f32e7400640 1 -- 172.21.15.107:0/1494006831 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32ec026480 con 0x5631dee83760 2024-09-19T04:21:59.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.007+0000 7f330b6c8740 10 client.25526 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:59.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.007+0000 7f330b6c8740 10 client.25526 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:21:59.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.007+0000 7f330b6c8740 20 client.25526 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:59.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.007+0000 7f330b6c8740 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5631dee9de90 0x5631deea0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.007+0000 7f330b6c8740 4 client.25526 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:21:59.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.007+0000 7f330b6c8740 1 -- 172.21.15.107:0/1494006831 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631dedf6950 con 0x5631dee9de90 2024-09-19T04:21:59.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.007+0000 7f3301e00640 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5631dee9de90 0x5631deea0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.008+0000 7f3301e00640 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5631dee9de90 0x5631deea0300 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c8a4cb20539a061 server_cookie=e7b6e1f490812dfe in_seq=0 out_seq=0 2024-09-19T04:21:59.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.008+0000 7f32e7400640 10 client.25526 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:21:59.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.008+0000 7f32e7400640 1 -- 172.21.15.107:0/1494006831 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631dedf6950 con 0x5631dee9de90 2024-09-19T04:21:59.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.008+0000 7f32e7400640 10 client.25526 handle_command_reply: tid=0 2024-09-19T04:21:59.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.008+0000 7f330b6c8740 1 client.25526 shutdown 2024-09-19T04:21:59.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 1 -- 172.21.15.107:0/1494006831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 msgr2=0x5631dee8b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8b400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f32ec012730 tx=0x7f32ec012bf0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 1 -- 172.21.15.107:0/1494006831 shutdown_connections 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5631dee9de90 0x5631deea0300 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee8e020 0x5631dee8b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee8b400 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 1 --2- 172.21.15.107:0/1494006831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee82e40 0x5631dee8aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 1 -- 172.21.15.107:0/1494006831 >> 172.21.15.107:0/1494006831 conn(0x5631dee7f3d0 msgr2=0x5631dee850b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f32e7400640 0 client.25526 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 1 -- 172.21.15.107:0/1494006831 shutdown_connections 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 1 -- 172.21.15.107:0/1494006831 wait complete. 2024-09-19T04:21:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.009+0000 7f330b6c8740 20 client.25526 trim_cache size 0 max 16384 2024-09-19T04:21:59.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.024+0000 7f330b6c8740 1 Processor -- start 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.024+0000 7f330b6c8740 1 -- start start 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee7a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee8e020 0x5631dee8ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f330b6c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000e650 con 0x5631dee8e020 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f330b6c8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f32e0010190 con 0x5631dee83760 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f330b6c8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000ed20 con 0x5631dee82e40 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f3301e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f32fbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee7a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f3301e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee858a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42916/0 (socket says 172.21.15.107:42916) 2024-09-19T04:21:59.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f3301e00640 1 -- 172.21.15.107:0/2836527501 learned_addr learned my addr 172.21.15.107:0/2836527501 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:59.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f3302800640 1 --2- 172.21.15.107:0/2836527501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee8e020 0x5631dee8ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f3301e00640 1 -- 172.21.15.107:0/2836527501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 msgr2=0x5631dee7a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:59.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f3301e00640 1 --2- 172.21.15.107:0/2836527501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee7a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f3301e00640 1 -- 172.21.15.107:0/2836527501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee8e020 msgr2=0x5631dee8ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:59.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f3301e00640 1 --2- 172.21.15.107:0/2836527501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee8e020 0x5631dee8ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.025+0000 7f3301e00640 1 -- 172.21.15.107:0/2836527501 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631dee89e70 con 0x5631dee82e40 2024-09-19T04:21:59.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.026+0000 7f32fbe00640 1 --2- 172.21.15.107:0/2836527501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee7a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:21:59.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.026+0000 7f3301e00640 1 --2- 172.21.15.107:0/2836527501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee858a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f32f4005a80 tx=0x7f32f40f80e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2e5039ec734c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:59.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.026+0000 7f32f2600640 1 -- 172.21.15.107:0/2836527501 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4049040 con 0x5631dee82e40 2024-09-19T04:21:59.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.026+0000 7f32f2600640 1 -- 172.21.15.107:0/2836527501 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f40bc830 con 0x5631dee82e40 2024-09-19T04:21:59.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.026+0000 7f32f2600640 1 -- 172.21.15.107:0/2836527501 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f403b460 con 0x5631dee82e40 2024-09-19T04:21:59.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.027+0000 7f330b6c8740 1 -- 172.21.15.107:0/2836527501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 msgr2=0x5631dee858a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:59.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.027+0000 7f330b6c8740 1 --2- 172.21.15.107:0/2836527501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee858a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f32f4005a80 tx=0x7f32f40f80e0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.027+0000 7f330b6c8740 1 -- 172.21.15.107:0/2836527501 shutdown_connections 2024-09-19T04:21:59.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.027+0000 7f330b6c8740 1 --2- 172.21.15.107:0/2836527501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee8e020 0x5631dee8ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.027+0000 7f330b6c8740 1 --2- 172.21.15.107:0/2836527501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee83760 0x5631dee7a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.027+0000 7f330b6c8740 1 --2- 172.21.15.107:0/2836527501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee858a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.027+0000 7f330b6c8740 1 -- 172.21.15.107:0/2836527501 >> 172.21.15.107:0/2836527501 conn(0x5631dee7f3d0 msgr2=0x5631dee8c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:59.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.027+0000 7f330b6c8740 1 -- 172.21.15.107:0/2836527501 shutdown_connections 2024-09-19T04:21:59.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.027+0000 7f330b6c8740 1 -- 172.21.15.107:0/2836527501 wait complete. 2024-09-19T04:21:59.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.028+0000 7f330b6c8740 1 -- start start 2024-09-19T04:21:59.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.028+0000 7f330b6c8740 1 Processor -- start 2024-09-19T04:21:59.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.028+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.028+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee83760 0x5631dee91b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.028+0000 7f330b6c8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee8e020 0x5631dee95b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.028+0000 7f330b6c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000fac0 con 0x5631dee83760 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.028+0000 7f330b6c8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000f3f0 con 0x5631dee8e020 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.028+0000 7f330b6c8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000c630 con 0x5631dee82e40 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.028+0000 7f330b6c8740 10 client.0 _ll_register_callbacks cb 0x7f32ec0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.029+0000 7f3301e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.029+0000 7f3302800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee8e020 0x5631dee95b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.029+0000 7f3301e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee915c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42930/0 (socket says 172.21.15.107:42930) 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.029+0000 7f3301e00640 1 -- 172.21.15.107:0/40673734 learned_addr learned my addr 172.21.15.107:0/40673734 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.029+0000 7f3301e00640 1 -- 172.21.15.107:0/40673734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee8e020 msgr2=0x5631dee95b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.029+0000 7f3301e00640 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee8e020 0x5631dee95b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.029+0000 7f3301e00640 1 -- 172.21.15.107:0/40673734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee83760 msgr2=0x5631dee91b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:21:59.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.029+0000 7f3301e00640 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee83760 0x5631dee91b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.029+0000 7f3301e00640 1 -- 172.21.15.107:0/40673734 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631dee83590 con 0x5631dee82e40 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f3301e00640 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee915c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f32f40f33b0 tx=0x7f32f4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2b6f9c92af1f2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f32e7400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f32e7400640 1 -- 172.21.15.107:0/40673734 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40d4070 con 0x5631dee82e40 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f32e7400640 1 -- 172.21.15.107:0/40673734 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f32f40f36e0 con 0x5631dee82e40 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f330b6c8740 1 -- 172.21.15.107:0/40673734 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f40bc830 con 0x5631dee82e40 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f32e7400640 1 -- 172.21.15.107:0/40673734 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32f4049030 con 0x5631dee82e40 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f330b6c8740 10 client.25510 fetch_fsmap learned FSMap version 28 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f330b6c8740 1 -- 172.21.15.107:0/40673734 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631def084b0 con 0x5631dee82e40 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f32e7400640 1 -- 172.21.15.107:0/40673734 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f40bc830 con 0x5631dee82e40 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f32e7400640 1 -- 172.21.15.107:0/40673734 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32f404c060 con 0x5631dee82e40 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f330b6c8740 10 client.25510 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:21:59.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f330b6c8740 10 client.25510 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:21:59.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f330b6c8740 20 client.25510 populate_metadata read hostname 'smithi107' 2024-09-19T04:21:59.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f330b6c8740 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5631def08780 0x5631def0ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:59.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f330b6c8740 4 client.25510 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:21:59.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.030+0000 7f330b6c8740 1 -- 172.21.15.107:0/40673734 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631dedf0e60 con 0x5631def08780 2024-09-19T04:21:59.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.031+0000 7f32fbe00640 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5631def08780 0x5631def0ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:59.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.031+0000 7f32fbe00640 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5631def08780 0x5631def0ab70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2eba26ce43ab54c6 server_cookie=cab59f3088f935f0 in_seq=0 out_seq=0 2024-09-19T04:21:59.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.031+0000 7f32e7400640 10 client.25510 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:21:59.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f32e7400640 1 -- 172.21.15.107:0/40673734 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631dedf0e60 con 0x5631def08780 2024-09-19T04:21:59.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f32e7400640 10 client.25510 handle_command_reply: tid=0 2024-09-19T04:21:59.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 client.25510 shutdown 2024-09-19T04:21:59.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 -- 172.21.15.107:0/40673734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 msgr2=0x5631dee915c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:59.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee915c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f32f40f33b0 tx=0x7f32f4009260 comp rx=0 tx=0).stop 2024-09-19T04:21:59.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 -- 172.21.15.107:0/40673734 shutdown_connections 2024-09-19T04:21:59.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5631def08780 0x5631def0ab70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631dee8e020 0x5631dee95b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f32e7400640 0 client.25510 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:21:59.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dee83760 0x5631dee91b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 --2- 172.21.15.107:0/40673734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5631dee82e40 0x5631dee915c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 -- 172.21.15.107:0/40673734 >> 172.21.15.107:0/40673734 conn(0x5631dee7f3d0 msgr2=0x5631dee7e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:59.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 -- 172.21.15.107:0/40673734 shutdown_connections 2024-09-19T04:21:59.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 1 -- 172.21.15.107:0/40673734 wait complete. 2024-09-19T04:21:59.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.032+0000 7f330b6c8740 20 client.25510 trim_cache size 0 max 16384 2024-09-19T04:21:59.038 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:21:59.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 -- 172.21.15.107:0/422127324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32e80a3430 msgr2=0x7f32e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:59.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32e80a3430 0x7f32e80a58f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32fc005fd0 tx=0x7f32fc009500 comp rx=0 tx=0).stop 2024-09-19T04:21:59.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 -- 172.21.15.107:0/422127324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3304103e30 msgr2=0x7f330419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:59.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3304103e30 0x7f330419a480 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f32f4004770 tx=0x7f32f4003c10 comp rx=0 tx=0).stop 2024-09-19T04:21:59.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 -- 172.21.15.107:0/422127324 shutdown_connections 2024-09-19T04:21:59.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32e80a3430 0x7f32e80a58f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3304108f60 0x7f330419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33041047e0 0x7f330419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 --2- 172.21.15.107:0/422127324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3304103e30 0x7f330419a480 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:59.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 -- 172.21.15.107:0/422127324 >> 172.21.15.107:0/422127324 conn(0x7f33040ff910 msgr2=0x7f3304100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:59.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 -- 172.21.15.107:0/422127324 shutdown_connections 2024-09-19T04:21:59.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:21:59.033+0000 7f3303200640 1 -- 172.21.15.107:0/422127324 wait complete. 2024-09-19T04:21:59.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 47778 16.16 MB/sec warmup 17 sec latency 11.860 ms 2024-09-19T04:21:59.761 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:22:00.159 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:00.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 50406 16.05 MB/sec warmup 18 sec latency 10.837 ms 2024-09-19T04:22:00.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:00 smithi107 ceph-mon[30659]: pgmap v53: 97 pgs: 97 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-09-19T04:22:00.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:00 smithi115 ceph-mon[35477]: pgmap v53: 97 pgs: 97 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-09-19T04:22:00.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:00 smithi145 ceph-mon[35096]: pgmap v53: 97 pgs: 97 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-09-19T04:22:00.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.763+0000 7fb929600640 1 -- 172.21.15.107:0/2443108589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9241079c0 msgr2=0x7fb924107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:00.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.763+0000 7fb929600640 1 --2- 172.21.15.107:0/2443108589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9241079c0 0x7fb924107da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb914009930 tx=0x7fb91402a420 comp rx=0 tx=0).stop 2024-09-19T04:22:00.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.764+0000 7fb929600640 1 -- 172.21.15.107:0/2443108589 shutdown_connections 2024-09-19T04:22:00.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.764+0000 7fb929600640 1 --2- 172.21.15.107:0/2443108589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb924102360 0x7fb92410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.764+0000 7fb929600640 1 --2- 172.21.15.107:0/2443108589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9241019c0 0x7fb924101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.764+0000 7fb929600640 1 --2- 172.21.15.107:0/2443108589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9241079c0 0x7fb924107da0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.764+0000 7fb929600640 1 -- 172.21.15.107:0/2443108589 >> 172.21.15.107:0/2443108589 conn(0x7fb9240fd740 msgr2=0x7fb9240ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:00.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.764+0000 7fb929600640 1 -- 172.21.15.107:0/2443108589 shutdown_connections 2024-09-19T04:22:00.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.764+0000 7fb929600640 1 -- 172.21.15.107:0/2443108589 wait complete. 2024-09-19T04:22:00.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.765+0000 7fb929600640 1 Processor -- start 2024-09-19T04:22:00.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.765+0000 7fb929600640 1 -- start start 2024-09-19T04:22:00.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb929600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9241019c0 0x7fb92419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb929600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb924102360 0x7fb92419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb929600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9241079c0 0x7fb92419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb929600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb924111dd0 con 0x7fb9241079c0 2024-09-19T04:22:00.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb929600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb924111c50 con 0x7fb924102360 2024-09-19T04:22:00.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb929600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb924111f50 con 0x7fb9241019c0 2024-09-19T04:22:00.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb923e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9241019c0 0x7fb92419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb923e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9241019c0 0x7fb92419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42954/0 (socket says 172.21.15.107:42954) 2024-09-19T04:22:00.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb923e00640 1 -- 172.21.15.107:0/509253141 learned_addr learned my addr 172.21.15.107:0/509253141 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:00.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb928c00640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9241079c0 0x7fb92419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb91be00640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb924102360 0x7fb92419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb923e00640 1 -- 172.21.15.107:0/509253141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb924102360 msgr2=0x7fb92419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb923e00640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb924102360 0x7fb92419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb923e00640 1 -- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9241079c0 msgr2=0x7fb92419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb923e00640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9241079c0 0x7fb92419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb923e00640 1 -- 172.21.15.107:0/509253141 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92419f490 con 0x7fb9241019c0 2024-09-19T04:22:00.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.766+0000 7fb928c00640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9241079c0 0x7fb92419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.767+0000 7fb923e00640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9241019c0 0x7fb92419a4e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb91402a940 tx=0x7fb914003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.767+0000 7fb921600640 1 -- 172.21.15.107:0/509253141 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91403f070 con 0x7fb9241019c0 2024-09-19T04:22:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.767+0000 7fb921600640 1 -- 172.21.15.107:0/509253141 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb914031c90 con 0x7fb9241019c0 2024-09-19T04:22:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.767+0000 7fb921600640 1 -- 172.21.15.107:0/509253141 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914035030 con 0x7fb9241019c0 2024-09-19T04:22:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.767+0000 7fb929600640 1 -- 172.21.15.107:0/509253141 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92419f720 con 0x7fb9241019c0 2024-09-19T04:22:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.767+0000 7fb929600640 1 -- 172.21.15.107:0/509253141 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241a7060 con 0x7fb9241019c0 2024-09-19T04:22:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.770+0000 7fb921600640 1 -- 172.21.15.107:0/509253141 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fb914042070 con 0x7fb9241019c0 2024-09-19T04:22:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.770+0000 7fb92d87a740 1 Processor -- start 2024-09-19T04:22:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.771+0000 7fb92d87a740 1 -- start start 2024-09-19T04:22:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.771+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b7c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.771+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b86ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.771+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b87020 0x5604a3b89410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.771+0000 7fb92d87a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5604a3ae23e0 con 0x5604a3b7c760 2024-09-19T04:22:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.771+0000 7fb92d87a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5604a3a89d80 con 0x5604a3b87020 2024-09-19T04:22:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.771+0000 7fb92d87a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5604a3ade3a0 con 0x5604a3b7be40 2024-09-19T04:22:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.771+0000 7fb923e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b86ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.771+0000 7fb928c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b7c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb923e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b86ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34362/0 (socket says 172.21.15.107:34362) 2024-09-19T04:22:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb923e00640 1 -- 172.21.15.107:0/3610576914 learned_addr learned my addr 172.21.15.107:0/3610576914 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb928c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b7c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42960/0 (socket says 172.21.15.107:42960) 2024-09-19T04:22:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb928c00640 1 -- 172.21.15.107:0/3610576914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b87020 msgr2=0x5604a3b89410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:22:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb928c00640 1 --2- 172.21.15.107:0/3610576914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b87020 0x5604a3b89410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb928c00640 1 -- 172.21.15.107:0/3610576914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 msgr2=0x5604a3b86ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb928c00640 1 --2- 172.21.15.107:0/3610576914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b86ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb928c00640 1 -- 172.21.15.107:0/3610576914 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604a3aa8ae0 con 0x5604a3b7be40 2024-09-19T04:22:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb921600640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9080a3500 0x7fb9080a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb928c00640 1 --2- 172.21.15.107:0/3610576914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b7c220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009f10 tx=0x7fb91c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=342709408d0566f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.772+0000 7fb921600640 1 -- 172.21.15.107:0/509253141 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fb9140eff90 con 0x7fb9241019c0 2024-09-19T04:22:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.773+0000 7fb923e00640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9080a3500 0x7fb9080a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.773+0000 7fb919200640 1 -- 172.21.15.107:0/3610576914 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c01c680 con 0x5604a3b7be40 2024-09-19T04:22:00.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.773+0000 7fb919200640 1 -- 172.21.15.107:0/3610576914 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91c01cd00 con 0x5604a3b7be40 2024-09-19T04:22:00.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.773+0000 7fb919200640 1 -- 172.21.15.107:0/3610576914 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c01a380 con 0x5604a3b7be40 2024-09-19T04:22:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.773+0000 7fb923e00640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9080a3500 0x7fb9080a59c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9140bbf60 tx=0x7fb9140f3a40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.773+0000 7fb92d87a740 1 -- 172.21.15.107:0/3610576914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 msgr2=0x5604a3b7c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.773+0000 7fb92d87a740 1 --2- 172.21.15.107:0/3610576914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b7c220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009f10 tx=0x7fb91c0107a0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.774+0000 7fb92d87a740 1 -- 172.21.15.107:0/3610576914 shutdown_connections 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.774+0000 7fb92d87a740 1 --2- 172.21.15.107:0/3610576914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b87020 0x5604a3b89410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.774+0000 7fb92d87a740 1 --2- 172.21.15.107:0/3610576914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b86ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.774+0000 7fb92d87a740 1 --2- 172.21.15.107:0/3610576914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b7c220 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.774+0000 7fb92d87a740 1 -- 172.21.15.107:0/3610576914 >> 172.21.15.107:0/3610576914 conn(0x5604a3b783d0 msgr2=0x5604a3b787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.774+0000 7fb92d87a740 1 -- 172.21.15.107:0/3610576914 shutdown_connections 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.774+0000 7fb92d87a740 1 -- 172.21.15.107:0/3610576914 wait complete. 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.774+0000 7fb92d87a740 1 -- start start 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.775+0000 7fb92d87a740 1 Processor -- start 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.775+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7be40 0x5604a3b83f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.775+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.775+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b87020 0x5604a3b849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.775+0000 7fb92d87a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5604a3b89bf0 con 0x5604a3b7c760 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.775+0000 7fb92d87a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5604a3a9d0f0 con 0x5604a3b7be40 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.775+0000 7fb92d87a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5604a3b89d70 con 0x5604a3b87020 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.775+0000 7fb92d87a740 10 client.0 _ll_register_callbacks cb 0x5604a3a84bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb928c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7be40 0x5604a3b83f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb91be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb928c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7be40 0x5604a3b83f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51458/0 (socket says 172.21.15.107:51458) 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb928c00640 1 -- 172.21.15.107:0/169305342 learned_addr learned my addr 172.21.15.107:0/169305342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb928c00640 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b87020 0x5604a3b849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb91be00640 1 -- 172.21.15.107:0/169305342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b87020 msgr2=0x5604a3b849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb91be00640 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b87020 0x5604a3b849e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb91be00640 1 -- 172.21.15.107:0/169305342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7be40 msgr2=0x5604a3b83f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb91be00640 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7be40 0x5604a3b83f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb91be00640 1 -- 172.21.15.107:0/169305342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604a38533c0 con 0x5604a3b7c760 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb91be00640 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b844a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0116c0 tx=0x7fb90c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b6d1bb822e44be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.776+0000 7fb911600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.777+0000 7fb911600640 1 -- 172.21.15.107:0/169305342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c01e060 con 0x5604a3b7c760 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.777+0000 7fb911600640 1 -- 172.21.15.107:0/169305342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fb90c0035f0 con 0x5604a3b7c760 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.777+0000 7fb92d87a740 1 -- 172.21.15.107:0/169305342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb91c01cd00 con 0x5604a3b7c760 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.777+0000 7fb911600640 1 -- 172.21.15.107:0/169305342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb90c0037f0 con 0x5604a3b7c760 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.777+0000 7fb92d87a740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.777+0000 7fb92d87a740 1 -- 172.21.15.107:0/169305342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604a3b95d30 con 0x5604a3b7c760 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.777+0000 7fb911600640 1 -- 172.21.15.107:0/169305342 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91c01cd00 con 0x5604a3b7c760 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.778+0000 7fb911600640 1 -- 172.21.15.107:0/169305342 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb90c00fc10 con 0x5604a3b7c760 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.778+0000 7fb92d87a740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:22:00.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.778+0000 7fb92d87a740 10 client.15486 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.778+0000 7fb92d87a740 20 client.15486 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.778+0000 7fb92d87a740 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5604a3b971f0 0x5604a3b99620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.778+0000 7fb92d87a740 4 client.15486 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.778+0000 7fb92d87a740 1 -- 172.21.15.107:0/169305342 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604a3aef950 con 0x5604a3b971f0 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.778+0000 7fb928c00640 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5604a3b971f0 0x5604a3b99620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.779+0000 7fb928c00640 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5604a3b971f0 0x5604a3b99620 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e844238401ed142 server_cookie=bbde63091af94b25 in_seq=0 out_seq=0 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.779+0000 7fb911600640 10 client.15486 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.779+0000 7fb911600640 1 -- 172.21.15.107:0/169305342 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604a3aef950 con 0x5604a3b971f0 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.779+0000 7fb911600640 10 client.15486 handle_command_reply: tid=0 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.779+0000 7fb92d87a740 1 client.15486 shutdown 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 1 -- 172.21.15.107:0/169305342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 msgr2=0x5604a3b844a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:00.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b844a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0116c0 tx=0x7fb90c011ac0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 1 -- 172.21.15.107:0/169305342 shutdown_connections 2024-09-19T04:22:00.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5604a3b971f0 0x5604a3b99620 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b87020 0x5604a3b849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b844a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 1 --2- 172.21.15.107:0/169305342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7be40 0x5604a3b83f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 1 -- 172.21.15.107:0/169305342 >> 172.21.15.107:0/169305342 conn(0x5604a3b783d0 msgr2=0x5604a3b7e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:00.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb911600640 0 client.15486 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:22:00.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 1 -- 172.21.15.107:0/169305342 shutdown_connections 2024-09-19T04:22:00.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 1 -- 172.21.15.107:0/169305342 wait complete. 2024-09-19T04:22:00.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.780+0000 7fb92d87a740 20 client.15486 trim_cache size 0 max 16384 2024-09-19T04:22:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.796+0000 7fb92d87a740 1 Processor -- start 2024-09-19T04:22:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.796+0000 7fb92d87a740 1 -- start start 2024-09-19T04:22:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.796+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b77380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b9ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b87020 0x5604a3b9f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb92d87a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241af7b0 con 0x5604a3b7c760 2024-09-19T04:22:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb92d87a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241b12f0 con 0x5604a3b87020 2024-09-19T04:22:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb92d87a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241afe80 con 0x5604a3b7be40 2024-09-19T04:22:00.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb928c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b77380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb928c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b77380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42976/0 (socket says 172.21.15.107:42976) 2024-09-19T04:22:00.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb928c00640 1 -- 172.21.15.107:0/4252102770 learned_addr learned my addr 172.21.15.107:0/4252102770 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:00.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb928c00640 1 --2- 172.21.15.107:0/4252102770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b87020 0x5604a3b9f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb91be00640 1 --2- 172.21.15.107:0/4252102770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b9ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb928c00640 1 -- 172.21.15.107:0/4252102770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b87020 msgr2=0x5604a3b9f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:00.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb928c00640 1 --2- 172.21.15.107:0/4252102770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b87020 0x5604a3b9f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb928c00640 1 -- 172.21.15.107:0/4252102770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 msgr2=0x5604a3b9ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:00.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb928c00640 1 --2- 172.21.15.107:0/4252102770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b9ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.797+0000 7fb928c00640 1 -- 172.21.15.107:0/4252102770 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604a3b7bc70 con 0x5604a3b7be40 2024-09-19T04:22:00.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.798+0000 7fb928c00640 1 --2- 172.21.15.107:0/4252102770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b77380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb91c013010 tx=0x7fb91c013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcd2062e8de3ac0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:00.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.798+0000 7fb913e00640 1 -- 172.21.15.107:0/4252102770 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c031070 con 0x5604a3b7be40 2024-09-19T04:22:00.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.798+0000 7fb913e00640 1 -- 172.21.15.107:0/4252102770 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91c032040 con 0x5604a3b7be40 2024-09-19T04:22:00.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.798+0000 7fb913e00640 1 -- 172.21.15.107:0/4252102770 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c0057c0 con 0x5604a3b7be40 2024-09-19T04:22:00.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 -- 172.21.15.107:0/4252102770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 msgr2=0x5604a3b77380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:00.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 --2- 172.21.15.107:0/4252102770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b77380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb91c013010 tx=0x7fb91c013040 comp rx=0 tx=0).stop 2024-09-19T04:22:00.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 -- 172.21.15.107:0/4252102770 shutdown_connections 2024-09-19T04:22:00.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 --2- 172.21.15.107:0/4252102770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b87020 0x5604a3b9f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 --2- 172.21.15.107:0/4252102770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b7c760 0x5604a3b9ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 --2- 172.21.15.107:0/4252102770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b77380 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 -- 172.21.15.107:0/4252102770 >> 172.21.15.107:0/4252102770 conn(0x5604a3b783d0 msgr2=0x5604a3b76c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:00.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 -- 172.21.15.107:0/4252102770 shutdown_connections 2024-09-19T04:22:00.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 -- 172.21.15.107:0/4252102770 wait complete. 2024-09-19T04:22:00.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 -- start start 2024-09-19T04:22:00.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.799+0000 7fb92d87a740 1 Processor -- start 2024-09-19T04:22:00.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b8b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7c760 0x5604a3b8b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb92d87a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b87020 0x5604a3c00f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb92d87a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241b0c20 con 0x5604a3b87020 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb92d87a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241b0550 con 0x5604a3b7c760 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb92d87a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241ad790 con 0x5604a3b7be40 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb92d87a740 10 client.0 _ll_register_callbacks cb 0x7fb90c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb91be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7c760 0x5604a3b8b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb91be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7c760 0x5604a3b8b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51484/0 (socket says 172.21.15.107:51484) 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb91be00640 1 -- 172.21.15.107:0/1046685012 learned_addr learned my addr 172.21.15.107:0/1046685012 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb928c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b8b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb928c00640 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b87020 0x5604a3c00f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb91be00640 1 -- 172.21.15.107:0/1046685012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 msgr2=0x5604a3b8b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.800+0000 7fb91be00640 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b8b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.801+0000 7fb91be00640 1 -- 172.21.15.107:0/1046685012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b87020 msgr2=0x5604a3c00f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.801+0000 7fb91be00640 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b87020 0x5604a3c00f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.801+0000 7fb91be00640 1 -- 172.21.15.107:0/1046685012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604a3b79060 con 0x5604a3b7c760 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.801+0000 7fb928c00640 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b87020 0x5604a3c00f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.801+0000 7fb91be00640 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7c760 0x5604a3b8b8f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x5604a3b837f0 tx=0x7fb90c004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b4db8f3ea1d12a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.801+0000 7fb911600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.801+0000 7fb911600640 1 -- 172.21.15.107:0/1046685012 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00ea10 con 0x5604a3b7c760 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.801+0000 7fb92d87a740 1 -- 172.21.15.107:0/1046685012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb91c032040 con 0x5604a3b7c760 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.801+0000 7fb911600640 1 -- 172.21.15.107:0/1046685012 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fb90c00ebb0 con 0x5604a3b7c760 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.801+0000 7fb911600640 1 -- 172.21.15.107:0/1046685012 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb90c00edb0 con 0x5604a3b7c760 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb92d87a740 10 client.25547 fetch_fsmap learned FSMap version 28 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb92d87a740 1 -- 172.21.15.107:0/1046685012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604a3b79480 con 0x5604a3b7c760 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb911600640 1 -- 172.21.15.107:0/1046685012 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91c032040 con 0x5604a3b7c760 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb911600640 1 -- 172.21.15.107:0/1046685012 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb90c004680 con 0x5604a3b7c760 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb92d87a740 10 client.25547 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:22:00.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb92d87a740 10 client.25547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:22:00.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb92d87a740 20 client.25547 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:00.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb92d87a740 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5604a3c029c0 0x5604a3c04de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:00.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb92d87a740 4 client.25547 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:00.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb92d87a740 1 -- 172.21.15.107:0/1046685012 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604a3ae9e60 con 0x5604a3c029c0 2024-09-19T04:22:00.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.802+0000 7fb928c00640 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5604a3c029c0 0x5604a3c04de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:00.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb928c00640 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5604a3c029c0 0x5604a3c04de0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7f409518d533688 server_cookie=cf69906cf96eebc0 in_seq=0 out_seq=0 2024-09-19T04:22:00.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb911600640 10 client.25547 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:22:00.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb911600640 1 -- 172.21.15.107:0/1046685012 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604a3ae9e60 con 0x5604a3c029c0 2024-09-19T04:22:00.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb911600640 10 client.25547 handle_command_reply: tid=0 2024-09-19T04:22:00.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb92d87a740 1 client.25547 shutdown 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb92d87a740 1 -- 172.21.15.107:0/1046685012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7c760 msgr2=0x5604a3b8b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb92d87a740 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7c760 0x5604a3b8b8f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x5604a3b837f0 tx=0x7fb90c004550 comp rx=0 tx=0).stop 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb92d87a740 1 -- 172.21.15.107:0/1046685012 shutdown_connections 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb92d87a740 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5604a3c029c0 0x5604a3c04de0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb92d87a740 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604a3b87020 0x5604a3c00f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb92d87a740 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604a3b7c760 0x5604a3b8b8f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb92d87a740 1 --2- 172.21.15.107:0/1046685012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604a3b7be40 0x5604a3b8b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb92d87a740 1 -- 172.21.15.107:0/1046685012 >> 172.21.15.107:0/1046685012 conn(0x5604a3b783d0 msgr2=0x5604a3b84740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.803+0000 7fb911600640 0 client.25547 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb92d87a740 1 -- 172.21.15.107:0/1046685012 shutdown_connections 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb92d87a740 1 -- 172.21.15.107:0/1046685012 wait complete. 2024-09-19T04:22:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb92d87a740 20 client.25547 trim_cache size 0 max 16384 2024-09-19T04:22:00.811 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:22:00.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb929600640 1 -- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9080a3500 msgr2=0x7fb9080a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:00.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb929600640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9080a3500 0x7fb9080a59c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9140bbf60 tx=0x7fb9140f3a40 comp rx=0 tx=0).stop 2024-09-19T04:22:00.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb929600640 1 -- 172.21.15.107:0/509253141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9241019c0 msgr2=0x7fb92419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:00.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb929600640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9241019c0 0x7fb92419a4e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb91402a940 tx=0x7fb914003b20 comp rx=0 tx=0).stop 2024-09-19T04:22:00.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb929600640 1 -- 172.21.15.107:0/509253141 shutdown_connections 2024-09-19T04:22:00.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb929600640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9080a3500 0x7fb9080a59c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb929600640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9241079c0 0x7fb92419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb929600640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb924102360 0x7fb92419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb929600640 1 --2- 172.21.15.107:0/509253141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9241019c0 0x7fb92419a4e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:00.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.804+0000 7fb929600640 1 -- 172.21.15.107:0/509253141 >> 172.21.15.107:0/509253141 conn(0x7fb9240fd740 msgr2=0x7fb92410cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:00.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.805+0000 7fb929600640 1 -- 172.21.15.107:0/509253141 shutdown_connections 2024-09-19T04:22:00.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:00.805+0000 7fb929600640 1 -- 172.21.15.107:0/509253141 wait complete. 2024-09-19T04:22:01.572 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:22:01.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 52702 15.86 MB/sec warmup 19 sec latency 20.039 ms 2024-09-19T04:22:01.975 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:02.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.567+0000 7f2c8c000640 1 -- 172.21.15.107:0/3509576825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c841049a0 msgr2=0x7f2c841089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.567+0000 7f2c8c000640 1 --2- 172.21.15.107:0/3509576825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c841049a0 0x7f2c841089f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009900 tx=0x7f2c7002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.567+0000 7f2c8c000640 1 -- 172.21.15.107:0/3509576825 shutdown_connections 2024-09-19T04:22:02.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.567+0000 7f2c8c000640 1 --2- 172.21.15.107:0/3509576825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c84109120 0x7f2c84110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.567+0000 7f2c8c000640 1 --2- 172.21.15.107:0/3509576825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c841049a0 0x7f2c841089f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.567+0000 7f2c8c000640 1 --2- 172.21.15.107:0/3509576825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84103ff0 0x7f2c841043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.567+0000 7f2c8c000640 1 -- 172.21.15.107:0/3509576825 >> 172.21.15.107:0/3509576825 conn(0x7f2c840ffa90 msgr2=0x7f2c84101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.568+0000 7f2c8c000640 1 -- 172.21.15.107:0/3509576825 shutdown_connections 2024-09-19T04:22:02.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.568+0000 7f2c8c000640 1 -- 172.21.15.107:0/3509576825 wait complete. 2024-09-19T04:22:02.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.568+0000 7f2c8c000640 1 Processor -- start 2024-09-19T04:22:02.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c8c000640 1 -- start start 2024-09-19T04:22:02.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c8c000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84103ff0 0x7f2c8419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c8c000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c841049a0 0x7f2c8419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c8c000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84109120 0x7f2c8419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c8c000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84114170 con 0x7f2c841049a0 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c8c000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84113ff0 con 0x7f2c84109120 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c8c000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841142f0 con 0x7f2c84103ff0 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c89600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84103ff0 0x7f2c8419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c8a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84109120 0x7f2c8419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c89600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84103ff0 0x7f2c8419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43000/0 (socket says 172.21.15.107:43000) 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c89600640 1 -- 172.21.15.107:0/3291365701 learned_addr learned my addr 172.21.15.107:0/3291365701 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.569+0000 7f2c88c00640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c841049a0 0x7f2c8419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c89600640 1 -- 172.21.15.107:0/3291365701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84109120 msgr2=0x7f2c8419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c89600640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84109120 0x7f2c8419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c89600640 1 -- 172.21.15.107:0/3291365701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c841049a0 msgr2=0x7f2c8419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c89600640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c841049a0 0x7f2c8419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c89600640 1 -- 172.21.15.107:0/3291365701 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8419f600 con 0x7f2c84103ff0 2024-09-19T04:22:02.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c8a000640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84109120 0x7f2c8419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:02.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c89600640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84103ff0 0x7f2c8419a650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800e9f0 tx=0x7f2c7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c82000640 1 -- 172.21.15.107:0/3291365701 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7800cd80 con 0x7f2c84103ff0 2024-09-19T04:22:02.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c8c000640 1 -- 172.21.15.107:0/3291365701 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8419f8f0 con 0x7f2c84103ff0 2024-09-19T04:22:02.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c82000640 1 -- 172.21.15.107:0/3291365701 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c78010910 con 0x7f2c84103ff0 2024-09-19T04:22:02.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c82000640 1 -- 172.21.15.107:0/3291365701 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78018990 con 0x7f2c84103ff0 2024-09-19T04:22:02.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.570+0000 7f2c8c000640 1 -- 172.21.15.107:0/3291365701 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841a71d0 con 0x7f2c84103ff0 2024-09-19T04:22:02.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.573+0000 7f2c82000640 1 -- 172.21.15.107:0/3291365701 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f2c78010ac0 con 0x7f2c84103ff0 2024-09-19T04:22:02.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.574+0000 7f2c8ec6f740 1 Processor -- start 2024-09-19T04:22:02.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.574+0000 7f2c8ec6f740 1 -- start start 2024-09-19T04:22:02.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edc3c70 0x555c3edc4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3edce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edcee50 0x555c3edd1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555c3ed2a3e0 con 0x555c3edc3c70 2024-09-19T04:22:02.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555c3ecd1d80 con 0x555c3edc4590 2024-09-19T04:22:02.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555c3ed263a0 con 0x555c3edcee50 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c89600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3edce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c88c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edcee50 0x555c3edd1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c89600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3edce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51502/0 (socket says 172.21.15.107:51502) 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c89600640 1 -- 172.21.15.107:0/3921048981 learned_addr learned my addr 172.21.15.107:0/3921048981 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c8a000640 1 --2- 172.21.15.107:0/3921048981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edc3c70 0x555c3edc4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c89600640 1 -- 172.21.15.107:0/3921048981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edcee50 msgr2=0x555c3edd1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c89600640 1 --2- 172.21.15.107:0/3921048981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edcee50 0x555c3edd1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c89600640 1 -- 172.21.15.107:0/3921048981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edc3c70 msgr2=0x555c3edc4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c89600640 1 --2- 172.21.15.107:0/3921048981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edc3c70 0x555c3edc4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.575+0000 7f2c89600640 1 -- 172.21.15.107:0/3921048981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3ecf0ae0 con 0x555c3edc4590 2024-09-19T04:22:02.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.576+0000 7f2c89600640 1 --2- 172.21.15.107:0/3921048981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3edce910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2c780cd6e0 tx=0x7f2c780d86f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddfc45904289c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.576+0000 7f2c82000640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2c6c0a33b0 0x7f2c6c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.576+0000 7f2c76600640 1 -- 172.21.15.107:0/3921048981 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780e3070 con 0x555c3edc4590 2024-09-19T04:22:02.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.576+0000 7f2c76600640 1 -- 172.21.15.107:0/3921048981 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7809c3e0 con 0x555c3edc4590 2024-09-19T04:22:02.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.576+0000 7f2c76600640 1 -- 172.21.15.107:0/3921048981 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7809c1b0 con 0x555c3edc4590 2024-09-19T04:22:02.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.576+0000 7f2c82000640 1 -- 172.21.15.107:0/3291365701 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f2c780d0060 con 0x7f2c84103ff0 2024-09-19T04:22:02.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.576+0000 7f2c8a000640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2c6c0a33b0 0x7f2c6c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.577+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/3921048981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 msgr2=0x555c3edce910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.577+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/3921048981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3edce910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2c780cd6e0 tx=0x7f2c780d86f0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.577+0000 7f2c8a000640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2c6c0a33b0 0x7f2c6c0a5870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009fd0 tx=0x7f2c7c009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.577+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/3921048981 shutdown_connections 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.577+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/3921048981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edcee50 0x555c3edd1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.577+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/3921048981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3edce910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.577+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/3921048981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edc3c70 0x555c3edc4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.577+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/3921048981 >> 172.21.15.107:0/3921048981 conn(0x555c3edc0200 msgr2=0x555c3edc05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.577+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/3921048981 shutdown_connections 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.577+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/3921048981 wait complete. 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.578+0000 7f2c8ec6f740 1 -- start start 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.578+0000 7f2c8ec6f740 1 Processor -- start 2024-09-19T04:22:02.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.578+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edcbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.578+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 0x555c3edcc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.578+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 0x555c3edcc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.578+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555c3edd1a20 con 0x555c3edcee50 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.578+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555c3ece50f0 con 0x555c3edc3c70 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.578+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555c3edd1ba0 con 0x555c3edc4590 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.578+0000 7f2c8ec6f740 10 client.0 _ll_register_callbacks cb 0x555c3ecccbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c88c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 0x555c3edcc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c89600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edcbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c88c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 0x555c3edcc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43026/0 (socket says 172.21.15.107:43026) 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c89600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edcbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51506/0 (socket says 172.21.15.107:51506) 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c88c00640 1 -- 172.21.15.107:0/1805809132 learned_addr learned my addr 172.21.15.107:0/1805809132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c8a000640 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 0x555c3edcc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c89600640 1 -- 172.21.15.107:0/1805809132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 msgr2=0x555c3edcc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c89600640 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 0x555c3edcc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c89600640 1 -- 172.21.15.107:0/1805809132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 msgr2=0x555c3edcc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c89600640 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 0x555c3edcc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c89600640 1 -- 172.21.15.107:0/1805809132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c3ea9b3c0 con 0x555c3edc3c70 2024-09-19T04:22:02.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c88c00640 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 0x555c3edcc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:02.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c8a000640 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 0x555c3edcc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:02.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c89600640 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edcbcf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c780cd6e0 tx=0x7f2c78009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2066a1a2e2618501 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.579+0000 7f2c6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:02.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c6aa00640 1 -- 172.21.15.107:0/1805809132 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780561a0 con 0x555c3edc3c70 2024-09-19T04:22:02.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c6aa00640 1 -- 172.21.15.107:0/1805809132 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f2c780d3c20 con 0x555c3edc3c70 2024-09-19T04:22:02.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/1805809132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7809c3e0 con 0x555c3edc3c70 2024-09-19T04:22:02.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c6aa00640 1 -- 172.21.15.107:0/1805809132 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2c78053fd0 con 0x555c3edc3c70 2024-09-19T04:22:02.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c8ec6f740 10 client.25559 fetch_fsmap learned FSMap version 28 2024-09-19T04:22:02.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/1805809132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c3edd1a20 con 0x555c3edc3c70 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c6aa00640 1 -- 172.21.15.107:0/1805809132 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7809c3e0 con 0x555c3edc3c70 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c6aa00640 1 -- 172.21.15.107:0/1805809132 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2c780cda20 con 0x555c3edc3c70 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c8ec6f740 10 client.25559 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c8ec6f740 10 client.25559 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c8ec6f740 20 client.25559 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x555c3eddeb90 0x555c3ede0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c8ec6f740 4 client.25559 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.580+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/1805809132 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c3ed37950 con 0x555c3eddeb90 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.581+0000 7f2c88c00640 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x555c3eddeb90 0x555c3ede0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.581+0000 7f2c88c00640 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x555c3eddeb90 0x555c3ede0f80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79c8db44fc45761c server_cookie=c7d25c3d68057fbc in_seq=0 out_seq=0 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.581+0000 7f2c6aa00640 10 client.25559 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c6aa00640 1 -- 172.21.15.107:0/1805809132 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c3ed37950 con 0x555c3eddeb90 2024-09-19T04:22:02.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c6aa00640 10 client.25559 handle_command_reply: tid=0 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c8ec6f740 1 client.25559 shutdown 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/1805809132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 msgr2=0x555c3edcbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edcbcf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c780cd6e0 tx=0x7f2c78009310 comp rx=0 tx=0).stop 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/1805809132 shutdown_connections 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x555c3eddeb90 0x555c3ede0f80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 0x555c3edcc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 0x555c3edcc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/1805809132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edcbcf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c6aa00640 0 client.25559 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.582+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/1805809132 >> 172.21.15.107:0/1805809132 conn(0x555c3edc0200 msgr2=0x555c3edc5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.583+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/1805809132 shutdown_connections 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.583+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/1805809132 wait complete. 2024-09-19T04:22:02.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.583+0000 7f2c8ec6f740 20 client.25559 trim_cache size 0 max 16384 2024-09-19T04:22:02.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.597+0000 7f2c8ec6f740 1 Processor -- start 2024-09-19T04:22:02.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.597+0000 7f2c8ec6f740 1 -- start start 2024-09-19T04:22:02.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.597+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edc3c70 0x555c3edc66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.597+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3ede6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.597+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edcee50 0x555c3ede6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.597+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841afa70 con 0x555c3edc3c70 2024-09-19T04:22:02.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.597+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841b15b0 con 0x555c3edc4590 2024-09-19T04:22:02.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.597+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841b0140 con 0x555c3edcee50 2024-09-19T04:22:02.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c88c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3ede6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c88c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3ede6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51518/0 (socket says 172.21.15.107:51518) 2024-09-19T04:22:02.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c88c00640 1 -- 172.21.15.107:0/3117158558 learned_addr learned my addr 172.21.15.107:0/3117158558 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:02.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c8a000640 1 --2- 172.21.15.107:0/3117158558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edcee50 0x555c3ede6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c89600640 1 --2- 172.21.15.107:0/3117158558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edc3c70 0x555c3edc66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c88c00640 1 -- 172.21.15.107:0/3117158558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edcee50 msgr2=0x555c3ede6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c88c00640 1 --2- 172.21.15.107:0/3117158558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edcee50 0x555c3ede6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c88c00640 1 -- 172.21.15.107:0/3117158558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edc3c70 msgr2=0x555c3edc66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c88c00640 1 --2- 172.21.15.107:0/3117158558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edc3c70 0x555c3edc66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c88c00640 1 -- 172.21.15.107:0/3117158558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3edc0030 con 0x555c3edc4590 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.598+0000 7f2c88c00640 1 --2- 172.21.15.107:0/3117158558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3ede6610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009850 tx=0x7f2c70005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a1729aea52e531 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.599+0000 7f2c75c00640 1 -- 172.21.15.107:0/3117158558 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70035040 con 0x555c3edc4590 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.599+0000 7f2c75c00640 1 -- 172.21.15.107:0/3117158558 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7004b070 con 0x555c3edc4590 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.599+0000 7f2c75c00640 1 -- 172.21.15.107:0/3117158558 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70031810 con 0x555c3edc4590 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.599+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/3117158558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 msgr2=0x555c3ede6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.599+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/3117158558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3ede6610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009850 tx=0x7f2c70005b50 comp rx=0 tx=0).stop 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.599+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/3117158558 shutdown_connections 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.599+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/3117158558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edcee50 0x555c3ede6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.599+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/3117158558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc4590 0x555c3ede6610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.599+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/3117158558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edc3c70 0x555c3edc66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.599+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/3117158558 >> 172.21.15.107:0/3117158558 conn(0x555c3edc0200 msgr2=0x555c3edcdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.600+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/3117158558 shutdown_connections 2024-09-19T04:22:02.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.600+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/3117158558 wait complete. 2024-09-19T04:22:02.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.600+0000 7f2c8ec6f740 1 -- start start 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.600+0000 7f2c8ec6f740 1 Processor -- start 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edd6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 0x555c3edd7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c8ec6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 0x555c3edd2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841b0ee0 con 0x555c3edcee50 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841b0810 con 0x555c3edc3c70 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c8ec6f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841ada50 con 0x555c3edc4590 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c8ec6f740 10 client.0 _ll_register_callbacks cb 0x7f2c78096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c89600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edd6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c89600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edd6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51528/0 (socket says 172.21.15.107:51528) 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c89600640 1 -- 172.21.15.107:0/2844895364 learned_addr learned my addr 172.21.15.107:0/2844895364 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:02.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c88c00640 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 0x555c3edd7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c8a000640 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 0x555c3edd2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c89600640 1 -- 172.21.15.107:0/2844895364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 msgr2=0x555c3edd7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c89600640 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 0x555c3edd7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c89600640 1 -- 172.21.15.107:0/2844895364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 msgr2=0x555c3edd2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c89600640 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 0x555c3edd2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.601+0000 7f2c89600640 1 -- 172.21.15.107:0/2844895364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c3edd2b80 con 0x555c3edc3c70 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c89600640 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edd6ec0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c78098db0 tx=0x7f2c780d86f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c0b24762eecc45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c6aa00640 1 -- 172.21.15.107:0/2844895364 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780541d0 con 0x555c3edc3c70 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c6aa00640 1 -- 172.21.15.107:0/2844895364 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f2c780d35e0 con 0x555c3edc3c70 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/2844895364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7004b070 con 0x555c3edc3c70 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c6aa00640 1 -- 172.21.15.107:0/2844895364 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2c780516f0 con 0x555c3edc3c70 2024-09-19T04:22:02.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c8ec6f740 10 client.25565 fetch_fsmap learned FSMap version 28 2024-09-19T04:22:02.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/2844895364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c3ee49040 con 0x555c3edc3c70 2024-09-19T04:22:02.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c6aa00640 1 -- 172.21.15.107:0/2844895364 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7004b070 con 0x555c3edc3c70 2024-09-19T04:22:02.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c6aa00640 1 -- 172.21.15.107:0/2844895364 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2c780a4b10 con 0x555c3edc3c70 2024-09-19T04:22:02.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c8ec6f740 10 client.25565 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:22:02.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c8ec6f740 10 client.25565 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:22:02.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c8ec6f740 20 client.25565 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:02.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x555c3ee49510 0x555c3ee4b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:02.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c8ec6f740 4 client.25565 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:02.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.602+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/2844895364 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c3ed31e60 con 0x555c3ee49510 2024-09-19T04:22:02.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.603+0000 7f2c88c00640 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x555c3ee49510 0x555c3ee4b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:02.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c88c00640 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x555c3ee49510 0x555c3ee4b900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb4799214611357a server_cookie=f1d2e41f51b5d9cb in_seq=0 out_seq=0 2024-09-19T04:22:02.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c6aa00640 10 client.25565 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:22:02.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c6aa00640 1 -- 172.21.15.107:0/2844895364 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c3ed31e60 con 0x555c3ee49510 2024-09-19T04:22:02.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c6aa00640 10 client.25565 handle_command_reply: tid=0 2024-09-19T04:22:02.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 client.25565 shutdown 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/2844895364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 msgr2=0x555c3edd6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edd6ec0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c78098db0 tx=0x7f2c780d86f0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/2844895364 shutdown_connections 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x555c3ee49510 0x555c3ee4b900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c3edcee50 0x555c3edd2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c3edc4590 0x555c3edd7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 --2- 172.21.15.107:0/2844895364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c3edc3c70 0x555c3edd6ec0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/2844895364 >> 172.21.15.107:0/2844895364 conn(0x555c3edc0200 msgr2=0x555c3edbeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c6aa00640 0 client.25565 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/2844895364 shutdown_connections 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.604+0000 7f2c8ec6f740 1 -- 172.21.15.107:0/2844895364 wait complete. 2024-09-19T04:22:02.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8ec6f740 20 client.25565 trim_cache size 0 max 16384 2024-09-19T04:22:02.610 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:22:02.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 -- 172.21.15.107:0/3291365701 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2c6c0a33b0 msgr2=0x7f2c6c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2c6c0a33b0 0x7f2c6c0a5870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009fd0 tx=0x7f2c7c009500 comp rx=0 tx=0).stop 2024-09-19T04:22:02.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 -- 172.21.15.107:0/3291365701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84103ff0 msgr2=0x7f2c8419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:02.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84103ff0 0x7f2c8419a650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800e9f0 tx=0x7f2c7800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 -- 172.21.15.107:0/3291365701 shutdown_connections 2024-09-19T04:22:02.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2c6c0a33b0 0x7f2c6c0a5870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c84109120 0x7f2c8419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c841049a0 0x7f2c8419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 --2- 172.21.15.107:0/3291365701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84103ff0 0x7f2c8419a650 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:02.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 -- 172.21.15.107:0/3291365701 >> 172.21.15.107:0/3291365701 conn(0x7f2c840ffa90 msgr2=0x7f2c84100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:02.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.605+0000 7f2c8c000640 1 -- 172.21.15.107:0/3291365701 shutdown_connections 2024-09-19T04:22:02.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:02.606+0000 7f2c8c000640 1 -- 172.21.15.107:0/3291365701 wait complete. 2024-09-19T04:22:02.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 54885 15.65 MB/sec warmup 20 sec latency 20.562 ms 2024-09-19T04:22:02.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:02 smithi115 ceph-mon[35477]: pgmap v54: 97 pgs: 97 active+clean; 25 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 256 op/s 2024-09-19T04:22:02.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:02 smithi145 ceph-mon[35096]: pgmap v54: 97 pgs: 97 active+clean; 25 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 256 op/s 2024-09-19T04:22:02.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:02 smithi107 ceph-mon[30659]: pgmap v54: 97 pgs: 97 active+clean; 25 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 256 op/s 2024-09-19T04:22:03.356 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:22:03.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 57442 15.57 MB/sec warmup 21 sec latency 28.537 ms 2024-09-19T04:22:03.796 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:04.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.470+0000 7f4b85600640 1 -- 172.21.15.107:0/1835157315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b801049c0 msgr2=0x7f4b80108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.470+0000 7f4b85600640 1 --2- 172.21.15.107:0/1835157315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b801049c0 0x7f4b80108a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009900 tx=0x7f4b6c02a410 comp rx=0 tx=0).stop 2024-09-19T04:22:04.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.470+0000 7f4b85600640 1 -- 172.21.15.107:0/1835157315 shutdown_connections 2024-09-19T04:22:04.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.470+0000 7f4b85600640 1 --2- 172.21.15.107:0/1835157315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b80109140 0x7f4b80110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.470+0000 7f4b85600640 1 --2- 172.21.15.107:0/1835157315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b801049c0 0x7f4b80108a10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.470+0000 7f4b85600640 1 --2- 172.21.15.107:0/1835157315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b80104010 0x7f4b801043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.470+0000 7f4b85600640 1 -- 172.21.15.107:0/1835157315 >> 172.21.15.107:0/1835157315 conn(0x7f4b800ffa50 msgr2=0x7f4b80101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:04.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.470+0000 7f4b85600640 1 -- 172.21.15.107:0/1835157315 shutdown_connections 2024-09-19T04:22:04.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.470+0000 7f4b85600640 1 -- 172.21.15.107:0/1835157315 wait complete. 2024-09-19T04:22:04.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.471+0000 7f4b85600640 1 Processor -- start 2024-09-19T04:22:04.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.471+0000 7f4b85600640 1 -- start start 2024-09-19T04:22:04.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b85600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80104010 0x7f4b801a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b85600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b801049c0 0x7f4b801a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b85600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b80109140 0x7f4b8019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b85600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80114190 con 0x7f4b801049c0 2024-09-19T04:22:04.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b85600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80114010 con 0x7f4b80104010 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b85600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80114310 con 0x7f4b80109140 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b7ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80104010 0x7f4b801a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b7ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80104010 0x7f4b801a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51554/0 (socket says 172.21.15.107:51554) 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b7ea00640 1 -- 172.21.15.107:0/3748465249 learned_addr learned my addr 172.21.15.107:0/3748465249 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b7f400640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b80109140 0x7f4b8019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b7e000640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b801049c0 0x7f4b801a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b7ea00640 1 -- 172.21.15.107:0/3748465249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b80109140 msgr2=0x7f4b8019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b80109140 0x7f4b8019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b7ea00640 1 -- 172.21.15.107:0/3748465249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b801049c0 msgr2=0x7f4b801a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b801049c0 0x7f4b801a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.472+0000 7f4b7ea00640 1 -- 172.21.15.107:0/3748465249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8019acd0 con 0x7f4b80104010 2024-09-19T04:22:04.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.473+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80104010 0x7f4b801a03e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000ea60 tx=0x7f4b7000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:04.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.473+0000 7f4b77400640 1 -- 172.21.15.107:0/3748465249 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000cde0 con 0x7f4b80104010 2024-09-19T04:22:04.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.473+0000 7f4b77400640 1 -- 172.21.15.107:0/3748465249 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b700035f0 con 0x7f4b80104010 2024-09-19T04:22:04.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.473+0000 7f4b77400640 1 -- 172.21.15.107:0/3748465249 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70010670 con 0x7f4b80104010 2024-09-19T04:22:04.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.473+0000 7f4b85600640 1 -- 172.21.15.107:0/3748465249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8019afc0 con 0x7f4b80104010 2024-09-19T04:22:04.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.473+0000 7f4b85600640 1 -- 172.21.15.107:0/3748465249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b801a72b0 con 0x7f4b80104010 2024-09-19T04:22:04.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.476+0000 7f4b8820a740 1 Processor -- start 2024-09-19T04:22:04.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.476+0000 7f4b77400640 1 -- 172.21.15.107:0/3748465249 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f4b700037a0 con 0x7f4b80104010 2024-09-19T04:22:04.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.476+0000 7f4b8820a740 1 -- start start 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 0x55f9c35ea050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c35f4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 0x55f9c35f7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b8820a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f9c35503e0 con 0x55f9c35e9c70 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b8820a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f9c34f7d80 con 0x55f9c35f4e50 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b8820a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f9c354c3a0 con 0x55f9c35ea590 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b7ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c35f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b7f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 0x55f9c35ea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b7e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 0x55f9c35f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b7f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 0x55f9c35ea050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34466/0 (socket says 172.21.15.107:34466) 2024-09-19T04:22:04.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b7ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c35f4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43062/0 (socket says 172.21.15.107:43062) 2024-09-19T04:22:04.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.477+0000 7f4b7f400640 1 -- 172.21.15.107:0/1668500659 learned_addr learned my addr 172.21.15.107:0/1668500659 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:04.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.478+0000 7f4b7e000640 1 -- 172.21.15.107:0/1668500659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 msgr2=0x55f9c35f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:04.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.478+0000 7f4b7e000640 1 --2- 172.21.15.107:0/1668500659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c35f4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.478+0000 7f4b7e000640 1 -- 172.21.15.107:0/1668500659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 msgr2=0x55f9c35ea050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:04.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.478+0000 7f4b7e000640 1 --2- 172.21.15.107:0/1668500659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 0x55f9c35ea050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.478+0000 7f4b7e000640 1 -- 172.21.15.107:0/1668500659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c3516ae0 con 0x55f9c35f4e50 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.478+0000 7f4b7e000640 1 --2- 172.21.15.107:0/1668500659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 0x55f9c35f7240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c02aa00 tx=0x7f4b6c005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aab789d1449e373 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.478+0000 7f4b6b400640 1 -- 172.21.15.107:0/1668500659 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c035030 con 0x55f9c35f4e50 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.478+0000 7f4b6b400640 1 -- 172.21.15.107:0/1668500659 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c0043c0 con 0x55f9c35f4e50 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.478+0000 7f4b6b400640 1 -- 172.21.15.107:0/1668500659 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0035f0 con 0x55f9c35f4e50 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.478+0000 7f4b77400640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4b640a3430 0x7f4b640a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b77400640 1 -- 172.21.15.107:0/3748465249 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4b700cfff0 con 0x7f4b80104010 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b7f400640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4b640a3430 0x7f4b640a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b8820a740 1 -- 172.21.15.107:0/1668500659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 msgr2=0x55f9c35f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b8820a740 1 --2- 172.21.15.107:0/1668500659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 0x55f9c35f7240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c02aa00 tx=0x7f4b6c005e60 comp rx=0 tx=0).stop 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b8820a740 1 -- 172.21.15.107:0/1668500659 shutdown_connections 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b8820a740 1 --2- 172.21.15.107:0/1668500659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 0x55f9c35f7240 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b8820a740 1 --2- 172.21.15.107:0/1668500659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c35f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b8820a740 1 --2- 172.21.15.107:0/1668500659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 0x55f9c35ea050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b8820a740 1 -- 172.21.15.107:0/1668500659 >> 172.21.15.107:0/1668500659 conn(0x55f9c35e6200 msgr2=0x55f9c35e65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:04.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b8820a740 1 -- 172.21.15.107:0/1668500659 shutdown_connections 2024-09-19T04:22:04.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.479+0000 7f4b8820a740 1 -- 172.21.15.107:0/1668500659 wait complete. 2024-09-19T04:22:04.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.480+0000 7f4b7f400640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4b640a3430 0x7f4b640a58f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b78011630 tx=0x7f4b78009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:04.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.480+0000 7f4b8820a740 1 -- start start 2024-09-19T04:22:04.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.480+0000 7f4b8820a740 1 Processor -- start 2024-09-19T04:22:04.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 0x55f9c35f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c35f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35f4e50 0x55f9c35f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b8820a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f9c35f7a20 con 0x55f9c35f4e50 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b8820a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f9c350b0f0 con 0x55f9c35ea590 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b8820a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f9c35f7ba0 con 0x55f9c35e9c70 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b8820a740 10 client.0 _ll_register_callbacks cb 0x55f9c34f2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b7e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c35f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b7e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c35f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51578/0 (socket says 172.21.15.107:51578) 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b7e000640 1 -- 172.21.15.107:0/1106189490 learned_addr learned my addr 172.21.15.107:0/1106189490 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 0x55f9c35f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b7e000640 1 -- 172.21.15.107:0/1106189490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 msgr2=0x55f9c35f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b7e000640 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 0x55f9c35f1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.481+0000 7f4b7e000640 1 -- 172.21.15.107:0/1106189490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35f4e50 msgr2=0x55f9c35f27e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.482+0000 7f4b7e000640 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35f4e50 0x55f9c35f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.482+0000 7f4b7e000640 1 -- 172.21.15.107:0/1106189490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c32c13c0 con 0x55f9c35ea590 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.482+0000 7f4b7e000640 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c35f22a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c000c00 tx=0x7f4b6c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3acc9e15b448bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.482+0000 7f4b5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.482+0000 7f4b5fe00640 1 -- 172.21.15.107:0/1106189490 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c035070 con 0x55f9c35ea590 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.482+0000 7f4b5fe00640 1 -- 172.21.15.107:0/1106189490 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4b6c003b20 con 0x55f9c35ea590 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.482+0000 7f4b8820a740 1 -- 172.21.15.107:0/1106189490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b6c0043c0 con 0x55f9c35ea590 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b5fe00640 1 -- 172.21.15.107:0/1106189490 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b6c009520 con 0x55f9c35ea590 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b8820a740 10 client.25577 fetch_fsmap learned FSMap version 28 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b8820a740 1 -- 172.21.15.107:0/1106189490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9c3603a90 con 0x55f9c35ea590 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b5fe00640 1 -- 172.21.15.107:0/1106189490 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c0043c0 con 0x55f9c35ea590 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b5fe00640 1 -- 172.21.15.107:0/1106189490 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4b6c056430 con 0x55f9c35ea590 2024-09-19T04:22:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b8820a740 10 client.25577 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:22:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b8820a740 10 client.25577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b8820a740 20 client.25577 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b8820a740 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9c3604de0 0x55f9c36071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b8820a740 4 client.25577 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.483+0000 7f4b8820a740 1 -- 172.21.15.107:0/1106189490 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9c355d950 con 0x55f9c3604de0 2024-09-19T04:22:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.484+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9c3604de0 0x55f9c36071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.484+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9c3604de0 0x55f9c36071d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30d7b30cfefa1e28 server_cookie=60405bdd419e4fdb in_seq=0 out_seq=0 2024-09-19T04:22:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.484+0000 7f4b5fe00640 10 client.25577 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:04.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.485+0000 7f4b5fe00640 1 -- 172.21.15.107:0/1106189490 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9c355d950 con 0x55f9c3604de0 2024-09-19T04:22:04.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.485+0000 7f4b5fe00640 10 client.25577 handle_command_reply: tid=0 2024-09-19T04:22:04.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 client.25577 shutdown 2024-09-19T04:22:04.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 -- 172.21.15.107:0/1106189490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 msgr2=0x55f9c35f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c35f22a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c000c00 tx=0x7f4b6c003820 comp rx=0 tx=0).stop 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 -- 172.21.15.107:0/1106189490 shutdown_connections 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9c3604de0 0x55f9c36071d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35f4e50 0x55f9c35f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c35f22a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 --2- 172.21.15.107:0/1106189490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 0x55f9c35f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b5fe00640 0 client.25577 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 -- 172.21.15.107:0/1106189490 >> 172.21.15.107:0/1106189490 conn(0x55f9c35e6200 msgr2=0x55f9c35ebf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 -- 172.21.15.107:0/1106189490 shutdown_connections 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 1 -- 172.21.15.107:0/1106189490 wait complete. 2024-09-19T04:22:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.486+0000 7f4b8820a740 20 client.25577 trim_cache size 0 max 16384 2024-09-19T04:22:04.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.503+0000 7f4b8820a740 1 Processor -- start 2024-09-19T04:22:04.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.503+0000 7f4b8820a740 1 -- start start 2024-09-19T04:22:04.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.503+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 0x55f9c360a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c360a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35f4e50 0x55f9c35ef020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b8820a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801af990 con 0x55f9c35f4e50 2024-09-19T04:22:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b8820a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801b14d0 con 0x55f9c35ea590 2024-09-19T04:22:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b8820a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801b0060 con 0x55f9c35e9c70 2024-09-19T04:22:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b7ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 0x55f9c360a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b7ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 0x55f9c360a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43084/0 (socket says 172.21.15.107:43084) 2024-09-19T04:22:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b7ea00640 1 -- 172.21.15.107:0/2672767095 learned_addr learned my addr 172.21.15.107:0/2672767095 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:04.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b7f400640 1 --2- 172.21.15.107:0/2672767095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35f4e50 0x55f9c35ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b7e000640 1 --2- 172.21.15.107:0/2672767095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c360a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b7ea00640 1 -- 172.21.15.107:0/2672767095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 msgr2=0x55f9c360a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:04.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/2672767095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c360a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b7ea00640 1 -- 172.21.15.107:0/2672767095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35f4e50 msgr2=0x55f9c35ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:04.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/2672767095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35f4e50 0x55f9c35ef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.504+0000 7f4b7ea00640 1 -- 172.21.15.107:0/2672767095 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c35e9aa0 con 0x55f9c35e9c70 2024-09-19T04:22:04.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.505+0000 7f4b7e000640 1 --2- 172.21.15.107:0/2672767095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c360a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:04.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.505+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/2672767095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 0x55f9c360a1f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000ea30 tx=0x7f4b700d3b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c039bb071b7b780 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:04.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.505+0000 7f4b6aa00640 1 -- 172.21.15.107:0/2672767095 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7009be60 con 0x55f9c35e9c70 2024-09-19T04:22:04.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.505+0000 7f4b6aa00640 1 -- 172.21.15.107:0/2672767095 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b7009ca20 con 0x55f9c35e9c70 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.505+0000 7f4b6aa00640 1 -- 172.21.15.107:0/2672767095 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7009bc80 con 0x55f9c35e9c70 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.506+0000 7f4b8820a740 1 -- 172.21.15.107:0/2672767095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 msgr2=0x55f9c360a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.506+0000 7f4b8820a740 1 --2- 172.21.15.107:0/2672767095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 0x55f9c360a1f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000ea30 tx=0x7f4b700d3b40 comp rx=0 tx=0).stop 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.506+0000 7f4b8820a740 1 -- 172.21.15.107:0/2672767095 shutdown_connections 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.506+0000 7f4b8820a740 1 --2- 172.21.15.107:0/2672767095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35f4e50 0x55f9c35ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.506+0000 7f4b8820a740 1 --2- 172.21.15.107:0/2672767095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35ea590 0x55f9c360a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.506+0000 7f4b8820a740 1 --2- 172.21.15.107:0/2672767095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35e9c70 0x55f9c360a1f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.506+0000 7f4b8820a740 1 -- 172.21.15.107:0/2672767095 >> 172.21.15.107:0/2672767095 conn(0x55f9c35e6200 msgr2=0x55f9c35f3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.506+0000 7f4b8820a740 1 -- 172.21.15.107:0/2672767095 shutdown_connections 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.506+0000 7f4b8820a740 1 -- 172.21.15.107:0/2672767095 wait complete. 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.507+0000 7f4b8820a740 1 -- start start 2024-09-19T04:22:04.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.507+0000 7f4b8820a740 1 Processor -- start 2024-09-19T04:22:04.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.507+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 0x55f9c36083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.507+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c36088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.507+0000 7f4b8820a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 0x55f9c366e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.507+0000 7f4b8820a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801b0e00 con 0x55f9c35e9c70 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.507+0000 7f4b8820a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801b0730 con 0x55f9c35f4e50 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.507+0000 7f4b8820a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801ad970 con 0x55f9c35ea590 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.507+0000 7f4b8820a740 10 client.0 _ll_register_callbacks cb 0x7f4b78009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c36088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c36088e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43098/0 (socket says 172.21.15.107:43098) 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7e000640 1 -- 172.21.15.107:0/3084057207 learned_addr learned my addr 172.21.15.107:0/3084057207 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7f400640 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 0x55f9c366e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7e000640 1 -- 172.21.15.107:0/3084057207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 msgr2=0x55f9c366e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7e000640 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 0x55f9c366e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7e000640 1 -- 172.21.15.107:0/3084057207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 msgr2=0x55f9c36083a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 0x55f9c36083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7e000640 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 0x55f9c36083a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7e000640 1 -- 172.21.15.107:0/3084057207 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c35e1770 con 0x55f9c35ea590 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 0x55f9c36083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.508+0000 7f4b7e000640 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c36088e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0096c0 tx=0x7f4b6c056c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4c6329ead34c595 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.509+0000 7f4b5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.509+0000 7f4b5fe00640 1 -- 172.21.15.107:0/3084057207 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c035040 con 0x55f9c35ea590 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.509+0000 7f4b5fe00640 1 -- 172.21.15.107:0/3084057207 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4b6c003d20 con 0x55f9c35ea590 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.509+0000 7f4b8820a740 1 -- 172.21.15.107:0/3084057207 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b7009ca20 con 0x55f9c35ea590 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.509+0000 7f4b5fe00640 1 -- 172.21.15.107:0/3084057207 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b6c004490 con 0x55f9c35ea590 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.509+0000 7f4b8820a740 10 client.25561 fetch_fsmap learned FSMap version 28 2024-09-19T04:22:04.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.509+0000 7f4b8820a740 1 -- 172.21.15.107:0/3084057207 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9c366fc50 con 0x55f9c35ea590 2024-09-19T04:22:04.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.510+0000 7f4b5fe00640 1 -- 172.21.15.107:0/3084057207 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b7009ca20 con 0x55f9c35ea590 2024-09-19T04:22:04.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.510+0000 7f4b5fe00640 1 -- 172.21.15.107:0/3084057207 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4b6c05d030 con 0x55f9c35ea590 2024-09-19T04:22:04.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.510+0000 7f4b8820a740 10 client.25561 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:22:04.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.510+0000 7f4b8820a740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:04.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.510+0000 7f4b8820a740 20 client.25561 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:04.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.510+0000 7f4b8820a740 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9c3670260 0x55f9c3672680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:04.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.510+0000 7f4b8820a740 4 client.25561 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:04.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.510+0000 7f4b8820a740 1 -- 172.21.15.107:0/3084057207 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9c3557e60 con 0x55f9c3670260 2024-09-19T04:22:04.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.510+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9c3670260 0x55f9c3672680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:04.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b7ea00640 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9c3670260 0x55f9c3672680 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dcb267d316bc967 server_cookie=19c6ed57bef4ff88 in_seq=0 out_seq=0 2024-09-19T04:22:04.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b5fe00640 10 client.25561 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:04.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b5fe00640 1 -- 172.21.15.107:0/3084057207 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9c3557e60 con 0x55f9c3670260 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b5fe00640 10 client.25561 handle_command_reply: tid=0 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 client.25561 shutdown 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 -- 172.21.15.107:0/3084057207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 msgr2=0x55f9c36088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c36088e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0096c0 tx=0x7f4b6c056c70 comp rx=0 tx=0).stop 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 -- 172.21.15.107:0/3084057207 shutdown_connections 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9c3670260 0x55f9c3672680 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9c35f4e50 0x55f9c366e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f9c35ea590 0x55f9c36088e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 --2- 172.21.15.107:0/3084057207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c35e9c70 0x55f9c36083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b5fe00640 0 client.25561 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 -- 172.21.15.107:0/3084057207 >> 172.21.15.107:0/3084057207 conn(0x55f9c35e6200 msgr2=0x55f9c35e4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 -- 172.21.15.107:0/3084057207 shutdown_connections 2024-09-19T04:22:04.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 1 -- 172.21.15.107:0/3084057207 wait complete. 2024-09-19T04:22:04.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.511+0000 7f4b8820a740 20 client.25561 trim_cache size 0 max 16384 2024-09-19T04:22:04.517 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:22:04.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 -- 172.21.15.107:0/3748465249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4b640a3430 msgr2=0x7f4b640a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4b640a3430 0x7f4b640a58f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b78011630 tx=0x7f4b78009500 comp rx=0 tx=0).stop 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 -- 172.21.15.107:0/3748465249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80104010 msgr2=0x7f4b801a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80104010 0x7f4b801a03e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000ea60 tx=0x7f4b7000ef20 comp rx=0 tx=0).stop 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 -- 172.21.15.107:0/3748465249 shutdown_connections 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4b640a3430 0x7f4b640a58f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b80109140 0x7f4b8019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b801049c0 0x7f4b801a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 --2- 172.21.15.107:0/3748465249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80104010 0x7f4b801a03e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 -- 172.21.15.107:0/3748465249 >> 172.21.15.107:0/3748465249 conn(0x7f4b800ffa50 msgr2=0x7f4b80100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 -- 172.21.15.107:0/3748465249 shutdown_connections 2024-09-19T04:22:04.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:04.512+0000 7f4b85600640 1 -- 172.21.15.107:0/3748465249 wait complete. 2024-09-19T04:22:04.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 60602 15.71 MB/sec warmup 22 sec latency 10.482 ms 2024-09-19T04:22:04.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:04 smithi115 ceph-mon[35477]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-09-19T04:22:04.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:04 smithi145 ceph-mon[35096]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-09-19T04:22:04.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:04 smithi107 ceph-mon[30659]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-09-19T04:22:05.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 63244 15.59 MB/sec warmup 23 sec latency 18.972 ms 2024-09-19T04:22:06.284 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:22:06.284 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:22:06.613 INFO:tasks.workunit.client.0.smithi107.stdout: 1 65375 15.43 MB/sec warmup 24 sec latency 21.307 ms 2024-09-19T04:22:06.692 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:06.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:06 smithi115 ceph-mon[35477]: pgmap v56: 97 pgs: 97 active+clean; 32 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-09-19T04:22:06.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:06 smithi145 ceph-mon[35096]: pgmap v56: 97 pgs: 97 active+clean; 32 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-09-19T04:22:06.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:06 smithi107 ceph-mon[30659]: pgmap v56: 97 pgs: 97 active+clean; 32 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-09-19T04:22:07.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.276+0000 7f4662800640 1 -- 172.21.15.107:0/1163976985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f465c103ff0 msgr2=0x7f465c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:07.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.276+0000 7f4662800640 1 --2- 172.21.15.107:0/1163976985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f465c103ff0 0x7f465c1043d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f464c009900 tx=0x7f464c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.276+0000 7f4662800640 1 -- 172.21.15.107:0/1163976985 shutdown_connections 2024-09-19T04:22:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.276+0000 7f4662800640 1 --2- 172.21.15.107:0/1163976985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465c109120 0x7f465c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.276+0000 7f4662800640 1 --2- 172.21.15.107:0/1163976985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f465c1049a0 0x7f465c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.276+0000 7f4662800640 1 --2- 172.21.15.107:0/1163976985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f465c103ff0 0x7f465c1043d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.276+0000 7f4662800640 1 -- 172.21.15.107:0/1163976985 >> 172.21.15.107:0/1163976985 conn(0x7f465c0ffa90 msgr2=0x7f465c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.276+0000 7f4662800640 1 -- 172.21.15.107:0/1163976985 shutdown_connections 2024-09-19T04:22:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.276+0000 7f4662800640 1 -- 172.21.15.107:0/1163976985 wait complete. 2024-09-19T04:22:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.277+0000 7f4662800640 1 Processor -- start 2024-09-19T04:22:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f4662800640 1 -- start start 2024-09-19T04:22:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f4662800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f465c103ff0 0x7f465c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f4662800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465c1049a0 0x7f465c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f4662800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f465c109120 0x7f465c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f4662800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f465c114150 con 0x7f465c109120 2024-09-19T04:22:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f4662800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f465c113fd0 con 0x7f465c103ff0 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f4662800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f465c1142d0 con 0x7f465c1049a0 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f465b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f465c103ff0 0x7f465c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f465aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465c1049a0 0x7f465c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f465be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f465c109120 0x7f465c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f465b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f465c103ff0 0x7f465c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51612/0 (socket says 172.21.15.107:51612) 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f465aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465c1049a0 0x7f465c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43110/0 (socket says 172.21.15.107:43110) 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.278+0000 7f465b400640 1 -- 172.21.15.107:0/525911076 learned_addr learned my addr 172.21.15.107:0/525911076 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f465aa00640 1 -- 172.21.15.107:0/525911076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f465c103ff0 msgr2=0x7f465c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f465aa00640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f465c103ff0 0x7f465c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f465aa00640 1 -- 172.21.15.107:0/525911076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f465c109120 msgr2=0x7f465c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f465aa00640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f465c109120 0x7f465c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f465aa00640 1 -- 172.21.15.107:0/525911076 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c19f620 con 0x7f465c1049a0 2024-09-19T04:22:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f465b400640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f465c103ff0 0x7f465c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f465aa00640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465c1049a0 0x7f465c19abb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f464800e9f0 tx=0x7f464800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f4653e00640 1 -- 172.21.15.107:0/525911076 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464800cd80 con 0x7f465c1049a0 2024-09-19T04:22:07.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f4662800640 1 -- 172.21.15.107:0/525911076 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465c19f910 con 0x7f465c1049a0 2024-09-19T04:22:07.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f4653e00640 1 -- 172.21.15.107:0/525911076 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46480035f0 con 0x7f465c1049a0 2024-09-19T04:22:07.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f4653e00640 1 -- 172.21.15.107:0/525911076 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648010670 con 0x7f465c1049a0 2024-09-19T04:22:07.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.279+0000 7f4662800640 1 -- 172.21.15.107:0/525911076 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465c1a71f0 con 0x7f465c1049a0 2024-09-19T04:22:07.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.283+0000 7f466547b740 1 Processor -- start 2024-09-19T04:22:07.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.283+0000 7f466547b740 1 -- start start 2024-09-19T04:22:07.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.283+0000 7f466547b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a81ac0 0x558b35a81ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.283+0000 7f466547b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a8c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.283+0000 7f466547b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a8cca0 0x558b35a8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.283+0000 7f4653e00640 1 -- 172.21.15.107:0/525911076 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f46480037a0 con 0x7f465c1049a0 2024-09-19T04:22:07.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.283+0000 7f466547b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558b359e8290 con 0x558b35a81ac0 2024-09-19T04:22:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.283+0000 7f466547b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558b3598fb40 con 0x558b35a8cca0 2024-09-19T04:22:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.283+0000 7f466547b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558b359e4250 con 0x558b35a823e0 2024-09-19T04:22:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.284+0000 7f465b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a8c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.284+0000 7f465b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a8c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43120/0 (socket says 172.21.15.107:43120) 2024-09-19T04:22:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.284+0000 7f465b400640 1 -- 172.21.15.107:0/3866435557 learned_addr learned my addr 172.21.15.107:0/3866435557 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.284+0000 7f465be00640 1 --2- 172.21.15.107:0/3866435557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a81ac0 0x558b35a81ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.284+0000 7f465b400640 1 -- 172.21.15.107:0/3866435557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a8cca0 msgr2=0x558b35a8f090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:22:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.284+0000 7f465b400640 1 --2- 172.21.15.107:0/3866435557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a8cca0 0x558b35a8f090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.284+0000 7f465b400640 1 -- 172.21.15.107:0/3866435557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a81ac0 msgr2=0x558b35a81ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.284+0000 7f465b400640 1 --2- 172.21.15.107:0/3866435557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a81ac0 0x558b35a81ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.284+0000 7f465b400640 1 -- 172.21.15.107:0/3866435557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b357596a0 con 0x558b35a823e0 2024-09-19T04:22:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.285+0000 7f465b400640 1 --2- 172.21.15.107:0/3866435557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a8c760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f464c009a20 tx=0x7f464c005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dcd490329b652ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.285+0000 7f4650c00640 1 -- 172.21.15.107:0/3866435557 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c040070 con 0x558b35a823e0 2024-09-19T04:22:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.285+0000 7f4650c00640 1 -- 172.21.15.107:0/3866435557 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464c031970 con 0x558b35a823e0 2024-09-19T04:22:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.285+0000 7f4650c00640 1 -- 172.21.15.107:0/3866435557 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c035040 con 0x558b35a823e0 2024-09-19T04:22:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.286+0000 7f4653e00640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46400a33b0 0x7f46400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.286+0000 7f4653e00640 1 -- 172.21.15.107:0/525911076 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f46480cff50 con 0x7f465c1049a0 2024-09-19T04:22:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.286+0000 7f465be00640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46400a33b0 0x7f46400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.286+0000 7f466547b740 1 -- 172.21.15.107:0/3866435557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 msgr2=0x558b35a8c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.286+0000 7f466547b740 1 --2- 172.21.15.107:0/3866435557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a8c760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f464c009a20 tx=0x7f464c005dd0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.287+0000 7f466547b740 1 -- 172.21.15.107:0/3866435557 shutdown_connections 2024-09-19T04:22:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.287+0000 7f466547b740 1 --2- 172.21.15.107:0/3866435557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a8cca0 0x558b35a8f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.287+0000 7f466547b740 1 --2- 172.21.15.107:0/3866435557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a8c760 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.287+0000 7f466547b740 1 --2- 172.21.15.107:0/3866435557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a81ac0 0x558b35a81ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.287+0000 7f466547b740 1 -- 172.21.15.107:0/3866435557 >> 172.21.15.107:0/3866435557 conn(0x558b35a7e050 msgr2=0x558b35a7e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.287+0000 7f466547b740 1 -- 172.21.15.107:0/3866435557 shutdown_connections 2024-09-19T04:22:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.287+0000 7f466547b740 1 -- 172.21.15.107:0/3866435557 wait complete. 2024-09-19T04:22:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.287+0000 7f465be00640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46400a33b0 0x7f46400a5870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4654005fd0 tx=0x7f4654009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.287+0000 7f466547b740 1 -- start start 2024-09-19T04:22:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f466547b740 1 Processor -- start 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f466547b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a81ac0 0x558b35a8c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f466547b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a823e0 0x558b35a898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f466547b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a8cca0 0x558b35a89e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f466547b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558b35a8f910 con 0x558b35a8cca0 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f466547b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558b359a2fa0 con 0x558b35a823e0 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f466547b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558b35a8fa90 con 0x558b35a81ac0 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f466547b740 10 client.0 _ll_register_callbacks cb 0x558b356f6ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f465aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a823e0 0x558b35a898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f465aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a823e0 0x558b35a898a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51638/0 (socket says 172.21.15.107:51638) 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f465aa00640 1 -- 172.21.15.107:0/3448216812 learned_addr learned my addr 172.21.15.107:0/3448216812 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.288+0000 7f465be00640 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a8cca0 0x558b35a89e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f465b400640 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a81ac0 0x558b35a8c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f465aa00640 1 -- 172.21.15.107:0/3448216812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a81ac0 msgr2=0x558b35a8c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f465aa00640 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a81ac0 0x558b35a8c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f465aa00640 1 -- 172.21.15.107:0/3448216812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a8cca0 msgr2=0x558b35a89e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f465aa00640 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a8cca0 0x558b35a89e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f465aa00640 1 -- 172.21.15.107:0/3448216812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b35a8a630 con 0x558b35a823e0 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f465be00640 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a8cca0 0x558b35a89e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f465b400640 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a81ac0 0x558b35a8c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f465aa00640 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a823e0 0x558b35a898a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f46480cdcf0 tx=0x7f464809bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc50f12b5ad2ce06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f4645600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f4645600640 1 -- 172.21.15.107:0/3448216812 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46480a4ef0 con 0x558b35a823e0 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f4645600640 1 -- 172.21.15.107:0/3448216812 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4648003b30 con 0x558b35a823e0 2024-09-19T04:22:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f466547b740 1 -- 172.21.15.107:0/3448216812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f464c031970 con 0x558b35a823e0 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f4645600640 1 -- 172.21.15.107:0/3448216812 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f464807ae60 con 0x558b35a823e0 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f466547b740 10 client.25592 fetch_fsmap learned FSMap version 28 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.289+0000 7f466547b740 1 -- 172.21.15.107:0/3448216812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b35aa4090 con 0x558b35a823e0 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.290+0000 7f4645600640 1 -- 172.21.15.107:0/3448216812 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464c031970 con 0x558b35a823e0 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.290+0000 7f4645600640 1 -- 172.21.15.107:0/3448216812 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f46480bc0c0 con 0x558b35a823e0 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.290+0000 7f466547b740 10 client.25592 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.290+0000 7f466547b740 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.290+0000 7f466547b740 20 client.25592 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.290+0000 7f466547b740 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558b35aa54f0 0x558b35aa7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.290+0000 7f466547b740 4 client.25592 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.290+0000 7f466547b740 1 -- 172.21.15.107:0/3448216812 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b359f5800 con 0x558b35aa54f0 2024-09-19T04:22:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.291+0000 7f465b400640 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558b35aa54f0 0x558b35aa7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.291+0000 7f4645600640 10 client.25592 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:07.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.291+0000 7f465b400640 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558b35aa54f0 0x558b35aa7930 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb510245199705ec server_cookie=2ed4553c30ff0aad in_seq=0 out_seq=0 2024-09-19T04:22:07.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f4645600640 1 -- 172.21.15.107:0/3448216812 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b359f5800 con 0x558b35aa54f0 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f4645600640 10 client.25592 handle_command_reply: tid=0 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 client.25592 shutdown 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 -- 172.21.15.107:0/3448216812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a823e0 msgr2=0x558b35a898a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a823e0 0x558b35a898a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f46480cdcf0 tx=0x7f464809bcd0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 -- 172.21.15.107:0/3448216812 shutdown_connections 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558b35aa54f0 0x558b35aa7930 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a8cca0 0x558b35a89e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a823e0 0x558b35a898a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f4645600640 0 client.25592 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 --2- 172.21.15.107:0/3448216812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a81ac0 0x558b35a8c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 -- 172.21.15.107:0/3448216812 >> 172.21.15.107:0/3448216812 conn(0x558b35a7e050 msgr2=0x558b35a83d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 -- 172.21.15.107:0/3448216812 shutdown_connections 2024-09-19T04:22:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.292+0000 7f466547b740 1 -- 172.21.15.107:0/3448216812 wait complete. 2024-09-19T04:22:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.293+0000 7f466547b740 20 client.25592 trim_cache size 0 max 16384 2024-09-19T04:22:07.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.312+0000 7f466547b740 1 Processor -- start 2024-09-19T04:22:07.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.312+0000 7f466547b740 1 -- start start 2024-09-19T04:22:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f466547b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a7e050 0x558b35a8b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f466547b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a81ac0 0x558b35aad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f466547b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35aad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f466547b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f465c1ab550 con 0x558b35a7e050 2024-09-19T04:22:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f466547b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f465c1ad090 con 0x558b35a81ac0 2024-09-19T04:22:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f466547b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f465c1abc20 con 0x558b35a823e0 2024-09-19T04:22:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f465be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35aad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f465be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35aad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43152/0 (socket says 172.21.15.107:43152) 2024-09-19T04:22:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f465be00640 1 -- 172.21.15.107:0/4166876473 learned_addr learned my addr 172.21.15.107:0/4166876473 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f465aa00640 1 --2- 172.21.15.107:0/4166876473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a81ac0 0x558b35aad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f465b400640 1 --2- 172.21.15.107:0/4166876473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a7e050 0x558b35a8b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.313+0000 7f465be00640 1 -- 172.21.15.107:0/4166876473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a81ac0 msgr2=0x558b35aad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.314+0000 7f465be00640 1 --2- 172.21.15.107:0/4166876473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a81ac0 0x558b35aad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.314+0000 7f465be00640 1 -- 172.21.15.107:0/4166876473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a7e050 msgr2=0x558b35a8b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.314+0000 7f465be00640 1 --2- 172.21.15.107:0/4166876473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a7e050 0x558b35a8b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.314+0000 7f465be00640 1 -- 172.21.15.107:0/4166876473 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b35a8a170 con 0x558b35a823e0 2024-09-19T04:22:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.314+0000 7f465be00640 1 --2- 172.21.15.107:0/4166876473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35aad740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4654003d20 tx=0x7f4654016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=226f689cf6bd3b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.314+0000 7f4647e00640 1 -- 172.21.15.107:0/4166876473 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654003e10 con 0x558b35a823e0 2024-09-19T04:22:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.314+0000 7f4647e00640 1 -- 172.21.15.107:0/4166876473 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46540045c0 con 0x558b35a823e0 2024-09-19T04:22:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.314+0000 7f4647e00640 1 -- 172.21.15.107:0/4166876473 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654005800 con 0x558b35a823e0 2024-09-19T04:22:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.315+0000 7f466547b740 1 -- 172.21.15.107:0/4166876473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 msgr2=0x558b35aad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.315+0000 7f466547b740 1 --2- 172.21.15.107:0/4166876473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35aad740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4654003d20 tx=0x7f4654016000 comp rx=0 tx=0).stop 2024-09-19T04:22:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.315+0000 7f466547b740 1 -- 172.21.15.107:0/4166876473 shutdown_connections 2024-09-19T04:22:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.315+0000 7f466547b740 1 --2- 172.21.15.107:0/4166876473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35aad740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.315+0000 7f466547b740 1 --2- 172.21.15.107:0/4166876473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a81ac0 0x558b35aad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.315+0000 7f466547b740 1 --2- 172.21.15.107:0/4166876473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a7e050 0x558b35a8b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.315+0000 7f466547b740 1 -- 172.21.15.107:0/4166876473 >> 172.21.15.107:0/4166876473 conn(0x558b35a7ead0 msgr2=0x558b35a8b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.315+0000 7f466547b740 1 -- 172.21.15.107:0/4166876473 shutdown_connections 2024-09-19T04:22:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.315+0000 7f466547b740 1 -- 172.21.15.107:0/4166876473 wait complete. 2024-09-19T04:22:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.316+0000 7f466547b740 1 -- start start 2024-09-19T04:22:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.316+0000 7f466547b740 1 Processor -- start 2024-09-19T04:22:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f466547b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a7e050 0x558b35a86a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f466547b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a81ac0 0x558b35a86fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f466547b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a87500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f466547b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f465c1ac9c0 con 0x558b35a81ac0 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f466547b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f465c1ac2f0 con 0x558b35a7e050 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f466547b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f465c1a9550 con 0x558b35a823e0 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f466547b740 10 client.0 _ll_register_callbacks cb 0x558b35a8a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f465b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a7e050 0x558b35a86a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f465b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a7e050 0x558b35a86a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51650/0 (socket says 172.21.15.107:51650) 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f465b400640 1 -- 172.21.15.107:0/4092346608 learned_addr learned my addr 172.21.15.107:0/4092346608 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f465be00640 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a87500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f465aa00640 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a81ac0 0x558b35a86fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f465b400640 1 -- 172.21.15.107:0/4092346608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 msgr2=0x558b35a87500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f465b400640 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a87500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f465b400640 1 -- 172.21.15.107:0/4092346608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a81ac0 msgr2=0x558b35a86fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f465b400640 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a81ac0 0x558b35a86fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.317+0000 7f465b400640 1 -- 172.21.15.107:0/4092346608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b35b0f330 con 0x558b35a7e050 2024-09-19T04:22:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.318+0000 7f465be00640 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a87500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.318+0000 7f465b400640 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a7e050 0x558b35a86a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f464c005500 tx=0x7f464c005490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f920de41d7bf25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.318+0000 7f4645600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.318+0000 7f4645600640 1 -- 172.21.15.107:0/4092346608 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c040040 con 0x558b35a7e050 2024-09-19T04:22:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.318+0000 7f4645600640 1 -- 172.21.15.107:0/4092346608 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f464c004040 con 0x558b35a7e050 2024-09-19T04:22:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.318+0000 7f466547b740 1 -- 172.21.15.107:0/4092346608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46540045c0 con 0x558b35a7e050 2024-09-19T04:22:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f4645600640 1 -- 172.21.15.107:0/4092346608 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f464c035040 con 0x558b35a7e050 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f466547b740 10 client.25598 fetch_fsmap learned FSMap version 28 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f466547b740 1 -- 172.21.15.107:0/4092346608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b35b106b0 con 0x558b35a7e050 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f4645600640 1 -- 172.21.15.107:0/4092346608 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46540045c0 con 0x558b35a7e050 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f4645600640 1 -- 172.21.15.107:0/4092346608 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f464c05f480 con 0x558b35a7e050 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f466547b740 10 client.25598 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f466547b740 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f466547b740 20 client.25598 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f466547b740 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558b35a8cca0 0x558b35b12950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f466547b740 4 client.25598 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f466547b740 1 -- 172.21.15.107:0/4092346608 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558b359efd10 con 0x558b35a8cca0 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.319+0000 7f465aa00640 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558b35a8cca0 0x558b35b12950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.320+0000 7f465aa00640 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558b35a8cca0 0x558b35b12950 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785fdfb15b304187 server_cookie=d3eaf70e7e5d8e7 in_seq=0 out_seq=0 2024-09-19T04:22:07.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.320+0000 7f4645600640 10 client.25598 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:07.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f4645600640 1 -- 172.21.15.107:0/4092346608 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b359efd10 con 0x558b35a8cca0 2024-09-19T04:22:07.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f4645600640 10 client.25598 handle_command_reply: tid=0 2024-09-19T04:22:07.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f466547b740 1 client.25598 shutdown 2024-09-19T04:22:07.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f466547b740 1 -- 172.21.15.107:0/4092346608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a7e050 msgr2=0x558b35a86a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f466547b740 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a7e050 0x558b35a86a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f464c005500 tx=0x7f464c005490 comp rx=0 tx=0).stop 2024-09-19T04:22:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f466547b740 1 -- 172.21.15.107:0/4092346608 shutdown_connections 2024-09-19T04:22:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f466547b740 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558b35a8cca0 0x558b35b12950 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f466547b740 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b35a823e0 0x558b35a87500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f466547b740 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b35a81ac0 0x558b35a86fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f466547b740 1 --2- 172.21.15.107:0/4092346608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558b35a7e050 0x558b35a86a80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.322+0000 7f466547b740 1 -- 172.21.15.107:0/4092346608 >> 172.21.15.107:0/4092346608 conn(0x558b35a7ead0 msgr2=0x558b35a89980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:07.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.323+0000 7f4645600640 0 client.25598 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:07.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.323+0000 7f466547b740 1 -- 172.21.15.107:0/4092346608 shutdown_connections 2024-09-19T04:22:07.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.323+0000 7f466547b740 1 -- 172.21.15.107:0/4092346608 wait complete. 2024-09-19T04:22:07.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.323+0000 7f466547b740 20 client.25598 trim_cache size 0 max 16384 2024-09-19T04:22:07.326 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:22:07.326 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:22:07.327 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "8c2a45b6-5930-4e24-b41e-d8e7fd8111f7", 2024-09-19T04:22:07.327 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:22:07.327 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:22:07.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.324+0000 7f4662800640 1 -- 172.21.15.107:0/525911076 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46400a33b0 msgr2=0x7f46400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:07.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.324+0000 7f4662800640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46400a33b0 0x7f46400a5870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4654005fd0 tx=0x7f4654009500 comp rx=0 tx=0).stop 2024-09-19T04:22:07.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.324+0000 7f4662800640 1 -- 172.21.15.107:0/525911076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465c1049a0 msgr2=0x7f465c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:07.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.324+0000 7f4662800640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465c1049a0 0x7f465c19abb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f464800e9f0 tx=0x7f464800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.325+0000 7f4662800640 1 -- 172.21.15.107:0/525911076 shutdown_connections 2024-09-19T04:22:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.325+0000 7f4662800640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46400a33b0 0x7f46400a5870 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.325+0000 7f4662800640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f465c109120 0x7f465c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.325+0000 7f4662800640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465c1049a0 0x7f465c19abb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.325+0000 7f4662800640 1 --2- 172.21.15.107:0/525911076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f465c103ff0 0x7f465c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.325+0000 7f4662800640 1 -- 172.21.15.107:0/525911076 >> 172.21.15.107:0/525911076 conn(0x7f465c0ffa90 msgr2=0x7f465c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.325+0000 7f4662800640 1 -- 172.21.15.107:0/525911076 shutdown_connections 2024-09-19T04:22:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:07.325+0000 7f4662800640 1 -- 172.21.15.107:0/525911076 wait complete. 2024-09-19T04:22:07.614 INFO:tasks.workunit.client.0.smithi107.stdout: 1 67887 15.37 MB/sec warmup 25 sec latency 22.427 ms 2024-09-19T04:22:08.248 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:22:08.614 INFO:tasks.workunit.client.0.smithi107.stdout: 1 70788 15.34 MB/sec warmup 26 sec latency 11.541 ms 2024-09-19T04:22:08.660 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:08.799 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:08 smithi107 ceph-mon[30659]: pgmap v57: 97 pgs: 97 active+clean; 32 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-09-19T04:22:08.799 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:08 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:22:08.799 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:08 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:08.799 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:08 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:22:08.800 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:08 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:08.800 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:08 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:22:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:08 smithi115 ceph-mon[35477]: pgmap v57: 97 pgs: 97 active+clean; 32 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-09-19T04:22:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:08 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:22:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:08 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:08 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:22:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:08 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:08.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:08 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:22:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:08 smithi145 ceph-mon[35096]: pgmap v57: 97 pgs: 97 active+clean; 32 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-09-19T04:22:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:08 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:22:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:08 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:08 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:22:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:08 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:08 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:22:09.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.220+0000 7f77c3200640 1 -- 172.21.15.107:0/3938982210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c4076720 msgr2=0x7f77c4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:09.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.220+0000 7f77c3200640 1 --2- 172.21.15.107:0/3938982210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c4076720 0x7f77c4076b00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009900 tx=0x7f77b402a410 comp rx=0 tx=0).stop 2024-09-19T04:22:09.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.220+0000 7f77c3200640 1 -- 172.21.15.107:0/3938982210 shutdown_connections 2024-09-19T04:22:09.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.220+0000 7f77c3200640 1 --2- 172.21.15.107:0/3938982210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77c4112870 0x7f77c4114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.220+0000 7f77c3200640 1 --2- 172.21.15.107:0/3938982210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c4077040 0x7f77c4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.220+0000 7f77c3200640 1 --2- 172.21.15.107:0/3938982210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c4076720 0x7f77c4076b00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.220+0000 7f77c3200640 1 -- 172.21.15.107:0/3938982210 >> 172.21.15.107:0/3938982210 conn(0x7f77c40ff910 msgr2=0x7f77c4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:09.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.220+0000 7f77c3200640 1 -- 172.21.15.107:0/3938982210 shutdown_connections 2024-09-19T04:22:09.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.220+0000 7f77c3200640 1 -- 172.21.15.107:0/3938982210 wait complete. 2024-09-19T04:22:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.221+0000 7f77c3200640 1 Processor -- start 2024-09-19T04:22:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.221+0000 7f77c3200640 1 -- start start 2024-09-19T04:22:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c3200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c4076720 0x7f77c41a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77c4077040 0x7f77c41a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c4112870 0x7f77c419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4118400 con 0x7f77c4112870 2024-09-19T04:22:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c3200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4118280 con 0x7f77c4076720 2024-09-19T04:22:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4118580 con 0x7f77c4077040 2024-09-19T04:22:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c4076720 0x7f77c41a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c4076720 0x7f77c41a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:52628/0 (socket says 172.21.15.107:52628) 2024-09-19T04:22:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c4112870 0x7f77c419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c1e00640 1 -- 172.21.15.107:0/3903843702 learned_addr learned my addr 172.21.15.107:0/3903843702 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c1400640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77c4077040 0x7f77c41a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c2800640 1 -- 172.21.15.107:0/3903843702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77c4077040 msgr2=0x7f77c41a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c2800640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77c4077040 0x7f77c41a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c2800640 1 -- 172.21.15.107:0/3903843702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c4076720 msgr2=0x7f77c41a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.222+0000 7f77c2800640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c4076720 0x7f77c41a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.223+0000 7f77c2800640 1 -- 172.21.15.107:0/3903843702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c419efa0 con 0x7f77c4112870 2024-09-19T04:22:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.223+0000 7f77c2800640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c4112870 0x7f77c419e730 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77c407aa10 tx=0x7f77bc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.223+0000 7f77baa00640 1 -- 172.21.15.107:0/3903843702 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc00cdf0 con 0x7f77c4112870 2024-09-19T04:22:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.223+0000 7f77baa00640 1 -- 172.21.15.107:0/3903843702 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77bc005020 con 0x7f77c4112870 2024-09-19T04:22:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.223+0000 7f77baa00640 1 -- 172.21.15.107:0/3903843702 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc010670 con 0x7f77c4112870 2024-09-19T04:22:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.223+0000 7f77c3200640 1 -- 172.21.15.107:0/3903843702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c419f290 con 0x7f77c4112870 2024-09-19T04:22:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.223+0000 7f77c3200640 1 -- 172.21.15.107:0/3903843702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c41ab5e0 con 0x7f77c4112870 2024-09-19T04:22:09.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.227+0000 7f77cba73740 1 Processor -- start 2024-09-19T04:22:09.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77baa00640 1 -- 172.21.15.107:0/3903843702 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f77bc0054d0 con 0x7f77c4112870 2024-09-19T04:22:09.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77cba73740 1 -- start start 2024-09-19T04:22:09.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77cba73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 0x557a07e06130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77cba73740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77cba73740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e13320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77cba73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557a07d6c3e0 con 0x557a07e05d50 2024-09-19T04:22:09.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77cba73740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557a07d13d80 con 0x557a07e06670 2024-09-19T04:22:09.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77cba73740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557a07d683a0 con 0x557a07e10f30 2024-09-19T04:22:09.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77c1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 0x557a07e06130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77c1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 0x557a07e06130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49088/0 (socket says 172.21.15.107:49088) 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.228+0000 7f77c1400640 1 -- 172.21.15.107:0/550344512 learned_addr learned my addr 172.21.15.107:0/550344512 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77c1e00640 1 --2- 172.21.15.107:0/550344512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77c1400640 1 --2- 172.21.15.107:0/550344512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e13320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77c1e00640 1 -- 172.21.15.107:0/550344512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 msgr2=0x557a07e13320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77c1e00640 1 --2- 172.21.15.107:0/550344512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e13320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77c1e00640 1 -- 172.21.15.107:0/550344512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 msgr2=0x557a07e06130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77c1e00640 1 --2- 172.21.15.107:0/550344512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 0x557a07e06130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77c1e00640 1 -- 172.21.15.107:0/550344512 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a07d32ae0 con 0x557a07e06670 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77c1400640 1 --2- 172.21.15.107:0/550344512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77baa00640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77a80a3430 0x7f77a80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77c1e00640 1 --2- 172.21.15.107:0/550344512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e109f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009230 tx=0x7f77b4005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7962864b81353914 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.229+0000 7f77baa00640 1 -- 172.21.15.107:0/3903843702 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f77bc0d02e0 con 0x7f77c4112870 2024-09-19T04:22:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.230+0000 7f77c1400640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77a80a3430 0x7f77a80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.230+0000 7f77af400640 1 -- 172.21.15.107:0/550344512 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4040070 con 0x557a07e06670 2024-09-19T04:22:09.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.230+0000 7f77af400640 1 -- 172.21.15.107:0/550344512 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b4004030 con 0x557a07e06670 2024-09-19T04:22:09.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.230+0000 7f77af400640 1 -- 172.21.15.107:0/550344512 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4035040 con 0x557a07e06670 2024-09-19T04:22:09.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.230+0000 7f77cba73740 1 -- 172.21.15.107:0/550344512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 msgr2=0x557a07e109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.230+0000 7f77cba73740 1 --2- 172.21.15.107:0/550344512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e109f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009230 tx=0x7f77b4005dd0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.230+0000 7f77c1400640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77a80a3430 0x7f77a80a58f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x557a07e03660 tx=0x7f77b0015040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.230+0000 7f77cba73740 1 -- 172.21.15.107:0/550344512 shutdown_connections 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.231+0000 7f77cba73740 1 --2- 172.21.15.107:0/550344512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.231+0000 7f77cba73740 1 --2- 172.21.15.107:0/550344512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e109f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.231+0000 7f77cba73740 1 --2- 172.21.15.107:0/550344512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 0x557a07e06130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.231+0000 7f77cba73740 1 -- 172.21.15.107:0/550344512 >> 172.21.15.107:0/550344512 conn(0x557a07e022e0 msgr2=0x557a07e026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.231+0000 7f77cba73740 1 -- 172.21.15.107:0/550344512 shutdown_connections 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.231+0000 7f77cba73740 1 -- 172.21.15.107:0/550344512 wait complete. 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.231+0000 7f77cba73740 1 -- start start 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.231+0000 7f77cba73740 1 Processor -- start 2024-09-19T04:22:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77cba73740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e0de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77cba73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e06670 0x557a07e0e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77cba73740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e0e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77cba73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557a07e13ba0 con 0x557a07e06670 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77cba73740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557a07d270f0 con 0x557a07e05d50 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77cba73740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557a07e13d20 con 0x557a07e10f30 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77cba73740 10 client.0 _ll_register_callbacks cb 0x557a07d189e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e06670 0x557a07e0e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e0de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e06670 0x557a07e0e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49102/0 (socket says 172.21.15.107:49102) 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e0de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:52652/0 (socket says 172.21.15.107:52652) 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c1400640 1 -- 172.21.15.107:0/3237504327 learned_addr learned my addr 172.21.15.107:0/3237504327 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c2800640 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e0e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c1e00640 1 -- 172.21.15.107:0/3237504327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 msgr2=0x557a07e0e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c1e00640 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e0e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c1e00640 1 -- 172.21.15.107:0/3237504327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e06670 msgr2=0x557a07e0e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c1e00640 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e06670 0x557a07e0e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.232+0000 7f77c1e00640 1 -- 172.21.15.107:0/3237504327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a07add3c0 con 0x557a07e05d50 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.233+0000 7f77c1e00640 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e0de40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009230 tx=0x7f77b4009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10a7655e7942fb8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.233+0000 7f77a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.233+0000 7f77a3e00640 1 -- 172.21.15.107:0/3237504327 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4040050 con 0x557a07e05d50 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.233+0000 7f77cba73740 1 -- 172.21.15.107:0/3237504327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77b4004030 con 0x557a07e05d50 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.233+0000 7f77a3e00640 1 -- 172.21.15.107:0/3237504327 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f77b40054c0 con 0x557a07e05d50 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.233+0000 7f77a3e00640 1 -- 172.21.15.107:0/3237504327 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f77b4035040 con 0x557a07e05d50 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.233+0000 7f77cba73740 10 client.25610 fetch_fsmap learned FSMap version 29 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.233+0000 7f77cba73740 1 -- 172.21.15.107:0/3237504327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a07e1f880 con 0x557a07e05d50 2024-09-19T04:22:09.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.233+0000 7f77a3e00640 1 -- 172.21.15.107:0/3237504327 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b4004030 con 0x557a07e05d50 2024-09-19T04:22:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.234+0000 7f77a3e00640 1 -- 172.21.15.107:0/3237504327 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f77b404c070 con 0x557a07e05d50 2024-09-19T04:22:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.234+0000 7f77cba73740 10 client.25610 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:22:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.234+0000 7f77cba73740 10 client.25610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.234+0000 7f77cba73740 20 client.25610 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.234+0000 7f77cba73740 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557a07e20bd0 0x557a07e22fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.234+0000 7f77cba73740 4 client.25610 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.234+0000 7f77cba73740 1 -- 172.21.15.107:0/3237504327 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a07d79950 con 0x557a07e20bd0 2024-09-19T04:22:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.234+0000 7f77c1400640 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557a07e20bd0 0x557a07e22fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.235+0000 7f77c1400640 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557a07e20bd0 0x557a07e22fc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a78524938e5cfbd server_cookie=dc277e45590b82e7 in_seq=0 out_seq=0 2024-09-19T04:22:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.235+0000 7f77a3e00640 10 client.25610 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.235+0000 7f77a3e00640 1 -- 172.21.15.107:0/3237504327 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a07d79950 con 0x557a07e20bd0 2024-09-19T04:22:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.235+0000 7f77a3e00640 10 client.25610 handle_command_reply: tid=0 2024-09-19T04:22:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.235+0000 7f77cba73740 1 client.25610 shutdown 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.235+0000 7f77cba73740 1 -- 172.21.15.107:0/3237504327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 msgr2=0x557a07e0de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.235+0000 7f77cba73740 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e0de40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009230 tx=0x7f77b4009680 comp rx=0 tx=0).stop 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.236+0000 7f77cba73740 1 -- 172.21.15.107:0/3237504327 shutdown_connections 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.236+0000 7f77cba73740 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557a07e20bd0 0x557a07e22fc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.236+0000 7f77cba73740 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e0e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.236+0000 7f77cba73740 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e06670 0x557a07e0e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.236+0000 7f77a3e00640 0 client.25610 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.236+0000 7f77cba73740 1 --2- 172.21.15.107:0/3237504327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e0de40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.236+0000 7f77cba73740 1 -- 172.21.15.107:0/3237504327 >> 172.21.15.107:0/3237504327 conn(0x557a07e022e0 msgr2=0x557a07e08010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.236+0000 7f77cba73740 1 -- 172.21.15.107:0/3237504327 shutdown_connections 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.236+0000 7f77cba73740 1 -- 172.21.15.107:0/3237504327 wait complete. 2024-09-19T04:22:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.236+0000 7f77cba73740 20 client.25610 trim_cache size 0 max 16384 2024-09-19T04:22:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77cba73740 1 Processor -- start 2024-09-19T04:22:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77cba73740 1 -- start start 2024-09-19T04:22:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77cba73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 0x557a07e28650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77cba73740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e28b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77cba73740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e0b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77cba73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b3b80 con 0x557a07e05d50 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77cba73740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b56f0 con 0x557a07e06670 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77cba73740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b4250 con 0x557a07e10f30 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77c1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e28b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77c1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e28b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:52668/0 (socket says 172.21.15.107:52668) 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77c1400640 1 -- 172.21.15.107:0/3386618823 learned_addr learned my addr 172.21.15.107:0/3386618823 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.254+0000 7f77c1e00640 1 --2- 172.21.15.107:0/3386618823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 0x557a07e28650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.255+0000 7f77c2800640 1 --2- 172.21.15.107:0/3386618823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e0b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.255+0000 7f77c1400640 1 -- 172.21.15.107:0/3386618823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 msgr2=0x557a07e0b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.255+0000 7f77c1400640 1 --2- 172.21.15.107:0/3386618823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e0b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.255+0000 7f77c1400640 1 -- 172.21.15.107:0/3386618823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 msgr2=0x557a07e28650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.255+0000 7f77c1400640 1 --2- 172.21.15.107:0/3386618823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 0x557a07e28650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.255+0000 7f77c1400640 1 -- 172.21.15.107:0/3386618823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a07e02110 con 0x557a07e06670 2024-09-19T04:22:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.255+0000 7f77c1400640 1 --2- 172.21.15.107:0/3386618823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e28b90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x557a07e235f0 tx=0x7f77b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24049c03c52b8bc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.255+0000 7f77aea00640 1 -- 172.21.15.107:0/3386618823 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0004180 con 0x557a07e06670 2024-09-19T04:22:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.255+0000 7f77aea00640 1 -- 172.21.15.107:0/3386618823 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b0005500 con 0x557a07e06670 2024-09-19T04:22:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.255+0000 7f77aea00640 1 -- 172.21.15.107:0/3386618823 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b000f5a0 con 0x557a07e06670 2024-09-19T04:22:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.256+0000 7f77cba73740 1 -- 172.21.15.107:0/3386618823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 msgr2=0x557a07e28b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.256+0000 7f77cba73740 1 --2- 172.21.15.107:0/3386618823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e28b90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x557a07e235f0 tx=0x7f77b00046c0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.256+0000 7f77cba73740 1 -- 172.21.15.107:0/3386618823 shutdown_connections 2024-09-19T04:22:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.256+0000 7f77cba73740 1 --2- 172.21.15.107:0/3386618823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e10f30 0x557a07e0b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.256+0000 7f77cba73740 1 --2- 172.21.15.107:0/3386618823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e06670 0x557a07e28b90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.256+0000 7f77cba73740 1 --2- 172.21.15.107:0/3386618823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e05d50 0x557a07e28650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.256+0000 7f77cba73740 1 -- 172.21.15.107:0/3386618823 >> 172.21.15.107:0/3386618823 conn(0x557a07e022e0 msgr2=0x557a07e0fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.256+0000 7f77cba73740 1 -- 172.21.15.107:0/3386618823 shutdown_connections 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.256+0000 7f77cba73740 1 -- 172.21.15.107:0/3386618823 wait complete. 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.257+0000 7f77cba73740 1 -- start start 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.257+0000 7f77cba73740 1 Processor -- start 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.257+0000 7f77cba73740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e14ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.257+0000 7f77cba73740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e06670 0x557a07e15530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.257+0000 7f77cba73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e10f30 0x557a07e8a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.257+0000 7f77cba73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b4ff0 con 0x557a07e10f30 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.257+0000 7f77cba73740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b4920 con 0x557a07e05d50 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.257+0000 7f77cba73740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b1b60 con 0x557a07e06670 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.257+0000 7f77cba73740 10 client.0 _ll_register_callbacks cb 0x7f77b4009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e14ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e10f30 0x557a07e8a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e14ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:52676/0 (socket says 172.21.15.107:52676) 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c1e00640 1 -- 172.21.15.107:0/3993968758 learned_addr learned my addr 172.21.15.107:0/3993968758 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c1400640 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e06670 0x557a07e15530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c1e00640 1 -- 172.21.15.107:0/3993968758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e06670 msgr2=0x557a07e15530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c1e00640 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e06670 0x557a07e15530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c1e00640 1 -- 172.21.15.107:0/3993968758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e10f30 msgr2=0x557a07e8a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c1e00640 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e10f30 0x557a07e8a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c1e00640 1 -- 172.21.15.107:0/3993968758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a07e01b90 con 0x557a07e05d50 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77c1e00640 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e14ff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009a20 tx=0x7f77b40317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e914e8fb138e11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77a3e00640 1 -- 172.21.15.107:0/3993968758 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4040040 con 0x557a07e05d50 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77a3e00640 1 -- 172.21.15.107:0/3993968758 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f77b40037f0 con 0x557a07e05d50 2024-09-19T04:22:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.258+0000 7f77cba73740 1 -- 172.21.15.107:0/3993968758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77b0005500 con 0x557a07e05d50 2024-09-19T04:22:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77a3e00640 1 -- 172.21.15.107:0/3993968758 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f77b4003a10 con 0x557a07e05d50 2024-09-19T04:22:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77cba73740 10 client.25616 fetch_fsmap learned FSMap version 29 2024-09-19T04:22:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77cba73740 1 -- 172.21.15.107:0/3993968758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a07e8ba60 con 0x557a07e05d50 2024-09-19T04:22:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77a3e00640 1 -- 172.21.15.107:0/3993968758 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b0005500 con 0x557a07e05d50 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77a3e00640 1 -- 172.21.15.107:0/3993968758 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f77b405d070 con 0x557a07e05d50 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77cba73740 10 client.25616 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77cba73740 10 client.25616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77cba73740 20 client.25616 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77cba73740 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557a07e8bfe0 0x557a07e8e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77cba73740 4 client.25616 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.259+0000 7f77cba73740 1 -- 172.21.15.107:0/3993968758 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557a07d73e60 con 0x557a07e8bfe0 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77c1400640 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557a07e8bfe0 0x557a07e8e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77c1400640 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557a07e8bfe0 0x557a07e8e3d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918900d4ad095bc0 server_cookie=3ef23a0025b71f0 in_seq=0 out_seq=0 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77a3e00640 10 client.25616 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77a3e00640 1 -- 172.21.15.107:0/3993968758 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557a07d73e60 con 0x557a07e8bfe0 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77a3e00640 10 client.25616 handle_command_reply: tid=0 2024-09-19T04:22:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77cba73740 1 client.25616 shutdown 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77cba73740 1 -- 172.21.15.107:0/3993968758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 msgr2=0x557a07e14ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77cba73740 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e14ff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009a20 tx=0x7f77b40317e0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77cba73740 1 -- 172.21.15.107:0/3993968758 shutdown_connections 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77cba73740 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557a07e8bfe0 0x557a07e8e3d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77cba73740 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a07e10f30 0x557a07e8a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77cba73740 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a07e06670 0x557a07e15530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77cba73740 1 --2- 172.21.15.107:0/3993968758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a07e05d50 0x557a07e14ff0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77cba73740 1 -- 172.21.15.107:0/3993968758 >> 172.21.15.107:0/3993968758 conn(0x557a07e022e0 msgr2=0x557a07e00b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.260+0000 7f77a3e00640 0 client.25616 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.261+0000 7f77cba73740 1 -- 172.21.15.107:0/3993968758 shutdown_connections 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.261+0000 7f77cba73740 1 -- 172.21.15.107:0/3993968758 wait complete. 2024-09-19T04:22:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.261+0000 7f77cba73740 20 client.25616 trim_cache size 0 max 16384 2024-09-19T04:22:09.267 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:22:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:22:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:22:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "8c2a45b6-5930-4e24-b41e-d8e7fd8111f7": { 2024-09-19T04:22:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:22:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "8c2a45b6-5930-4e24-b41e-d8e7fd8111f7", 2024-09-19T04:22:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:22:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:22:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:22:09.267 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:22:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.261+0000 7f77c3200640 1 -- 172.21.15.107:0/3903843702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77a80a3430 msgr2=0x7f77a80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.261+0000 7f77c3200640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77a80a3430 0x7f77a80a58f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x557a07e03660 tx=0x7f77b0015040 comp rx=0 tx=0).stop 2024-09-19T04:22:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.261+0000 7f77c3200640 1 -- 172.21.15.107:0/3903843702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c4112870 msgr2=0x7f77c419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.261+0000 7f77c3200640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c4112870 0x7f77c419e730 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77c407aa10 tx=0x7f77bc00ef10 comp rx=0 tx=0).stop 2024-09-19T04:22:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.262+0000 7f77c3200640 1 -- 172.21.15.107:0/3903843702 shutdown_connections 2024-09-19T04:22:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.262+0000 7f77c3200640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77a80a3430 0x7f77a80a58f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.262+0000 7f77c3200640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c4112870 0x7f77c419e730 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.262+0000 7f77c3200640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77c4077040 0x7f77c41a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.262+0000 7f77c3200640 1 --2- 172.21.15.107:0/3903843702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c4076720 0x7f77c41a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.262+0000 7f77c3200640 1 -- 172.21.15.107:0/3903843702 >> 172.21.15.107:0/3903843702 conn(0x7f77c40ff910 msgr2=0x7f77c4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.262+0000 7f77c3200640 1 -- 172.21.15.107:0/3903843702 shutdown_connections 2024-09-19T04:22:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:09.262+0000 7f77c3200640 1 -- 172.21.15.107:0/3903843702 wait complete. 2024-09-19T04:22:09.614 INFO:tasks.workunit.client.0.smithi107.stdout: 1 70945 14.88 MB/sec warmup 27 sec latency 949.311 ms 2024-09-19T04:22:10.614 INFO:tasks.workunit.client.0.smithi107.stdout: 1 70945 14.35 MB/sec warmup 28 sec latency 1949.387 ms 2024-09-19T04:22:10.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:10 smithi115 ceph-mon[35477]: pgmap v58: 97 pgs: 97 active+clean; 32 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-09-19T04:22:10.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:10 smithi145 ceph-mon[35096]: pgmap v58: 97 pgs: 97 active+clean; 32 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-09-19T04:22:10.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:10 smithi107 ceph-mon[30659]: pgmap v58: 97 pgs: 97 active+clean; 32 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-09-19T04:22:11.614 INFO:tasks.workunit.client.0.smithi107.stdout: 1 70945 13.85 MB/sec warmup 29 sec latency 2949.479 ms 2024-09-19T04:22:11.931 INFO:tasks.cephfs.filesystem:scrub status for tag:8c2a45b6-5930-4e24-b41e-d8e7fd8111f7 - {'path': '/', 'tag': '8c2a45b6-5930-4e24-b41e-d8e7fd8111f7', 'options': 'recursive,force'} 2024-09-19T04:22:12.614 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71035 13.40 MB/sec warmup 30 sec latency 3192.866 ms 2024-09-19T04:22:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:12 smithi115 ceph-mon[35477]: pgmap v59: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-09-19T04:22:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:22:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:12 smithi145 ceph-mon[35096]: pgmap v59: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-09-19T04:22:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:22:12.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:12 smithi107 ceph-mon[30659]: pgmap v59: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-09-19T04:22:12.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:22:13.614 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71035 12.97 MB/sec warmup 31 sec latency 1709.537 ms 2024-09-19T04:22:13.678 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:13 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:22:13.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:13 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:22:13.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:13 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:22:14.614 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71035 12.56 MB/sec warmup 32 sec latency 2709.654 ms 2024-09-19T04:22:14.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:14 smithi115 ceph-mon[35477]: pgmap v60: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-09-19T04:22:14.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:14 smithi145 ceph-mon[35096]: pgmap v60: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-09-19T04:22:14.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:14 smithi107 ceph-mon[30659]: pgmap v60: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-09-19T04:22:15.614 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71035 12.18 MB/sec warmup 33 sec latency 3709.787 ms 2024-09-19T04:22:16.614 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71035 11.82 MB/sec warmup 34 sec latency 4709.898 ms 2024-09-19T04:22:16.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:16 smithi115 ceph-mon[35477]: pgmap v61: 97 pgs: 97 active+clean; 56 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.9 MiB/s wr, 193 op/s 2024-09-19T04:22:16.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:16 smithi145 ceph-mon[35096]: pgmap v61: 97 pgs: 97 active+clean; 56 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.9 MiB/s wr, 193 op/s 2024-09-19T04:22:16.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:16 smithi107 ceph-mon[30659]: pgmap v61: 97 pgs: 97 active+clean; 56 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.9 MiB/s wr, 193 op/s 2024-09-19T04:22:17.615 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71337 11.53 MB/sec warmup 35 sec latency 4956.630 ms 2024-09-19T04:22:18.615 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71337 11.21 MB/sec warmup 36 sec latency 1424.434 ms 2024-09-19T04:22:18.761 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:18 smithi107 ceph-mon[30659]: pgmap v62: 97 pgs: 97 active+clean; 56 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-19T04:22:18.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:18 smithi115 ceph-mon[35477]: pgmap v62: 97 pgs: 97 active+clean; 56 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-19T04:22:18.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:18 smithi145 ceph-mon[35096]: pgmap v62: 97 pgs: 97 active+clean; 56 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-19T04:22:19.615 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71337 10.90 MB/sec warmup 37 sec latency 2424.475 ms 2024-09-19T04:22:20.615 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71337 10.62 MB/sec warmup 38 sec latency 3424.542 ms 2024-09-19T04:22:20.774 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:20 smithi107 ceph-mon[30659]: pgmap v63: 97 pgs: 97 active+clean; 56 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-19T04:22:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:20 smithi115 ceph-mon[35477]: pgmap v63: 97 pgs: 97 active+clean; 56 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-19T04:22:20.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:20 smithi145 ceph-mon[35096]: pgmap v63: 97 pgs: 97 active+clean; 56 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-19T04:22:21.615 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71337 10.34 MB/sec warmup 39 sec latency 4424.679 ms 2024-09-19T04:22:22.615 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71876 10.10 MB/sec warmup 40 sec latency 4669.112 ms 2024-09-19T04:22:22.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:22 smithi115 ceph-mon[35477]: pgmap v64: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 3.3 MiB/s wr, 133 op/s 2024-09-19T04:22:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:22 smithi145 ceph-mon[35096]: pgmap v64: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 3.3 MiB/s wr, 133 op/s 2024-09-19T04:22:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:22 smithi107 ceph-mon[30659]: pgmap v64: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 3.3 MiB/s wr, 133 op/s 2024-09-19T04:22:23.617 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71876 9.86 MB/sec warmup 41 sec latency 1285.981 ms 2024-09-19T04:22:24.617 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71876 9.62 MB/sec warmup 42 sec latency 2286.060 ms 2024-09-19T04:22:24.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:24 smithi107 ceph-mon[30659]: pgmap v65: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.8 MiB/s wr, 71 op/s 2024-09-19T04:22:24.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:24 smithi115 ceph-mon[35477]: pgmap v65: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.8 MiB/s wr, 71 op/s 2024-09-19T04:22:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:24 smithi145 ceph-mon[35096]: pgmap v65: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.8 MiB/s wr, 71 op/s 2024-09-19T04:22:25.618 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71876 9.40 MB/sec warmup 43 sec latency 3286.110 ms 2024-09-19T04:22:26.618 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71941 9.19 MB/sec warmup 44 sec latency 3703.739 ms 2024-09-19T04:22:26.837 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:26 smithi107 ceph-mon[30659]: pgmap v66: 97 pgs: 97 active+clean; 65 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.8 MiB/s wr, 105 op/s 2024-09-19T04:22:26.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:26 smithi115 ceph-mon[35477]: pgmap v66: 97 pgs: 97 active+clean; 65 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.8 MiB/s wr, 105 op/s 2024-09-19T04:22:26.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:26 smithi145 ceph-mon[35096]: pgmap v66: 97 pgs: 97 active+clean; 65 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.8 MiB/s wr, 105 op/s 2024-09-19T04:22:27.618 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71941 8.98 MB/sec warmup 45 sec latency 1555.462 ms 2024-09-19T04:22:28.618 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71941 8.79 MB/sec warmup 46 sec latency 2555.638 ms 2024-09-19T04:22:28.845 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:28 smithi107 ceph-mon[30659]: pgmap v67: 97 pgs: 97 active+clean; 65 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.2 MiB/s wr, 87 op/s 2024-09-19T04:22:28.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:28 smithi115 ceph-mon[35477]: pgmap v67: 97 pgs: 97 active+clean; 65 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.2 MiB/s wr, 87 op/s 2024-09-19T04:22:28.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:28 smithi145 ceph-mon[35096]: pgmap v67: 97 pgs: 97 active+clean; 65 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.2 MiB/s wr, 87 op/s 2024-09-19T04:22:29.624 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71941 8.60 MB/sec warmup 47 sec latency 3561.554 ms 2024-09-19T04:22:30.624 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71941 8.42 MB/sec warmup 48 sec latency 4561.621 ms 2024-09-19T04:22:30.824 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:30 smithi107 ceph-mon[30659]: pgmap v68: 97 pgs: 97 active+clean; 65 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.2 MiB/s wr, 87 op/s 2024-09-19T04:22:30.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:30 smithi115 ceph-mon[35477]: pgmap v68: 97 pgs: 97 active+clean; 65 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.2 MiB/s wr, 87 op/s 2024-09-19T04:22:30.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:30 smithi145 ceph-mon[35096]: pgmap v68: 97 pgs: 97 active+clean; 65 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.2 MiB/s wr, 87 op/s 2024-09-19T04:22:31.624 INFO:tasks.workunit.client.0.smithi107.stdout: 1 71941 8.25 MB/sec warmup 49 sec latency 5561.659 ms 2024-09-19T04:22:32.624 INFO:tasks.workunit.client.0.smithi107.stdout: 1 73755 8.29 MB/sec warmup 50 sec latency 5920.195 ms 2024-09-19T04:22:32.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:32 smithi115 ceph-mon[35477]: pgmap v69: 97 pgs: 97 active+clean; 63 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.2 MiB/s wr, 131 op/s 2024-09-19T04:22:32.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:32 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:22:32.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:32 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:22:32.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:32 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:22:32.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:32 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:22:32.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:32 smithi145 ceph-mon[35096]: pgmap v69: 97 pgs: 97 active+clean; 63 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.2 MiB/s wr, 131 op/s 2024-09-19T04:22:32.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:32 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:22:32.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:32 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:22:32.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:32 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:22:32.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:32 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:22:32.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:32 smithi107 ceph-mon[30659]: pgmap v69: 97 pgs: 97 active+clean; 63 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.2 MiB/s wr, 131 op/s 2024-09-19T04:22:32.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:32 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:22:32.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:32 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:22:32.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:32 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:22:32.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:32 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:22:33.624 INFO:tasks.workunit.client.0.smithi107.stdout: 1 75210 8.29 MB/sec warmup 51 sec latency 43.689 ms 2024-09-19T04:22:34.625 INFO:tasks.workunit.client.0.smithi107.stdout: 1 75210 8.13 MB/sec warmup 52 sec latency 1043.757 ms 2024-09-19T04:22:34.682 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:34 smithi107 ceph-mon[30659]: pgmap v70: 97 pgs: 97 active+clean; 63 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.0 MiB/s wr, 78 op/s 2024-09-19T04:22:34.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:34 smithi115 ceph-mon[35477]: pgmap v70: 97 pgs: 97 active+clean; 63 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.0 MiB/s wr, 78 op/s 2024-09-19T04:22:34.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:34 smithi145 ceph-mon[35096]: pgmap v70: 97 pgs: 97 active+clean; 63 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.0 MiB/s wr, 78 op/s 2024-09-19T04:22:35.624 INFO:tasks.workunit.client.0.smithi107.stdout: 1 75210 7.97 MB/sec warmup 53 sec latency 2043.821 ms 2024-09-19T04:22:36.625 INFO:tasks.workunit.client.0.smithi107.stdout: 1 77151 7.97 MB/sec warmup 54 sec latency 2242.892 ms 2024-09-19T04:22:36.924 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:36 smithi107 ceph-mon[30659]: pgmap v71: 97 pgs: 97 active+clean; 62 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 2.9 MiB/s wr, 144 op/s 2024-09-19T04:22:36.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:36 smithi115 ceph-mon[35477]: pgmap v71: 97 pgs: 97 active+clean; 62 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 2.9 MiB/s wr, 144 op/s 2024-09-19T04:22:36.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:36 smithi145 ceph-mon[35096]: pgmap v71: 97 pgs: 97 active+clean; 62 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 2.9 MiB/s wr, 144 op/s 2024-09-19T04:22:37.625 INFO:tasks.workunit.client.0.smithi107.stdout: 1 78685 8.03 MB/sec warmup 55 sec latency 20.923 ms 2024-09-19T04:22:38.625 INFO:tasks.workunit.client.0.smithi107.stdout: 1 81259 8.12 MB/sec warmup 56 sec latency 9.682 ms 2024-09-19T04:22:38.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:38 smithi115 ceph-mon[35477]: pgmap v72: 97 pgs: 97 active+clean; 62 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-19T04:22:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:38 smithi107 ceph-mon[30659]: pgmap v72: 97 pgs: 97 active+clean; 62 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-19T04:22:38.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:38 smithi145 ceph-mon[35096]: pgmap v72: 97 pgs: 97 active+clean; 62 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-19T04:22:39.632 INFO:tasks.workunit.client.0.smithi107.stdout: 1 81866 8.08 MB/sec warmup 57 sec latency 674.643 ms 2024-09-19T04:22:40.632 INFO:tasks.workunit.client.0.smithi107.stdout: 1 81866 7.94 MB/sec warmup 58 sec latency 1674.711 ms 2024-09-19T04:22:40.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:40 smithi107 ceph-mon[30659]: pgmap v73: 97 pgs: 97 active+clean; 62 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-19T04:22:40.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:40 smithi115 ceph-mon[35477]: pgmap v73: 97 pgs: 97 active+clean; 62 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-19T04:22:40.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:40 smithi145 ceph-mon[35096]: pgmap v73: 97 pgs: 97 active+clean; 62 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-19T04:22:41.632 INFO:tasks.workunit.client.0.smithi107.stdout: 1 82436 7.82 MB/sec warmup 59 sec latency 2068.088 ms 2024-09-19T04:22:41.932 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:22:42.322 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:42.632 INFO:tasks.workunit.client.0.smithi107.stdout: 1 85022 7.92 MB/sec warmup 60 sec latency 11.117 ms 2024-09-19T04:22:42.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:42 smithi107 ceph-mon[30659]: pgmap v74: 97 pgs: 97 active+clean; 55 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.3 MiB/s wr, 225 op/s 2024-09-19T04:22:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.859+0000 7f5973a00640 1 -- 172.21.15.107:0/489610225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f596c102790 msgr2=0x7f596c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.859+0000 7f5973a00640 1 --2- 172.21.15.107:0/489610225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f596c102790 0x7f596c1067e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5954009900 tx=0x7f595402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.859+0000 7f5973a00640 1 -- 172.21.15.107:0/489610225 shutdown_connections 2024-09-19T04:22:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.859+0000 7f5973a00640 1 --2- 172.21.15.107:0/489610225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c106f10 0x7f596c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.859+0000 7f5973a00640 1 --2- 172.21.15.107:0/489610225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f596c102790 0x7f596c1067e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.859+0000 7f5973a00640 1 --2- 172.21.15.107:0/489610225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f596c101de0 0x7f596c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.859+0000 7f5973a00640 1 -- 172.21.15.107:0/489610225 >> 172.21.15.107:0/489610225 conn(0x7f596c0fd8c0 msgr2=0x7f596c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:42.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.859+0000 7f5973a00640 1 -- 172.21.15.107:0/489610225 shutdown_connections 2024-09-19T04:22:42.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.859+0000 7f5973a00640 1 -- 172.21.15.107:0/489610225 wait complete. 2024-09-19T04:22:42.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.860+0000 7f5973a00640 1 Processor -- start 2024-09-19T04:22:42.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.860+0000 7f5973a00640 1 -- start start 2024-09-19T04:22:42.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.860+0000 7f5973a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c101de0 0x7f596c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.860+0000 7f5973a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f596c102790 0x7f596c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.860+0000 7f5973a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f596c106f10 0x7f596c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.860+0000 7f5973a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f596c111f70 con 0x7f596c106f10 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.860+0000 7f5973a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f596c111df0 con 0x7f596c101de0 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.860+0000 7f5973a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1120f0 con 0x7f596c102790 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5971000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c101de0 0x7f596c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5963e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f596c102790 0x7f596c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5963e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f596c102790 0x7f596c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46808/0 (socket says 172.21.15.107:46808) 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5963e00640 1 -- 172.21.15.107:0/2266674137 learned_addr learned my addr 172.21.15.107:0/2266674137 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5971000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c101de0 0x7f596c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36404/0 (socket says 172.21.15.107:36404) 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5971a00640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f596c106f10 0x7f596c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5971000640 1 -- 172.21.15.107:0/2266674137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f596c102790 msgr2=0x7f596c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5971000640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f596c102790 0x7f596c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5971000640 1 -- 172.21.15.107:0/2266674137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f596c106f10 msgr2=0x7f596c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5971000640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f596c106f10 0x7f596c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5971000640 1 -- 172.21.15.107:0/2266674137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c19f5f0 con 0x7f596c101de0 2024-09-19T04:22:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.861+0000 7f5971000640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c101de0 0x7f596c19a640 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f595c005fd0 tx=0x7f595c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.862+0000 7f596a000640 1 -- 172.21.15.107:0/2266674137 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c009c30 con 0x7f596c101de0 2024-09-19T04:22:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.862+0000 7f5973a00640 1 -- 172.21.15.107:0/2266674137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c19f8e0 con 0x7f596c101de0 2024-09-19T04:22:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.862+0000 7f596a000640 1 -- 172.21.15.107:0/2266674137 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595c009dd0 con 0x7f596c101de0 2024-09-19T04:22:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.862+0000 7f596a000640 1 -- 172.21.15.107:0/2266674137 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c010460 con 0x7f596c101de0 2024-09-19T04:22:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.862+0000 7f5973a00640 1 -- 172.21.15.107:0/2266674137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1a71c0 con 0x7f596c101de0 2024-09-19T04:22:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.865+0000 7f597662a740 1 Processor -- start 2024-09-19T04:22:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.865+0000 7f596a000640 1 -- 172.21.15.107:0/2266674137 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f595c003f50 con 0x7f596c101de0 2024-09-19T04:22:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.865+0000 7f597662a740 1 -- start start 2024-09-19T04:22:42.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.865+0000 7f597662a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2eb90 0x55a449e2ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.865+0000 7f597662a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2f4b0 0x55a449e39830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.865+0000 7f597662a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e39d70 0x55a449e3c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.865+0000 7f597662a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a449d953e0 con 0x55a449e39d70 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.865+0000 7f597662a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a449d3cd80 con 0x55a449e2eb90 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.865+0000 7f597662a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a449d913a0 con 0x55a449e2f4b0 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5963e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e39d70 0x55a449e3c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5971000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2f4b0 0x55a449e39830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5971a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2eb90 0x55a449e2ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5971000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2f4b0 0x55a449e39830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46820/0 (socket says 172.21.15.107:46820) 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5963e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e39d70 0x55a449e3c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43350/0 (socket says 172.21.15.107:43350) 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5971000640 1 -- 172.21.15.107:0/1204112916 learned_addr learned my addr 172.21.15.107:0/1204112916 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5971000640 1 -- 172.21.15.107:0/1204112916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2eb90 msgr2=0x55a449e2ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:42.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5971000640 1 --2- 172.21.15.107:0/1204112916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2eb90 0x55a449e2ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5971000640 1 -- 172.21.15.107:0/1204112916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e39d70 msgr2=0x55a449e3c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:42.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5971000640 1 --2- 172.21.15.107:0/1204112916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e39d70 0x55a449e3c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5971000640 1 -- 172.21.15.107:0/1204112916 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a449d5bae0 con 0x55a449e2f4b0 2024-09-19T04:22:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f5971000640 1 --2- 172.21.15.107:0/1204112916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2f4b0 0x55a449e39830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f595c0cd240 tx=0x7f595c0d3df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae464952d7d96af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.866+0000 7f596a000640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59500a33b0 0x7f59500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.867+0000 7f596a000640 1 -- 172.21.15.107:0/2266674137 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f595c0cfbc0 con 0x7f596c101de0 2024-09-19T04:22:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.867+0000 7f5962000640 1 -- 172.21.15.107:0/1204112916 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0df020 con 0x55a449e2f4b0 2024-09-19T04:22:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.867+0000 7f5962000640 1 -- 172.21.15.107:0/1204112916 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595c09bf40 con 0x55a449e2f4b0 2024-09-19T04:22:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.867+0000 7f5962000640 1 -- 172.21.15.107:0/1204112916 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c09b8a0 con 0x55a449e2f4b0 2024-09-19T04:22:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.867+0000 7f5971a00640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59500a33b0 0x7f59500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.867+0000 7f597662a740 1 -- 172.21.15.107:0/1204112916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2f4b0 msgr2=0x55a449e39830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.867+0000 7f597662a740 1 --2- 172.21.15.107:0/1204112916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2f4b0 0x55a449e39830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f595c0cd240 tx=0x7f595c0d3df0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.868+0000 7f5971a00640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59500a33b0 0x7f59500a5870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5964005fd0 tx=0x7f5964009600 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.868+0000 7f597662a740 1 -- 172.21.15.107:0/1204112916 shutdown_connections 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.868+0000 7f597662a740 1 --2- 172.21.15.107:0/1204112916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e39d70 0x55a449e3c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.868+0000 7f597662a740 1 --2- 172.21.15.107:0/1204112916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2f4b0 0x55a449e39830 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.868+0000 7f597662a740 1 --2- 172.21.15.107:0/1204112916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2eb90 0x55a449e2ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.868+0000 7f597662a740 1 -- 172.21.15.107:0/1204112916 >> 172.21.15.107:0/1204112916 conn(0x55a449e2b120 msgr2=0x55a449e2b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.868+0000 7f597662a740 1 -- 172.21.15.107:0/1204112916 shutdown_connections 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.868+0000 7f597662a740 1 -- 172.21.15.107:0/1204112916 wait complete. 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.868+0000 7f597662a740 1 -- start start 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.868+0000 7f597662a740 1 Processor -- start 2024-09-19T04:22:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f597662a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2eb90 0x55a449e36c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f597662a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2f4b0 0x55a449e371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f597662a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e39d70 0x55a449e37700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f597662a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a449e3c9e0 con 0x55a449e2f4b0 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f597662a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a449d500f0 con 0x55a449e39d70 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f597662a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a449e3cb60 con 0x55a449e2eb90 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f597662a740 10 client.0 _ll_register_callbacks cb 0x55a449d419e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f5963e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2f4b0 0x55a449e371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f5971000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2eb90 0x55a449e36c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f5963e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2f4b0 0x55a449e371c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43362/0 (socket says 172.21.15.107:43362) 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f5963e00640 1 -- 172.21.15.107:0/2213007227 learned_addr learned my addr 172.21.15.107:0/2213007227 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.869+0000 7f5971a00640 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e39d70 0x55a449e37700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f5971000640 1 -- 172.21.15.107:0/2213007227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e39d70 msgr2=0x55a449e37700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f5971000640 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e39d70 0x55a449e37700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f5971000640 1 -- 172.21.15.107:0/2213007227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2f4b0 msgr2=0x55a449e371c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f5971000640 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2f4b0 0x55a449e371c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f5971000640 1 -- 172.21.15.107:0/2213007227 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a449b063c0 con 0x55a449e2eb90 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f5971a00640 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e39d70 0x55a449e37700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f5971000640 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2eb90 0x55a449e36c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f595c0d3270 tx=0x7f595c005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2390172d8383a36e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f595aa00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f595aa00640 1 -- 172.21.15.107:0/2213007227 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0df070 con 0x55a449e2eb90 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f595aa00640 1 -- 172.21.15.107:0/2213007227 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f595c0985b0 con 0x55a449e2eb90 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f597662a740 1 -- 172.21.15.107:0/2213007227 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595c09bf40 con 0x55a449e2eb90 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f595aa00640 1 -- 172.21.15.107:0/2213007227 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f595c020070 con 0x55a449e2eb90 2024-09-19T04:22:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f597662a740 10 client.25606 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.870+0000 7f597662a740 1 -- 172.21.15.107:0/2213007227 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a449e3c9e0 con 0x55a449e2eb90 2024-09-19T04:22:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.871+0000 7f595aa00640 1 -- 172.21.15.107:0/2213007227 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595c09bf40 con 0x55a449e2eb90 2024-09-19T04:22:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.871+0000 7f595aa00640 1 -- 172.21.15.107:0/2213007227 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f595c0cd580 con 0x55a449e2eb90 2024-09-19T04:22:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.871+0000 7f597662a740 10 client.25606 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.871+0000 7f597662a740 10 client.25606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.871+0000 7f597662a740 20 client.25606 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.871+0000 7f597662a740 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a449e49840 0x55a449e4bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.871+0000 7f597662a740 4 client.25606 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.871+0000 7f597662a740 1 -- 172.21.15.107:0/2213007227 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a449da2950 con 0x55a449e49840 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.871+0000 7f5963e00640 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a449e49840 0x55a449e4bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.872+0000 7f5963e00640 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a449e49840 0x55a449e4bc30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=975e8bef22f2688b server_cookie=ec5988f2a2f9722b in_seq=0 out_seq=0 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.872+0000 7f595aa00640 10 client.25606 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.872+0000 7f595aa00640 1 -- 172.21.15.107:0/2213007227 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a449da2950 con 0x55a449e49840 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.872+0000 7f595aa00640 10 client.25606 handle_command_reply: tid=0 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.872+0000 7f597662a740 1 client.25606 shutdown 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 1 -- 172.21.15.107:0/2213007227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2eb90 msgr2=0x55a449e36c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2eb90 0x55a449e36c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f595c0d3270 tx=0x7f595c005b90 comp rx=0 tx=0).stop 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 1 -- 172.21.15.107:0/2213007227 shutdown_connections 2024-09-19T04:22:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a449e49840 0x55a449e4bc30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e39d70 0x55a449e37700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2f4b0 0x55a449e371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 1 --2- 172.21.15.107:0/2213007227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e2eb90 0x55a449e36c80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 1 -- 172.21.15.107:0/2213007227 >> 172.21.15.107:0/2213007227 conn(0x55a449e2b120 msgr2=0x55a449e30e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:42.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f595aa00640 0 client.25606 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:42.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 1 -- 172.21.15.107:0/2213007227 shutdown_connections 2024-09-19T04:22:42.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 1 -- 172.21.15.107:0/2213007227 wait complete. 2024-09-19T04:22:42.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.873+0000 7f597662a740 20 client.25606 trim_cache size 0 max 16384 2024-09-19T04:22:42.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.890+0000 7f597662a740 1 Processor -- start 2024-09-19T04:22:42.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.890+0000 7f597662a740 1 -- start start 2024-09-19T04:22:42.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f597662a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f597662a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e2a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f597662a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e39d70 0x55a449e51480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f597662a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1af9f0 con 0x55a449e2eb90 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f597662a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1b1620 con 0x55a449e2f4b0 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f597662a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1b00c0 con 0x55a449e39d70 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5971000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5963e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e2a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5963e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e2a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36422/0 (socket says 172.21.15.107:36422) 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5963e00640 1 -- 172.21.15.107:0/2088609529 learned_addr learned my addr 172.21.15.107:0/2088609529 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5971a00640 1 --2- 172.21.15.107:0/2088609529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e39d70 0x55a449e51480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5963e00640 1 -- 172.21.15.107:0/2088609529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e39d70 msgr2=0x55a449e51480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5963e00640 1 --2- 172.21.15.107:0/2088609529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e39d70 0x55a449e51480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5963e00640 1 -- 172.21.15.107:0/2088609529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 msgr2=0x55a449e315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5963e00640 1 --2- 172.21.15.107:0/2088609529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e315f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5963e00640 1 -- 172.21.15.107:0/2088609529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a449e2f8b0 con 0x55a449e2f4b0 2024-09-19T04:22:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.891+0000 7f5971000640 1 --2- 172.21.15.107:0/2088609529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.892+0000 7f5963e00640 1 --2- 172.21.15.107:0/2088609529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e2a0b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5954009230 tx=0x7f5954005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81e10d99350589fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.892+0000 7f5961600640 1 -- 172.21.15.107:0/2088609529 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954035040 con 0x55a449e2f4b0 2024-09-19T04:22:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.892+0000 7f5961600640 1 -- 172.21.15.107:0/2088609529 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5954032a60 con 0x55a449e2f4b0 2024-09-19T04:22:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.892+0000 7f5961600640 1 -- 172.21.15.107:0/2088609529 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954004740 con 0x55a449e2f4b0 2024-09-19T04:22:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 -- 172.21.15.107:0/2088609529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 msgr2=0x55a449e2a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 --2- 172.21.15.107:0/2088609529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e2a0b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5954009230 tx=0x7f5954005a20 comp rx=0 tx=0).stop 2024-09-19T04:22:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 -- 172.21.15.107:0/2088609529 shutdown_connections 2024-09-19T04:22:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 --2- 172.21.15.107:0/2088609529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e39d70 0x55a449e51480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 --2- 172.21.15.107:0/2088609529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e2a0b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 --2- 172.21.15.107:0/2088609529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 -- 172.21.15.107:0/2088609529 >> 172.21.15.107:0/2088609529 conn(0x55a449e2b120 msgr2=0x55a449e299a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 -- 172.21.15.107:0/2088609529 shutdown_connections 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 -- 172.21.15.107:0/2088609529 wait complete. 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 -- start start 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.893+0000 7f597662a740 1 Processor -- start 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f597662a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e42500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f597662a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e3d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f597662a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e39d70 0x55a449e3dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f597662a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1b0e90 con 0x55a449e2eb90 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f597662a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1b0790 con 0x55a449e2f4b0 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f597662a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1ad9d0 con 0x55a449e39d70 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f597662a740 10 client.0 _ll_register_callbacks cb 0x7f595c0d3bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5971000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e42500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5963e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e3d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5971000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e42500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43376/0 (socket says 172.21.15.107:43376) 2024-09-19T04:22:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5971000640 1 -- 172.21.15.107:0/2576465472 learned_addr learned my addr 172.21.15.107:0/2576465472 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5963e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e3d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36438/0 (socket says 172.21.15.107:36438) 2024-09-19T04:22:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5971a00640 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e39d70 0x55a449e3dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5963e00640 1 -- 172.21.15.107:0/2576465472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e39d70 msgr2=0x55a449e3dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5963e00640 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e39d70 0x55a449e3dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5963e00640 1 -- 172.21.15.107:0/2576465472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 msgr2=0x55a449e42500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5963e00640 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e42500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.894+0000 7f5963e00640 1 -- 172.21.15.107:0/2576465472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a449e3e550 con 0x55a449e2f4b0 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.895+0000 7f5971000640 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e42500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.895+0000 7f5963e00640 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e3d830 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55a449e2bf00 tx=0x7f595402aaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef01dadd09b0cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.895+0000 7f595aa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.895+0000 7f595aa00640 1 -- 172.21.15.107:0/2576465472 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954035050 con 0x55a449e2f4b0 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.895+0000 7f595aa00640 1 -- 172.21.15.107:0/2576465472 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f595404b040 con 0x55a449e2f4b0 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.895+0000 7f597662a740 1 -- 172.21.15.107:0/2576465472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5954032a60 con 0x55a449e2f4b0 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.895+0000 7f595aa00640 1 -- 172.21.15.107:0/2576465472 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5954004040 con 0x55a449e2f4b0 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.895+0000 7f597662a740 10 client.25634 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.895+0000 7f597662a740 1 -- 172.21.15.107:0/2576465472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a449eb4470 con 0x55a449e2f4b0 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.896+0000 7f595aa00640 1 -- 172.21.15.107:0/2576465472 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5954032a60 con 0x55a449e2f4b0 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.896+0000 7f595aa00640 1 -- 172.21.15.107:0/2576465472 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5954031960 con 0x55a449e2f4b0 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.896+0000 7f597662a740 10 client.25634 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.896+0000 7f597662a740 10 client.25634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.896+0000 7f597662a740 20 client.25634 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.896+0000 7f597662a740 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a449eb4b30 0x55a449eb6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.896+0000 7f597662a740 4 client.25634 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.896+0000 7f597662a740 1 -- 172.21.15.107:0/2576465472 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a449d9ce60 con 0x55a449eb4b30 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.896+0000 7f5971000640 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a449eb4b30 0x55a449eb6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f5971000640 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a449eb4b30 0x55a449eb6f50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b437ae8a9688909 server_cookie=dbca08176a8dcf3b in_seq=0 out_seq=0 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f595aa00640 10 client.25634 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f595aa00640 1 -- 172.21.15.107:0/2576465472 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a449d9ce60 con 0x55a449eb4b30 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f595aa00640 10 client.25634 handle_command_reply: tid=0 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 client.25634 shutdown 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 -- 172.21.15.107:0/2576465472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 msgr2=0x55a449e3d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e3d830 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55a449e2bf00 tx=0x7f595402aaf0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 -- 172.21.15.107:0/2576465472 shutdown_connections 2024-09-19T04:22:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a449eb4b30 0x55a449eb6f50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a449e39d70 0x55a449e3dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a449e2f4b0 0x55a449e3d830 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 --2- 172.21.15.107:0/2576465472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a449e2eb90 0x55a449e42500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f595aa00640 0 client.25634 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 -- 172.21.15.107:0/2576465472 >> 172.21.15.107:0/2576465472 conn(0x55a449e2b120 msgr2=0x55a449e2a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 -- 172.21.15.107:0/2576465472 shutdown_connections 2024-09-19T04:22:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 1 -- 172.21.15.107:0/2576465472 wait complete. 2024-09-19T04:22:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.897+0000 7f597662a740 20 client.25634 trim_cache size 0 max 16384 2024-09-19T04:22:42.904 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:22:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:22:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:22:42.905 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:22:42.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 -- 172.21.15.107:0/2266674137 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59500a33b0 msgr2=0x7f59500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:42.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59500a33b0 0x7f59500a5870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5964005fd0 tx=0x7f5964009600 comp rx=0 tx=0).stop 2024-09-19T04:22:42.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 -- 172.21.15.107:0/2266674137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c101de0 msgr2=0x7f596c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c101de0 0x7f596c19a640 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f595c005fd0 tx=0x7f595c00dde0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 -- 172.21.15.107:0/2266674137 shutdown_connections 2024-09-19T04:22:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59500a33b0 0x7f59500a5870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f596c106f10 0x7f596c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f596c102790 0x7f596c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 --2- 172.21.15.107:0/2266674137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c101de0 0x7f596c19a640 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 -- 172.21.15.107:0/2266674137 >> 172.21.15.107:0/2266674137 conn(0x7f596c0fd8c0 msgr2=0x7f596c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 -- 172.21.15.107:0/2266674137 shutdown_connections 2024-09-19T04:22:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:42.898+0000 7f5973a00640 1 -- 172.21.15.107:0/2266674137 wait complete. 2024-09-19T04:22:42.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:42 smithi115 ceph-mon[35477]: pgmap v74: 97 pgs: 97 active+clean; 55 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.3 MiB/s wr, 225 op/s 2024-09-19T04:22:42.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:42 smithi145 ceph-mon[35096]: pgmap v74: 97 pgs: 97 active+clean; 55 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.3 MiB/s wr, 225 op/s 2024-09-19T04:22:43.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 85123 7.84 MB/sec warmup 61 sec latency 996.488 ms 2024-09-19T04:22:43.886 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:43 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:22:43.923 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:22:43.923 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:22:43.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:43 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:22:43.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:43 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:22:44.300 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:44.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 85123 7.71 MB/sec warmup 62 sec latency 1996.893 ms 2024-09-19T04:22:44.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:44 smithi107 ceph-mon[30659]: pgmap v75: 97 pgs: 97 active+clean; 55 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 2.3 MiB/s wr, 181 op/s 2024-09-19T04:22:44.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.872+0000 7fc186200640 1 -- 172.21.15.107:0/347080477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1801049c0 msgr2=0x7fc180108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.872+0000 7fc186200640 1 --2- 172.21.15.107:0/347080477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1801049c0 0x7fc180108a10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009900 tx=0x7fc16c02a400 comp rx=0 tx=0).stop 2024-09-19T04:22:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.872+0000 7fc186200640 1 -- 172.21.15.107:0/347080477 shutdown_connections 2024-09-19T04:22:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.872+0000 7fc186200640 1 --2- 172.21.15.107:0/347080477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc180109140 0x7fc180110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.872+0000 7fc186200640 1 --2- 172.21.15.107:0/347080477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1801049c0 0x7fc180108a10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.872+0000 7fc186200640 1 --2- 172.21.15.107:0/347080477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc180104010 0x7fc1801043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.872+0000 7fc186200640 1 -- 172.21.15.107:0/347080477 >> 172.21.15.107:0/347080477 conn(0x7fc1800ffa50 msgr2=0x7fc180101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.872+0000 7fc186200640 1 -- 172.21.15.107:0/347080477 shutdown_connections 2024-09-19T04:22:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.872+0000 7fc186200640 1 -- 172.21.15.107:0/347080477 wait complete. 2024-09-19T04:22:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.873+0000 7fc186200640 1 Processor -- start 2024-09-19T04:22:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.873+0000 7fc186200640 1 -- start start 2024-09-19T04:22:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.873+0000 7fc186200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc180104010 0x7fc18019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.873+0000 7fc186200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1801049c0 0x7fc18019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc186200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc180109140 0x7fc18019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc186200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc180114190 con 0x7fc180104010 2024-09-19T04:22:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc186200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc180114010 con 0x7fc180109140 2024-09-19T04:22:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc186200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc180114310 con 0x7fc1801049c0 2024-09-19T04:22:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc17f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc180104010 0x7fc18019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc17f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc180104010 0x7fc18019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43398/0 (socket says 172.21.15.107:43398) 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc17f400640 1 -- 172.21.15.107:0/2303640741 learned_addr learned my addr 172.21.15.107:0/2303640741 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc17ea00640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1801049c0 0x7fc18019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc17fe00640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc180109140 0x7fc18019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc17f400640 1 -- 172.21.15.107:0/2303640741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1801049c0 msgr2=0x7fc18019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc17f400640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1801049c0 0x7fc18019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc17f400640 1 -- 172.21.15.107:0/2303640741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc180109140 msgr2=0x7fc18019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc17f400640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc180109140 0x7fc18019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.874+0000 7fc17f400640 1 -- 172.21.15.107:0/2303640741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18019f6a0 con 0x7fc180104010 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.875+0000 7fc17f400640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc180104010 0x7fc18019a6f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc17000e9f0 tx=0x7fc17000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.875+0000 7fc177e00640 1 -- 172.21.15.107:0/2303640741 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17000cd80 con 0x7fc180104010 2024-09-19T04:22:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.875+0000 7fc177e00640 1 -- 172.21.15.107:0/2303640741 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc170010910 con 0x7fc180104010 2024-09-19T04:22:44.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.875+0000 7fc186200640 1 -- 172.21.15.107:0/2303640741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18019f990 con 0x7fc180104010 2024-09-19T04:22:44.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.875+0000 7fc177e00640 1 -- 172.21.15.107:0/2303640741 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170018990 con 0x7fc180104010 2024-09-19T04:22:44.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.875+0000 7fc186200640 1 -- 172.21.15.107:0/2303640741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1801a7270 con 0x7fc180104010 2024-09-19T04:22:44.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.876+0000 7fc186200640 1 -- 172.21.15.107:0/2303640741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1801043f0 con 0x7fc180104010 2024-09-19T04:22:44.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.878+0000 7fc177e00640 1 -- 172.21.15.107:0/2303640741 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fc170010ac0 con 0x7fc180104010 2024-09-19T04:22:44.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.879+0000 7fc177e00640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc1640a33b0 0x7fc1640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:44.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.879+0000 7fc177e00640 1 -- 172.21.15.107:0/2303640741 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fc1700cffe0 con 0x7fc180104010 2024-09-19T04:22:44.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.879+0000 7fc17ea00640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc1640a33b0 0x7fc1640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:44.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.880+0000 7fc17ea00640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc1640a33b0 0x7fc1640a5870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc16c000c00 tx=0x7fc16c005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:44.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:44.883+0000 7fc177e00640 1 -- 172.21.15.107:0/2303640741 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc17009c3c0 con 0x7fc180104010 2024-09-19T04:22:44.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:44 smithi115 ceph-mon[35477]: pgmap v75: 97 pgs: 97 active+clean; 55 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 2.3 MiB/s wr, 181 op/s 2024-09-19T04:22:44.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:44 smithi145 ceph-mon[35096]: pgmap v75: 97 pgs: 97 active+clean; 55 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 2.3 MiB/s wr, 181 op/s 2024-09-19T04:22:45.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.093+0000 7fc186200640 1 -- 172.21.15.107:0/2303640741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc18019ba50 con 0x7fc180104010 2024-09-19T04:22:45.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.094+0000 7fc177e00640 1 -- 172.21.15.107:0/2303640741 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12000 (secure 0 0 0) 0x7fc1700a06c0 con 0x7fc180104010 2024-09-19T04:22:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:22:45.096 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":31,"btime":"2024-09-19T04:22:42:592499+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:22:41.847132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:22:45.096 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 31 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.096+0000 7fc186200640 1 -- 172.21.15.107:0/2303640741 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc1640a33b0 msgr2=0x7fc1640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.096+0000 7fc186200640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc1640a33b0 0x7fc1640a5870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc16c000c00 tx=0x7fc16c005b50 comp rx=0 tx=0).stop 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.096+0000 7fc186200640 1 -- 172.21.15.107:0/2303640741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc180104010 msgr2=0x7fc18019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.096+0000 7fc186200640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc180104010 0x7fc18019a6f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc17000e9f0 tx=0x7fc17000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.097+0000 7fc186200640 1 -- 172.21.15.107:0/2303640741 shutdown_connections 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.097+0000 7fc186200640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc1640a33b0 0x7fc1640a5870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.097+0000 7fc186200640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc180109140 0x7fc18019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.097+0000 7fc186200640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1801049c0 0x7fc18019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.097+0000 7fc186200640 1 --2- 172.21.15.107:0/2303640741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc180104010 0x7fc18019a6f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.097+0000 7fc186200640 1 -- 172.21.15.107:0/2303640741 >> 172.21.15.107:0/2303640741 conn(0x7fc1800ffa50 msgr2=0x7fc180100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.097+0000 7fc186200640 1 -- 172.21.15.107:0/2303640741 shutdown_connections 2024-09-19T04:22:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:45.097+0000 7fc186200640 1 -- 172.21.15.107:0/2303640741 wait complete. 2024-09-19T04:22:45.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 85123 7.59 MB/sec warmup 63 sec latency 2997.045 ms 2024-09-19T04:22:45.845 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:45 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2303640741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:22:45.883 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:22:45.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:45 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2303640741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:22:45.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:45 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2303640741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:22:46.250 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:46.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 85123 7.47 MB/sec warmup 64 sec latency 3997.168 ms 2024-09-19T04:22:46.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:46 smithi107 ceph-mon[30659]: pgmap v76: 97 pgs: 97 active+clean; 57 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-09-19T04:22:46.875 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:46 smithi145 ceph-mon[35096]: pgmap v76: 97 pgs: 97 active+clean; 57 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-09-19T04:22:46.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.893+0000 7fc51ce00640 1 -- 172.21.15.107:0/2354446079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5181079c0 msgr2=0x7fc518107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:46.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.893+0000 7fc51ce00640 1 --2- 172.21.15.107:0/2354446079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5181079c0 0x7fc518107da0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc508009930 tx=0x7fc50802a420 comp rx=0 tx=0).stop 2024-09-19T04:22:46.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.893+0000 7fc51ce00640 1 -- 172.21.15.107:0/2354446079 shutdown_connections 2024-09-19T04:22:46.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.893+0000 7fc51ce00640 1 --2- 172.21.15.107:0/2354446079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc518102360 0x7fc51810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.893+0000 7fc51ce00640 1 --2- 172.21.15.107:0/2354446079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5181019c0 0x7fc518101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.893+0000 7fc51ce00640 1 --2- 172.21.15.107:0/2354446079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5181079c0 0x7fc518107da0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.893+0000 7fc51ce00640 1 -- 172.21.15.107:0/2354446079 >> 172.21.15.107:0/2354446079 conn(0x7fc5180fd740 msgr2=0x7fc5180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:46.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.894+0000 7fc51ce00640 1 -- 172.21.15.107:0/2354446079 shutdown_connections 2024-09-19T04:22:46.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.894+0000 7fc51ce00640 1 -- 172.21.15.107:0/2354446079 wait complete. 2024-09-19T04:22:46.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.895+0000 7fc51ce00640 1 Processor -- start 2024-09-19T04:22:46.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.895+0000 7fc51ce00640 1 -- start start 2024-09-19T04:22:46.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.895+0000 7fc51ce00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5181019c0 0x7fc51819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.895+0000 7fc51ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc518102360 0x7fc51819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.895+0000 7fc51ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5181079c0 0x7fc51819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.895+0000 7fc51ce00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc518111e10 con 0x7fc5181079c0 2024-09-19T04:22:46.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.895+0000 7fc51ce00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc518111c90 con 0x7fc5181019c0 2024-09-19T04:22:46.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.895+0000 7fc51ce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc518111f90 con 0x7fc518102360 2024-09-19T04:22:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.895+0000 7fc517400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5181019c0 0x7fc51819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.895+0000 7fc516a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc518102360 0x7fc51819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc516a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc518102360 0x7fc51819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46892/0 (socket says 172.21.15.107:46892) 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc517400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5181019c0 0x7fc51819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36480/0 (socket says 172.21.15.107:36480) 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc516a00640 1 -- 172.21.15.107:0/1895382021 learned_addr learned my addr 172.21.15.107:0/1895382021 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc517e00640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5181079c0 0x7fc51819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc517400640 1 -- 172.21.15.107:0/1895382021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc518102360 msgr2=0x7fc51819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc517400640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc518102360 0x7fc51819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc517400640 1 -- 172.21.15.107:0/1895382021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5181079c0 msgr2=0x7fc51819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc517400640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5181079c0 0x7fc51819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc517400640 1 -- 172.21.15.107:0/1895382021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51819f4a0 con 0x7fc5181019c0 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc517400640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5181019c0 0x7fc51819a4f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc50802a940 tx=0x7fc508003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.896+0000 7fc50fe00640 1 -- 172.21.15.107:0/1895382021 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50803f070 con 0x7fc5181019c0 2024-09-19T04:22:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.897+0000 7fc50fe00640 1 -- 172.21.15.107:0/1895382021 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc508031c90 con 0x7fc5181019c0 2024-09-19T04:22:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.897+0000 7fc50fe00640 1 -- 172.21.15.107:0/1895382021 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508035030 con 0x7fc5181019c0 2024-09-19T04:22:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.897+0000 7fc51ce00640 1 -- 172.21.15.107:0/1895382021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51819f730 con 0x7fc5181019c0 2024-09-19T04:22:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.897+0000 7fc51ce00640 1 -- 172.21.15.107:0/1895382021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51819fbc0 con 0x7fc5181019c0 2024-09-19T04:22:46.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc520ffb740 1 Processor -- start 2024-09-19T04:22:46.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc520ffb740 1 -- start start 2024-09-19T04:22:46.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc50fe00640 1 -- 172.21.15.107:0/1895382021 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fc508042070 con 0x7fc5181019c0 2024-09-19T04:22:46.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 0x55bad0dcd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0dda410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc520ffb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bad0d333e0 con 0x55bad0dd8020 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc520ffb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bad0cdad80 con 0x55bad0dcd760 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc520ffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bad0d2f3a0 con 0x55bad0dcce40 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc517e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 0x55bad0dcd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc517400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc517e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 0x55bad0dcd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46902/0 (socket says 172.21.15.107:46902) 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.900+0000 7fc517e00640 1 -- 172.21.15.107:0/4110462686 learned_addr learned my addr 172.21.15.107:0/4110462686 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.901+0000 7fc516a00640 1 --2- 172.21.15.107:0/4110462686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0dda410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.901+0000 7fc517e00640 1 -- 172.21.15.107:0/4110462686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 msgr2=0x55bad0dd7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:46.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.901+0000 7fc517e00640 1 --2- 172.21.15.107:0/4110462686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.901+0000 7fc517e00640 1 -- 172.21.15.107:0/4110462686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 msgr2=0x55bad0dda410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:46.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.901+0000 7fc517e00640 1 --2- 172.21.15.107:0/4110462686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0dda410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.901+0000 7fc517e00640 1 -- 172.21.15.107:0/4110462686 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad0cf9ae0 con 0x55bad0dcce40 2024-09-19T04:22:46.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.901+0000 7fc517e00640 1 --2- 172.21.15.107:0/4110462686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 0x55bad0dcd220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc510009f50 tx=0x7fc5100118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9ffb0691b31c6fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:46.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.901+0000 7fc50cc00640 1 -- 172.21.15.107:0/4110462686 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510004d20 con 0x55bad0dcce40 2024-09-19T04:22:46.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc50cc00640 1 -- 172.21.15.107:0/4110462686 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc510004050 con 0x55bad0dcce40 2024-09-19T04:22:46.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc50cc00640 1 -- 172.21.15.107:0/4110462686 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510005280 con 0x55bad0dcce40 2024-09-19T04:22:46.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc50fe00640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc4fc0a33b0 0x7fc4fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc50fe00640 1 -- 172.21.15.107:0/1895382021 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fc5080f04f0 con 0x7fc5181019c0 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc517400640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc4fc0a33b0 0x7fc4fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc520ffb740 1 -- 172.21.15.107:0/4110462686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 msgr2=0x55bad0dcd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc520ffb740 1 --2- 172.21.15.107:0/4110462686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 0x55bad0dcd220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc510009f50 tx=0x7fc5100118d0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc520ffb740 1 -- 172.21.15.107:0/4110462686 shutdown_connections 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc520ffb740 1 --2- 172.21.15.107:0/4110462686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0dda410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc520ffb740 1 --2- 172.21.15.107:0/4110462686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc520ffb740 1 --2- 172.21.15.107:0/4110462686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 0x55bad0dcd220 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.902+0000 7fc520ffb740 1 -- 172.21.15.107:0/4110462686 >> 172.21.15.107:0/4110462686 conn(0x55bad0dc93d0 msgr2=0x55bad0dc97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.903+0000 7fc520ffb740 1 -- 172.21.15.107:0/4110462686 shutdown_connections 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.903+0000 7fc520ffb740 1 -- 172.21.15.107:0/4110462686 wait complete. 2024-09-19T04:22:46.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.903+0000 7fc517400640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc4fc0a33b0 0x7fc4fc0a5870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc5080edd40 tx=0x7fc5080f60e0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.903+0000 7fc520ffb740 1 -- start start 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.903+0000 7fc520ffb740 1 Processor -- start 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcce40 0x55bad0dd4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcd760 0x55bad0dd5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0dd5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc520ffb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bad0ddabf0 con 0x55bad0dd8020 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc520ffb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bad0cee0f0 con 0x55bad0dcce40 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc520ffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bad0ddad70 con 0x55bad0dcd760 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc520ffb740 10 client.0 _ll_register_callbacks cb 0x55bad0cd5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc517e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcce40 0x55bad0dd4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc516a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcd760 0x55bad0dd5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc516a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcd760 0x55bad0dd5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46908/0 (socket says 172.21.15.107:46908) 2024-09-19T04:22:46.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc516a00640 1 -- 172.21.15.107:0/4158504250 learned_addr learned my addr 172.21.15.107:0/4158504250 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:46.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.904+0000 7fc517e00640 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0dd5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc516a00640 1 -- 172.21.15.107:0/4158504250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcce40 msgr2=0x55bad0dd4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:46.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc516a00640 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcce40 0x55bad0dd4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc516a00640 1 -- 172.21.15.107:0/4158504250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 msgr2=0x55bad0dd5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:46.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc516a00640 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0dd5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc516a00640 1 -- 172.21.15.107:0/4158504250 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad0aa43c0 con 0x55bad0dcd760 2024-09-19T04:22:46.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc517e00640 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0dd5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:46.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc516a00640 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcd760 0x55bad0dd5450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc5040127c0 tx=0x7fc504012c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4376992930ac1d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:46.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc501600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:22:46.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc501600640 1 -- 172.21.15.107:0/4158504250 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50400f920 con 0x55bad0dcd760 2024-09-19T04:22:46.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc520ffb740 1 -- 172.21.15.107:0/4158504250 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc510004050 con 0x55bad0dcd760 2024-09-19T04:22:46.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc501600640 1 -- 172.21.15.107:0/4158504250 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fc50400fac0 con 0x55bad0dcd760 2024-09-19T04:22:46.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.905+0000 7fc501600640 1 -- 172.21.15.107:0/4158504250 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc50400fcc0 con 0x55bad0dcd760 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.906+0000 7fc520ffb740 10 client.25630 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.906+0000 7fc520ffb740 1 -- 172.21.15.107:0/4158504250 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bad0de6ce0 con 0x55bad0dcd760 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.906+0000 7fc501600640 1 -- 172.21.15.107:0/4158504250 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc510004050 con 0x55bad0dcd760 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.906+0000 7fc501600640 1 -- 172.21.15.107:0/4158504250 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fc50401c500 con 0x55bad0dcd760 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.906+0000 7fc520ffb740 10 client.25630 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.906+0000 7fc520ffb740 10 client.25630 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.906+0000 7fc520ffb740 20 client.25630 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.906+0000 7fc520ffb740 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55bad0de80f0 0x55bad0dea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.906+0000 7fc520ffb740 4 client.25630 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.906+0000 7fc520ffb740 1 -- 172.21.15.107:0/4158504250 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bad0d40950 con 0x55bad0de80f0 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.907+0000 7fc517e00640 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55bad0de80f0 0x55bad0dea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.907+0000 7fc517e00640 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55bad0de80f0 0x55bad0dea510 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=621d02a334bdd900 server_cookie=74ee914b61516283 in_seq=0 out_seq=0 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.907+0000 7fc501600640 10 client.25630 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc501600640 1 -- 172.21.15.107:0/4158504250 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bad0d40950 con 0x55bad0de80f0 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc501600640 10 client.25630 handle_command_reply: tid=0 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 client.25630 shutdown 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 -- 172.21.15.107:0/4158504250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcd760 msgr2=0x55bad0dd5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcd760 0x55bad0dd5450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc5040127c0 tx=0x7fc504012c80 comp rx=0 tx=0).stop 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 -- 172.21.15.107:0/4158504250 shutdown_connections 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55bad0de80f0 0x55bad0dea510 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0dd5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcd760 0x55bad0dd5450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 --2- 172.21.15.107:0/4158504250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcce40 0x55bad0dd4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc501600640 0 client.25630 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:22:46.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 -- 172.21.15.107:0/4158504250 >> 172.21.15.107:0/4158504250 conn(0x55bad0dc93d0 msgr2=0x55bad0dcf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:46.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 -- 172.21.15.107:0/4158504250 shutdown_connections 2024-09-19T04:22:46.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.908+0000 7fc520ffb740 1 -- 172.21.15.107:0/4158504250 wait complete. 2024-09-19T04:22:46.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.909+0000 7fc520ffb740 20 client.25630 trim_cache size 0 max 16384 2024-09-19T04:22:46.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.922+0000 7fc520ffb740 1 Processor -- start 2024-09-19T04:22:46.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.922+0000 7fc520ffb740 1 -- start start 2024-09-19T04:22:46.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 0x55bad0dd37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0defde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc520ffb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5181af8a0 con 0x55bad0dd8020 2024-09-19T04:22:46.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc520ffb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc5181b14d0 con 0x55bad0dcd760 2024-09-19T04:22:46.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc520ffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc5181aff70 con 0x55bad0dcce40 2024-09-19T04:22:46.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc516a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc516a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36500/0 (socket says 172.21.15.107:36500) 2024-09-19T04:22:46.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc516a00640 1 -- 172.21.15.107:0/3491427086 learned_addr learned my addr 172.21.15.107:0/3491427086 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:46.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc517e00640 1 --2- 172.21.15.107:0/3491427086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 0x55bad0dd37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc517e00640 1 --2- 172.21.15.107:0/3491427086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0defde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc516a00640 1 -- 172.21.15.107:0/3491427086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 msgr2=0x55bad0dd37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc516a00640 1 --2- 172.21.15.107:0/3491427086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 0x55bad0dd37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc516a00640 1 -- 172.21.15.107:0/3491427086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 msgr2=0x55bad0defde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc516a00640 1 --2- 172.21.15.107:0/3491427086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0defde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc516a00640 1 -- 172.21.15.107:0/3491427086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad0dc8e70 con 0x55bad0dcd760 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.923+0000 7fc517e00640 1 --2- 172.21.15.107:0/3491427086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0defde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.924+0000 7fc516a00640 1 --2- 172.21.15.107:0/3491427086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd57e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc504001eb0 tx=0x7fc5040059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d6df87c47f3f684 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.924+0000 7fc503e00640 1 -- 172.21.15.107:0/3491427086 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50402b030 con 0x55bad0dcd760 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.924+0000 7fc503e00640 1 -- 172.21.15.107:0/3491427086 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc504005510 con 0x55bad0dcd760 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.924+0000 7fc503e00640 1 -- 172.21.15.107:0/3491427086 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50401c470 con 0x55bad0dcd760 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 -- 172.21.15.107:0/3491427086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 msgr2=0x55bad0dd57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 --2- 172.21.15.107:0/3491427086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd57e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc504001eb0 tx=0x7fc5040059b0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 -- 172.21.15.107:0/3491427086 shutdown_connections 2024-09-19T04:22:46.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 --2- 172.21.15.107:0/3491427086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dd8020 0x55bad0defde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 --2- 172.21.15.107:0/3491427086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd57e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 --2- 172.21.15.107:0/3491427086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dcce40 0x55bad0dd37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 -- 172.21.15.107:0/3491427086 >> 172.21.15.107:0/3491427086 conn(0x55bad0dc93d0 msgr2=0x55bad0dd6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:46.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 -- 172.21.15.107:0/3491427086 shutdown_connections 2024-09-19T04:22:46.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 -- 172.21.15.107:0/3491427086 wait complete. 2024-09-19T04:22:46.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 -- start start 2024-09-19T04:22:46.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.925+0000 7fc520ffb740 1 Processor -- start 2024-09-19T04:22:46.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dcce40 0x55bad0de0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc520ffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dd8020 0x55bad0dd1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc520ffb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5181b0d40 con 0x55bad0dcce40 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc520ffb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc5181b0640 con 0x55bad0dcd760 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc520ffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc5181ad880 con 0x55bad0dd8020 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc520ffb740 10 client.0 _ll_register_callbacks cb 0x7fc504009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc517e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dcce40 0x55bad0de0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc516a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc517e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dcce40 0x55bad0de0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43484/0 (socket says 172.21.15.107:43484) 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc517e00640 1 -- 172.21.15.107:0/2513054343 learned_addr learned my addr 172.21.15.107:0/2513054343 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc516a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36502/0 (socket says 172.21.15.107:36502) 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.926+0000 7fc517e00640 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dd8020 0x55bad0dd1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.927+0000 7fc516a00640 1 -- 172.21.15.107:0/2513054343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dd8020 msgr2=0x55bad0dd1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.927+0000 7fc516a00640 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dd8020 0x55bad0dd1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.927+0000 7fc516a00640 1 -- 172.21.15.107:0/2513054343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dcce40 msgr2=0x55bad0de0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.927+0000 7fc516a00640 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dcce40 0x55bad0de0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.927+0000 7fc516a00640 1 -- 172.21.15.107:0/2513054343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad0dd2570 con 0x55bad0dcd760 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.927+0000 7fc516a00640 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd1a40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55bad0ddadb0 tx=0x7fc50400f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ffa0928912d0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.927+0000 7fc501600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.927+0000 7fc501600640 1 -- 172.21.15.107:0/2513054343 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504034060 con 0x55bad0dcd760 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.927+0000 7fc501600640 1 -- 172.21.15.107:0/2513054343 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fc504026690 con 0x55bad0dcd760 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.927+0000 7fc520ffb740 1 -- 172.21.15.107:0/2513054343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc504005510 con 0x55bad0dcd760 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc501600640 1 -- 172.21.15.107:0/2513054343 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc50401caa0 con 0x55bad0dcd760 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc520ffb740 10 client.25658 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc520ffb740 1 -- 172.21.15.107:0/2513054343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bad0e52bb0 con 0x55bad0dcd760 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc501600640 1 -- 172.21.15.107:0/2513054343 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc504005510 con 0x55bad0dcd760 2024-09-19T04:22:46.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc501600640 1 -- 172.21.15.107:0/2513054343 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fc504039670 con 0x55bad0dcd760 2024-09-19T04:22:46.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc520ffb740 10 client.25658 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:46.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc520ffb740 10 client.25658 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:22:46.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc520ffb740 20 client.25658 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:46.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc520ffb740 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55bad0e531b0 0x55bad0e555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:46.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc520ffb740 4 client.25658 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:46.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.928+0000 7fc520ffb740 1 -- 172.21.15.107:0/2513054343 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bad0d3ae60 con 0x55bad0e531b0 2024-09-19T04:22:46.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.929+0000 7fc517e00640 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55bad0e531b0 0x55bad0e555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:46.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.929+0000 7fc517e00640 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55bad0e531b0 0x55bad0e555a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=261643b467155865 server_cookie=1ef64b12fb9ccde7 in_seq=0 out_seq=0 2024-09-19T04:22:46.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.929+0000 7fc501600640 10 client.25658 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:22:46.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.929+0000 7fc501600640 1 -- 172.21.15.107:0/2513054343 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bad0d3ae60 con 0x55bad0e531b0 2024-09-19T04:22:46.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.929+0000 7fc501600640 10 client.25658 handle_command_reply: tid=0 2024-09-19T04:22:46.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.929+0000 7fc520ffb740 1 client.25658 shutdown 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.929+0000 7fc520ffb740 1 -- 172.21.15.107:0/2513054343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 msgr2=0x55bad0dd1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.929+0000 7fc520ffb740 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd1a40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55bad0ddadb0 tx=0x7fc50400f850 comp rx=0 tx=0).stop 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.929+0000 7fc520ffb740 1 -- 172.21.15.107:0/2513054343 shutdown_connections 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.929+0000 7fc520ffb740 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55bad0e531b0 0x55bad0e555a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.930+0000 7fc520ffb740 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bad0dd8020 0x55bad0dd1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.930+0000 7fc520ffb740 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bad0dcd760 0x55bad0dd1a40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.930+0000 7fc520ffb740 1 --2- 172.21.15.107:0/2513054343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bad0dcce40 0x55bad0de0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.930+0000 7fc501600640 0 client.25658 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.930+0000 7fc520ffb740 1 -- 172.21.15.107:0/2513054343 >> 172.21.15.107:0/2513054343 conn(0x55bad0dc93d0 msgr2=0x55bad0dce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.930+0000 7fc520ffb740 1 -- 172.21.15.107:0/2513054343 shutdown_connections 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.930+0000 7fc520ffb740 1 -- 172.21.15.107:0/2513054343 wait complete. 2024-09-19T04:22:46.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.930+0000 7fc520ffb740 20 client.25658 trim_cache size 0 max 16384 2024-09-19T04:22:46.936 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:22:46.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 -- 172.21.15.107:0/1895382021 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc4fc0a33b0 msgr2=0x7fc4fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:46.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc4fc0a33b0 0x7fc4fc0a5870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc5080edd40 tx=0x7fc5080f60e0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 -- 172.21.15.107:0/1895382021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5181019c0 msgr2=0x7fc51819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:46.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5181019c0 0x7fc51819a4f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc50802a940 tx=0x7fc508003b20 comp rx=0 tx=0).stop 2024-09-19T04:22:46.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 -- 172.21.15.107:0/1895382021 shutdown_connections 2024-09-19T04:22:46.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc4fc0a33b0 0x7fc4fc0a5870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5181079c0 0x7fc51819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc518102360 0x7fc51819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 --2- 172.21.15.107:0/1895382021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5181019c0 0x7fc51819a4f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:46.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 -- 172.21.15.107:0/1895382021 >> 172.21.15.107:0/1895382021 conn(0x7fc5180fd740 msgr2=0x7fc51810d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:46.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 -- 172.21.15.107:0/1895382021 shutdown_connections 2024-09-19T04:22:46.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:46.931+0000 7fc51ce00640 1 -- 172.21.15.107:0/1895382021 wait complete. 2024-09-19T04:22:46.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:46 smithi115 ceph-mon[35477]: pgmap v76: 97 pgs: 97 active+clean; 57 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-09-19T04:22:47.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 85843 7.38 MB/sec warmup 65 sec latency 4220.253 ms 2024-09-19T04:22:47.891 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:22:48.262 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:48.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 85843 7.27 MB/sec warmup 66 sec latency 1058.795 ms 2024-09-19T04:22:48.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:48 smithi107 ceph-mon[30659]: pgmap v77: 97 pgs: 97 active+clean; 57 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T04:22:48.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.935+0000 7f0ceea00640 1 -- 172.21.15.107:0/2085820242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 msgr2=0x7f0ce8114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:48.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.935+0000 7f0ceea00640 1 --2- 172.21.15.107:0/2085820242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce8114f80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009ca0 tx=0x7f0ce002a950 comp rx=0 tx=0).stop 2024-09-19T04:22:48.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.935+0000 7f0ceea00640 1 -- 172.21.15.107:0/2085820242 shutdown_connections 2024-09-19T04:22:48.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.935+0000 7f0ceea00640 1 --2- 172.21.15.107:0/2085820242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce8114f80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.935+0000 7f0ceea00640 1 --2- 172.21.15.107:0/2085820242 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074eb0 0x7f0ce8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.935+0000 7f0ceea00640 1 --2- 172.21.15.107:0/2085820242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074590 0x7f0ce8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.935+0000 7f0ceea00640 1 -- 172.21.15.107:0/2085820242 >> 172.21.15.107:0/2085820242 conn(0x7f0ce80fd8c0 msgr2=0x7f0ce80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:48.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.935+0000 7f0ceea00640 1 -- 172.21.15.107:0/2085820242 shutdown_connections 2024-09-19T04:22:48.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.935+0000 7f0ceea00640 1 -- 172.21.15.107:0/2085820242 wait complete. 2024-09-19T04:22:48.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.936+0000 7f0ceea00640 1 Processor -- start 2024-09-19T04:22:48.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ceea00640 1 -- start start 2024-09-19T04:22:48.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ceea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074590 0x7f0ce819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ceea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074eb0 0x7f0ce819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ceea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce81a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ceea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81185c0 con 0x7f0ce8112b90 2024-09-19T04:22:48.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ceea00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8118440 con 0x7f0ce8074eb0 2024-09-19T04:22:48.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ceea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8118740 con 0x7f0ce8074590 2024-09-19T04:22:48.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074590 0x7f0ce819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ceca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce81a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074590 0x7f0ce819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57000/0 (socket says 172.21.15.107:57000) 2024-09-19T04:22:48.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0cdf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074eb0 0x7f0ce819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.937+0000 7f0ce7e00640 1 -- 172.21.15.107:0/1520622279 learned_addr learned my addr 172.21.15.107:0/1520622279 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:48.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.938+0000 7f0ceca00640 1 -- 172.21.15.107:0/1520622279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074590 msgr2=0x7f0ce819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:48.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.938+0000 7f0ceca00640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074590 0x7f0ce819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.938+0000 7f0ceca00640 1 -- 172.21.15.107:0/1520622279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074eb0 msgr2=0x7f0ce819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:48.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.938+0000 7f0ceca00640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074eb0 0x7f0ce819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.938+0000 7f0ceca00640 1 -- 172.21.15.107:0/1520622279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce81a3b90 con 0x7f0ce8112b90 2024-09-19T04:22:48.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.938+0000 7f0cdf400640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074eb0 0x7f0ce819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:48.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.938+0000 7f0ceca00640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce81a3460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0ce002af40 tx=0x7f0ce0003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:48.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.939+0000 7f0ce5600640 1 -- 172.21.15.107:0/1520622279 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0034040 con 0x7f0ce8112b90 2024-09-19T04:22:48.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.939+0000 7f0ce5600640 1 -- 172.21.15.107:0/1520622279 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce002f940 con 0x7f0ce8112b90 2024-09-19T04:22:48.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.939+0000 7f0ceea00640 1 -- 172.21.15.107:0/1520622279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce81a3e20 con 0x7f0ce8112b90 2024-09-19T04:22:48.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.939+0000 7f0ce5600640 1 -- 172.21.15.107:0/1520622279 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00045a0 con 0x7f0ce8112b90 2024-09-19T04:22:48.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.939+0000 7f0ceea00640 1 -- 172.21.15.107:0/1520622279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce81ab710 con 0x7f0ce8112b90 2024-09-19T04:22:48.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.942+0000 7f0cf146d740 1 Processor -- start 2024-09-19T04:22:48.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.942+0000 7f0ce5600640 1 -- 172.21.15.107:0/1520622279 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f0ce0037020 con 0x7f0ce8112b90 2024-09-19T04:22:48.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.942+0000 7f0cf146d740 1 -- start start 2024-09-19T04:22:48.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.942+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d744c70 0x55868d745050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.942+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d745590 0x55868d74f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.942+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d752240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.942+0000 7f0cf146d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55868d6ab3e0 con 0x55868d74fe50 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.942+0000 7f0cf146d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55868d652d80 con 0x55868d745590 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.942+0000 7f0cf146d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55868d6a73a0 con 0x55868d744c70 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d745590 0x55868d74f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d745590 0x55868d74f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:48106/0 (socket says 172.21.15.107:48106) 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0ce7e00640 1 -- 172.21.15.107:0/2020719099 learned_addr learned my addr 172.21.15.107:0/2020719099 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0cdf400640 1 --2- 172.21.15.107:0/2020719099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d744c70 0x55868d745050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0cdf400640 1 --2- 172.21.15.107:0/2020719099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d752240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0ce7e00640 1 -- 172.21.15.107:0/2020719099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d744c70 msgr2=0x55868d745050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/2020719099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d744c70 0x55868d745050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0ce7e00640 1 -- 172.21.15.107:0/2020719099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 msgr2=0x55868d752240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/2020719099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d752240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0ce7e00640 1 -- 172.21.15.107:0/2020719099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868d671ae0 con 0x55868d745590 2024-09-19T04:22:48.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.943+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/2020719099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d745590 0x55868d74f910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009dc0 tx=0x7f0cd8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd9453fad3dd1aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:48.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.944+0000 7f0ce5600640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ccc0a3430 0x7f0ccc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.944+0000 7f0cdd200640 1 -- 172.21.15.107:0/2020719099 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8005220 con 0x55868d745590 2024-09-19T04:22:48.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.944+0000 7f0cdd200640 1 -- 172.21.15.107:0/2020719099 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8004c80 con 0x55868d745590 2024-09-19T04:22:48.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.944+0000 7f0cdd200640 1 -- 172.21.15.107:0/2020719099 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8019910 con 0x55868d745590 2024-09-19T04:22:48.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.944+0000 7f0ce5600640 1 -- 172.21.15.107:0/1520622279 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0ce00f08a0 con 0x7f0ce8112b90 2024-09-19T04:22:48.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:48 smithi115 ceph-mon[35477]: pgmap v77: 97 pgs: 97 active+clean; 57 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T04:22:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.944+0000 7f0cdf400640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ccc0a3430 0x7f0ccc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cf146d740 1 -- 172.21.15.107:0/2020719099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d745590 msgr2=0x55868d74f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cdf400640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ccc0a3430 0x7f0ccc0a58f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55868d742580 tx=0x7f0cd0005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cf146d740 1 --2- 172.21.15.107:0/2020719099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d745590 0x55868d74f910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009dc0 tx=0x7f0cd8011810 comp rx=0 tx=0).stop 2024-09-19T04:22:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cf146d740 1 -- 172.21.15.107:0/2020719099 shutdown_connections 2024-09-19T04:22:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cf146d740 1 --2- 172.21.15.107:0/2020719099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d752240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cf146d740 1 --2- 172.21.15.107:0/2020719099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d745590 0x55868d74f910 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cf146d740 1 --2- 172.21.15.107:0/2020719099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d744c70 0x55868d745050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cf146d740 1 -- 172.21.15.107:0/2020719099 >> 172.21.15.107:0/2020719099 conn(0x55868d741200 msgr2=0x55868d7415e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cf146d740 1 -- 172.21.15.107:0/2020719099 shutdown_connections 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cf146d740 1 -- 172.21.15.107:0/2020719099 wait complete. 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.945+0000 7f0cf146d740 1 -- start start 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0cf146d740 1 Processor -- start 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d74d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d74d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0cf146d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55868d752a20 con 0x55868d74fe50 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0cf146d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55868d6660f0 con 0x55868d744c70 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0cf146d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55868d752ba0 con 0x55868d745590 2024-09-19T04:22:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0cf146d740 10 client.0 _ll_register_callbacks cb 0x55868d64dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0ceca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d74d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:48120/0 (socket says 172.21.15.107:48120) 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0ce7e00640 1 -- 172.21.15.107:0/616494746 learned_addr learned my addr 172.21.15.107:0/616494746 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.946+0000 7f0cdf400640 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d74d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.947+0000 7f0ce7e00640 1 -- 172.21.15.107:0/616494746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 msgr2=0x55868d74d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.947+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d74d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.947+0000 7f0ce7e00640 1 -- 172.21.15.107:0/616494746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 msgr2=0x55868d74d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.947+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d74d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.947+0000 7f0ce7e00640 1 -- 172.21.15.107:0/616494746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868d41c3c0 con 0x55868d744c70 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.947+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74cd60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80097b0 tx=0x7f0cd80048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfaedbd4a30d0081 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:48.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.947+0000 7f0cd5600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.947+0000 7f0cd5600640 1 -- 172.21.15.107:0/616494746 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8027070 con 0x55868d744c70 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.947+0000 7f0cd5600640 1 -- 172.21.15.107:0/616494746 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0cd8004450 con 0x55868d744c70 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.947+0000 7f0cf146d740 1 -- 172.21.15.107:0/616494746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd8004c80 con 0x55868d744c70 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cd5600640 1 -- 172.21.15.107:0/616494746 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0cd8033050 con 0x55868d744c70 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cf146d740 10 client.25670 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cf146d740 1 -- 172.21.15.107:0/616494746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55868d75ea90 con 0x55868d744c70 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cd5600640 1 -- 172.21.15.107:0/616494746 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8004c80 con 0x55868d744c70 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cd5600640 1 -- 172.21.15.107:0/616494746 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0cd80035f0 con 0x55868d744c70 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cf146d740 10 client.25670 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cf146d740 10 client.25670 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cf146d740 20 client.25670 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cf146d740 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55868d75fea0 0x55868d762290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cf146d740 4 client.25670 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.948+0000 7f0cf146d740 1 -- 172.21.15.107:0/616494746 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55868d6b8950 con 0x55868d75fea0 2024-09-19T04:22:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.949+0000 7f0cdf400640 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55868d75fea0 0x55868d762290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.949+0000 7f0cdf400640 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55868d75fea0 0x55868d762290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=226049603561fc4f server_cookie=adba67b99af8a783 in_seq=0 out_seq=0 2024-09-19T04:22:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.949+0000 7f0cd5600640 10 client.25670 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:22:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cd5600640 1 -- 172.21.15.107:0/616494746 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55868d6b8950 con 0x55868d75fea0 2024-09-19T04:22:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cd5600640 10 client.25670 handle_command_reply: tid=0 2024-09-19T04:22:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cf146d740 1 client.25670 shutdown 2024-09-19T04:22:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cf146d740 1 -- 172.21.15.107:0/616494746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 msgr2=0x55868d74cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cf146d740 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74cd60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80097b0 tx=0x7f0cd80048f0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cf146d740 1 -- 172.21.15.107:0/616494746 shutdown_connections 2024-09-19T04:22:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cf146d740 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55868d75fea0 0x55868d762290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cf146d740 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d74d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cf146d740 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d74d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cd5600640 0 client.25670 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:22:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cf146d740 1 --2- 172.21.15.107:0/616494746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74cd60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cf146d740 1 -- 172.21.15.107:0/616494746 >> 172.21.15.107:0/616494746 conn(0x55868d741200 msgr2=0x55868d746ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.950+0000 7f0cf146d740 1 -- 172.21.15.107:0/616494746 shutdown_connections 2024-09-19T04:22:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.951+0000 7f0cf146d740 1 -- 172.21.15.107:0/616494746 wait complete. 2024-09-19T04:22:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.951+0000 7f0cf146d740 20 client.25670 trim_cache size 0 max 16384 2024-09-19T04:22:48.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.964+0000 7f0cf146d740 1 Processor -- start 2024-09-19T04:22:48.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.964+0000 7f0cf146d740 1 -- start start 2024-09-19T04:22:48.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d767ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d768000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0cf146d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81b3d00 con 0x55868d74fe50 2024-09-19T04:22:48.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0cf146d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81b5930 con 0x55868d744c70 2024-09-19T04:22:48.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0cf146d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81b43d0 con 0x55868d745590 2024-09-19T04:22:48.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0ceca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d768000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0ceca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d768000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52080/0 (socket says 172.21.15.107:52080) 2024-09-19T04:22:48.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0ceca00640 1 -- 172.21.15.107:0/3240410096 learned_addr learned my addr 172.21.15.107:0/3240410096 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:48.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0cdf400640 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d767ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0ce7e00640 1 -- 172.21.15.107:0/3240410096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 msgr2=0x55868d767ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:48.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d767ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0ce7e00640 1 -- 172.21.15.107:0/3240410096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 msgr2=0x55868d768000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:48.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d768000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0ce7e00640 1 -- 172.21.15.107:0/3240410096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868d73c770 con 0x55868d744c70 2024-09-19T04:22:48.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0cdf400640 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d767ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:48.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.965+0000 7f0ceca00640 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d768000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:48.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.966+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74f090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8011850 tx=0x7f0cd8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d314884885afe776 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:48.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.966+0000 7f0cd7e00640 1 -- 172.21.15.107:0/3240410096 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8030070 con 0x55868d744c70 2024-09-19T04:22:48.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.966+0000 7f0cd7e00640 1 -- 172.21.15.107:0/3240410096 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8004960 con 0x55868d744c70 2024-09-19T04:22:48.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.966+0000 7f0cd7e00640 1 -- 172.21.15.107:0/3240410096 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8027070 con 0x55868d744c70 2024-09-19T04:22:48.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.966+0000 7f0cf146d740 1 -- 172.21.15.107:0/3240410096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 msgr2=0x55868d74f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:48.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.966+0000 7f0cf146d740 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74f090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8011850 tx=0x7f0cd8005d50 comp rx=0 tx=0).stop 2024-09-19T04:22:48.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.967+0000 7f0cf146d740 1 -- 172.21.15.107:0/3240410096 shutdown_connections 2024-09-19T04:22:48.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.967+0000 7f0cf146d740 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d74fe50 0x55868d768000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.967+0000 7f0cf146d740 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d767ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.967+0000 7f0cf146d740 1 --2- 172.21.15.107:0/3240410096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d744c70 0x55868d74f090 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.967+0000 7f0cf146d740 1 -- 172.21.15.107:0/3240410096 >> 172.21.15.107:0/3240410096 conn(0x55868d741200 msgr2=0x55868d746ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:48.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.967+0000 7f0cf146d740 1 -- 172.21.15.107:0/3240410096 shutdown_connections 2024-09-19T04:22:48.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.967+0000 7f0cf146d740 1 -- 172.21.15.107:0/3240410096 wait complete. 2024-09-19T04:22:48.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.967+0000 7f0cf146d740 1 -- start start 2024-09-19T04:22:48.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.967+0000 7f0cf146d740 1 Processor -- start 2024-09-19T04:22:48.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d744c70 0x55868d758490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d7537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0cf146d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d74fe50 0x55868d753d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0cf146d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81b51a0 con 0x55868d744c70 2024-09-19T04:22:48.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0cf146d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81b4aa0 con 0x55868d74fe50 2024-09-19T04:22:48.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0cf146d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81b1ce0 con 0x55868d745590 2024-09-19T04:22:48.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0cf146d740 10 client.0 _ll_register_callbacks cb 0x7f0cd8009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:48.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d744c70 0x55868d758490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d744c70 0x55868d758490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52086/0 (socket says 172.21.15.107:52086) 2024-09-19T04:22:48.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0ce7e00640 1 -- 172.21.15.107:0/259647753 learned_addr learned my addr 172.21.15.107:0/259647753 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:48.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0cdf400640 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d7537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0ceca00640 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d74fe50 0x55868d753d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0ce7e00640 1 -- 172.21.15.107:0/259647753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 msgr2=0x55868d7537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:48.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d7537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0ce7e00640 1 -- 172.21.15.107:0/259647753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d74fe50 msgr2=0x55868d753d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:48.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.968+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d74fe50 0x55868d753d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.969+0000 7f0ce7e00640 1 -- 172.21.15.107:0/259647753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868d754530 con 0x55868d744c70 2024-09-19T04:22:48.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.969+0000 7f0ceca00640 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d74fe50 0x55868d753d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:22:48.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.969+0000 7f0cdf400640 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d7537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:48.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.969+0000 7f0ce7e00640 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d744c70 0x55868d758490 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80059f0 tx=0x7f0cd8003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc8f52f478e9dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:48.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.969+0000 7f0cd5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:22:48.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.969+0000 7f0cd5600640 1 -- 172.21.15.107:0/259647753 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8030040 con 0x55868d744c70 2024-09-19T04:22:48.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.969+0000 7f0cd5600640 1 -- 172.21.15.107:0/259647753 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0cd8004450 con 0x55868d744c70 2024-09-19T04:22:48.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.969+0000 7f0cf146d740 1 -- 172.21.15.107:0/259647753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd8004960 con 0x55868d744c70 2024-09-19T04:22:48.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.969+0000 7f0cd5600640 1 -- 172.21.15.107:0/259647753 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0cd8044020 con 0x55868d744c70 2024-09-19T04:22:48.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.970+0000 7f0cf146d740 10 client.15618 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:48.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.970+0000 7f0cf146d740 1 -- 172.21.15.107:0/259647753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55868d7caa00 con 0x55868d744c70 2024-09-19T04:22:48.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.970+0000 7f0cd5600640 1 -- 172.21.15.107:0/259647753 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8004960 con 0x55868d744c70 2024-09-19T04:22:48.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.970+0000 7f0cd5600640 1 -- 172.21.15.107:0/259647753 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0cd8019a50 con 0x55868d744c70 2024-09-19T04:22:48.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.970+0000 7f0cf146d740 10 client.15618 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:48.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.970+0000 7f0cf146d740 10 client.15618 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:22:48.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.970+0000 7f0cf146d740 20 client.15618 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:48.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.970+0000 7f0cf146d740 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55868d7cb010 0x55868d7cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:48.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.970+0000 7f0cf146d740 4 client.15618 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:48.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.970+0000 7f0cf146d740 1 -- 172.21.15.107:0/259647753 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55868d6b2e60 con 0x55868d7cb010 2024-09-19T04:22:48.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.971+0000 7f0cdf400640 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55868d7cb010 0x55868d7cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:48.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.971+0000 7f0cdf400640 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55868d7cb010 0x55868d7cd400 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f582d512eade1e76 server_cookie=fc70da823873c64d in_seq=0 out_seq=0 2024-09-19T04:22:48.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.971+0000 7f0cd5600640 10 client.15618 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:22:48.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.971+0000 7f0cd5600640 1 -- 172.21.15.107:0/259647753 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55868d6b2e60 con 0x55868d7cb010 2024-09-19T04:22:48.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.971+0000 7f0cd5600640 10 client.15618 handle_command_reply: tid=0 2024-09-19T04:22:48.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.971+0000 7f0cf146d740 1 client.15618 shutdown 2024-09-19T04:22:48.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 1 -- 172.21.15.107:0/259647753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d744c70 msgr2=0x55868d758490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:48.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d744c70 0x55868d758490 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80059f0 tx=0x7f0cd8003d20 comp rx=0 tx=0).stop 2024-09-19T04:22:48.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 1 -- 172.21.15.107:0/259647753 shutdown_connections 2024-09-19T04:22:48.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55868d7cb010 0x55868d7cd400 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55868d74fe50 0x55868d753d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55868d745590 0x55868d7537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 1 --2- 172.21.15.107:0/259647753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55868d744c70 0x55868d758490 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 1 -- 172.21.15.107:0/259647753 >> 172.21.15.107:0/259647753 conn(0x55868d741200 msgr2=0x55868d740380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:48.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cd5600640 0 client.15618 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:22:48.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 1 -- 172.21.15.107:0/259647753 shutdown_connections 2024-09-19T04:22:48.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 1 -- 172.21.15.107:0/259647753 wait complete. 2024-09-19T04:22:48.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.972+0000 7f0cf146d740 20 client.15618 trim_cache size 0 max 16384 2024-09-19T04:22:48.992 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:22:48.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 -- 172.21.15.107:0/1520622279 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ccc0a3430 msgr2=0x7f0ccc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:48.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ccc0a3430 0x7f0ccc0a58f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55868d742580 tx=0x7f0cd0005f90 comp rx=0 tx=0).stop 2024-09-19T04:22:48.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 -- 172.21.15.107:0/1520622279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 msgr2=0x7f0ce81a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:48.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce81a3460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0ce002af40 tx=0x7f0ce0003b80 comp rx=0 tx=0).stop 2024-09-19T04:22:48.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 -- 172.21.15.107:0/1520622279 shutdown_connections 2024-09-19T04:22:48.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ccc0a3430 0x7f0ccc0a58f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce8112b90 0x7f0ce81a3460 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce8074eb0 0x7f0ce819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 --2- 172.21.15.107:0/1520622279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce8074590 0x7f0ce819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:48.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 -- 172.21.15.107:0/1520622279 >> 172.21.15.107:0/1520622279 conn(0x7f0ce80fd8c0 msgr2=0x7f0ce80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:48.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 -- 172.21.15.107:0/1520622279 shutdown_connections 2024-09-19T04:22:48.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:48.973+0000 7f0ceea00640 1 -- 172.21.15.107:0/1520622279 wait complete. 2024-09-19T04:22:48.996 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:48 smithi145 ceph-mon[35096]: pgmap v77: 97 pgs: 97 active+clean; 57 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T04:22:49.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 86586 7.24 MB/sec warmup 67 sec latency 1637.526 ms 2024-09-19T04:22:49.706 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:22:50.094 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:50.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 88998 7.33 MB/sec warmup 68 sec latency 12.405 ms 2024-09-19T04:22:50.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.725+0000 7f3995c00640 1 -- 172.21.15.107:0/2471197344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3990108f60 msgr2=0x7f3990110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:50.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.725+0000 7f3995c00640 1 --2- 172.21.15.107:0/2471197344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3990108f60 0x7f3990110ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3988009be0 tx=0x7f398802a6d0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.725+0000 7f3995c00640 1 -- 172.21.15.107:0/2471197344 shutdown_connections 2024-09-19T04:22:50.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.725+0000 7f3995c00640 1 --2- 172.21.15.107:0/2471197344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3990108f60 0x7f3990110ae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.725+0000 7f3995c00640 1 --2- 172.21.15.107:0/2471197344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39901047e0 0x7f3990108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.725+0000 7f3995c00640 1 --2- 172.21.15.107:0/2471197344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3990103e30 0x7f3990104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.725+0000 7f3995c00640 1 -- 172.21.15.107:0/2471197344 >> 172.21.15.107:0/2471197344 conn(0x7f39900ff910 msgr2=0x7f3990101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:50.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.725+0000 7f3995c00640 1 -- 172.21.15.107:0/2471197344 shutdown_connections 2024-09-19T04:22:50.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.725+0000 7f3995c00640 1 -- 172.21.15.107:0/2471197344 wait complete. 2024-09-19T04:22:50.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.726+0000 7f3995c00640 1 Processor -- start 2024-09-19T04:22:50.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f3995c00640 1 -- start start 2024-09-19T04:22:50.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f3995c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3990103e30 0x7f399019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f3995c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39901047e0 0x7f399019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f3995c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3990108f60 0x7f399019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f3995c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3990113fc0 con 0x7f3990103e30 2024-09-19T04:22:50.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f3995c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3990113e40 con 0x7f3990108f60 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f3995c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3990114140 con 0x7f39901047e0 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f398fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3990103e30 0x7f399019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f398f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39901047e0 0x7f399019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f398f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39901047e0 0x7f399019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57062/0 (socket says 172.21.15.107:57062) 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f398f400640 1 -- 172.21.15.107:0/3741251202 learned_addr learned my addr 172.21.15.107:0/3741251202 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.727+0000 7f3995200640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3990108f60 0x7f399019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f398f400640 1 -- 172.21.15.107:0/3741251202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3990108f60 msgr2=0x7f399019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f398f400640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3990108f60 0x7f399019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f398f400640 1 -- 172.21.15.107:0/3741251202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3990103e30 msgr2=0x7f399019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f398f400640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3990103e30 0x7f399019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f398f400640 1 -- 172.21.15.107:0/3741251202 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399019f3d0 con 0x7f39901047e0 2024-09-19T04:22:50.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f398fe00640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3990103e30 0x7f399019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:50.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f398f400640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39901047e0 0x7f399019a960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f397c00e9f0 tx=0x7f397c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:50.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f398cc00640 1 -- 172.21.15.107:0/3741251202 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c00ccc0 con 0x7f39901047e0 2024-09-19T04:22:50.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f3995c00640 1 -- 172.21.15.107:0/3741251202 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399019f6c0 con 0x7f39901047e0 2024-09-19T04:22:50.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f398cc00640 1 -- 172.21.15.107:0/3741251202 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f397c005500 con 0x7f39901047e0 2024-09-19T04:22:50.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f398cc00640 1 -- 172.21.15.107:0/3741251202 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c010690 con 0x7f39901047e0 2024-09-19T04:22:50.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.728+0000 7f3995c00640 1 -- 172.21.15.107:0/3741251202 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39901a6fa0 con 0x7f39901047e0 2024-09-19T04:22:50.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.732+0000 7f3999c84740 1 Processor -- start 2024-09-19T04:22:50.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.732+0000 7f3999c84740 1 -- start start 2024-09-19T04:22:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f3999c84740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 0x5616a1c28220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f3999c84740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c32ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f3999c84740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 0x5616a1c35410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f3999c84740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5616a1b8e3e0 con 0x5616a1c27e40 2024-09-19T04:22:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f3999c84740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5616a1b35d80 con 0x5616a1c33020 2024-09-19T04:22:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f3999c84740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5616a1b8a3a0 con 0x5616a1c28760 2024-09-19T04:22:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f398cc00640 1 -- 172.21.15.107:0/3741251202 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f397c0046c0 con 0x7f39901047e0 2024-09-19T04:22:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f398fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c32ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f3995200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 0x5616a1c28220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f398fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c32ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57064/0 (socket says 172.21.15.107:57064) 2024-09-19T04:22:50.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f398fe00640 1 -- 172.21.15.107:0/354872945 learned_addr learned my addr 172.21.15.107:0/354872945 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:50.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.733+0000 7f398f400640 1 --2- 172.21.15.107:0/354872945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 0x5616a1c35410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f3995200640 1 -- 172.21.15.107:0/354872945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 msgr2=0x5616a1c32ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:50.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f3995200640 1 --2- 172.21.15.107:0/354872945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c32ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f3995200640 1 -- 172.21.15.107:0/354872945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 msgr2=0x5616a1c35410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:50.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f3995200640 1 --2- 172.21.15.107:0/354872945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 0x5616a1c35410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f3995200640 1 -- 172.21.15.107:0/354872945 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616a1b54ae0 con 0x5616a1c27e40 2024-09-19T04:22:50.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f398f400640 1 --2- 172.21.15.107:0/354872945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 0x5616a1c35410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:50.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f398fe00640 1 --2- 172.21.15.107:0/354872945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c32ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f3995200640 1 --2- 172.21.15.107:0/354872945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 0x5616a1c28220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3988009b30 tx=0x7f3988030a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896cecd0f6adbfec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f3985600640 1 -- 172.21.15.107:0/354872945 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988041030 con 0x5616a1c27e40 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f3985600640 1 -- 172.21.15.107:0/354872945 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3988030c60 con 0x5616a1c27e40 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.734+0000 7f3985600640 1 -- 172.21.15.107:0/354872945 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988036040 con 0x5616a1c27e40 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f398cc00640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39740a3430 0x7f39740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f398cc00640 1 -- 172.21.15.107:0/3741251202 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f397c0cfea0 con 0x7f39901047e0 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f398fe00640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39740a3430 0x7f39740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f3999c84740 1 -- 172.21.15.107:0/354872945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 msgr2=0x5616a1c28220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f3999c84740 1 --2- 172.21.15.107:0/354872945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 0x5616a1c28220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3988009b30 tx=0x7f3988030a30 comp rx=0 tx=0).stop 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f3999c84740 1 -- 172.21.15.107:0/354872945 shutdown_connections 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f3999c84740 1 --2- 172.21.15.107:0/354872945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 0x5616a1c35410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f3999c84740 1 --2- 172.21.15.107:0/354872945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c32ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f3999c84740 1 --2- 172.21.15.107:0/354872945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 0x5616a1c28220 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f3999c84740 1 -- 172.21.15.107:0/354872945 >> 172.21.15.107:0/354872945 conn(0x5616a1c243d0 msgr2=0x5616a1c247b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:50.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.735+0000 7f3999c84740 1 -- 172.21.15.107:0/354872945 shutdown_connections 2024-09-19T04:22:50.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.736+0000 7f3999c84740 1 -- 172.21.15.107:0/354872945 wait complete. 2024-09-19T04:22:50.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.736+0000 7f398fe00640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39740a3430 0x7f39740a58f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5616a1c25230 tx=0x7f3980005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:50.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.736+0000 7f3999c84740 1 -- start start 2024-09-19T04:22:50.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.736+0000 7f3999c84740 1 Processor -- start 2024-09-19T04:22:50.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3999c84740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 0x5616a1c2fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3999c84740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c30400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3999c84740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 0x5616a1c30940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3999c84740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5616a1c35bf0 con 0x5616a1c27e40 2024-09-19T04:22:50.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3999c84740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5616a1b490f0 con 0x5616a1c33020 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3999c84740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5616a1c35d70 con 0x5616a1c28760 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3999c84740 10 client.0 _ll_register_callbacks cb 0x5616a1b30bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f398f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c30400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f398fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 0x5616a1c2fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f398f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c30400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57066/0 (socket says 172.21.15.107:57066) 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f398f400640 1 -- 172.21.15.107:0/787161848 learned_addr learned my addr 172.21.15.107:0/787161848 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3995200640 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 0x5616a1c30940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3995200640 1 -- 172.21.15.107:0/787161848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 msgr2=0x5616a1c30400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3995200640 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c30400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3995200640 1 -- 172.21.15.107:0/787161848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 msgr2=0x5616a1c2fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:50.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.737+0000 7f3995200640 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 0x5616a1c2fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.738+0000 7f3995200640 1 -- 172.21.15.107:0/787161848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616a18ff3c0 con 0x5616a1c33020 2024-09-19T04:22:50.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.738+0000 7f398f400640 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c30400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:50.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.738+0000 7f3995200640 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 0x5616a1c30940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3988009b30 tx=0x7f39880098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c80a0fe6f96349 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:50.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.738+0000 7f397a000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.738+0000 7f397a000640 1 -- 172.21.15.107:0/787161848 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988036050 con 0x5616a1c33020 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.738+0000 7f3999c84740 1 -- 172.21.15.107:0/787161848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3988030c60 con 0x5616a1c33020 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.738+0000 7f397a000640 1 -- 172.21.15.107:0/787161848 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f3988003b70 con 0x5616a1c33020 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.738+0000 7f397a000640 1 -- 172.21.15.107:0/787161848 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3988041040 con 0x5616a1c33020 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f3999c84740 10 client.25685 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f3999c84740 1 -- 172.21.15.107:0/787161848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5616a1c41be0 con 0x5616a1c33020 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f397a000640 1 -- 172.21.15.107:0/787161848 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3988030c60 con 0x5616a1c33020 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f397a000640 1 -- 172.21.15.107:0/787161848 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f398803dbd0 con 0x5616a1c33020 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f3999c84740 10 client.25685 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f3999c84740 10 client.25685 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f3999c84740 20 client.25685 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f3999c84740 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5616a1c42f30 0x5616a1c45320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f3999c84740 4 client.25685 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f3999c84740 1 -- 172.21.15.107:0/787161848 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616a1b9b950 con 0x5616a1c42f30 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.739+0000 7f398fe00640 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5616a1c42f30 0x5616a1c45320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.740+0000 7f398fe00640 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5616a1c42f30 0x5616a1c45320 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41b2dcdf04bb3f73 server_cookie=cb51addd9197fb92 in_seq=0 out_seq=0 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.740+0000 7f397a000640 10 client.25685 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.740+0000 7f397a000640 1 -- 172.21.15.107:0/787161848 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616a1b9b950 con 0x5616a1c42f30 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.740+0000 7f397a000640 10 client.25685 handle_command_reply: tid=0 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 client.25685 shutdown 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 -- 172.21.15.107:0/787161848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 msgr2=0x5616a1c30940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 0x5616a1c30940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3988009b30 tx=0x7f39880098b0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 -- 172.21.15.107:0/787161848 shutdown_connections 2024-09-19T04:22:50.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5616a1c42f30 0x5616a1c45320 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c33020 0x5616a1c30940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f397a000640 0 client.25685 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:22:50.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c30400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 --2- 172.21.15.107:0/787161848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c27e40 0x5616a1c2fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 -- 172.21.15.107:0/787161848 >> 172.21.15.107:0/787161848 conn(0x5616a1c243d0 msgr2=0x5616a1c2a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:50.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 -- 172.21.15.107:0/787161848 shutdown_connections 2024-09-19T04:22:50.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 1 -- 172.21.15.107:0/787161848 wait complete. 2024-09-19T04:22:50.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.741+0000 7f3999c84740 20 client.25685 trim_cache size 0 max 16384 2024-09-19T04:22:50.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.755+0000 7f3999c84740 1 Processor -- start 2024-09-19T04:22:50.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.755+0000 7f3999c84740 1 -- start start 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.755+0000 7f3999c84740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c2d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.755+0000 7f3999c84740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c28760 0x5616a1c2d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.755+0000 7f3999c84740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c33020 0x5616a1c3dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.755+0000 7f3999c84740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39901af6a0 con 0x5616a1c28760 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.755+0000 7f3999c84740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39901b12d0 con 0x5616a1c27e40 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.755+0000 7f3999c84740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39901afd70 con 0x5616a1c33020 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c2d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c28760 0x5616a1c2d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f3995200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c33020 0x5616a1c3dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c2d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:48180/0 (socket says 172.21.15.107:48180) 2024-09-19T04:22:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398fe00640 1 -- 172.21.15.107:0/475935110 learned_addr learned my addr 172.21.15.107:0/475935110 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:50.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398fe00640 1 -- 172.21.15.107:0/475935110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c33020 msgr2=0x5616a1c3dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:50.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398fe00640 1 --2- 172.21.15.107:0/475935110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c33020 0x5616a1c3dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398fe00640 1 -- 172.21.15.107:0/475935110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c28760 msgr2=0x5616a1c2d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:50.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398fe00640 1 --2- 172.21.15.107:0/475935110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c28760 0x5616a1c2d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398fe00640 1 -- 172.21.15.107:0/475935110 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616a1c27c70 con 0x5616a1c27e40 2024-09-19T04:22:50.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398f400640 1 --2- 172.21.15.107:0/475935110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c28760 0x5616a1c2d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:50.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f398fe00640 1 --2- 172.21.15.107:0/475935110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c2d120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3980009fd0 tx=0x7f398001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb7ccebf8fe395f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:50.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.756+0000 7f3984c00640 1 -- 172.21.15.107:0/475935110 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980005020 con 0x5616a1c27e40 2024-09-19T04:22:50.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3984c00640 1 -- 172.21.15.107:0/475935110 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39800047a0 con 0x5616a1c27e40 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3984c00640 1 -- 172.21.15.107:0/475935110 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398002b740 con 0x5616a1c27e40 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3999c84740 1 -- 172.21.15.107:0/475935110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 msgr2=0x5616a1c2d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3999c84740 1 --2- 172.21.15.107:0/475935110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c2d120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3980009fd0 tx=0x7f398001b000 comp rx=0 tx=0).stop 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3999c84740 1 -- 172.21.15.107:0/475935110 shutdown_connections 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3999c84740 1 --2- 172.21.15.107:0/475935110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c33020 0x5616a1c3dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3999c84740 1 --2- 172.21.15.107:0/475935110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c28760 0x5616a1c2d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3999c84740 1 --2- 172.21.15.107:0/475935110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c2d120 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3999c84740 1 -- 172.21.15.107:0/475935110 >> 172.21.15.107:0/475935110 conn(0x5616a1c243d0 msgr2=0x5616a1c2a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3999c84740 1 -- 172.21.15.107:0/475935110 shutdown_connections 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.757+0000 7f3999c84740 1 -- 172.21.15.107:0/475935110 wait complete. 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.758+0000 7f3999c84740 1 -- start start 2024-09-19T04:22:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.758+0000 7f3999c84740 1 Processor -- start 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f3999c84740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c48840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f3999c84740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c45b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f3999c84740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c33020 0x5616a1c460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f3999c84740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39901b0b40 con 0x5616a1c33020 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f3999c84740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39901b0440 con 0x5616a1c27e40 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f3999c84740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39901ad680 con 0x5616a1c28760 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f3999c84740 10 client.0 _ll_register_callbacks cb 0x7f397c0d3820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f398fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c48840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f398f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c45b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f398fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c48840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:48186/0 (socket says 172.21.15.107:48186) 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f398fe00640 1 -- 172.21.15.107:0/3343640793 learned_addr learned my addr 172.21.15.107:0/3343640793 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:50.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f3995200640 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c33020 0x5616a1c460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f398f400640 1 -- 172.21.15.107:0/3343640793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 msgr2=0x5616a1c48840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f398f400640 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c48840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f398f400640 1 -- 172.21.15.107:0/3343640793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c33020 msgr2=0x5616a1c460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f398f400640 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c33020 0x5616a1c460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f398f400640 1 -- 172.21.15.107:0/3343640793 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616a1c25060 con 0x5616a1c28760 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f398fe00640 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c48840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.759+0000 7f3995200640 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c33020 0x5616a1c460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f398f400640 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c45b90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f397c0cd640 tx=0x7f397c0d3570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22cdd004e0f115cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f397a000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f397a000640 1 -- 172.21.15.107:0/3343640793 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0056b0 con 0x5616a1c28760 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f397a000640 1 -- 172.21.15.107:0/3343640793 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f397c0a4950 con 0x5616a1c28760 2024-09-19T04:22:50.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f3999c84740 1 -- 172.21.15.107:0/3343640793 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39800047a0 con 0x5616a1c28760 2024-09-19T04:22:50.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f397a000640 1 -- 172.21.15.107:0/3343640793 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f397c0cda60 con 0x5616a1c28760 2024-09-19T04:22:50.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f3999c84740 10 client.25672 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:50.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f3999c84740 1 -- 172.21.15.107:0/3343640793 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5616a1cad490 con 0x5616a1c28760 2024-09-19T04:22:50.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f397a000640 1 -- 172.21.15.107:0/3343640793 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39800047a0 con 0x5616a1c28760 2024-09-19T04:22:50.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f397a000640 1 -- 172.21.15.107:0/3343640793 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f397c096ce0 con 0x5616a1c28760 2024-09-19T04:22:50.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f3999c84740 10 client.25672 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:50.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f3999c84740 10 client.25672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.760+0000 7f3999c84740 20 client.25672 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.761+0000 7f3999c84740 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5616a1c42f30 0x5616a1cafc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.761+0000 7f3999c84740 4 client.25672 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.761+0000 7f3999c84740 1 -- 172.21.15.107:0/3343640793 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616a1b95e60 con 0x5616a1c42f30 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.761+0000 7f398fe00640 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5616a1c42f30 0x5616a1cafc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.761+0000 7f398fe00640 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5616a1c42f30 0x5616a1cafc50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d4410d1ca4abf90 server_cookie=f9b16bddf781e95 in_seq=0 out_seq=0 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.761+0000 7f397a000640 10 client.25672 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f397a000640 1 -- 172.21.15.107:0/3343640793 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616a1b95e60 con 0x5616a1c42f30 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f397a000640 10 client.25672 handle_command_reply: tid=0 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 client.25672 shutdown 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 -- 172.21.15.107:0/3343640793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 msgr2=0x5616a1c45b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c45b90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f397c0cd640 tx=0x7f397c0d3570 comp rx=0 tx=0).stop 2024-09-19T04:22:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 -- 172.21.15.107:0/3343640793 shutdown_connections 2024-09-19T04:22:50.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5616a1c42f30 0x5616a1cafc50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616a1c33020 0x5616a1c460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616a1c28760 0x5616a1c45b90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 --2- 172.21.15.107:0/3343640793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616a1c27e40 0x5616a1c48840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f397a000640 0 client.25672 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:22:50.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 -- 172.21.15.107:0/3343640793 >> 172.21.15.107:0/3343640793 conn(0x5616a1c243d0 msgr2=0x5616a1c3dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:50.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 -- 172.21.15.107:0/3343640793 shutdown_connections 2024-09-19T04:22:50.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 1 -- 172.21.15.107:0/3343640793 wait complete. 2024-09-19T04:22:50.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.762+0000 7f3999c84740 20 client.25672 trim_cache size 0 max 16384 2024-09-19T04:22:50.768 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:22:50.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 -- 172.21.15.107:0/3741251202 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39740a3430 msgr2=0x7f39740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39740a3430 0x7f39740a58f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5616a1c25230 tx=0x7f3980005f90 comp rx=0 tx=0).stop 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 -- 172.21.15.107:0/3741251202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39901047e0 msgr2=0x7f399019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39901047e0 0x7f399019a960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f397c00e9f0 tx=0x7f397c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 -- 172.21.15.107:0/3741251202 shutdown_connections 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39740a3430 0x7f39740a58f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3990108f60 0x7f399019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39901047e0 0x7f399019a960 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 --2- 172.21.15.107:0/3741251202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3990103e30 0x7f399019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 -- 172.21.15.107:0/3741251202 >> 172.21.15.107:0/3741251202 conn(0x7f39900ff910 msgr2=0x7f3990100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 -- 172.21.15.107:0/3741251202 shutdown_connections 2024-09-19T04:22:50.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:50.763+0000 7f3995c00640 1 -- 172.21.15.107:0/3741251202 wait complete. 2024-09-19T04:22:50.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:50 smithi115 ceph-mon[35477]: pgmap v78: 97 pgs: 97 active+clean; 57 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T04:22:50.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:50 smithi145 ceph-mon[35096]: pgmap v78: 97 pgs: 97 active+clean; 57 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T04:22:50.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:50 smithi107 ceph-mon[30659]: pgmap v78: 97 pgs: 97 active+clean; 57 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T04:22:51.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 91051 7.33 MB/sec warmup 69 sec latency 9.772 ms 2024-09-19T04:22:51.638 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:22:52.028 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:52.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.598+0000 7f784a400640 1 -- 172.21.15.107:0/725656728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7844106f00 msgr2=0x7f784410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:52.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.598+0000 7f784a400640 1 --2- 172.21.15.107:0/725656728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7844106f00 0x7f784410ea80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f783c009ca0 tx=0x7f783c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.599+0000 7f784a400640 1 -- 172.21.15.107:0/725656728 shutdown_connections 2024-09-19T04:22:52.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.599+0000 7f784a400640 1 --2- 172.21.15.107:0/725656728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7844106f00 0x7f784410ea80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.599+0000 7f784a400640 1 --2- 172.21.15.107:0/725656728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7844102780 0x7f78441067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.599+0000 7f784a400640 1 --2- 172.21.15.107:0/725656728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844101dd0 0x7f78441021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.599+0000 7f784a400640 1 -- 172.21.15.107:0/725656728 >> 172.21.15.107:0/725656728 conn(0x7f78440fd8b0 msgr2=0x7f78440ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:52.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.599+0000 7f784a400640 1 -- 172.21.15.107:0/725656728 shutdown_connections 2024-09-19T04:22:52.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.599+0000 7f784a400640 1 -- 172.21.15.107:0/725656728 wait complete. 2024-09-19T04:22:52.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.599+0000 7f784a400640 1 Processor -- start 2024-09-19T04:22:52.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f784a400640 1 -- start start 2024-09-19T04:22:52.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f784a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7844101dd0 0x7f784419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f784a400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102780 0x7f784419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f784a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7844106f00 0x7f784419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f784a400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7844111f60 con 0x7f7844106f00 2024-09-19T04:22:52.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f784a400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7844111de0 con 0x7f7844102780 2024-09-19T04:22:52.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f784a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78441120e0 con 0x7f7844101dd0 2024-09-19T04:22:52.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f7843400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7844101dd0 0x7f784419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f7843e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7844106f00 0x7f784419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f7843400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7844101dd0 0x7f784419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57116/0 (socket says 172.21.15.107:57116) 2024-09-19T04:22:52.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.600+0000 7f7843400640 1 -- 172.21.15.107:0/541644652 learned_addr learned my addr 172.21.15.107:0/541644652 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:52.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f7843400640 1 -- 172.21.15.107:0/541644652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102780 msgr2=0x7f784419abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:22:52.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f7843400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102780 0x7f784419abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f7843400640 1 -- 172.21.15.107:0/541644652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7844106f00 msgr2=0x7f784419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:52.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f7843400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7844106f00 0x7f784419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f7843400640 1 -- 172.21.15.107:0/541644652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784419f630 con 0x7f7844101dd0 2024-09-19T04:22:52.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f7843400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7844101dd0 0x7f784419a680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f783400d950 tx=0x7f783400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f783be00640 1 -- 172.21.15.107:0/541644652 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834017070 con 0x7f7844101dd0 2024-09-19T04:22:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f783be00640 1 -- 172.21.15.107:0/541644652 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7834009c30 con 0x7f7844101dd0 2024-09-19T04:22:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f784a400640 1 -- 172.21.15.107:0/541644652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784419f920 con 0x7f7844101dd0 2024-09-19T04:22:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f783be00640 1 -- 172.21.15.107:0/541644652 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783400f6b0 con 0x7f7844101dd0 2024-09-19T04:22:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.601+0000 7f784a400640 1 -- 172.21.15.107:0/541644652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441a7200 con 0x7f7844101dd0 2024-09-19T04:22:52.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f784d000740 1 Processor -- start 2024-09-19T04:22:52.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f783be00640 1 -- 172.21.15.107:0/541644652 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f78340109d0 con 0x7f7844101dd0 2024-09-19T04:22:52.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f784d000740 1 -- start start 2024-09-19T04:22:52.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f784d000740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 0x55589fab8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f784d000740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fab8590 0x55589fac2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f784d000740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fac2e50 0x55589fac5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f784d000740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55589fa1e3e0 con 0x55589fac2e50 2024-09-19T04:22:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f784d000740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55589f9c5d80 con 0x55589fab8590 2024-09-19T04:22:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f784d000740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55589fa1a3a0 con 0x55589fab7c70 2024-09-19T04:22:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f7843e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 0x55589fab8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.606+0000 7f7843400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fab8590 0x55589fac2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.607+0000 7f7843e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 0x55589fab8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57120/0 (socket says 172.21.15.107:57120) 2024-09-19T04:22:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.607+0000 7f7843e00640 1 -- 172.21.15.107:0/3696430529 learned_addr learned my addr 172.21.15.107:0/3696430529 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.607+0000 7f7842a00640 1 --2- 172.21.15.107:0/3696430529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fac2e50 0x55589fac5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.607+0000 7f7843e00640 1 -- 172.21.15.107:0/3696430529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fab8590 msgr2=0x55589fac2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.607+0000 7f7843e00640 1 --2- 172.21.15.107:0/3696430529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fab8590 0x55589fac2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.607+0000 7f7843e00640 1 -- 172.21.15.107:0/3696430529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fac2e50 msgr2=0x55589fac5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.607+0000 7f7843e00640 1 --2- 172.21.15.107:0/3696430529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fac2e50 0x55589fac5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.607+0000 7f7843e00640 1 -- 172.21.15.107:0/3696430529 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589f9e4ae0 con 0x55589fab7c70 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.607+0000 7f7843e00640 1 --2- 172.21.15.107:0/3696430529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 0x55589fab8050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f783c009ca0 tx=0x7f783c009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec516cf07a4c538f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.607+0000 7f783be00640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f78280a3430 0x7f78280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.608+0000 7f7838c00640 1 -- 172.21.15.107:0/3696430529 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c031db0 con 0x55589fab7c70 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.608+0000 7f7838c00640 1 -- 172.21.15.107:0/3696430529 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783c036040 con 0x55589fab7c70 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.608+0000 7f7838c00640 1 -- 172.21.15.107:0/3696430529 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c0043c0 con 0x55589fab7c70 2024-09-19T04:22:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.608+0000 7f783be00640 1 -- 172.21.15.107:0/541644652 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f78340cf8c0 con 0x7f7844101dd0 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.608+0000 7f7843400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f78280a3430 0x7f78280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.608+0000 7f7843400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f78280a3430 0x7f78280a58f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78340cd110 tx=0x7f78340d5550 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.608+0000 7f784d000740 1 -- 172.21.15.107:0/3696430529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 msgr2=0x55589fab8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.608+0000 7f784d000740 1 --2- 172.21.15.107:0/3696430529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 0x55589fab8050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f783c009ca0 tx=0x7f783c009800 comp rx=0 tx=0).stop 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.609+0000 7f784d000740 1 -- 172.21.15.107:0/3696430529 shutdown_connections 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.609+0000 7f784d000740 1 --2- 172.21.15.107:0/3696430529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fac2e50 0x55589fac5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.609+0000 7f784d000740 1 --2- 172.21.15.107:0/3696430529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fab8590 0x55589fac2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.609+0000 7f784d000740 1 --2- 172.21.15.107:0/3696430529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 0x55589fab8050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.609+0000 7f784d000740 1 -- 172.21.15.107:0/3696430529 >> 172.21.15.107:0/3696430529 conn(0x55589fab4200 msgr2=0x55589fab45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.609+0000 7f784d000740 1 -- 172.21.15.107:0/3696430529 shutdown_connections 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.609+0000 7f784d000740 1 -- 172.21.15.107:0/3696430529 wait complete. 2024-09-19T04:22:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.609+0000 7f784d000740 1 -- start start 2024-09-19T04:22:52.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.609+0000 7f784d000740 1 Processor -- start 2024-09-19T04:22:52.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f784d000740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fab7c70 0x55589fabfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f784d000740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fac0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f784d000740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fac2e50 0x55589fac0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f784d000740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55589fac58a0 con 0x55589fac2e50 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f784d000740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55589f9d90f0 con 0x55589fab7c70 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f784d000740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55589fac5a20 con 0x55589fab8590 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f784d000740 10 client.0 _ll_register_callbacks cb 0x55589f9c0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f7842a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fac0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f7842a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fac0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57126/0 (socket says 172.21.15.107:57126) 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f7842a00640 1 -- 172.21.15.107:0/1337950082 learned_addr learned my addr 172.21.15.107:0/1337950082 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f7843e00640 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fab7c70 0x55589fabfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.610+0000 7f7843e00640 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fac2e50 0x55589fac0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f7842a00640 1 -- 172.21.15.107:0/1337950082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fab7c70 msgr2=0x55589fabfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:52.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f7842a00640 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fab7c70 0x55589fabfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f7842a00640 1 -- 172.21.15.107:0/1337950082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fac2e50 msgr2=0x55589fac0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:52.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f7842a00640 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fac2e50 0x55589fac0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f7842a00640 1 -- 172.21.15.107:0/1337950082 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589f78f3c0 con 0x55589fab8590 2024-09-19T04:22:52.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f7842a00640 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fac0240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7830011440 tx=0x7f7830011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2091fd2d9ec21c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:52.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f782d600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:22:52.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f782d600640 1 -- 172.21.15.107:0/1337950082 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830005220 con 0x55589fab8590 2024-09-19T04:22:52.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f784d000740 1 -- 172.21.15.107:0/1337950082 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f783c036040 con 0x55589fab8590 2024-09-19T04:22:52.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f782d600640 1 -- 172.21.15.107:0/1337950082 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f78300053c0 con 0x55589fab8590 2024-09-19T04:22:52.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f782d600640 1 -- 172.21.15.107:0/1337950082 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f78300055c0 con 0x55589fab8590 2024-09-19T04:22:52.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f784d000740 10 client.25684 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:52.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.611+0000 7f784d000740 1 -- 172.21.15.107:0/1337950082 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55589fad1960 con 0x55589fab8590 2024-09-19T04:22:52.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.612+0000 7f782d600640 1 -- 172.21.15.107:0/1337950082 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783c036040 con 0x55589fab8590 2024-09-19T04:22:52.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.612+0000 7f782d600640 1 -- 172.21.15.107:0/1337950082 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f783001a400 con 0x55589fab8590 2024-09-19T04:22:52.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.612+0000 7f784d000740 10 client.25684 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:52.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.612+0000 7f784d000740 10 client.25684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:22:52.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.612+0000 7f784d000740 20 client.25684 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:52.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.612+0000 7f784d000740 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55589fad2ec0 0x55589fad52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.612+0000 7f784d000740 4 client.25684 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:52.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.612+0000 7f784d000740 1 -- 172.21.15.107:0/1337950082 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55589fa2b950 con 0x55589fad2ec0 2024-09-19T04:22:52.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.612+0000 7f7843e00640 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55589fad2ec0 0x55589fad52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.613+0000 7f7843e00640 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55589fad2ec0 0x55589fad52b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ca4d1c69efdd78f server_cookie=5a386c86f3d9d72a in_seq=0 out_seq=0 2024-09-19T04:22:52.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.613+0000 7f782d600640 10 client.25684 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:22:52.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f782d600640 1 -- 172.21.15.107:0/1337950082 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55589fa2b950 con 0x55589fad2ec0 2024-09-19T04:22:52.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f782d600640 10 client.25684 handle_command_reply: tid=0 2024-09-19T04:22:52.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 client.25684 shutdown 2024-09-19T04:22:52.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 -- 172.21.15.107:0/1337950082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 msgr2=0x55589fac0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:52.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fac0240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7830011440 tx=0x7f7830011900 comp rx=0 tx=0).stop 2024-09-19T04:22:52.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 -- 172.21.15.107:0/1337950082 shutdown_connections 2024-09-19T04:22:52.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55589fad2ec0 0x55589fad52b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fac2e50 0x55589fac0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fac0240 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f782d600640 0 client.25684 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:22:52.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 --2- 172.21.15.107:0/1337950082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fab7c70 0x55589fabfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 -- 172.21.15.107:0/1337950082 >> 172.21.15.107:0/1337950082 conn(0x55589fab4200 msgr2=0x55589fab3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:52.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 -- 172.21.15.107:0/1337950082 shutdown_connections 2024-09-19T04:22:52.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 1 -- 172.21.15.107:0/1337950082 wait complete. 2024-09-19T04:22:52.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.614+0000 7f784d000740 20 client.25684 trim_cache size 0 max 16384 2024-09-19T04:22:52.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.629+0000 7f784d000740 1 Processor -- start 2024-09-19T04:22:52.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.629+0000 7f784d000740 1 -- start start 2024-09-19T04:22:52.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.629+0000 7f784d000740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 0x55589fac05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.629+0000 7f784d000740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab8590 0x55589fadaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.629+0000 7f784d000740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fac2e50 0x55589fadb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.629+0000 7f784d000740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78441af920 con 0x55589fab8590 2024-09-19T04:22:52.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.629+0000 7f784d000740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f78441b1550 con 0x55589fac2e50 2024-09-19T04:22:52.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.629+0000 7f784d000740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78441afff0 con 0x55589fab7c70 2024-09-19T04:22:52.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7843e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 0x55589fac05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7842a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab8590 0x55589fadaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7842a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab8590 0x55589fadaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52200/0 (socket says 172.21.15.107:52200) 2024-09-19T04:22:52.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7842a00640 1 -- 172.21.15.107:0/3127597977 learned_addr learned my addr 172.21.15.107:0/3127597977 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:52.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7843e00640 1 --2- 172.21.15.107:0/3127597977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fac2e50 0x55589fadb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7842a00640 1 -- 172.21.15.107:0/3127597977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 msgr2=0x55589fac05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:52.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7842a00640 1 --2- 172.21.15.107:0/3127597977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 0x55589fac05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7842a00640 1 -- 172.21.15.107:0/3127597977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fac2e50 msgr2=0x55589fadb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:52.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7842a00640 1 --2- 172.21.15.107:0/3127597977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fac2e50 0x55589fadb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7842a00640 1 -- 172.21.15.107:0/3127597977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589fab4e90 con 0x55589fab8590 2024-09-19T04:22:52.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.630+0000 7f7842a00640 1 --2- 172.21.15.107:0/3127597977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab8590 0x55589fadaba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f783001aa30 tx=0x7f7830011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe01338c09c0b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:52.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.631+0000 7f782fe00640 1 -- 172.21.15.107:0/3127597977 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830027070 con 0x55589fab8590 2024-09-19T04:22:52.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.631+0000 7f782fe00640 1 -- 172.21.15.107:0/3127597977 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783001acb0 con 0x55589fab8590 2024-09-19T04:22:52.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.631+0000 7f782fe00640 1 -- 172.21.15.107:0/3127597977 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830012d50 con 0x55589fab8590 2024-09-19T04:22:52.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.631+0000 7f784d000740 1 -- 172.21.15.107:0/3127597977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab8590 msgr2=0x55589fadaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:52.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.631+0000 7f784d000740 1 --2- 172.21.15.107:0/3127597977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab8590 0x55589fadaba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f783001aa30 tx=0x7f7830011e80 comp rx=0 tx=0).stop 2024-09-19T04:22:52.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.631+0000 7f784d000740 1 -- 172.21.15.107:0/3127597977 shutdown_connections 2024-09-19T04:22:52.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.631+0000 7f784d000740 1 --2- 172.21.15.107:0/3127597977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fac2e50 0x55589fadb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.631+0000 7f784d000740 1 --2- 172.21.15.107:0/3127597977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab8590 0x55589fadaba0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.631+0000 7f784d000740 1 --2- 172.21.15.107:0/3127597977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab7c70 0x55589fac05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.631+0000 7f784d000740 1 -- 172.21.15.107:0/3127597977 >> 172.21.15.107:0/3127597977 conn(0x55589fab4200 msgr2=0x55589fac1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:52.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.632+0000 7f784d000740 1 -- 172.21.15.107:0/3127597977 shutdown_connections 2024-09-19T04:22:52.638 INFO:tasks.workunit.client.0.smithi107.stdout: 1 92693 7.40 MB/sec warmup 70 sec latency 11.438 ms 2024-09-19T04:22:52.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.632+0000 7f784d000740 1 -- 172.21.15.107:0/3127597977 wait complete. 2024-09-19T04:22:52.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.632+0000 7f784d000740 1 -- start start 2024-09-19T04:22:52.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.632+0000 7f784d000740 1 Processor -- start 2024-09-19T04:22:52.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f784d000740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab7c70 0x55589fabceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f784d000740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fabd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f784d000740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fac2e50 0x55589fabd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f784d000740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78441b0dc0 con 0x55589fab7c70 2024-09-19T04:22:52.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f784d000740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f78441b06c0 con 0x55589fac2e50 2024-09-19T04:22:52.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f784d000740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78441ad900 con 0x55589fab8590 2024-09-19T04:22:52.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f784d000740 10 client.0 _ll_register_callbacks cb 0x7f7830009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:52.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7842a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fabd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7842a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fabd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57152/0 (socket says 172.21.15.107:57152) 2024-09-19T04:22:52.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7842a00640 1 -- 172.21.15.107:0/38512899 learned_addr learned my addr 172.21.15.107:0/38512899 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:52.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7843e00640 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab7c70 0x55589fabceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7843e00640 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fac2e50 0x55589fabd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7842a00640 1 -- 172.21.15.107:0/38512899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fac2e50 msgr2=0x55589fabd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:52.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7842a00640 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fac2e50 0x55589fabd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7842a00640 1 -- 172.21.15.107:0/38512899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab7c70 msgr2=0x55589fabceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:52.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7842a00640 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab7c70 0x55589fabceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7842a00640 1 -- 172.21.15.107:0/38512899 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589fabaae0 con 0x55589fab8590 2024-09-19T04:22:52.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.633+0000 7f7843e00640 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab7c70 0x55589fabceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:52.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.634+0000 7f7842a00640 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fabd3f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f783001a8c0 tx=0x7f7830005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef368d9010396322 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:52.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.634+0000 7f782d600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:22:52.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.634+0000 7f782d600640 1 -- 172.21.15.107:0/38512899 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830027070 con 0x55589fab8590 2024-09-19T04:22:52.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.634+0000 7f782d600640 1 -- 172.21.15.107:0/38512899 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f7830012be0 con 0x55589fab8590 2024-09-19T04:22:52.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.634+0000 7f784d000740 1 -- 172.21.15.107:0/38512899 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f783001acb0 con 0x55589fab8590 2024-09-19T04:22:52.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.634+0000 7f782d600640 1 -- 172.21.15.107:0/38512899 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f783003b050 con 0x55589fab8590 2024-09-19T04:22:52.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.634+0000 7f784d000740 10 client.25690 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.634+0000 7f784d000740 1 -- 172.21.15.107:0/38512899 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55589fb3de20 con 0x55589fab8590 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.635+0000 7f782d600640 1 -- 172.21.15.107:0/38512899 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783001acb0 con 0x55589fab8590 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.635+0000 7f782d600640 1 -- 172.21.15.107:0/38512899 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f7830023430 con 0x55589fab8590 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.635+0000 7f784d000740 10 client.25690 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.635+0000 7f784d000740 10 client.25690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.635+0000 7f784d000740 20 client.25690 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.635+0000 7f784d000740 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55589fb3e560 0x55589fb40980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.635+0000 7f784d000740 4 client.25690 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.635+0000 7f784d000740 1 -- 172.21.15.107:0/38512899 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55589fa25e60 con 0x55589fb3e560 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.635+0000 7f7843e00640 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55589fb3e560 0x55589fb40980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f7843e00640 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55589fb3e560 0x55589fb40980 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e00c29357c9dc045 server_cookie=5cdad40551c17fef in_seq=0 out_seq=0 2024-09-19T04:22:52.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f782d600640 10 client.25690 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:22:52.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f782d600640 1 -- 172.21.15.107:0/38512899 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55589fa25e60 con 0x55589fb3e560 2024-09-19T04:22:52.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f782d600640 10 client.25690 handle_command_reply: tid=0 2024-09-19T04:22:52.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f784d000740 1 client.25690 shutdown 2024-09-19T04:22:52.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f784d000740 1 -- 172.21.15.107:0/38512899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 msgr2=0x55589fabd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:52.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f784d000740 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fabd3f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f783001a8c0 tx=0x7f7830005cc0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f784d000740 1 -- 172.21.15.107:0/38512899 shutdown_connections 2024-09-19T04:22:52.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f784d000740 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55589fb3e560 0x55589fb40980 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f784d000740 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55589fac2e50 0x55589fabd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f784d000740 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55589fab8590 0x55589fabd3f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f782d600640 0 client.25690 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:22:52.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f784d000740 1 --2- 172.21.15.107:0/38512899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55589fab7c70 0x55589fabceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.636+0000 7f784d000740 1 -- 172.21.15.107:0/38512899 >> 172.21.15.107:0/38512899 conn(0x55589fab4200 msgr2=0x55589fab2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:52.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784d000740 1 -- 172.21.15.107:0/38512899 shutdown_connections 2024-09-19T04:22:52.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784d000740 1 -- 172.21.15.107:0/38512899 wait complete. 2024-09-19T04:22:52.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784d000740 20 client.25690 trim_cache size 0 max 16384 2024-09-19T04:22:52.646 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784a400640 1 -- 172.21.15.107:0/541644652 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f78280a3430 msgr2=0x7f78280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784a400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f78280a3430 0x7f78280a58f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78340cd110 tx=0x7f78340d5550 comp rx=0 tx=0).stop 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784a400640 1 -- 172.21.15.107:0/541644652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7844101dd0 msgr2=0x7f784419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784a400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7844101dd0 0x7f784419a680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f783400d950 tx=0x7f783400de10 comp rx=0 tx=0).stop 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784a400640 1 -- 172.21.15.107:0/541644652 shutdown_connections 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784a400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f78280a3430 0x7f78280a58f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784a400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7844106f00 0x7f784419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784a400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102780 0x7f784419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784a400640 1 --2- 172.21.15.107:0/541644652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7844101dd0 0x7f784419a680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.637+0000 7f784a400640 1 -- 172.21.15.107:0/541644652 >> 172.21.15.107:0/541644652 conn(0x7f78440fd8b0 msgr2=0x7f78440fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.638+0000 7f784a400640 1 -- 172.21.15.107:0/541644652 shutdown_connections 2024-09-19T04:22:52.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:52.638+0000 7f784a400640 1 -- 172.21.15.107:0/541644652 wait complete. 2024-09-19T04:22:52.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:52 smithi107 ceph-mon[30659]: pgmap v79: 97 pgs: 97 active+clean; 59 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-19T04:22:52.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:52 smithi115 ceph-mon[35477]: pgmap v79: 97 pgs: 97 active+clean; 59 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-19T04:22:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:52 smithi145 ceph-mon[35096]: pgmap v79: 97 pgs: 97 active+clean; 59 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-19T04:22:53.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 92936 7.30 MB/sec warmup 71 sec latency 760.572 ms 2024-09-19T04:22:53.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:22:54.016 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:54.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.569+0000 7f39dc800640 1 -- 172.21.15.107:0/2538673285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39d4074510 msgr2=0x7f39d40748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:54.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.569+0000 7f39dc800640 1 --2- 172.21.15.107:0/2538673285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39d4074510 0x7f39d40748f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009860 tx=0x7f39c802a3d0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.569+0000 7f39dc800640 1 -- 172.21.15.107:0/2538673285 shutdown_connections 2024-09-19T04:22:54.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.569+0000 7f39dc800640 1 --2- 172.21.15.107:0/2538673285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d410e440 0x7f39d4110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.569+0000 7f39dc800640 1 --2- 172.21.15.107:0/2538673285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d4074e30 0x7f39d410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.569+0000 7f39dc800640 1 --2- 172.21.15.107:0/2538673285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39d4074510 0x7f39d40748f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.569+0000 7f39dc800640 1 -- 172.21.15.107:0/2538673285 >> 172.21.15.107:0/2538673285 conn(0x7f39d40fd8c0 msgr2=0x7f39d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:54.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.569+0000 7f39dc800640 1 -- 172.21.15.107:0/2538673285 shutdown_connections 2024-09-19T04:22:54.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.569+0000 7f39dc800640 1 -- 172.21.15.107:0/2538673285 wait complete. 2024-09-19T04:22:54.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.570+0000 7f39dc800640 1 Processor -- start 2024-09-19T04:22:54.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.570+0000 7f39dc800640 1 -- start start 2024-09-19T04:22:54.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.570+0000 7f39dc800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d4074510 0x7f39d419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.570+0000 7f39dc800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d4074e30 0x7f39d419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.570+0000 7f39dc800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39d410e440 0x7f39d419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39dc800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41141c0 con 0x7f39d4074510 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39dc800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39d4114040 con 0x7f39d4074e30 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39dc800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39d4114340 con 0x7f39d410e440 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39d9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d4074510 0x7f39d419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39d9400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d4074e30 0x7f39d419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39d9400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d4074e30 0x7f39d419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:48256/0 (socket says 172.21.15.107:48256) 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39d9400640 1 -- 172.21.15.107:0/1941959138 learned_addr learned my addr 172.21.15.107:0/1941959138 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39da800640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39d410e440 0x7f39d419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39da800640 1 -- 172.21.15.107:0/1941959138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d4074e30 msgr2=0x7f39d419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39da800640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d4074e30 0x7f39d419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39da800640 1 -- 172.21.15.107:0/1941959138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d4074510 msgr2=0x7f39d419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:54.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39da800640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d4074510 0x7f39d419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39da800640 1 -- 172.21.15.107:0/1941959138 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d419f670 con 0x7f39d410e440 2024-09-19T04:22:54.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39d9e00640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d4074510 0x7f39d419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:54.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.571+0000 7f39d9400640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d4074e30 0x7f39d419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:54.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.572+0000 7f39da800640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39d410e440 0x7f39d419ef90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f39d0005fd0 tx=0x7f39d000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:54.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.572+0000 7f39cea00640 1 -- 172.21.15.107:0/1941959138 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d000cdf0 con 0x7f39d410e440 2024-09-19T04:22:54.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.572+0000 7f39cea00640 1 -- 172.21.15.107:0/1941959138 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d0005020 con 0x7f39d410e440 2024-09-19T04:22:54.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.572+0000 7f39dc800640 1 -- 172.21.15.107:0/1941959138 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d419f960 con 0x7f39d410e440 2024-09-19T04:22:54.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.572+0000 7f39cea00640 1 -- 172.21.15.107:0/1941959138 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d00106d0 con 0x7f39d410e440 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.572+0000 7f39dc800640 1 -- 172.21.15.107:0/1941959138 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d41a7240 con 0x7f39d410e440 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.575+0000 7f39df2a9740 1 Processor -- start 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.577+0000 7f39cea00640 1 -- 172.21.15.107:0/1941959138 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f39d00054d0 con 0x7f39d410e440 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.577+0000 7f39df2a9740 1 -- start start 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.577+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.577+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200d0590 0x564f200da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.577+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200dae50 0x564f200dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.577+0000 7f39df2a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564f200363e0 con 0x564f200dae50 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.577+0000 7f39df2a9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f1ffddd80 con 0x564f200d0590 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.577+0000 7f39df2a9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564f200323a0 con 0x564f200cfc70 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200d0590 0x564f200da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200d0590 0x564f200da910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:48260/0 (socket says 172.21.15.107:48260) 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9e00640 1 -- 172.21.15.107:0/399349838 learned_addr learned my addr 172.21.15.107:0/399349838 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9400640 1 --2- 172.21.15.107:0/399349838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200dae50 0x564f200dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9e00640 1 -- 172.21.15.107:0/399349838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 msgr2=0x564f200d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9e00640 1 --2- 172.21.15.107:0/399349838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200d0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9e00640 1 -- 172.21.15.107:0/399349838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200dae50 msgr2=0x564f200dd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9e00640 1 --2- 172.21.15.107:0/399349838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200dae50 0x564f200dd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9e00640 1 -- 172.21.15.107:0/399349838 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f1fffcae0 con 0x564f200d0590 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9400640 1 --2- 172.21.15.107:0/399349838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.578+0000 7f39d9e00640 1 --2- 172.21.15.107:0/399349838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200d0590 0x564f200da910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f39c8034000 tx=0x7f39c8030e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb3e4ac78048816c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.579+0000 7f39cea00640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39bc0a3430 0x7f39bc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.579+0000 7f39cea00640 1 -- 172.21.15.107:0/1941959138 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f39d00d02d0 con 0x7f39d410e440 2024-09-19T04:22:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.579+0000 7f39c3000640 1 -- 172.21.15.107:0/399349838 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8004230 con 0x564f200d0590 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.579+0000 7f39c3000640 1 -- 172.21.15.107:0/399349838 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c8041030 con 0x564f200d0590 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.579+0000 7f39d9400640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39bc0a3430 0x7f39bc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.579+0000 7f39c3000640 1 -- 172.21.15.107:0/399349838 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80046c0 con 0x564f200d0590 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.580+0000 7f39d9400640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39bc0a3430 0x7f39bc0a58f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x564f200cd580 tx=0x7f39c4004730 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.580+0000 7f39df2a9740 1 -- 172.21.15.107:0/399349838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200d0590 msgr2=0x564f200da910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.580+0000 7f39df2a9740 1 --2- 172.21.15.107:0/399349838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200d0590 0x564f200da910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f39c8034000 tx=0x7f39c8030e60 comp rx=0 tx=0).stop 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.580+0000 7f39df2a9740 1 -- 172.21.15.107:0/399349838 shutdown_connections 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.580+0000 7f39df2a9740 1 --2- 172.21.15.107:0/399349838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200dae50 0x564f200dd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.580+0000 7f39df2a9740 1 --2- 172.21.15.107:0/399349838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200d0590 0x564f200da910 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.580+0000 7f39df2a9740 1 --2- 172.21.15.107:0/399349838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.580+0000 7f39df2a9740 1 -- 172.21.15.107:0/399349838 >> 172.21.15.107:0/399349838 conn(0x564f200cc200 msgr2=0x564f200cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.580+0000 7f39df2a9740 1 -- 172.21.15.107:0/399349838 shutdown_connections 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.580+0000 7f39df2a9740 1 -- 172.21.15.107:0/399349838 wait complete. 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.581+0000 7f39df2a9740 1 -- start start 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.581+0000 7f39df2a9740 1 Processor -- start 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.581+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200d7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.581+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 0x564f200d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.581+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 0x564f200d87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.581+0000 7f39df2a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564f200dda20 con 0x564f200d0590 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.581+0000 7f39df2a9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f1fff10f0 con 0x564f200dae50 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.581+0000 7f39df2a9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564f200ddba0 con 0x564f200cfc70 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.581+0000 7f39df2a9740 10 client.0 _ll_register_callbacks cb 0x564f1ffd8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39da800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 0x564f200d87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39d9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200d7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39d9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 0x564f200d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39da800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 0x564f200d87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:48270/0 (socket says 172.21.15.107:48270) 2024-09-19T04:22:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39da800640 1 -- 172.21.15.107:0/3123452916 learned_addr learned my addr 172.21.15.107:0/3123452916 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39da800640 1 -- 172.21.15.107:0/3123452916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 msgr2=0x564f200d7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39da800640 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200d7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39da800640 1 -- 172.21.15.107:0/3123452916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 msgr2=0x564f200d82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39da800640 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 0x564f200d82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39da800640 1 -- 172.21.15.107:0/3123452916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f1fda73c0 con 0x564f200dae50 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39d9e00640 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200d7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.582+0000 7f39d9400640 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 0x564f200d82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.583+0000 7f39da800640 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 0x564f200d87e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39d0098a00 tx=0x7f39d009cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f41832d6f1084f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.583+0000 7f39b7400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.583+0000 7f39b7400640 1 -- 172.21.15.107:0/3123452916 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d00b4020 con 0x564f200dae50 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.583+0000 7f39b7400640 1 -- 172.21.15.107:0/3123452916 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f39d009a8a0 con 0x564f200dae50 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.583+0000 7f39df2a9740 1 -- 172.21.15.107:0/3123452916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c8041030 con 0x564f200dae50 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.583+0000 7f39b7400640 1 -- 172.21.15.107:0/3123452916 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f39d00ba050 con 0x564f200dae50 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.583+0000 7f39df2a9740 10 client.25718 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.583+0000 7f39df2a9740 1 -- 172.21.15.107:0/3123452916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564f200e9af0 con 0x564f200dae50 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.584+0000 7f39b7400640 1 -- 172.21.15.107:0/3123452916 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d007c020 con 0x564f200dae50 2024-09-19T04:22:54.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.584+0000 7f39b7400640 1 -- 172.21.15.107:0/3123452916 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f39d009b9f0 con 0x564f200dae50 2024-09-19T04:22:54.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.584+0000 7f39df2a9740 10 client.25718 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:54.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.584+0000 7f39df2a9740 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:54.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.584+0000 7f39df2a9740 20 client.25718 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:54.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.584+0000 7f39df2a9740 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f200eb050 0x564f200ed4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.584+0000 7f39df2a9740 4 client.25718 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:54.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.584+0000 7f39df2a9740 1 -- 172.21.15.107:0/3123452916 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f20043950 con 0x564f200eb050 2024-09-19T04:22:54.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.585+0000 7f39d9e00640 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f200eb050 0x564f200ed4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.585+0000 7f39d9e00640 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f200eb050 0x564f200ed4d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc984d8b13c3361 server_cookie=ae266fdd825d6e5a in_seq=0 out_seq=0 2024-09-19T04:22:54.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.585+0000 7f39b7400640 10 client.25718 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:54.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39b7400640 1 -- 172.21.15.107:0/3123452916 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f20043950 con 0x564f200eb050 2024-09-19T04:22:54.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39b7400640 10 client.25718 handle_command_reply: tid=0 2024-09-19T04:22:54.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39df2a9740 1 client.25718 shutdown 2024-09-19T04:22:54.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39df2a9740 1 -- 172.21.15.107:0/3123452916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 msgr2=0x564f200d87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39df2a9740 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 0x564f200d87e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39d0098a00 tx=0x7f39d009cd60 comp rx=0 tx=0).stop 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39df2a9740 1 -- 172.21.15.107:0/3123452916 shutdown_connections 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39df2a9740 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f200eb050 0x564f200ed4d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39df2a9740 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 0x564f200d87e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39df2a9740 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 0x564f200d82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39df2a9740 1 --2- 172.21.15.107:0/3123452916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200d7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39b7400640 0 client.25718 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.586+0000 7f39df2a9740 1 -- 172.21.15.107:0/3123452916 >> 172.21.15.107:0/3123452916 conn(0x564f200cc200 msgr2=0x564f200d1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.587+0000 7f39df2a9740 1 -- 172.21.15.107:0/3123452916 shutdown_connections 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.587+0000 7f39df2a9740 1 -- 172.21.15.107:0/3123452916 wait complete. 2024-09-19T04:22:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.587+0000 7f39df2a9740 20 client.25718 trim_cache size 0 max 16384 2024-09-19T04:22:54.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.600+0000 7f39df2a9740 1 Processor -- start 2024-09-19T04:22:54.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.600+0000 7f39df2a9740 1 -- start start 2024-09-19T04:22:54.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 0x564f200f3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 0x564f200d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39df2a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41afa30 con 0x564f200d0590 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39df2a9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41b1660 con 0x564f200dae50 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39df2a9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41b0100 con 0x564f200cfc70 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39d9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 0x564f200f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39d9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39d9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 0x564f200f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52246/0 (socket says 172.21.15.107:52246) 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39d9400640 1 -- 172.21.15.107:0/3004636356 learned_addr learned my addr 172.21.15.107:0/3004636356 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.601+0000 7f39da800640 1 --2- 172.21.15.107:0/3004636356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 0x564f200d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.602+0000 7f39d9e00640 1 -- 172.21.15.107:0/3004636356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 msgr2=0x564f200d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.602+0000 7f39d9e00640 1 --2- 172.21.15.107:0/3004636356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 0x564f200d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.602+0000 7f39d9e00640 1 -- 172.21.15.107:0/3004636356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 msgr2=0x564f200f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.602+0000 7f39d9e00640 1 --2- 172.21.15.107:0/3004636356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 0x564f200f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.602+0000 7f39d9e00640 1 -- 172.21.15.107:0/3004636356 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f200c7770 con 0x564f200cfc70 2024-09-19T04:22:54.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.602+0000 7f39d9e00640 1 --2- 172.21.15.107:0/3004636356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200f2de0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f39c8030ed0 tx=0x7f39c8005cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9afcc4b325bb20e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.602+0000 7f39c2600640 1 -- 172.21.15.107:0/3004636356 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80303a0 con 0x564f200cfc70 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.602+0000 7f39c2600640 1 -- 172.21.15.107:0/3004636356 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c80035f0 con 0x564f200cfc70 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.602+0000 7f39c2600640 1 -- 172.21.15.107:0/3004636356 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c803d690 con 0x564f200cfc70 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.603+0000 7f39df2a9740 1 -- 172.21.15.107:0/3004636356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 msgr2=0x564f200f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.603+0000 7f39df2a9740 1 --2- 172.21.15.107:0/3004636356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200f2de0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f39c8030ed0 tx=0x7f39c8005cb0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.603+0000 7f39df2a9740 1 -- 172.21.15.107:0/3004636356 shutdown_connections 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.603+0000 7f39df2a9740 1 --2- 172.21.15.107:0/3004636356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200dae50 0x564f200d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.603+0000 7f39df2a9740 1 --2- 172.21.15.107:0/3004636356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200d0590 0x564f200f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.603+0000 7f39df2a9740 1 --2- 172.21.15.107:0/3004636356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200cfc70 0x564f200f2de0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.603+0000 7f39df2a9740 1 -- 172.21.15.107:0/3004636356 >> 172.21.15.107:0/3004636356 conn(0x564f200cc200 msgr2=0x564f200d6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.603+0000 7f39df2a9740 1 -- 172.21.15.107:0/3004636356 shutdown_connections 2024-09-19T04:22:54.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.603+0000 7f39df2a9740 1 -- 172.21.15.107:0/3004636356 wait complete. 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.604+0000 7f39df2a9740 1 -- start start 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.604+0000 7f39df2a9740 1 Processor -- start 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.604+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200cfc70 0x564f200e3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200d0590 0x564f200de530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39df2a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200dae50 0x564f200dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39df2a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41b0ed0 con 0x564f200dae50 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39df2a9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41b07d0 con 0x564f200cfc70 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39df2a9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41ada30 con 0x564f200d0590 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39df2a9740 10 client.0 _ll_register_callbacks cb 0x7f39d0098180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200cfc70 0x564f200e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200cfc70 0x564f200e3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:48292/0 (socket says 172.21.15.107:48292) 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39d9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200d0590 0x564f200de530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39d9e00640 1 -- 172.21.15.107:0/1786425652 learned_addr learned my addr 172.21.15.107:0/1786425652 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39da800640 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200dae50 0x564f200dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39d9e00640 1 -- 172.21.15.107:0/1786425652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200d0590 msgr2=0x564f200de530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39d9e00640 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200d0590 0x564f200de530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39d9e00640 1 -- 172.21.15.107:0/1786425652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200dae50 msgr2=0x564f200dea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39d9e00640 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200dae50 0x564f200dea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.605+0000 7f39d9e00640 1 -- 172.21.15.107:0/1786425652 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f200defb0 con 0x564f200cfc70 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.606+0000 7f39d9e00640 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200cfc70 0x564f200e3200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009860 tx=0x7f39c8004330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=849fc50a9805a81e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.606+0000 7f39b7400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.606+0000 7f39b7400640 1 -- 172.21.15.107:0/1786425652 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8061060 con 0x564f200cfc70 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.606+0000 7f39b7400640 1 -- 172.21.15.107:0/1786425652 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f39c80305a0 con 0x564f200cfc70 2024-09-19T04:22:54.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.606+0000 7f39df2a9740 1 -- 172.21.15.107:0/1786425652 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c80035f0 con 0x564f200cfc70 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.606+0000 7f39b7400640 1 -- 172.21.15.107:0/1786425652 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f39c80307a0 con 0x564f200cfc70 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.606+0000 7f39df2a9740 10 client.25724 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.606+0000 7f39df2a9740 1 -- 172.21.15.107:0/1786425652 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564f20155a80 con 0x564f200cfc70 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.607+0000 7f39b7400640 1 -- 172.21.15.107:0/1786425652 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c80035f0 con 0x564f200cfc70 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.607+0000 7f39b7400640 1 -- 172.21.15.107:0/1786425652 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f39c803d550 con 0x564f200cfc70 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.607+0000 7f39df2a9740 10 client.25724 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.607+0000 7f39df2a9740 10 client.25724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.607+0000 7f39df2a9740 20 client.25724 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.607+0000 7f39df2a9740 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f20155e60 0x564f20158250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.607+0000 7f39df2a9740 4 client.25724 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.607+0000 7f39df2a9740 1 -- 172.21.15.107:0/1786425652 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f2003de60 con 0x564f20155e60 2024-09-19T04:22:54.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.607+0000 7f39d9400640 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f20155e60 0x564f20158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39b7400640 10 client.25724 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39d9400640 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f20155e60 0x564f20158250 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1465ffd817db7b server_cookie=bf899228c3920c83 in_seq=0 out_seq=0 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39b7400640 1 -- 172.21.15.107:0/1786425652 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f2003de60 con 0x564f20155e60 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39b7400640 10 client.25724 handle_command_reply: tid=0 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39df2a9740 1 client.25724 shutdown 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39df2a9740 1 -- 172.21.15.107:0/1786425652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200cfc70 msgr2=0x564f200e3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39df2a9740 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200cfc70 0x564f200e3200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009860 tx=0x7f39c8004330 comp rx=0 tx=0).stop 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39df2a9740 1 -- 172.21.15.107:0/1786425652 shutdown_connections 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39df2a9740 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f20155e60 0x564f20158250 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39df2a9740 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f200dae50 0x564f200dea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39df2a9740 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f200d0590 0x564f200de530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39df2a9740 1 --2- 172.21.15.107:0/1786425652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f200cfc70 0x564f200e3200 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39df2a9740 1 -- 172.21.15.107:0/1786425652 >> 172.21.15.107:0/1786425652 conn(0x564f200cc200 msgr2=0x564f200d95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:54.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39b7400640 0 client.25724 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:54.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.608+0000 7f39df2a9740 1 -- 172.21.15.107:0/1786425652 shutdown_connections 2024-09-19T04:22:54.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.609+0000 7f39df2a9740 1 -- 172.21.15.107:0/1786425652 wait complete. 2024-09-19T04:22:54.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.609+0000 7f39df2a9740 20 client.25724 trim_cache size 0 max 16384 2024-09-19T04:22:54.821 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:22:54.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.609+0000 7f39dc800640 1 -- 172.21.15.107:0/1941959138 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39bc0a3430 msgr2=0x7f39bc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:54.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.609+0000 7f39dc800640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39bc0a3430 0x7f39bc0a58f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x564f200cd580 tx=0x7f39c4004730 comp rx=0 tx=0).stop 2024-09-19T04:22:54.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.609+0000 7f39dc800640 1 -- 172.21.15.107:0/1941959138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39d410e440 msgr2=0x7f39d419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:54.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.609+0000 7f39dc800640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39d410e440 0x7f39d419ef90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f39d0005fd0 tx=0x7f39d000ef10 comp rx=0 tx=0).stop 2024-09-19T04:22:54.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.610+0000 7f39dc800640 1 -- 172.21.15.107:0/1941959138 shutdown_connections 2024-09-19T04:22:54.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.610+0000 7f39dc800640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f39bc0a3430 0x7f39bc0a58f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.610+0000 7f39dc800640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39d410e440 0x7f39d419ef90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.610+0000 7f39dc800640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d4074e30 0x7f39d419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.610+0000 7f39dc800640 1 --2- 172.21.15.107:0/1941959138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d4074510 0x7f39d419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:54.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.610+0000 7f39dc800640 1 -- 172.21.15.107:0/1941959138 >> 172.21.15.107:0/1941959138 conn(0x7f39d40fd8c0 msgr2=0x7f39d40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:54.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.610+0000 7f39dc800640 1 -- 172.21.15.107:0/1941959138 shutdown_connections 2024-09-19T04:22:54.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:54.610+0000 7f39dc800640 1 -- 172.21.15.107:0/1941959138 wait complete. 2024-09-19T04:22:54.823 INFO:tasks.workunit.client.0.smithi107.stdout: 1 92992 7.20 MB/sec warmup 72 sec latency 1336.703 ms 2024-09-19T04:22:54.823 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:54 smithi107 ceph-mon[30659]: pgmap v80: 97 pgs: 97 active+clean; 59 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 524 KiB/s wr, 83 op/s 2024-09-19T04:22:54.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:54 smithi115 ceph-mon[35477]: pgmap v80: 97 pgs: 97 active+clean; 59 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 524 KiB/s wr, 83 op/s 2024-09-19T04:22:54.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:54 smithi145 ceph-mon[35096]: pgmap v80: 97 pgs: 97 active+clean; 59 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 524 KiB/s wr, 83 op/s 2024-09-19T04:22:55.633 INFO:tasks.workunit.client.0.smithi107.stdout: 1 92992 7.10 MB/sec warmup 73 sec latency 1384.282 ms 2024-09-19T04:22:56.339 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:22:56.339 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:22:56.634 INFO:tasks.workunit.client.0.smithi107.stdout: 1 92992 7.00 MB/sec warmup 74 sec latency 2384.327 ms 2024-09-19T04:22:56.715 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:56.896 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:56 smithi107 ceph-mon[30659]: pgmap v81: 97 pgs: 97 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 821 KiB/s wr, 144 op/s 2024-09-19T04:22:56.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:56 smithi115 ceph-mon[35477]: pgmap v81: 97 pgs: 97 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 821 KiB/s wr, 144 op/s 2024-09-19T04:22:56.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:56 smithi145 ceph-mon[35096]: pgmap v81: 97 pgs: 97 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 821 KiB/s wr, 144 op/s 2024-09-19T04:22:57.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.288+0000 7efd82600640 1 -- 172.21.15.107:0/2521003406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd7c074f40 msgr2=0x7efd7c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.288+0000 7efd82600640 1 --2- 172.21.15.107:0/2521003406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd7c074f40 0x7efd7c10de80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efd68009900 tx=0x7efd6802a480 comp rx=0 tx=0).stop 2024-09-19T04:22:57.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.288+0000 7efd82600640 1 -- 172.21.15.107:0/2521003406 shutdown_connections 2024-09-19T04:22:57.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.288+0000 7efd82600640 1 --2- 172.21.15.107:0/2521003406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c10e3c0 0x7efd7c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.289+0000 7efd82600640 1 --2- 172.21.15.107:0/2521003406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd7c074f40 0x7efd7c10de80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.289+0000 7efd82600640 1 --2- 172.21.15.107:0/2521003406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c074620 0x7efd7c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.289+0000 7efd82600640 1 -- 172.21.15.107:0/2521003406 >> 172.21.15.107:0/2521003406 conn(0x7efd7c0fd8c0 msgr2=0x7efd7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.289+0000 7efd82600640 1 -- 172.21.15.107:0/2521003406 shutdown_connections 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.289+0000 7efd82600640 1 -- 172.21.15.107:0/2521003406 wait complete. 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.289+0000 7efd82600640 1 Processor -- start 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd82600640 1 -- start start 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd82600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c074620 0x7efd7c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd82600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd7c074f40 0x7efd7c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd82600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c10e3c0 0x7efd7c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd82600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c114140 con 0x7efd7c074f40 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd82600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c113fc0 con 0x7efd7c074620 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd82600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1142c0 con 0x7efd7c10e3c0 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd7aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd7c074f40 0x7efd7c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd7aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd7c074f40 0x7efd7c19ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52266/0 (socket says 172.21.15.107:52266) 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd7aa00640 1 -- 172.21.15.107:0/1146214291 learned_addr learned my addr 172.21.15.107:0/1146214291 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.290+0000 7efd7b400640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c074620 0x7efd7c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.291+0000 7efd7be00640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c10e3c0 0x7efd7c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.291+0000 7efd7aa00640 1 -- 172.21.15.107:0/1146214291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c10e3c0 msgr2=0x7efd7c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.291+0000 7efd7aa00640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c10e3c0 0x7efd7c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.291+0000 7efd7aa00640 1 -- 172.21.15.107:0/1146214291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c074620 msgr2=0x7efd7c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.291+0000 7efd7aa00640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c074620 0x7efd7c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.291+0000 7efd7aa00640 1 -- 172.21.15.107:0/1146214291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c19f690 con 0x7efd7c074f40 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.291+0000 7efd7b400640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c074620 0x7efd7c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.291+0000 7efd7aa00640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd7c074f40 0x7efd7c19ac20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efd68006010 tx=0x7efd68004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.291+0000 7efd73e00640 1 -- 172.21.15.107:0/1146214291 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6803f030 con 0x7efd7c074f40 2024-09-19T04:22:57.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.292+0000 7efd73e00640 1 -- 172.21.15.107:0/1146214291 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd68004440 con 0x7efd7c074f40 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.292+0000 7efd73e00640 1 -- 172.21.15.107:0/1146214291 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68035070 con 0x7efd7c074f40 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.292+0000 7efd82600640 1 -- 172.21.15.107:0/1146214291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c19f920 con 0x7efd7c074f40 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.292+0000 7efd82600640 1 -- 172.21.15.107:0/1146214291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c1a7260 con 0x7efd7c074f40 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.295+0000 7efd73e00640 1 -- 172.21.15.107:0/1146214291 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7efd68042030 con 0x7efd7c074f40 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.295+0000 7efd8506e740 1 Processor -- start 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.295+0000 7efd8506e740 1 -- start start 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.295+0000 7efd8506e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 0x55eacc2cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.295+0000 7efd8506e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.295+0000 7efd8506e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 0x55eacc2dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.295+0000 7efd8506e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eacc236290 con 0x55eacc2d03e0 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.295+0000 7efd8506e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55eacc1ddb40 con 0x55eacc2cfac0 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.295+0000 7efd8506e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55eacc232250 con 0x55eacc2daca0 2024-09-19T04:22:57.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.295+0000 7efd7b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd7b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2da760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52268/0 (socket says 172.21.15.107:52268) 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd7b400640 1 -- 172.21.15.107:0/2210073571 learned_addr learned my addr 172.21.15.107:0/2210073571 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd7aa00640 1 --2- 172.21.15.107:0/2210073571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 0x55eacc2dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd7be00640 1 --2- 172.21.15.107:0/2210073571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 0x55eacc2cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd7b400640 1 -- 172.21.15.107:0/2210073571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 msgr2=0x55eacc2dd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd7b400640 1 --2- 172.21.15.107:0/2210073571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 0x55eacc2dd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd7b400640 1 -- 172.21.15.107:0/2210073571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 msgr2=0x55eacc2cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd7b400640 1 --2- 172.21.15.107:0/2210073571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 0x55eacc2cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd7b400640 1 -- 172.21.15.107:0/2210073571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eacbfa76a0 con 0x55eacc2d03e0 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd7b400640 1 --2- 172.21.15.107:0/2210073571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2da760 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0115f0 tx=0x7efd6c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb195ca94ffb1fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd70c00640 1 -- 172.21.15.107:0/2210073571 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c01e070 con 0x55eacc2d03e0 2024-09-19T04:22:57.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.296+0000 7efd73e00640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd600a33b0 0x7efd600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd70c00640 1 -- 172.21.15.107:0/2210073571 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd6c00fb20 con 0x55eacc2d03e0 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd70c00640 1 -- 172.21.15.107:0/2210073571 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c019be0 con 0x55eacc2d03e0 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd73e00640 1 -- 172.21.15.107:0/1146214291 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7efd680efd70 con 0x7efd7c074f40 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd7be00640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd600a33b0 0x7efd600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd8506e740 1 -- 172.21.15.107:0/2210073571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 msgr2=0x55eacc2da760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd8506e740 1 --2- 172.21.15.107:0/2210073571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2da760 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0115f0 tx=0x7efd6c011ab0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd7be00640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd600a33b0 0x7efd600a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd74005fd0 tx=0x7efd74009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd8506e740 1 -- 172.21.15.107:0/2210073571 shutdown_connections 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd8506e740 1 --2- 172.21.15.107:0/2210073571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 0x55eacc2dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd8506e740 1 --2- 172.21.15.107:0/2210073571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2da760 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd8506e740 1 --2- 172.21.15.107:0/2210073571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 0x55eacc2cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.297+0000 7efd8506e740 1 -- 172.21.15.107:0/2210073571 >> 172.21.15.107:0/2210073571 conn(0x55eacc2cc050 msgr2=0x55eacc2cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.298+0000 7efd8506e740 1 -- 172.21.15.107:0/2210073571 shutdown_connections 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.298+0000 7efd8506e740 1 -- 172.21.15.107:0/2210073571 wait complete. 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.298+0000 7efd8506e740 1 -- start start 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.298+0000 7efd8506e740 1 Processor -- start 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd8506e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 0x55eacc2da5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd8506e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2d78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd8506e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 0x55eacc2d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd8506e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eacc2dd910 con 0x55eacc2d03e0 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd8506e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55eacc1f0fa0 con 0x55eacc2cfac0 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd8506e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55eacc2dda90 con 0x55eacc2daca0 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd8506e740 10 client.0 _ll_register_callbacks cb 0x55eacbf44ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2d78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2d78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52272/0 (socket says 172.21.15.107:52272) 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7aa00640 1 -- 172.21.15.107:0/2435524399 learned_addr learned my addr 172.21.15.107:0/2435524399 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7b400640 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 0x55eacc2da5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7be00640 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 0x55eacc2d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7aa00640 1 -- 172.21.15.107:0/2435524399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 msgr2=0x55eacc2d7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7aa00640 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 0x55eacc2d7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7aa00640 1 -- 172.21.15.107:0/2435524399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 msgr2=0x55eacc2da5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7aa00640 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 0x55eacc2da5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7aa00640 1 -- 172.21.15.107:0/2435524399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eacc2d8680 con 0x55eacc2d03e0 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7be00640 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 0x55eacc2d7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.299+0000 7efd7b400640 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 0x55eacc2da5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.300+0000 7efd7aa00640 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2d78d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efd680b8550 tx=0x7efd68009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520099d682e69637 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.300+0000 7efd65600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.300+0000 7efd65600640 1 -- 172.21.15.107:0/2435524399 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68049070 con 0x55eacc2d03e0 2024-09-19T04:22:57.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.300+0000 7efd65600640 1 -- 172.21.15.107:0/2435524399 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7efd680b8760 con 0x55eacc2d03e0 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.300+0000 7efd8506e740 1 -- 172.21.15.107:0/2435524399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd6c00fb20 con 0x55eacc2d03e0 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.300+0000 7efd65600640 1 -- 172.21.15.107:0/2435524399 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efd680bbc30 con 0x55eacc2d03e0 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.300+0000 7efd8506e740 10 client.15684 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.300+0000 7efd8506e740 1 -- 172.21.15.107:0/2435524399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55eacc2e9860 con 0x55eacc2d03e0 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.300+0000 7efd65600640 1 -- 172.21.15.107:0/2435524399 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd6c00fb20 con 0x55eacc2d03e0 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.301+0000 7efd65600640 1 -- 172.21.15.107:0/2435524399 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7efd680f3030 con 0x55eacc2d03e0 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.301+0000 7efd8506e740 10 client.15684 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.301+0000 7efd8506e740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.301+0000 7efd8506e740 20 client.15684 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.301+0000 7efd8506e740 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55eacc2eacf0 0x55eacc2ed170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.301+0000 7efd8506e740 4 client.15684 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.301+0000 7efd8506e740 1 -- 172.21.15.107:0/2435524399 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eacc243800 con 0x55eacc2eacf0 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.301+0000 7efd7b400640 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55eacc2eacf0 0x55eacc2ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.302+0000 7efd7b400640 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55eacc2eacf0 0x55eacc2ed170 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633bbddbc16de1fb server_cookie=78b82b18a34c044d in_seq=0 out_seq=0 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.302+0000 7efd65600640 10 client.15684 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.302+0000 7efd65600640 1 -- 172.21.15.107:0/2435524399 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eacc243800 con 0x55eacc2eacf0 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.302+0000 7efd65600640 10 client.15684 handle_command_reply: tid=0 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.302+0000 7efd8506e740 1 client.15684 shutdown 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 1 -- 172.21.15.107:0/2435524399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 msgr2=0x55eacc2d78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2d78d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efd680b8550 tx=0x7efd68009560 comp rx=0 tx=0).stop 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 1 -- 172.21.15.107:0/2435524399 shutdown_connections 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55eacc2eacf0 0x55eacc2ed170 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2daca0 0x55eacc2d7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2d78d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 1 --2- 172.21.15.107:0/2435524399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2cfac0 0x55eacc2da5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 1 -- 172.21.15.107:0/2435524399 >> 172.21.15.107:0/2435524399 conn(0x55eacc2cc050 msgr2=0x55eacc2cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd65600640 0 client.15684 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:57.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 1 -- 172.21.15.107:0/2435524399 shutdown_connections 2024-09-19T04:22:57.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 1 -- 172.21.15.107:0/2435524399 wait complete. 2024-09-19T04:22:57.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.303+0000 7efd8506e740 20 client.15684 trim_cache size 0 max 16384 2024-09-19T04:22:57.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd8506e740 1 Processor -- start 2024-09-19T04:22:57.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd8506e740 1 -- start start 2024-09-19T04:22:57.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd8506e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 0x55eacc2d9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd8506e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd8506e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2daca0 0x55eacc2f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd8506e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1af9e0 con 0x55eacc2d03e0 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd8506e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1b1610 con 0x55eacc2daca0 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd8506e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1b00b0 con 0x55eacc2cfac0 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd7aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd7aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52282/0 (socket says 172.21.15.107:52282) 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd7aa00640 1 -- 172.21.15.107:0/1378223126 learned_addr learned my addr 172.21.15.107:0/1378223126 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.321+0000 7efd7be00640 1 --2- 172.21.15.107:0/1378223126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2daca0 0x55eacc2f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.322+0000 7efd7b400640 1 --2- 172.21.15.107:0/1378223126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 0x55eacc2d9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.322+0000 7efd7aa00640 1 -- 172.21.15.107:0/1378223126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 msgr2=0x55eacc2d9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.322+0000 7efd7aa00640 1 --2- 172.21.15.107:0/1378223126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 0x55eacc2d9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.322+0000 7efd7aa00640 1 -- 172.21.15.107:0/1378223126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2daca0 msgr2=0x55eacc2f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.322+0000 7efd7aa00640 1 --2- 172.21.15.107:0/1378223126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2daca0 0x55eacc2f2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.322+0000 7efd7aa00640 1 -- 172.21.15.107:0/1378223126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eacc2d6880 con 0x55eacc2d03e0 2024-09-19T04:22:57.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.322+0000 7efd7aa00640 1 --2- 172.21.15.107:0/1378223126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2f2a80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efd680bbdf0 tx=0x7efd680ed590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8213f481d3a8f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.322+0000 7efd67e00640 1 -- 172.21.15.107:0/1378223126 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68049070 con 0x55eacc2d03e0 2024-09-19T04:22:57.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.322+0000 7efd67e00640 1 -- 172.21.15.107:0/1378223126 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd680f3510 con 0x55eacc2d03e0 2024-09-19T04:22:57.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.322+0000 7efd67e00640 1 -- 172.21.15.107:0/1378223126 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd680bba50 con 0x55eacc2d03e0 2024-09-19T04:22:57.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.323+0000 7efd8506e740 1 -- 172.21.15.107:0/1378223126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 msgr2=0x55eacc2f2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.323+0000 7efd8506e740 1 --2- 172.21.15.107:0/1378223126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2f2a80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efd680bbdf0 tx=0x7efd680ed590 comp rx=0 tx=0).stop 2024-09-19T04:22:57.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.323+0000 7efd8506e740 1 -- 172.21.15.107:0/1378223126 shutdown_connections 2024-09-19T04:22:57.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.323+0000 7efd8506e740 1 --2- 172.21.15.107:0/1378223126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2daca0 0x55eacc2f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.323+0000 7efd8506e740 1 --2- 172.21.15.107:0/1378223126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2d03e0 0x55eacc2f2a80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.323+0000 7efd8506e740 1 --2- 172.21.15.107:0/1378223126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 0x55eacc2d9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.323+0000 7efd8506e740 1 -- 172.21.15.107:0/1378223126 >> 172.21.15.107:0/1378223126 conn(0x55eacc2cc050 msgr2=0x55eacc2d9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.323+0000 7efd8506e740 1 -- 172.21.15.107:0/1378223126 shutdown_connections 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.323+0000 7efd8506e740 1 -- 172.21.15.107:0/1378223126 wait complete. 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.324+0000 7efd8506e740 1 -- start start 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.324+0000 7efd8506e740 1 Processor -- start 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.324+0000 7efd8506e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 0x55eacc2e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.324+0000 7efd8506e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2d03e0 0x55eacc2e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.324+0000 7efd8506e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2daca0 0x55eacc2e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.324+0000 7efd8506e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1b0e80 con 0x55eacc2daca0 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.324+0000 7efd8506e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1b0780 con 0x55eacc2d03e0 2024-09-19T04:22:57.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.324+0000 7efd8506e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1ad9c0 con 0x55eacc2cfac0 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.324+0000 7efd8506e740 10 client.0 _ll_register_callbacks cb 0x7efd6803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 0x55eacc2e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 0x55eacc2e0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57270/0 (socket says 172.21.15.107:57270) 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2d03e0 0x55eacc2e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7b400640 1 -- 172.21.15.107:0/1335353986 learned_addr learned my addr 172.21.15.107:0/1335353986 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7be00640 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2daca0 0x55eacc2e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7b400640 1 -- 172.21.15.107:0/1335353986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2d03e0 msgr2=0x55eacc2e11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7b400640 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2d03e0 0x55eacc2e11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7b400640 1 -- 172.21.15.107:0/1335353986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2daca0 msgr2=0x55eacc2e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7b400640 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2daca0 0x55eacc2e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7b400640 1 -- 172.21.15.107:0/1335353986 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eacc354c60 con 0x55eacc2cfac0 2024-09-19T04:22:57.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7aa00640 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2d03e0 0x55eacc2e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:57.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd7b400640 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 0x55eacc2e0ca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efd6c019890 tx=0x7efd6c0198c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0211267d5fd4beb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:57.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.325+0000 7efd65600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:22:57.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.326+0000 7efd65600640 1 -- 172.21.15.107:0/1335353986 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c01e040 con 0x55eacc2cfac0 2024-09-19T04:22:57.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.326+0000 7efd8506e740 1 -- 172.21.15.107:0/1335353986 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd680f3510 con 0x55eacc2cfac0 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.326+0000 7efd65600640 1 -- 172.21.15.107:0/1335353986 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7efd6c00f6c0 con 0x55eacc2cfac0 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.326+0000 7efd65600640 1 -- 172.21.15.107:0/1335353986 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efd6c030030 con 0x55eacc2cfac0 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.326+0000 7efd8506e740 10 client.25726 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.326+0000 7efd8506e740 1 -- 172.21.15.107:0/1335353986 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55eacc355fc0 con 0x55eacc2cfac0 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.326+0000 7efd65600640 1 -- 172.21.15.107:0/1335353986 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd680f3510 con 0x55eacc2cfac0 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.326+0000 7efd65600640 1 -- 172.21.15.107:0/1335353986 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7efd6c0050b0 con 0x55eacc2cfac0 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.327+0000 7efd8506e740 10 client.25726 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.327+0000 7efd8506e740 10 client.25726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.327+0000 7efd8506e740 20 client.25726 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.327+0000 7efd8506e740 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55eacc3565e0 0x55eacc358a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:57.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.327+0000 7efd8506e740 4 client.25726 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.327+0000 7efd8506e740 1 -- 172.21.15.107:0/1335353986 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eacc23dd10 con 0x55eacc3565e0 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.327+0000 7efd7aa00640 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55eacc3565e0 0x55eacc358a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.327+0000 7efd7aa00640 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55eacc3565e0 0x55eacc358a00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44fc8c7f2d22e8d server_cookie=ccf2def0064ca417 in_seq=0 out_seq=0 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.327+0000 7efd65600640 10 client.25726 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd65600640 1 -- 172.21.15.107:0/1335353986 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eacc23dd10 con 0x55eacc3565e0 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd65600640 10 client.25726 handle_command_reply: tid=0 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 client.25726 shutdown 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 -- 172.21.15.107:0/1335353986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 msgr2=0x55eacc2e0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 0x55eacc2e0ca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efd6c019890 tx=0x7efd6c0198c0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 -- 172.21.15.107:0/1335353986 shutdown_connections 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55eacc3565e0 0x55eacc358a00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eacc2daca0 0x55eacc2e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eacc2d03e0 0x55eacc2e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd65600640 0 client.25726 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:57.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 --2- 172.21.15.107:0/1335353986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eacc2cfac0 0x55eacc2e0ca0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 -- 172.21.15.107:0/1335353986 >> 172.21.15.107:0/1335353986 conn(0x55eacc2cc050 msgr2=0x55eacc2d9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 -- 172.21.15.107:0/1335353986 shutdown_connections 2024-09-19T04:22:57.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 1 -- 172.21.15.107:0/1335353986 wait complete. 2024-09-19T04:22:57.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.329+0000 7efd8506e740 20 client.25726 trim_cache size 0 max 16384 2024-09-19T04:22:57.334 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:22:57.334 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:22:57.334 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "83c413a3-eeaa-464c-9522-568b266fc10a", 2024-09-19T04:22:57.335 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:22:57.335 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:22:57.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 -- 172.21.15.107:0/1146214291 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd600a33b0 msgr2=0x7efd600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd600a33b0 0x7efd600a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd74005fd0 tx=0x7efd74009500 comp rx=0 tx=0).stop 2024-09-19T04:22:57.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 -- 172.21.15.107:0/1146214291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd7c074f40 msgr2=0x7efd7c19ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:57.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd7c074f40 0x7efd7c19ac20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efd68006010 tx=0x7efd68004040 comp rx=0 tx=0).stop 2024-09-19T04:22:57.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 -- 172.21.15.107:0/1146214291 shutdown_connections 2024-09-19T04:22:57.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd600a33b0 0x7efd600a5870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c10e3c0 0x7efd7c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd7c074f40 0x7efd7c19ac20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 --2- 172.21.15.107:0/1146214291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c074620 0x7efd7c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:57.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 -- 172.21.15.107:0/1146214291 >> 172.21.15.107:0/1146214291 conn(0x7efd7c0fd8c0 msgr2=0x7efd7c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:57.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.330+0000 7efd82600640 1 -- 172.21.15.107:0/1146214291 shutdown_connections 2024-09-19T04:22:57.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:57.331+0000 7efd82600640 1 -- 172.21.15.107:0/1146214291 wait complete. 2024-09-19T04:22:57.634 INFO:tasks.workunit.client.0.smithi107.stdout: 1 93456 6.94 MB/sec warmup 75 sec latency 3112.766 ms 2024-09-19T04:22:57.901 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:57 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:22:57.901 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:57 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:57.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:57 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:22:57.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:57 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:57.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:57 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:22:57.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:57 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.211 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:22:58.596 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:22:58.634 INFO:tasks.workunit.client.0.smithi107.stdout: 1 96070 7.06 MB/sec warmup 76 sec latency 12.621 ms 2024-09-19T04:22:58.899 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: pgmap v82: 97 pgs: 97 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 547 KiB/s wr, 104 op/s 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.900 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:22:58.901 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.901 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:22:58.901 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.901 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:22:58 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: pgmap v82: 97 pgs: 97 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 547 KiB/s wr, 104 op/s 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:22:58.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:22:58.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:22:58.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:22:58 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: pgmap v82: 97 pgs: 97 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 547 KiB/s wr, 104 op/s 2024-09-19T04:22:58.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:22:58.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:22:58.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:58.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:22:58 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:22:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.167+0000 7f8cc8c00640 1 -- 172.21.15.107:0/3450465145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc4101de0 msgr2=0x7f8cc41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.167+0000 7f8cc8c00640 1 --2- 172.21.15.107:0/3450465145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc4101de0 0x7f8cc41021c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009900 tx=0x7f8cb402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.167+0000 7f8cc8c00640 1 -- 172.21.15.107:0/3450465145 shutdown_connections 2024-09-19T04:22:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.167+0000 7f8cc8c00640 1 --2- 172.21.15.107:0/3450465145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cc4106f10 0x7f8cc410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.167+0000 7f8cc8c00640 1 --2- 172.21.15.107:0/3450465145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cc4102790 0x7f8cc41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.167+0000 7f8cc8c00640 1 --2- 172.21.15.107:0/3450465145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc4101de0 0x7f8cc41021c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.167+0000 7f8cc8c00640 1 -- 172.21.15.107:0/3450465145 >> 172.21.15.107:0/3450465145 conn(0x7f8cc40fd8c0 msgr2=0x7f8cc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.167+0000 7f8cc8c00640 1 -- 172.21.15.107:0/3450465145 shutdown_connections 2024-09-19T04:22:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.167+0000 7f8cc8c00640 1 -- 172.21.15.107:0/3450465145 wait complete. 2024-09-19T04:22:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc8c00640 1 Processor -- start 2024-09-19T04:22:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc8c00640 1 -- start start 2024-09-19T04:22:59.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc8c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc4101de0 0x7f8cc4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cc4102790 0x7f8cc4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc8c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cc4106f10 0x7f8cc419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc8c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc4111f70 con 0x7f8cc4106f10 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc8c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc4111df0 con 0x7f8cc4101de0 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc8c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc41120f0 con 0x7f8cc4102790 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc4101de0 0x7f8cc4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc4101de0 0x7f8cc4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:56590/0 (socket says 172.21.15.107:56590) 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc1e00640 1 -- 172.21.15.107:0/2789148538 learned_addr learned my addr 172.21.15.107:0/2789148538 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.168+0000 7f8cc1400640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cc4102790 0x7f8cc4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.169+0000 7f8cc2800640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cc4106f10 0x7f8cc419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.169+0000 7f8cc1e00640 1 -- 172.21.15.107:0/2789148538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cc4102790 msgr2=0x7f8cc4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.169+0000 7f8cc1e00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cc4102790 0x7f8cc4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.169+0000 7f8cc1e00640 1 -- 172.21.15.107:0/2789148538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cc4106f10 msgr2=0x7f8cc419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.169+0000 7f8cc1e00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cc4106f10 0x7f8cc419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.169+0000 7f8cc1e00640 1 -- 172.21.15.107:0/2789148538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc419d3e0 con 0x7f8cc4101de0 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.169+0000 7f8cc1e00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc4101de0 0x7f8cc4198430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4006010 tx=0x7f8cb4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.169+0000 7f8cbaa00640 1 -- 172.21.15.107:0/2789148538 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb403f030 con 0x7f8cc4101de0 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.169+0000 7f8cbaa00640 1 -- 172.21.15.107:0/2789148538 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb4004440 con 0x7f8cc4101de0 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.169+0000 7f8cc8c00640 1 -- 172.21.15.107:0/2789148538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc419d670 con 0x7f8cc4101de0 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.170+0000 7f8cbaa00640 1 -- 172.21.15.107:0/2789148538 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4035070 con 0x7f8cc4101de0 2024-09-19T04:22:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.170+0000 7f8cc8c00640 1 -- 172.21.15.107:0/2789148538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc41a4fb0 con 0x7f8cc4101de0 2024-09-19T04:22:59.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8ccb68b740 1 Processor -- start 2024-09-19T04:22:59.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8ccb68b740 1 -- start start 2024-09-19T04:22:59.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8cbaa00640 1 -- 172.21.15.107:0/2789148538 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f8cb4042030 con 0x7f8cc4101de0 2024-09-19T04:22:59.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c2cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 0x55a016c37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 0x55a016c3a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a016b933e0 con 0x55a016c37d70 2024-09-19T04:22:59.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a016b3ad80 con 0x55a016c2d4b0 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a016b8f3a0 con 0x55a016c2cb90 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8cc2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c2cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8cc2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c2cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:34470/0 (socket says 172.21.15.107:34470) 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8cc2800640 1 -- 172.21.15.107:0/2419689892 learned_addr learned my addr 172.21.15.107:0/2419689892 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.173+0000 7f8cc1400640 1 --2- 172.21.15.107:0/2419689892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 0x55a016c3a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.174+0000 7f8cc1e00640 1 --2- 172.21.15.107:0/2419689892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 0x55a016c37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.174+0000 7f8cc2800640 1 -- 172.21.15.107:0/2419689892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 msgr2=0x55a016c37830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.174+0000 7f8cc2800640 1 --2- 172.21.15.107:0/2419689892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 0x55a016c37830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.174+0000 7f8cc2800640 1 -- 172.21.15.107:0/2419689892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 msgr2=0x55a016c3a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.174+0000 7f8cc2800640 1 --2- 172.21.15.107:0/2419689892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 0x55a016c3a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.174+0000 7f8cc2800640 1 -- 172.21.15.107:0/2419689892 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a016b59ae0 con 0x55a016c2cb90 2024-09-19T04:22:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.174+0000 7f8cc2800640 1 --2- 172.21.15.107:0/2419689892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c2cf70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009f50 tx=0x7f8cbc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80ca870ddda9420d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.174+0000 7f8caf000640 1 -- 172.21.15.107:0/2419689892 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc004d20 con 0x55a016c2cb90 2024-09-19T04:22:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.174+0000 7f8caf000640 1 -- 172.21.15.107:0/2419689892 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cbc004050 con 0x55a016c2cb90 2024-09-19T04:22:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.174+0000 7f8caf000640 1 -- 172.21.15.107:0/2419689892 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc005280 con 0x55a016c2cb90 2024-09-19T04:22:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8cbaa00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ca80a33b0 0x7f8ca80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2419689892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 msgr2=0x55a016c2cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/2419689892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c2cf70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009f50 tx=0x7f8cbc0118d0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8cbaa00640 1 -- 172.21.15.107:0/2789148538 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f8cb40f0070 con 0x7f8cc4101de0 2024-09-19T04:22:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8cc1e00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ca80a33b0 0x7f8ca80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2419689892 shutdown_connections 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/2419689892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 0x55a016c3a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/2419689892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 0x55a016c37830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/2419689892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c2cf70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2419689892 >> 172.21.15.107:0/2419689892 conn(0x55a016c29120 msgr2=0x55a016c29500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2419689892 shutdown_connections 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.175+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2419689892 wait complete. 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.176+0000 7f8cc1e00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ca80a33b0 0x7f8ca80a5870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55a016c29f80 tx=0x7f8cb402a3f0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.176+0000 7f8ccb68b740 1 -- start start 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.176+0000 7f8ccb68b740 1 Processor -- start 2024-09-19T04:22:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2cb90 0x55a016c34c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c37d70 0x55a016c356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a016c3a9e0 con 0x55a016c2d4b0 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a016b4e0f0 con 0x55a016c2cb90 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a016c3ab60 con 0x55a016c37d70 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8ccb68b740 10 client.0 _ll_register_callbacks cb 0x55a016b3f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8cc1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8cc1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c351b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38112/0 (socket says 172.21.15.107:38112) 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8cc1400640 1 -- 172.21.15.107:0/2507161182 learned_addr learned my addr 172.21.15.107:0/2507161182 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8cc2800640 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c37d70 0x55a016c356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.177+0000 7f8cc2800640 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2cb90 0x55a016c34c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8cc1400640 1 -- 172.21.15.107:0/2507161182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c37d70 msgr2=0x55a016c356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8cc1400640 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c37d70 0x55a016c356f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8cc1400640 1 -- 172.21.15.107:0/2507161182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2cb90 msgr2=0x55a016c34c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8cc1400640 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2cb90 0x55a016c34c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8cc1400640 1 -- 172.21.15.107:0/2507161182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0169043c0 con 0x55a016c2d4b0 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8cc1400640 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c351b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00126e0 tx=0x7f8cb0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a682dc5f18bf729 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8ca3400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8ca3400640 1 -- 172.21.15.107:0/2507161182 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000f920 con 0x55a016c2d4b0 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8ca3400640 1 -- 172.21.15.107:0/2507161182 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f8cb000fac0 con 0x55a016c2d4b0 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2507161182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cbc004050 con 0x55a016c2d4b0 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.178+0000 7f8ca3400640 1 -- 172.21.15.107:0/2507161182 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8cb000fcc0 con 0x55a016c2d4b0 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.179+0000 7f8ccb68b740 10 client.15702 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.179+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2507161182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a016c46700 con 0x55a016c2d4b0 2024-09-19T04:22:59.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.179+0000 7f8ca3400640 1 -- 172.21.15.107:0/2507161182 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cbc004050 con 0x55a016c2d4b0 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.179+0000 7f8ca3400640 1 -- 172.21.15.107:0/2507161182 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f8cb001c4b0 con 0x55a016c2d4b0 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.179+0000 7f8ccb68b740 10 client.15702 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.179+0000 7f8ccb68b740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.179+0000 7f8ccb68b740 20 client.15702 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.179+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a016c47c10 0x55a016c4a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.179+0000 7f8ccb68b740 4 client.15702 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:22:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.179+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2507161182 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a016ba0950 con 0x55a016c47c10 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.180+0000 7f8cc2800640 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a016c47c10 0x55a016c4a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.180+0000 7f8cc2800640 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a016c47c10 0x55a016c4a090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266d3b3c00e37e0f server_cookie=3aad8e1fc5e9cbe in_seq=0 out_seq=0 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.180+0000 7f8ca3400640 10 client.15702 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ca3400640 1 -- 172.21.15.107:0/2507161182 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a016ba0950 con 0x55a016c47c10 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ca3400640 10 client.15702 handle_command_reply: tid=0 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 client.15702 shutdown 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2507161182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 msgr2=0x55a016c351b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c351b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00126e0 tx=0x7f8cb0012ba0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2507161182 shutdown_connections 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a016c47c10 0x55a016c4a090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c37d70 0x55a016c356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c351b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ca3400640 0 client.15702 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/2507161182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2cb90 0x55a016c34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2507161182 >> 172.21.15.107:0/2507161182 conn(0x55a016c29120 msgr2=0x55a016c2ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2507161182 shutdown_connections 2024-09-19T04:22:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 1 -- 172.21.15.107:0/2507161182 wait complete. 2024-09-19T04:22:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.181+0000 7f8ccb68b740 20 client.15702 trim_cache size 0 max 16384 2024-09-19T04:22:59.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.199+0000 7f8ccb68b740 1 Processor -- start 2024-09-19T04:22:59.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.199+0000 7f8ccb68b740 1 -- start start 2024-09-19T04:22:59.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.199+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c2f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.199+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 0x55a016c280d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.199+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 0x55a016c4f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc41af970 con 0x55a016c37d70 2024-09-19T04:22:59.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc41b15a0 con 0x55a016c2d4b0 2024-09-19T04:22:59.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc41b0040 con 0x55a016c2cb90 2024-09-19T04:22:59.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 0x55a016c280d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 0x55a016c280d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:56606/0 (socket says 172.21.15.107:56606) 2024-09-19T04:22:59.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc1400640 1 -- 172.21.15.107:0/4057408700 learned_addr learned my addr 172.21.15.107:0/4057408700 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:59.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc2800640 1 --2- 172.21.15.107:0/4057408700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c2f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc2800640 1 --2- 172.21.15.107:0/4057408700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 0x55a016c4f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc1400640 1 -- 172.21.15.107:0/4057408700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 msgr2=0x55a016c2f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc1400640 1 --2- 172.21.15.107:0/4057408700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c2f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc1400640 1 -- 172.21.15.107:0/4057408700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 msgr2=0x55a016c4f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc1400640 1 --2- 172.21.15.107:0/4057408700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 0x55a016c4f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc1400640 1 -- 172.21.15.107:0/4057408700 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a016c30860 con 0x55a016c2d4b0 2024-09-19T04:22:59.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.200+0000 7f8cc2800640 1 --2- 172.21.15.107:0/4057408700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 0x55a016c4f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:22:59.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.201+0000 7f8cc1400640 1 --2- 172.21.15.107:0/4057408700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 0x55a016c280d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8cb001cb10 tx=0x7f8cb00096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f5a0d3824f6af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.201+0000 7f8cae600640 1 -- 172.21.15.107:0/4057408700 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0029030 con 0x55a016c2d4b0 2024-09-19T04:22:59.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.201+0000 7f8cae600640 1 -- 172.21.15.107:0/4057408700 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb000f920 con 0x55a016c2d4b0 2024-09-19T04:22:59.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.201+0000 7f8cae600640 1 -- 172.21.15.107:0/4057408700 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb001c420 con 0x55a016c2d4b0 2024-09-19T04:22:59.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.202+0000 7f8ccb68b740 1 -- 172.21.15.107:0/4057408700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 msgr2=0x55a016c280d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.202+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/4057408700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 0x55a016c280d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8cb001cb10 tx=0x7f8cb00096a0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.202+0000 7f8ccb68b740 1 -- 172.21.15.107:0/4057408700 shutdown_connections 2024-09-19T04:22:59.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.202+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/4057408700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c37d70 0x55a016c4f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.202+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/4057408700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c2d4b0 0x55a016c280d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.202+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/4057408700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c2f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.202+0000 7f8ccb68b740 1 -- 172.21.15.107:0/4057408700 >> 172.21.15.107:0/4057408700 conn(0x55a016c29120 msgr2=0x55a016c279c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.202+0000 7f8ccb68b740 1 -- 172.21.15.107:0/4057408700 shutdown_connections 2024-09-19T04:22:59.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.202+0000 7f8ccb68b740 1 -- 172.21.15.107:0/4057408700 wait complete. 2024-09-19T04:22:59.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.203+0000 7f8ccb68b740 1 -- start start 2024-09-19T04:22:59.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.203+0000 7f8ccb68b740 1 Processor -- start 2024-09-19T04:22:59.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.203+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c3bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.203+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c3c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.203+0000 7f8ccb68b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c37d70 0x55a016c3c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.203+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc41b0e10 con 0x55a016c2d4b0 2024-09-19T04:22:59.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.203+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc41b0710 con 0x55a016c37d70 2024-09-19T04:22:59.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.203+0000 7f8ccb68b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc41ad950 con 0x55a016c2cb90 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.203+0000 7f8ccb68b740 10 client.0 _ll_register_callbacks cb 0x7f8cb0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c3c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc2800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c37d70 0x55a016c3c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c3c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38132/0 (socket says 172.21.15.107:38132) 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc1400640 1 -- 172.21.15.107:0/3808809104 learned_addr learned my addr 172.21.15.107:0/3808809104 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc2800640 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c3bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc1400640 1 -- 172.21.15.107:0/3808809104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 msgr2=0x55a016c3bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc1400640 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c3bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc1400640 1 -- 172.21.15.107:0/3808809104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c37d70 msgr2=0x55a016c3c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc1400640 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c37d70 0x55a016c3c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc1400640 1 -- 172.21.15.107:0/3808809104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a016cb19e0 con 0x55a016c2d4b0 2024-09-19T04:22:59.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc2800640 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c37d70 0x55a016c3c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8cc1400640 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c3c150 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8cb001c920 tx=0x7f8cb001c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6092db091449ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8ca3400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.204+0000 7f8ca3400640 1 -- 172.21.15.107:0/3808809104 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0029040 con 0x55a016c2d4b0 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ccb68b740 1 -- 172.21.15.107:0/3808809104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb000f920 con 0x55a016c2d4b0 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ca3400640 1 -- 172.21.15.107:0/3808809104 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f8cb00050d0 con 0x55a016c2d4b0 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ca3400640 1 -- 172.21.15.107:0/3808809104 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8cb003b050 con 0x55a016c2d4b0 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ccb68b740 10 client.15708 fetch_fsmap learned FSMap version 31 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ccb68b740 1 -- 172.21.15.107:0/3808809104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a016cb2cd0 con 0x55a016c2d4b0 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ca3400640 1 -- 172.21.15.107:0/3808809104 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb000f920 con 0x55a016c2d4b0 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ca3400640 1 -- 172.21.15.107:0/3808809104 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f8cb0025430 con 0x55a016c2d4b0 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ccb68b740 10 client.15708 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:22:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ccb68b740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:22:59.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ccb68b740 20 client.15708 populate_metadata read hostname 'smithi107' 2024-09-19T04:22:59.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a016cb34d0 0x55a016cb58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:22:59.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ccb68b740 4 client.15708 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:22:59.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.205+0000 7f8ccb68b740 1 -- 172.21.15.107:0/3808809104 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a016b9ae60 con 0x55a016cb34d0 2024-09-19T04:22:59.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.206+0000 7f8cc2800640 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a016cb34d0 0x55a016cb58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:22:59.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.206+0000 7f8cc2800640 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a016cb34d0 0x55a016cb58f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b39517533410a0b server_cookie=a238ebd90526eb51 in_seq=0 out_seq=0 2024-09-19T04:22:59.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.206+0000 7f8ca3400640 10 client.15708 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:59.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.206+0000 7f8ca3400640 1 -- 172.21.15.107:0/3808809104 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a016b9ae60 con 0x55a016cb34d0 2024-09-19T04:22:59.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.206+0000 7f8ca3400640 10 client.15708 handle_command_reply: tid=0 2024-09-19T04:22:59.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.206+0000 7f8ccb68b740 1 client.15708 shutdown 2024-09-19T04:22:59.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.206+0000 7f8ccb68b740 1 -- 172.21.15.107:0/3808809104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 msgr2=0x55a016c3c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.206+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c3c150 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8cb001c920 tx=0x7f8cb001c650 comp rx=0 tx=0).stop 2024-09-19T04:22:59.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8ccb68b740 1 -- 172.21.15.107:0/3808809104 shutdown_connections 2024-09-19T04:22:59.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a016cb34d0 0x55a016cb58f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a016c37d70 0x55a016c3c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a016c2d4b0 0x55a016c3c150 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8ccb68b740 1 --2- 172.21.15.107:0/3808809104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a016c2cb90 0x55a016c3bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8ca3400640 0 client.15708 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:22:59.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8ccb68b740 1 -- 172.21.15.107:0/3808809104 >> 172.21.15.107:0/3808809104 conn(0x55a016c29120 msgr2=0x55a016c35450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8ccb68b740 1 -- 172.21.15.107:0/3808809104 shutdown_connections 2024-09-19T04:22:59.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8ccb68b740 1 -- 172.21.15.107:0/3808809104 wait complete. 2024-09-19T04:22:59.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8ccb68b740 20 client.15708 trim_cache size 0 max 16384 2024-09-19T04:22:59.213 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:22:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:22:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:22:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "83c413a3-eeaa-464c-9522-568b266fc10a": { 2024-09-19T04:22:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:22:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "83c413a3-eeaa-464c-9522-568b266fc10a", 2024-09-19T04:22:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:22:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:22:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:22:59.213 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8cc8c00640 1 -- 172.21.15.107:0/2789148538 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ca80a33b0 msgr2=0x7f8ca80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8cc8c00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ca80a33b0 0x7f8ca80a5870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55a016c29f80 tx=0x7f8cb402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8cc8c00640 1 -- 172.21.15.107:0/2789148538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc4101de0 msgr2=0x7f8cc4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.207+0000 7f8cc8c00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc4101de0 0x7f8cc4198430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4006010 tx=0x7f8cb4004040 comp rx=0 tx=0).stop 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.208+0000 7f8cc8c00640 1 -- 172.21.15.107:0/2789148538 shutdown_connections 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.208+0000 7f8cc8c00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ca80a33b0 0x7f8ca80a5870 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.208+0000 7f8cc8c00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cc4106f10 0x7f8cc419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.208+0000 7f8cc8c00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cc4102790 0x7f8cc4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.208+0000 7f8cc8c00640 1 --2- 172.21.15.107:0/2789148538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc4101de0 0x7f8cc4198430 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.208+0000 7f8cc8c00640 1 -- 172.21.15.107:0/2789148538 >> 172.21.15.107:0/2789148538 conn(0x7f8cc40fd8c0 msgr2=0x7f8cc40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:22:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.208+0000 7f8cc8c00640 1 -- 172.21.15.107:0/2789148538 shutdown_connections 2024-09-19T04:22:59.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:22:59.208+0000 7f8cc8c00640 1 -- 172.21.15.107:0/2789148538 wait complete. 2024-09-19T04:22:59.634 INFO:tasks.workunit.client.0.smithi107.stdout: 1 97349 7.04 MB/sec warmup 77 sec latency 95.959 ms 2024-09-19T04:22:59.962 INFO:tasks.cephfs.filesystem:scrub status for tag:83c413a3-eeaa-464c-9522-568b266fc10a - {'path': '/', 'tag': '83c413a3-eeaa-464c-9522-568b266fc10a', 'options': 'recursive,force'} 2024-09-19T04:23:00.634 INFO:tasks.workunit.client.0.smithi107.stdout: 1 99665 7.13 MB/sec warmup 78 sec latency 10.104 ms 2024-09-19T04:23:00.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:00 smithi115 ceph-mon[35477]: pgmap v83: 97 pgs: 97 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 525 KiB/s wr, 102 op/s 2024-09-19T04:23:00.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:00 smithi145 ceph-mon[35096]: pgmap v83: 97 pgs: 97 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 525 KiB/s wr, 102 op/s 2024-09-19T04:23:00.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:00 smithi107 ceph-mon[30659]: pgmap v83: 97 pgs: 97 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 525 KiB/s wr, 102 op/s 2024-09-19T04:23:01.634 INFO:tasks.workunit.client.0.smithi107.stdout: 1 100084 7.05 MB/sec warmup 79 sec latency 544.382 ms 2024-09-19T04:23:02.634 INFO:tasks.workunit.client.0.smithi107.stdout: 1 100084 6.96 MB/sec warmup 80 sec latency 1544.549 ms 2024-09-19T04:23:02.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:02 smithi115 ceph-mon[35477]: pgmap v84: 97 pgs: 97 active+clean; 64 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 871 KiB/s wr, 165 op/s 2024-09-19T04:23:02.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:02 smithi145 ceph-mon[35096]: pgmap v84: 97 pgs: 97 active+clean; 64 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 871 KiB/s wr, 165 op/s 2024-09-19T04:23:02.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:02 smithi107 ceph-mon[30659]: pgmap v84: 97 pgs: 97 active+clean; 64 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 871 KiB/s wr, 165 op/s 2024-09-19T04:23:03.634 INFO:tasks.workunit.client.0.smithi107.stdout: 1 100084 6.87 MB/sec warmup 81 sec latency 2544.698 ms 2024-09-19T04:23:04.638 INFO:tasks.workunit.client.0.smithi107.stdout: 1 100393 6.80 MB/sec warmup 82 sec latency 3122.190 ms 2024-09-19T04:23:04.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:04 smithi115 ceph-mon[35477]: pgmap v85: 97 pgs: 97 active+clean; 64 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 642 KiB/s wr, 124 op/s 2024-09-19T04:23:04.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:04 smithi145 ceph-mon[35096]: pgmap v85: 97 pgs: 97 active+clean; 64 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 642 KiB/s wr, 124 op/s 2024-09-19T04:23:04.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:04 smithi107 ceph-mon[30659]: pgmap v85: 97 pgs: 97 active+clean; 64 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 642 KiB/s wr, 124 op/s 2024-09-19T04:23:05.638 INFO:tasks.workunit.client.0.smithi107.stdout: 1 101851 6.80 MB/sec warmup 83 sec latency 380.648 ms 2024-09-19T04:23:06.638 INFO:tasks.workunit.client.0.smithi107.stdout: 1 103387 6.85 MB/sec warmup 84 sec latency 12.237 ms 2024-09-19T04:23:06.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:06 smithi115 ceph-mon[35477]: pgmap v86: 97 pgs: 97 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-09-19T04:23:06.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:06 smithi145 ceph-mon[35096]: pgmap v86: 97 pgs: 97 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-09-19T04:23:06.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:06 smithi107 ceph-mon[30659]: pgmap v86: 97 pgs: 97 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-09-19T04:23:07.639 INFO:tasks.workunit.client.0.smithi107.stdout: 1 103700 6.77 MB/sec warmup 85 sec latency 776.032 ms 2024-09-19T04:23:07.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:07 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:23:07.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:07 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:23:07.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:07 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:23:07.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:07 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:23:07.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:07 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:23:07.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:07 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:23:07.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:07 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:23:07.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:07 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:23:07.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:07 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:23:07.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:07 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:23:07.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:07 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:23:07.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:07 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:23:07.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:07 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:23:07.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:07 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:23:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:07 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:23:08.638 INFO:tasks.workunit.client.0.smithi107.stdout: 1 103700 6.70 MB/sec warmup 86 sec latency 1776.169 ms 2024-09-19T04:23:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:08 smithi115 ceph-mon[35477]: pgmap v87: 97 pgs: 97 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 841 KiB/s wr, 135 op/s 2024-09-19T04:23:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:08 smithi145 ceph-mon[35096]: pgmap v87: 97 pgs: 97 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 841 KiB/s wr, 135 op/s 2024-09-19T04:23:08.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:08 smithi107 ceph-mon[30659]: pgmap v87: 97 pgs: 97 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 841 KiB/s wr, 135 op/s 2024-09-19T04:23:09.639 INFO:tasks.workunit.client.0.smithi107.stdout: 1 104605 6.67 MB/sec warmup 87 sec latency 2352.974 ms 2024-09-19T04:23:10.639 INFO:tasks.workunit.client.0.smithi107.stdout: 1 104804 6.60 MB/sec warmup 88 sec latency 671.989 ms 2024-09-19T04:23:10.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:10 smithi115 ceph-mon[35477]: pgmap v88: 97 pgs: 97 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 841 KiB/s wr, 133 op/s 2024-09-19T04:23:10.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:10 smithi145 ceph-mon[35096]: pgmap v88: 97 pgs: 97 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 841 KiB/s wr, 133 op/s 2024-09-19T04:23:10.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:10 smithi107 ceph-mon[30659]: pgmap v88: 97 pgs: 97 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 841 KiB/s wr, 133 op/s 2024-09-19T04:23:11.639 INFO:tasks.workunit.client.0.smithi107.stdout: 1 106692 6.68 MB/sec warmup 89 sec latency 381.679 ms 2024-09-19T04:23:12.639 INFO:tasks.workunit.client.0.smithi107.stdout: 1 108154 6.66 MB/sec warmup 90 sec latency 14.694 ms 2024-09-19T04:23:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:12 smithi115 ceph-mon[35477]: pgmap v89: 97 pgs: 97 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-09-19T04:23:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:12 smithi145 ceph-mon[35096]: pgmap v89: 97 pgs: 97 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-09-19T04:23:12.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:12 smithi107 ceph-mon[30659]: pgmap v89: 97 pgs: 97 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-09-19T04:23:13.639 INFO:tasks.workunit.client.0.smithi107.stdout: 1 110361 6.74 MB/sec warmup 91 sec latency 11.200 ms 2024-09-19T04:23:14.639 INFO:tasks.workunit.client.0.smithi107.stdout: 1 111558 6.73 MB/sec warmup 92 sec latency 229.373 ms 2024-09-19T04:23:14.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:14 smithi115 ceph-mon[35477]: pgmap v90: 97 pgs: 97 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-19T04:23:14.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:14 smithi145 ceph-mon[35096]: pgmap v90: 97 pgs: 97 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-19T04:23:14.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:14 smithi107 ceph-mon[30659]: pgmap v90: 97 pgs: 97 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-19T04:23:15.639 INFO:tasks.workunit.client.0.smithi107.stdout: 1 113347 6.75 MB/sec warmup 93 sec latency 247.905 ms 2024-09-19T04:23:16.639 INFO:tasks.workunit.client.0.smithi107.stdout: 1 114210 6.73 MB/sec warmup 94 sec latency 265.474 ms 2024-09-19T04:23:16.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:16 smithi115 ceph-mon[35477]: pgmap v91: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-09-19T04:23:16.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:16 smithi145 ceph-mon[35096]: pgmap v91: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-09-19T04:23:16.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:16 smithi107 ceph-mon[30659]: pgmap v91: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-09-19T04:23:17.639 INFO:tasks.workunit.client.0.smithi107.stdout: 1 114210 6.66 MB/sec warmup 95 sec latency 1265.577 ms 2024-09-19T04:23:18.640 INFO:tasks.workunit.client.0.smithi107.stdout: 1 114210 6.59 MB/sec warmup 96 sec latency 2265.688 ms 2024-09-19T04:23:18.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:18 smithi115 ceph-mon[35477]: pgmap v92: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-19T04:23:18.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:18 smithi145 ceph-mon[35096]: pgmap v92: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-19T04:23:18.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:18 smithi107 ceph-mon[30659]: pgmap v92: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-19T04:23:19.640 INFO:tasks.workunit.client.0.smithi107.stdout: 1 114266 6.53 MB/sec warmup 97 sec latency 2839.452 ms 2024-09-19T04:23:20.640 INFO:tasks.workunit.client.0.smithi107.stdout: 1 114266 6.46 MB/sec warmup 98 sec latency 1384.522 ms 2024-09-19T04:23:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:20 smithi115 ceph-mon[35477]: pgmap v93: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-19T04:23:20.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:20 smithi145 ceph-mon[35096]: pgmap v93: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-19T04:23:20.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:20 smithi107 ceph-mon[30659]: pgmap v93: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-19T04:23:21.640 INFO:tasks.workunit.client.0.smithi107.stdout: 1 114266 6.39 MB/sec warmup 99 sec latency 2384.626 ms 2024-09-19T04:23:22.640 INFO:tasks.workunit.client.0.smithi107.stdout: 1 114266 6.33 MB/sec warmup 100 sec latency 3384.749 ms 2024-09-19T04:23:22.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:22 smithi115 ceph-mon[35477]: pgmap v94: 97 pgs: 97 active+clean; 76 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-19T04:23:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:22 smithi145 ceph-mon[35096]: pgmap v94: 97 pgs: 97 active+clean; 76 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-19T04:23:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:22 smithi107 ceph-mon[30659]: pgmap v94: 97 pgs: 97 active+clean; 76 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-19T04:23:23.640 INFO:tasks.workunit.client.0.smithi107.stdout: 1 114266 6.27 MB/sec warmup 101 sec latency 4384.858 ms 2024-09-19T04:23:24.640 INFO:tasks.workunit.client.0.smithi107.stdout: 1 115066 6.25 MB/sec warmup 102 sec latency 4957.329 ms 2024-09-19T04:23:24.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:24 smithi115 ceph-mon[35477]: pgmap v95: 97 pgs: 97 active+clean; 76 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-19T04:23:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:24 smithi145 ceph-mon[35096]: pgmap v95: 97 pgs: 97 active+clean; 76 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-19T04:23:24.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:24 smithi107 ceph-mon[30659]: pgmap v95: 97 pgs: 97 active+clean; 76 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-19T04:23:25.640 INFO:tasks.workunit.client.0.smithi107.stdout: 1 116150 6.22 MB/sec warmup 103 sec latency 554.421 ms 2024-09-19T04:23:26.640 INFO:tasks.workunit.client.0.smithi107.stdout: 1 117522 6.27 MB/sec warmup 104 sec latency 744.076 ms 2024-09-19T04:23:26.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:26 smithi115 ceph-mon[35477]: pgmap v96: 97 pgs: 97 active+clean; 82 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-19T04:23:26.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:26 smithi145 ceph-mon[35096]: pgmap v96: 97 pgs: 97 active+clean; 82 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-19T04:23:26.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:26 smithi107 ceph-mon[30659]: pgmap v96: 97 pgs: 97 active+clean; 82 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-19T04:23:27.640 INFO:tasks.workunit.client.0.smithi107.stdout: 1 117756 6.21 MB/sec warmup 105 sec latency 782.374 ms 2024-09-19T04:23:28.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 117756 6.15 MB/sec warmup 106 sec latency 1782.510 ms 2024-09-19T04:23:28.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:28 smithi115 ceph-mon[35477]: pgmap v97: 97 pgs: 97 active+clean; 82 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 881 KiB/s wr, 76 op/s 2024-09-19T04:23:28.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:28 smithi145 ceph-mon[35096]: pgmap v97: 97 pgs: 97 active+clean; 82 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 881 KiB/s wr, 76 op/s 2024-09-19T04:23:28.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:28 smithi107 ceph-mon[30659]: pgmap v97: 97 pgs: 97 active+clean; 82 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 881 KiB/s wr, 76 op/s 2024-09-19T04:23:29.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 118562 6.14 MB/sec warmup 107 sec latency 2352.916 ms 2024-09-19T04:23:29.963 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:23:30.381 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:30.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 120011 6.15 MB/sec warmup 108 sec latency 496.328 ms 2024-09-19T04:23:30.816 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:30 smithi107 ceph-mon[30659]: pgmap v98: 97 pgs: 97 active+clean; 82 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 881 KiB/s wr, 76 op/s 2024-09-19T04:23:30.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:30 smithi145 ceph-mon[35096]: pgmap v98: 97 pgs: 97 active+clean; 82 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 881 KiB/s wr, 76 op/s 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.995+0000 7f4d53400640 1 -- 172.21.15.107:0/1909031603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d4c104010 msgr2=0x7f4d4c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.995+0000 7f4d53400640 1 --2- 172.21.15.107:0/1909031603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d4c104010 0x7f4d4c1043f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009900 tx=0x7f4d3c02a410 comp rx=0 tx=0).stop 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.995+0000 7f4d53400640 1 -- 172.21.15.107:0/1909031603 shutdown_connections 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.995+0000 7f4d53400640 1 --2- 172.21.15.107:0/1909031603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d4c109140 0x7f4d4c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.995+0000 7f4d53400640 1 --2- 172.21.15.107:0/1909031603 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d4c1049c0 0x7f4d4c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.995+0000 7f4d53400640 1 --2- 172.21.15.107:0/1909031603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d4c104010 0x7f4d4c1043f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.995+0000 7f4d53400640 1 -- 172.21.15.107:0/1909031603 >> 172.21.15.107:0/1909031603 conn(0x7f4d4c0ffa50 msgr2=0x7f4d4c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.995+0000 7f4d53400640 1 -- 172.21.15.107:0/1909031603 shutdown_connections 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.995+0000 7f4d53400640 1 -- 172.21.15.107:0/1909031603 wait complete. 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.996+0000 7f4d53400640 1 Processor -- start 2024-09-19T04:23:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.996+0000 7f4d53400640 1 -- start start 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d53400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d4c104010 0x7f4d4c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d53400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d4c1049c0 0x7f4d4c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d4c109140 0x7f4d4c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d53400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c114230 con 0x7f4d4c109140 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d53400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1140b0 con 0x7f4d4c104010 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d53400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1143b0 con 0x7f4d4c1049c0 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d51400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d4c109140 0x7f4d4c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d50a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d4c104010 0x7f4d4c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d51400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d4c109140 0x7f4d4c19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35946/0 (socket says 172.21.15.107:35946) 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d50a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d4c104010 0x7f4d4c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59116/0 (socket says 172.21.15.107:59116) 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.997+0000 7f4d51400640 1 -- 172.21.15.107:0/2477460013 learned_addr learned my addr 172.21.15.107:0/2477460013 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:31.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.998+0000 7f4d50a00640 1 -- 172.21.15.107:0/2477460013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d4c1049c0 msgr2=0x7f4d4c19ac90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:23:31.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.998+0000 7f4d50a00640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d4c1049c0 0x7f4d4c19ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.998+0000 7f4d50a00640 1 -- 172.21.15.107:0/2477460013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d4c109140 msgr2=0x7f4d4c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:31.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.998+0000 7f4d50a00640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d4c109140 0x7f4d4c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.998+0000 7f4d50a00640 1 -- 172.21.15.107:0/2477460013 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c19f700 con 0x7f4d4c104010 2024-09-19T04:23:31.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:30.998+0000 7f4d51400640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d4c109140 0x7f4d4c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:31.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.003+0000 7f4d50a00640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d4c104010 0x7f4d4c19a750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c02a930 tx=0x7f4d3c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:31.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.003+0000 7f4d4a000640 1 -- 172.21.15.107:0/2477460013 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c03f070 con 0x7f4d4c104010 2024-09-19T04:23:31.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.003+0000 7f4d4a000640 1 -- 172.21.15.107:0/2477460013 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c031c90 con 0x7f4d4c104010 2024-09-19T04:23:31.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.003+0000 7f4d53400640 1 -- 172.21.15.107:0/2477460013 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c19f990 con 0x7f4d4c104010 2024-09-19T04:23:31.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.003+0000 7f4d4a000640 1 -- 172.21.15.107:0/2477460013 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c035030 con 0x7f4d4c104010 2024-09-19T04:23:31.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.003+0000 7f4d53400640 1 -- 172.21.15.107:0/2477460013 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c1a72d0 con 0x7f4d4c104010 2024-09-19T04:23:31.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.007+0000 7f4d4a000640 1 -- 172.21.15.107:0/2477460013 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f4d3c042070 con 0x7f4d4c104010 2024-09-19T04:23:31.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.007+0000 7f4d56018740 1 Processor -- start 2024-09-19T04:23:31.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.008+0000 7f4d56018740 1 -- start start 2024-09-19T04:23:31.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.008+0000 7f4d4a000640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d300a33b0 0x7f4d300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.008+0000 7f4d4a000640 1 -- 172.21.15.107:0/2477460013 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4d3c0f0430 con 0x7f4d4c104010 2024-09-19T04:23:31.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.008+0000 7f4d51400640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d300a33b0 0x7f4d300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.008+0000 7f4d56018740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 0x5647220a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.008+0000 7f4d56018740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.008+0000 7f4d56018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 0x5647220b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.008+0000 7f4d56018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5647220073e0 con 0x5647220b4580 2024-09-19T04:23:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.008+0000 7f4d56018740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564721faed80 con 0x5647220a1390 2024-09-19T04:23:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.008+0000 7f4d56018740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5647220033a0 con 0x5647220a0a70 2024-09-19T04:23:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59130/0 (socket says 172.21.15.107:59130) 2024-09-19T04:23:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d51400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 0x5647220b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d43e00640 1 -- 172.21.15.107:0/3613278278 learned_addr learned my addr 172.21.15.107:0/3613278278 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d51400640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d300a33b0 0x7f4d300a5870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c560 tx=0x7f4d44009710 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d51400640 1 -- 172.21.15.107:0/3613278278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 msgr2=0x5647220a0e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:23:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d51400640 1 --2- 172.21.15.107:0/3613278278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 0x5647220a0e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d51400640 1 -- 172.21.15.107:0/3613278278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 msgr2=0x5647220b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d51400640 1 --2- 172.21.15.107:0/3613278278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d51400640 1 -- 172.21.15.107:0/3613278278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564721fcdae0 con 0x5647220b4580 2024-09-19T04:23:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.009+0000 7f4d51400640 1 --2- 172.21.15.107:0/3613278278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 0x5647220b6970 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400cdf0 tx=0x7f4d44019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78039fa48bbf99c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:31.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.010+0000 7f4d41c00640 1 -- 172.21.15.107:0/3613278278 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4401b040 con 0x5647220b4580 2024-09-19T04:23:31.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.010+0000 7f4d41c00640 1 -- 172.21.15.107:0/3613278278 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d44024070 con 0x5647220b4580 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.010+0000 7f4d41c00640 1 -- 172.21.15.107:0/3613278278 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4401fb50 con 0x5647220b4580 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.010+0000 7f4d56018740 1 -- 172.21.15.107:0/3613278278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 msgr2=0x5647220b6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.010+0000 7f4d56018740 1 --2- 172.21.15.107:0/3613278278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 0x5647220b6970 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400cdf0 tx=0x7f4d44019000 comp rx=0 tx=0).stop 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.011+0000 7f4d56018740 1 -- 172.21.15.107:0/3613278278 shutdown_connections 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.011+0000 7f4d56018740 1 --2- 172.21.15.107:0/3613278278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 0x5647220b6970 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.011+0000 7f4d56018740 1 --2- 172.21.15.107:0/3613278278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.011+0000 7f4d56018740 1 --2- 172.21.15.107:0/3613278278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 0x5647220a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.011+0000 7f4d56018740 1 -- 172.21.15.107:0/3613278278 >> 172.21.15.107:0/3613278278 conn(0x56472209d120 msgr2=0x56472209d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.011+0000 7f4d56018740 1 -- 172.21.15.107:0/3613278278 shutdown_connections 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.011+0000 7f4d56018740 1 -- 172.21.15.107:0/3613278278 wait complete. 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.011+0000 7f4d56018740 1 -- start start 2024-09-19T04:23:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.011+0000 7f4d56018740 1 Processor -- start 2024-09-19T04:23:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d56018740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a0a70 0x5647220b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d56018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220a1390 0x5647220b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d56018740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220b4580 0x5647220b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d56018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5647220b71f0 con 0x5647220a1390 2024-09-19T04:23:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d56018740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564721fc20f0 con 0x5647220a0a70 2024-09-19T04:23:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d56018740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5647220b7370 con 0x5647220b4580 2024-09-19T04:23:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d56018740 10 client.0 _ll_register_callbacks cb 0x564721fb39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d50a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a0a70 0x5647220b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d50a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a0a70 0x5647220b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59144/0 (socket says 172.21.15.107:59144) 2024-09-19T04:23:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220a1390 0x5647220b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.012+0000 7f4d50a00640 1 -- 172.21.15.107:0/240379551 learned_addr learned my addr 172.21.15.107:0/240379551 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d50a00640 1 -- 172.21.15.107:0/240379551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220b4580 msgr2=0x5647220b1f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:23:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d50a00640 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220b4580 0x5647220b1f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d50a00640 1 -- 172.21.15.107:0/240379551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220a1390 msgr2=0x5647220b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d50a00640 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220a1390 0x5647220b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d50a00640 1 -- 172.21.15.107:0/240379551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564721d783c0 con 0x5647220a0a70 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d50a00640 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a0a70 0x5647220b1510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0edbd0 tx=0x7f4d3c0bc240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ab446f0f86b384d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d3a000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d3a000640 1 -- 172.21.15.107:0/240379551 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0d5060 con 0x5647220a0a70 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d3a000640 1 -- 172.21.15.107:0/240379551 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4d3c0b8d70 con 0x5647220a0a70 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d56018740 1 -- 172.21.15.107:0/240379551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d44024070 con 0x5647220a0a70 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d3a000640 1 -- 172.21.15.107:0/240379551 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4d3c049070 con 0x5647220a0a70 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d56018740 10 client.25769 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.013+0000 7f4d56018740 1 -- 172.21.15.107:0/240379551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5647220c2f80 con 0x5647220a0a70 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.014+0000 7f4d3a000640 1 -- 172.21.15.107:0/240379551 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d44024070 con 0x5647220a0a70 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.014+0000 7f4d3a000640 1 -- 172.21.15.107:0/240379551 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f4d3c0bc960 con 0x5647220a0a70 2024-09-19T04:23:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.014+0000 7f4d56018740 10 client.25769 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:31.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.014+0000 7f4d56018740 10 client.25769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:31.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.014+0000 7f4d56018740 20 client.25769 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:31.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.014+0000 7f4d56018740 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5647220c43a0 0x5647220c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.014+0000 7f4d56018740 4 client.25769 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:31.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.014+0000 7f4d56018740 1 -- 172.21.15.107:0/240379551 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564722014950 con 0x5647220c43a0 2024-09-19T04:23:31.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.027+0000 7f4d43e00640 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5647220c43a0 0x5647220c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.028+0000 7f4d43e00640 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5647220c43a0 0x5647220c6840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc76a609b178703 server_cookie=f646e3a4f6ff0b73 in_seq=0 out_seq=0 2024-09-19T04:23:31.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.028+0000 7f4d3a000640 10 client.25769 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:31.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.028+0000 7f4d3a000640 1 -- 172.21.15.107:0/240379551 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564722014950 con 0x5647220c43a0 2024-09-19T04:23:31.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.028+0000 7f4d3a000640 10 client.25769 handle_command_reply: tid=0 2024-09-19T04:23:31.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.028+0000 7f4d56018740 1 client.25769 shutdown 2024-09-19T04:23:31.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.028+0000 7f4d56018740 1 -- 172.21.15.107:0/240379551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a0a70 msgr2=0x5647220b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:31.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.028+0000 7f4d56018740 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a0a70 0x5647220b1510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0edbd0 tx=0x7f4d3c0bc240 comp rx=0 tx=0).stop 2024-09-19T04:23:31.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.029+0000 7f4d56018740 1 -- 172.21.15.107:0/240379551 shutdown_connections 2024-09-19T04:23:31.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.029+0000 7f4d56018740 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5647220c43a0 0x5647220c6840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.029+0000 7f4d56018740 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220b4580 0x5647220b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.029+0000 7f4d56018740 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220a1390 0x5647220b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.029+0000 7f4d56018740 1 --2- 172.21.15.107:0/240379551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a0a70 0x5647220b1510 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.029+0000 7f4d3a000640 0 client.25769 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:31.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.029+0000 7f4d56018740 1 -- 172.21.15.107:0/240379551 >> 172.21.15.107:0/240379551 conn(0x56472209d120 msgr2=0x5647220a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:31.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.029+0000 7f4d56018740 1 -- 172.21.15.107:0/240379551 shutdown_connections 2024-09-19T04:23:31.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.029+0000 7f4d56018740 1 -- 172.21.15.107:0/240379551 wait complete. 2024-09-19T04:23:31.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.029+0000 7f4d56018740 20 client.25769 trim_cache size 0 max 16384 2024-09-19T04:23:31.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.048+0000 7f4d56018740 1 Processor -- start 2024-09-19T04:23:31.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.048+0000 7f4d56018740 1 -- start start 2024-09-19T04:23:31.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.048+0000 7f4d56018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220a0a70 0x5647220afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.048+0000 7f4d56018740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a1390 0x5647220cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.048+0000 7f4d56018740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220b4580 0x5647220cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.048+0000 7f4d56018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1ab5b0 con 0x5647220a0a70 2024-09-19T04:23:31.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.048+0000 7f4d56018740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1ad1e0 con 0x5647220b4580 2024-09-19T04:23:31.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.048+0000 7f4d56018740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1abc80 con 0x5647220a1390 2024-09-19T04:23:31.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.048+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a1390 0x5647220cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d50a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220a0a70 0x5647220afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d51400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220b4580 0x5647220cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a1390 0x5647220cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:47480/0 (socket says 172.21.15.107:47480) 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d43e00640 1 -- 172.21.15.107:0/3326066685 learned_addr learned my addr 172.21.15.107:0/3326066685 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d43e00640 1 -- 172.21.15.107:0/3326066685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220b4580 msgr2=0x5647220cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d43e00640 1 --2- 172.21.15.107:0/3326066685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220b4580 0x5647220cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d43e00640 1 -- 172.21.15.107:0/3326066685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220a0a70 msgr2=0x5647220afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d43e00640 1 --2- 172.21.15.107:0/3326066685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220a0a70 0x5647220afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d43e00640 1 -- 172.21.15.107:0/3326066685 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647220afa60 con 0x5647220a1390 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d43e00640 1 --2- 172.21.15.107:0/3326066685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a1390 0x5647220cc110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5647220c70b0 tx=0x7f4d34014840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a454e0caefaaa5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d41200640 1 -- 172.21.15.107:0/3326066685 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d340142d0 con 0x5647220a1390 2024-09-19T04:23:31.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d41200640 1 -- 172.21.15.107:0/3326066685 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3401c890 con 0x5647220a1390 2024-09-19T04:23:31.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.049+0000 7f4d41200640 1 -- 172.21.15.107:0/3326066685 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34012a10 con 0x5647220a1390 2024-09-19T04:23:31.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.050+0000 7f4d56018740 1 -- 172.21.15.107:0/3326066685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a1390 msgr2=0x5647220cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:31.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.050+0000 7f4d56018740 1 --2- 172.21.15.107:0/3326066685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a1390 0x5647220cc110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5647220c70b0 tx=0x7f4d34014840 comp rx=0 tx=0).stop 2024-09-19T04:23:31.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.050+0000 7f4d56018740 1 -- 172.21.15.107:0/3326066685 shutdown_connections 2024-09-19T04:23:31.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.050+0000 7f4d56018740 1 --2- 172.21.15.107:0/3326066685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220b4580 0x5647220cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.050+0000 7f4d56018740 1 --2- 172.21.15.107:0/3326066685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a1390 0x5647220cc110 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.050+0000 7f4d56018740 1 --2- 172.21.15.107:0/3326066685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220a0a70 0x5647220afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.050+0000 7f4d56018740 1 -- 172.21.15.107:0/3326066685 >> 172.21.15.107:0/3326066685 conn(0x56472209d120 msgr2=0x56472209c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:31.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.050+0000 7f4d56018740 1 -- 172.21.15.107:0/3326066685 shutdown_connections 2024-09-19T04:23:31.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.050+0000 7f4d56018740 1 -- 172.21.15.107:0/3326066685 wait complete. 2024-09-19T04:23:31.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.051+0000 7f4d56018740 1 -- start start 2024-09-19T04:23:31.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.051+0000 7f4d56018740 1 Processor -- start 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.051+0000 7f4d56018740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 0x5647220b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.051+0000 7f4d56018740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.051+0000 7f4d56018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 0x56472212e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.051+0000 7f4d56018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1aca50 con 0x5647220b4580 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.051+0000 7f4d56018740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1ac350 con 0x5647220a1390 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.051+0000 7f4d56018740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1a9590 con 0x5647220a0a70 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d56018740 10 client.0 _ll_register_callbacks cb 0x7f4d3c0b89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d50a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 0x5647220b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59166/0 (socket says 172.21.15.107:59166) 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d50a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 0x5647220b89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:47490/0 (socket says 172.21.15.107:47490) 2024-09-19T04:23:31.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d43e00640 1 -- 172.21.15.107:0/3399716976 learned_addr learned my addr 172.21.15.107:0/3399716976 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d51400640 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 0x56472212e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d50a00640 1 -- 172.21.15.107:0/3399716976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 msgr2=0x5647220b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d50a00640 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d50a00640 1 -- 172.21.15.107:0/3399716976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 msgr2=0x56472212e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d50a00640 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 0x56472212e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d50a00640 1 -- 172.21.15.107:0/3399716976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472212e620 con 0x5647220a0a70 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d43e00640 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d50a00640 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 0x5647220b89b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0b90d0 tx=0x7f4d3c009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63d56e8c1345471e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.052+0000 7f4d3a000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d3a000640 1 -- 172.21.15.107:0/3399716976 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0bc060 con 0x5647220a0a70 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d3a000640 1 -- 172.21.15.107:0/3399716976 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f4d3c0bbb50 con 0x5647220a0a70 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d56018740 1 -- 172.21.15.107:0/3399716976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3401c890 con 0x5647220a0a70 2024-09-19T04:23:31.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d3a000640 1 -- 172.21.15.107:0/3399716976 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4d3c0bcb60 con 0x5647220a0a70 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d56018740 10 client.25753 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d56018740 1 -- 172.21.15.107:0/3399716976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56472212f840 con 0x5647220a0a70 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d3a000640 1 -- 172.21.15.107:0/3399716976 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3401c890 con 0x5647220a0a70 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d3a000640 1 -- 172.21.15.107:0/3399716976 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f4d3c0b8450 con 0x5647220a0a70 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d56018740 10 client.25753 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d56018740 10 client.25753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d56018740 20 client.25753 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d56018740 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56472212fe80 0x5647221322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d56018740 4 client.25753 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.053+0000 7f4d56018740 1 -- 172.21.15.107:0/3399716976 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56472200ee60 con 0x56472212fe80 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.054+0000 7f4d43e00640 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56472212fe80 0x5647221322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.054+0000 7f4d43e00640 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56472212fe80 0x5647221322a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f649b01ee9bb80 server_cookie=c059a31578634cb in_seq=0 out_seq=0 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.054+0000 7f4d3a000640 10 client.25753 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:31.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.054+0000 7f4d3a000640 1 -- 172.21.15.107:0/3399716976 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56472200ee60 con 0x56472212fe80 2024-09-19T04:23:31.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.054+0000 7f4d3a000640 10 client.25753 handle_command_reply: tid=0 2024-09-19T04:23:31.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.054+0000 7f4d56018740 1 client.25753 shutdown 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.054+0000 7f4d56018740 1 -- 172.21.15.107:0/3399716976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 msgr2=0x5647220b89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.054+0000 7f4d56018740 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 0x5647220b89b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0b90d0 tx=0x7f4d3c009680 comp rx=0 tx=0).stop 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d56018740 1 -- 172.21.15.107:0/3399716976 shutdown_connections 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d56018740 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56472212fe80 0x5647221322a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d56018740 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647220b4580 0x56472212e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d56018740 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647220a1390 0x5647220b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d56018740 1 --2- 172.21.15.107:0/3399716976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5647220a0a70 0x5647220b89b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d3a000640 0 client.25753 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d56018740 1 -- 172.21.15.107:0/3399716976 >> 172.21.15.107:0/3399716976 conn(0x56472209d120 msgr2=0x5647220a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d56018740 1 -- 172.21.15.107:0/3399716976 shutdown_connections 2024-09-19T04:23:31.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d56018740 1 -- 172.21.15.107:0/3399716976 wait complete. 2024-09-19T04:23:31.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d56018740 20 client.25753 trim_cache size 0 max 16384 2024-09-19T04:23:31.062 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:23:31.062 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:23:31.062 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:23:31.062 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:23:31.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d53400640 1 -- 172.21.15.107:0/2477460013 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d300a33b0 msgr2=0x7f4d300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:31.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.055+0000 7f4d53400640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d300a33b0 0x7f4d300a5870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c560 tx=0x7f4d44009710 comp rx=0 tx=0).stop 2024-09-19T04:23:31.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.056+0000 7f4d53400640 1 -- 172.21.15.107:0/2477460013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d4c104010 msgr2=0x7f4d4c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:31.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.056+0000 7f4d53400640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d4c104010 0x7f4d4c19a750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c02a930 tx=0x7f4d3c003b90 comp rx=0 tx=0).stop 2024-09-19T04:23:31.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.056+0000 7f4d53400640 1 -- 172.21.15.107:0/2477460013 shutdown_connections 2024-09-19T04:23:31.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.056+0000 7f4d53400640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4d300a33b0 0x7f4d300a5870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.056+0000 7f4d53400640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d4c109140 0x7f4d4c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.056+0000 7f4d53400640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d4c1049c0 0x7f4d4c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.056+0000 7f4d53400640 1 --2- 172.21.15.107:0/2477460013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d4c104010 0x7f4d4c19a750 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:31.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.056+0000 7f4d53400640 1 -- 172.21.15.107:0/2477460013 >> 172.21.15.107:0/2477460013 conn(0x7f4d4c0ffa50 msgr2=0x7f4d4c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:31.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.056+0000 7f4d53400640 1 -- 172.21.15.107:0/2477460013 shutdown_connections 2024-09-19T04:23:31.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:31.056+0000 7f4d53400640 1 -- 172.21.15.107:0/2477460013 wait complete. 2024-09-19T04:23:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:30 smithi115 ceph-mon[35477]: pgmap v98: 97 pgs: 97 active+clean; 82 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 881 KiB/s wr, 76 op/s 2024-09-19T04:23:31.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 121014 6.16 MB/sec warmup 109 sec latency 611.748 ms 2024-09-19T04:23:31.978 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:23:31.979 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:23:32.353 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:32.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 121356 6.10 MB/sec warmup 110 sec latency 649.495 ms 2024-09-19T04:23:32.811 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:32 smithi107 ceph-mon[30659]: pgmap v99: 97 pgs: 97 active+clean; 81 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-09-19T04:23:32.811 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:32 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:23:32.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.925+0000 7f1010200640 1 -- 172.21.15.107:0/726878711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1008100b30 msgr2=0x7f1008100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:32.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.925+0000 7f1010200640 1 --2- 172.21.15.107:0/726878711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1008100b30 0x7f1008100f10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009900 tx=0x7f0ffc02a470 comp rx=0 tx=0).stop 2024-09-19T04:23:32.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.925+0000 7f1010200640 1 -- 172.21.15.107:0/726878711 shutdown_connections 2024-09-19T04:23:32.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.925+0000 7f1010200640 1 --2- 172.21.15.107:0/726878711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1008105f40 0x7f10081063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:32.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.925+0000 7f1010200640 1 --2- 172.21.15.107:0/726878711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1008101450 0x7f1008105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:32.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.925+0000 7f1010200640 1 --2- 172.21.15.107:0/726878711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1008100b30 0x7f1008100f10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:32.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.925+0000 7f1010200640 1 -- 172.21.15.107:0/726878711 >> 172.21.15.107:0/726878711 conn(0x7f10080fc7a0 msgr2=0x7f10080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:32.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.926+0000 7f1010200640 1 -- 172.21.15.107:0/726878711 shutdown_connections 2024-09-19T04:23:32.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.926+0000 7f1010200640 1 -- 172.21.15.107:0/726878711 wait complete. 2024-09-19T04:23:32.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.927+0000 7f1010200640 1 Processor -- start 2024-09-19T04:23:32.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.927+0000 7f1010200640 1 -- start start 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.927+0000 7f1010200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1008100b30 0x7f10081a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.928+0000 7f1010200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1008101450 0x7f10081a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.928+0000 7f1010200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1008105f40 0x7f100819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.928+0000 7f1010200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1008118570 con 0x7f1008101450 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.928+0000 7f1010200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f10081183f0 con 0x7f1008105f40 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.928+0000 7f1010200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10081186f0 con 0x7f1008100b30 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.928+0000 7f100d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1008100b30 0x7f10081a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.928+0000 7f100d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1008100b30 0x7f10081a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:47506/0 (socket says 172.21.15.107:47506) 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.928+0000 7f100d800640 1 -- 172.21.15.107:0/1763980755 learned_addr learned my addr 172.21.15.107:0/1763980755 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.928+0000 7f100ce00640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1008101450 0x7f10081a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.928+0000 7f100e200640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1008105f40 0x7f100819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.929+0000 7f100d800640 1 -- 172.21.15.107:0/1763980755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1008105f40 msgr2=0x7f100819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.929+0000 7f100d800640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1008105f40 0x7f100819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.929+0000 7f100d800640 1 -- 172.21.15.107:0/1763980755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1008101450 msgr2=0x7f10081a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.929+0000 7f100d800640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1008101450 0x7f10081a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.929+0000 7f100d800640 1 -- 172.21.15.107:0/1763980755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100819f000 con 0x7f1008100b30 2024-09-19T04:23:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.929+0000 7f100e200640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1008105f40 0x7f100819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.929+0000 7f100d800640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1008100b30 0x7f10081a4710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc006010 tx=0x7f0ffc004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:32.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.929+0000 7f1006000640 1 -- 172.21.15.107:0/1763980755 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc03f030 con 0x7f1008100b30 2024-09-19T04:23:32.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.929+0000 7f1006000640 1 -- 172.21.15.107:0/1763980755 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ffc004440 con 0x7f1008100b30 2024-09-19T04:23:32.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.929+0000 7f1006000640 1 -- 172.21.15.107:0/1763980755 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc035070 con 0x7f1008100b30 2024-09-19T04:23:32.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.930+0000 7f1010200640 1 -- 172.21.15.107:0/1763980755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100819f290 con 0x7f1008100b30 2024-09-19T04:23:32.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.930+0000 7f1010200640 1 -- 172.21.15.107:0/1763980755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10081ab5e0 con 0x7f1008100b30 2024-09-19T04:23:32.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.931+0000 7f1010200640 1 -- 172.21.15.107:0/1763980755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100806b070 con 0x7f1008100b30 2024-09-19T04:23:32.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.932+0000 7f1006000640 1 -- 172.21.15.107:0/1763980755 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f0ffc042030 con 0x7f1008100b30 2024-09-19T04:23:32.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.934+0000 7f1006000640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ff00a3430 0x7f0ff00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:32.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.934+0000 7f1006000640 1 -- 172.21.15.107:0/1763980755 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0ffc0effe0 con 0x7f1008100b30 2024-09-19T04:23:32.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.934+0000 7f100ce00640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ff00a3430 0x7f0ff00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:32.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.935+0000 7f100ce00640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ff00a3430 0x7f0ff00a58f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10080fec40 tx=0x7f0ff4005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:32.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:32.938+0000 7f1006000640 1 -- 172.21.15.107:0/1763980755 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0ffc0bc380 con 0x7f1008100b30 2024-09-19T04:23:32.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:32 smithi145 ceph-mon[35096]: pgmap v99: 97 pgs: 97 active+clean; 81 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-09-19T04:23:32.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:32 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:23:33.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.158+0000 7f1010200640 1 -- 172.21.15.107:0/1763980755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10081ab810 con 0x7f1008100b30 2024-09-19T04:23:33.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.159+0000 7f1006000640 1 -- 172.21.15.107:0/1763980755 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12000 (secure 0 0 0) 0x7f0ffc0c0680 con 0x7f1008100b30 2024-09-19T04:23:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:23:33.160 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":31,"btime":"2024-09-19T04:22:42:592499+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:22:41.847132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:23:33.161 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 31 2024-09-19T04:23:33.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.161+0000 7f1010200640 1 -- 172.21.15.107:0/1763980755 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ff00a3430 msgr2=0x7f0ff00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.161+0000 7f1010200640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ff00a3430 0x7f0ff00a58f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10080fec40 tx=0x7f0ff4005f90 comp rx=0 tx=0).stop 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.161+0000 7f1010200640 1 -- 172.21.15.107:0/1763980755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1008100b30 msgr2=0x7f10081a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.161+0000 7f1010200640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1008100b30 0x7f10081a4710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc006010 tx=0x7f0ffc004040 comp rx=0 tx=0).stop 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.162+0000 7f1010200640 1 -- 172.21.15.107:0/1763980755 shutdown_connections 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.162+0000 7f1010200640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0ff00a3430 0x7f0ff00a58f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.162+0000 7f1010200640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1008105f40 0x7f100819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.162+0000 7f1010200640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1008101450 0x7f10081a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.162+0000 7f1010200640 1 --2- 172.21.15.107:0/1763980755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1008100b30 0x7f10081a4710 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.162+0000 7f1010200640 1 -- 172.21.15.107:0/1763980755 >> 172.21.15.107:0/1763980755 conn(0x7f10080fc7a0 msgr2=0x7f100810f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.162+0000 7f1010200640 1 -- 172.21.15.107:0/1763980755 shutdown_connections 2024-09-19T04:23:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:33.162+0000 7f1010200640 1 -- 172.21.15.107:0/1763980755 wait complete. 2024-09-19T04:23:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:32 smithi115 ceph-mon[35477]: pgmap v99: 97 pgs: 97 active+clean; 81 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-09-19T04:23:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:32 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:23:33.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 121356 6.05 MB/sec warmup 111 sec latency 1649.542 ms 2024-09-19T04:23:33.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:33 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1763980755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:23:33.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:33 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:23:33.953 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:33 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:23:33.953 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:33 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:23:33.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:33 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1763980755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:23:33.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:33 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:23:33.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:33 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:23:33.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:33 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:23:33.986 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:23:34.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:33 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1763980755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:23:34.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:33 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:23:34.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:33 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:23:34.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:33 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:23:34.360 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:34.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 122273 6.04 MB/sec warmup 112 sec latency 2219.198 ms 2024-09-19T04:23:34.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:34 smithi107 ceph-mon[30659]: pgmap v100: 97 pgs: 97 active+clean; 81 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-09-19T04:23:34.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:34 smithi145 ceph-mon[35096]: pgmap v100: 97 pgs: 97 active+clean; 81 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-09-19T04:23:35.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.006+0000 7f2152000640 1 -- 172.21.15.107:0/1682494274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f214c1019c0 msgr2=0x7f214c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:35.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.006+0000 7f2152000640 1 --2- 172.21.15.107:0/1682494274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f214c1019c0 0x7f214c101e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2138009900 tx=0x7f213802a410 comp rx=0 tx=0).stop 2024-09-19T04:23:35.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.006+0000 7f2152000640 1 -- 172.21.15.107:0/1682494274 shutdown_connections 2024-09-19T04:23:35.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.006+0000 7f2152000640 1 --2- 172.21.15.107:0/1682494274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214c102360 0x7f214c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.006+0000 7f2152000640 1 --2- 172.21.15.107:0/1682494274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f214c1019c0 0x7f214c101e20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.006+0000 7f2152000640 1 --2- 172.21.15.107:0/1682494274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f214c1079c0 0x7f214c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.006+0000 7f2152000640 1 -- 172.21.15.107:0/1682494274 >> 172.21.15.107:0/1682494274 conn(0x7f214c0fd6e0 msgr2=0x7f214c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:35.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.006+0000 7f2152000640 1 -- 172.21.15.107:0/1682494274 shutdown_connections 2024-09-19T04:23:35.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.006+0000 7f2152000640 1 -- 172.21.15.107:0/1682494274 wait complete. 2024-09-19T04:23:35.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.007+0000 7f2152000640 1 Processor -- start 2024-09-19T04:23:35.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.007+0000 7f2152000640 1 -- start start 2024-09-19T04:23:35.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f2152000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214c1019c0 0x7f214c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f2152000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f214c102360 0x7f214c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f2152000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f214c1079c0 0x7f214c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f2152000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f214c111dc0 con 0x7f214c1079c0 2024-09-19T04:23:35.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f2152000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f214c111c40 con 0x7f214c102360 2024-09-19T04:23:35.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f2152000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f214c111f40 con 0x7f214c1019c0 2024-09-19T04:23:35.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f214be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f214c102360 0x7f214c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f2150c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214c1019c0 0x7f214c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f214be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f214c102360 0x7f214c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59210/0 (socket says 172.21.15.107:59210) 2024-09-19T04:23:35.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f214be00640 1 -- 172.21.15.107:0/1471906713 learned_addr learned my addr 172.21.15.107:0/1471906713 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:35.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.008+0000 7f2151600640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f214c1079c0 0x7f214c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2150c00640 1 -- 172.21.15.107:0/1471906713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f214c102360 msgr2=0x7f214c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2150c00640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f214c102360 0x7f214c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2150c00640 1 -- 172.21.15.107:0/1471906713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f214c1079c0 msgr2=0x7f214c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2150c00640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f214c1079c0 0x7f214c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2150c00640 1 -- 172.21.15.107:0/1471906713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c1aabf0 con 0x7f214c1019c0 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f214be00640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f214c102360 0x7f214c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2150c00640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214c1019c0 0x7f214c0744f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f214000e9f0 tx=0x7f214000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2149600640 1 -- 172.21.15.107:0/1471906713 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214000ccc0 con 0x7f214c1019c0 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2152000640 1 -- 172.21.15.107:0/1471906713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214c1aae20 con 0x7f214c1019c0 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2149600640 1 -- 172.21.15.107:0/1471906713 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2140010910 con 0x7f214c1019c0 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2149600640 1 -- 172.21.15.107:0/1471906713 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140018960 con 0x7f214c1019c0 2024-09-19T04:23:35.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.009+0000 7f2152000640 1 -- 172.21.15.107:0/1471906713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214c1ab270 con 0x7f214c1019c0 2024-09-19T04:23:35.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.012+0000 7f2149600640 1 -- 172.21.15.107:0/1471906713 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f2140010ac0 con 0x7f214c1019c0 2024-09-19T04:23:35.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.012+0000 7f2156086740 1 Processor -- start 2024-09-19T04:23:35.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2156086740 1 -- start start 2024-09-19T04:23:35.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2156086740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13104e40 0x555f13105220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2156086740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13105760 0x555f1310fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2156086740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 0x555f13112410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2156086740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555f1306b3e0 con 0x555f13104e40 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2156086740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f13012d80 con 0x555f13110020 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2156086740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555f130673a0 con 0x555f13105760 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2150c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13105760 0x555f1310fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2150c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13105760 0x555f1310fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:47538/0 (socket says 172.21.15.107:47538) 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f214be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 0x555f13112410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2150c00640 1 -- 172.21.15.107:0/1868295734 learned_addr learned my addr 172.21.15.107:0/1868295734 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.013+0000 7f2151600640 1 --2- 172.21.15.107:0/1868295734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13104e40 0x555f13105220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.014+0000 7f2150c00640 1 -- 172.21.15.107:0/1868295734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 msgr2=0x555f13112410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.014+0000 7f2150c00640 1 --2- 172.21.15.107:0/1868295734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 0x555f13112410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.014+0000 7f2150c00640 1 -- 172.21.15.107:0/1868295734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13104e40 msgr2=0x555f13105220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.014+0000 7f2150c00640 1 --2- 172.21.15.107:0/1868295734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13104e40 0x555f13105220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.014+0000 7f2150c00640 1 -- 172.21.15.107:0/1868295734 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f13031ae0 con 0x555f13105760 2024-09-19T04:23:35.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.014+0000 7f2150c00640 1 --2- 172.21.15.107:0/1868295734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13105760 0x555f1310fae0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21400cd690 tx=0x7f21400d86f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3599fe9b4bcde5bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:35.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.014+0000 7f2149600640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f21340a33b0 0x7f21340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.014+0000 7f213dc00640 1 -- 172.21.15.107:0/1868295734 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21400e3070 con 0x555f13105760 2024-09-19T04:23:35.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.015+0000 7f213dc00640 1 -- 172.21.15.107:0/1868295734 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f214009c390 con 0x555f13105760 2024-09-19T04:23:35.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.015+0000 7f213dc00640 1 -- 172.21.15.107:0/1868295734 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140098630 con 0x555f13105760 2024-09-19T04:23:35.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.015+0000 7f2149600640 1 -- 172.21.15.107:0/1471906713 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f21400cfff0 con 0x7f214c1019c0 2024-09-19T04:23:35.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.015+0000 7f2151600640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f21340a33b0 0x7f21340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.015+0000 7f2156086740 1 -- 172.21.15.107:0/1868295734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13105760 msgr2=0x555f1310fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:35.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.015+0000 7f2151600640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f21340a33b0 0x7f21340a5870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2144005fd0 tx=0x7f2144009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:35.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.015+0000 7f2156086740 1 --2- 172.21.15.107:0/1868295734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13105760 0x555f1310fae0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21400cd690 tx=0x7f21400d86f0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.016+0000 7f2156086740 1 -- 172.21.15.107:0/1868295734 shutdown_connections 2024-09-19T04:23:35.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.016+0000 7f2156086740 1 --2- 172.21.15.107:0/1868295734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 0x555f13112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.016+0000 7f2156086740 1 --2- 172.21.15.107:0/1868295734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13105760 0x555f1310fae0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.016+0000 7f2156086740 1 --2- 172.21.15.107:0/1868295734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13104e40 0x555f13105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.016+0000 7f2156086740 1 -- 172.21.15.107:0/1868295734 >> 172.21.15.107:0/1868295734 conn(0x555f131013d0 msgr2=0x555f131017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:35.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.016+0000 7f2156086740 1 -- 172.21.15.107:0/1868295734 shutdown_connections 2024-09-19T04:23:35.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.016+0000 7f2156086740 1 -- 172.21.15.107:0/1868295734 wait complete. 2024-09-19T04:23:35.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.016+0000 7f2156086740 1 -- start start 2024-09-19T04:23:35.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f2156086740 1 Processor -- start 2024-09-19T04:23:35.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f2156086740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13104e40 0x555f1310cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f2156086740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13105760 0x555f1310d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f2156086740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 0x555f1310d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f2156086740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555f13112bf0 con 0x555f13105760 2024-09-19T04:23:35.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f2156086740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f130260f0 con 0x555f13110020 2024-09-19T04:23:35.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f2156086740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555f13112d70 con 0x555f13104e40 2024-09-19T04:23:35.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f2156086740 10 client.0 _ll_register_callbacks cb 0x555f1300dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f2150c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13104e40 0x555f1310cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f2151600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 0x555f1310d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.017+0000 7f214be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13105760 0x555f1310d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.018+0000 7f2150c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13104e40 0x555f1310cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:47540/0 (socket says 172.21.15.107:47540) 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.018+0000 7f2150c00640 1 -- 172.21.15.107:0/3598146504 learned_addr learned my addr 172.21.15.107:0/3598146504 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.018+0000 7f2151600640 1 -- 172.21.15.107:0/3598146504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13104e40 msgr2=0x555f1310cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.018+0000 7f2151600640 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13104e40 0x555f1310cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.018+0000 7f2151600640 1 -- 172.21.15.107:0/3598146504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13105760 msgr2=0x555f1310d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.018+0000 7f2151600640 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13105760 0x555f1310d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.018+0000 7f2151600640 1 -- 172.21.15.107:0/3598146504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f12ddc3c0 con 0x555f13110020 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.018+0000 7f2132000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:23:35.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.018+0000 7f2151600640 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 0x555f1310d9b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2144003f50 tx=0x7f2144004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb07ae46dea4aef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.018+0000 7f2132000640 1 -- 172.21.15.107:0/3598146504 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214400f7b0 con 0x555f13110020 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2156086740 1 -- 172.21.15.107:0/3598146504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214009c390 con 0x555f13110020 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2132000640 1 -- 172.21.15.107:0/3598146504 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f214400f950 con 0x555f13110020 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2132000640 1 -- 172.21.15.107:0/3598146504 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f214400fb50 con 0x555f13110020 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2156086740 10 client.25793 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2156086740 1 -- 172.21.15.107:0/3598146504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555f1311ed10 con 0x555f13110020 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2132000640 1 -- 172.21.15.107:0/3598146504 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f214009c390 con 0x555f13110020 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2132000640 1 -- 172.21.15.107:0/3598146504 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f2144026020 con 0x555f13110020 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2156086740 10 client.25793 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2156086740 10 client.25793 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2156086740 20 client.25793 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2156086740 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555f13120120 0x555f13122550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2156086740 4 client.25793 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:35.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.019+0000 7f2156086740 1 -- 172.21.15.107:0/3598146504 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f13078950 con 0x555f13120120 2024-09-19T04:23:35.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.020+0000 7f2150c00640 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555f13120120 0x555f13122550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.020+0000 7f2150c00640 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555f13120120 0x555f13122550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=122000889d865533 server_cookie=cb8c63beada2b1d8 in_seq=0 out_seq=0 2024-09-19T04:23:35.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.020+0000 7f2132000640 10 client.25793 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:23:35.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2132000640 1 -- 172.21.15.107:0/3598146504 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f13078950 con 0x555f13120120 2024-09-19T04:23:35.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2132000640 10 client.25793 handle_command_reply: tid=0 2024-09-19T04:23:35.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 client.25793 shutdown 2024-09-19T04:23:35.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 -- 172.21.15.107:0/3598146504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 msgr2=0x555f1310d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:35.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 0x555f1310d9b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2144003f50 tx=0x7f2144004ab0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 -- 172.21.15.107:0/3598146504 shutdown_connections 2024-09-19T04:23:35.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555f13120120 0x555f13122550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13110020 0x555f1310d9b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13105760 0x555f1310d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2132000640 0 client.25793 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:23:35.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 --2- 172.21.15.107:0/3598146504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13104e40 0x555f1310cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 -- 172.21.15.107:0/3598146504 >> 172.21.15.107:0/3598146504 conn(0x555f131013d0 msgr2=0x555f13107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:35.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 -- 172.21.15.107:0/3598146504 shutdown_connections 2024-09-19T04:23:35.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.021+0000 7f2156086740 1 -- 172.21.15.107:0/3598146504 wait complete. 2024-09-19T04:23:35.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.022+0000 7f2156086740 20 client.25793 trim_cache size 0 max 16384 2024-09-19T04:23:35.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.036+0000 7f2156086740 1 Processor -- start 2024-09-19T04:23:35.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.036+0000 7f2156086740 1 -- start start 2024-09-19T04:23:35.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.036+0000 7f2156086740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13104e40 0x555f131017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.036+0000 7f2156086740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13105760 0x555f13127e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.036+0000 7f2156086740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13110020 0x555f13128360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.036+0000 7f2156086740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b3ba0 con 0x555f13105760 2024-09-19T04:23:35.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.036+0000 7f2156086740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b57d0 con 0x555f13104e40 2024-09-19T04:23:35.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.036+0000 7f2156086740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b4270 con 0x555f13110020 2024-09-19T04:23:35.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.036+0000 7f2150c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13104e40 0x555f131017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f2150c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13104e40 0x555f131017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59234/0 (socket says 172.21.15.107:59234) 2024-09-19T04:23:35.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f2150c00640 1 -- 172.21.15.107:0/891226844 learned_addr learned my addr 172.21.15.107:0/891226844 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:35.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f214be00640 1 --2- 172.21.15.107:0/891226844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13105760 0x555f13127e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f2151600640 1 --2- 172.21.15.107:0/891226844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13110020 0x555f13128360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f2150c00640 1 -- 172.21.15.107:0/891226844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13110020 msgr2=0x555f13128360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:35.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f2150c00640 1 --2- 172.21.15.107:0/891226844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13110020 0x555f13128360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f2150c00640 1 -- 172.21.15.107:0/891226844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13105760 msgr2=0x555f13127e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:35.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f2150c00640 1 --2- 172.21.15.107:0/891226844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13105760 0x555f13127e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f2150c00640 1 -- 172.21.15.107:0/891226844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f1310bee0 con 0x555f13104e40 2024-09-19T04:23:35.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f2150c00640 1 --2- 172.21.15.107:0/891226844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13104e40 0x555f131017b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f21400d8bf0 tx=0x7f21400bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca30d9b571b3fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:35.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f213d200640 1 -- 172.21.15.107:0/891226844 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140053ec0 con 0x555f13104e40 2024-09-19T04:23:35.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f213d200640 1 -- 172.21.15.107:0/891226844 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21400d35f0 con 0x555f13104e40 2024-09-19T04:23:35.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.037+0000 7f213d200640 1 -- 172.21.15.107:0/891226844 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21400540c0 con 0x555f13104e40 2024-09-19T04:23:35.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.038+0000 7f2156086740 1 -- 172.21.15.107:0/891226844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13104e40 msgr2=0x555f131017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:35.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.038+0000 7f2156086740 1 --2- 172.21.15.107:0/891226844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13104e40 0x555f131017b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f21400d8bf0 tx=0x7f21400bc040 comp rx=0 tx=0).stop 2024-09-19T04:23:35.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.038+0000 7f2156086740 1 -- 172.21.15.107:0/891226844 shutdown_connections 2024-09-19T04:23:35.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.038+0000 7f2156086740 1 --2- 172.21.15.107:0/891226844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13110020 0x555f13128360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.038+0000 7f2156086740 1 --2- 172.21.15.107:0/891226844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13105760 0x555f13127e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.038+0000 7f2156086740 1 --2- 172.21.15.107:0/891226844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13104e40 0x555f131017b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.038+0000 7f2156086740 1 -- 172.21.15.107:0/891226844 >> 172.21.15.107:0/891226844 conn(0x555f131013d0 msgr2=0x555f1310efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:35.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.038+0000 7f2156086740 1 -- 172.21.15.107:0/891226844 shutdown_connections 2024-09-19T04:23:35.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.038+0000 7f2156086740 1 -- 172.21.15.107:0/891226844 wait complete. 2024-09-19T04:23:35.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.039+0000 7f2156086740 1 -- start start 2024-09-19T04:23:35.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.039+0000 7f2156086740 1 Processor -- start 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.039+0000 7f2156086740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13104e40 0x555f13114060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.039+0000 7f2156086740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13105760 0x555f131145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.039+0000 7f2156086740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13110020 0x555f13189df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.039+0000 7f2156086740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b5040 con 0x555f13104e40 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.039+0000 7f2156086740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b4940 con 0x555f13105760 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.039+0000 7f2156086740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b1b80 con 0x555f13110020 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.039+0000 7f2156086740 10 client.0 _ll_register_callbacks cb 0x7f2144009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f214be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13105760 0x555f131145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f2150c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13104e40 0x555f13114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f214be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13105760 0x555f131145a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59248/0 (socket says 172.21.15.107:59248) 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f214be00640 1 -- 172.21.15.107:0/114738758 learned_addr learned my addr 172.21.15.107:0/114738758 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f214be00640 1 -- 172.21.15.107:0/114738758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13110020 msgr2=0x555f13189df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:35.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f214be00640 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13110020 0x555f13189df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f214be00640 1 -- 172.21.15.107:0/114738758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13104e40 msgr2=0x555f13114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:35.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f214be00640 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13104e40 0x555f13114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f214be00640 1 -- 172.21.15.107:0/114738758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f13114e80 con 0x555f13105760 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f214be00640 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13105760 0x555f131145a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2138006010 tx=0x7f2138003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6b520b9601ddf42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.040+0000 7f2132000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.041+0000 7f2132000640 1 -- 172.21.15.107:0/114738758 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213804b060 con 0x555f13105760 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.041+0000 7f2132000640 1 -- 172.21.15.107:0/114738758 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f2138031540 con 0x555f13105760 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.041+0000 7f2156086740 1 -- 172.21.15.107:0/114738758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21400d35f0 con 0x555f13105760 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.041+0000 7f2132000640 1 -- 172.21.15.107:0/114738758 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2138035040 con 0x555f13105760 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.041+0000 7f2156086740 10 client.25799 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.041+0000 7f2156086740 1 -- 172.21.15.107:0/114738758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555f1318b440 con 0x555f13105760 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.041+0000 7f2132000640 1 -- 172.21.15.107:0/114738758 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21400d35f0 con 0x555f13105760 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.041+0000 7f2132000640 1 -- 172.21.15.107:0/114738758 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f2138050850 con 0x555f13105760 2024-09-19T04:23:35.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2156086740 10 client.25799 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2156086740 10 client.25799 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2156086740 20 client.25799 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2156086740 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555f1318bbf0 0x555f1318e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2156086740 4 client.25799 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2156086740 1 -- 172.21.15.107:0/114738758 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f13072e60 con 0x555f1318bbf0 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2150c00640 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555f1318bbf0 0x555f1318e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2150c00640 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555f1318bbf0 0x555f1318e010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6286614c3ea3b146 server_cookie=9795cb1af83dedd6 in_seq=0 out_seq=0 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2132000640 10 client.25799 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2132000640 1 -- 172.21.15.107:0/114738758 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f13072e60 con 0x555f1318bbf0 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.042+0000 7f2132000640 10 client.25799 handle_command_reply: tid=0 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 client.25799 shutdown 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 -- 172.21.15.107:0/114738758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13105760 msgr2=0x555f131145a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:35.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13105760 0x555f131145a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2138006010 tx=0x7f2138003960 comp rx=0 tx=0).stop 2024-09-19T04:23:35.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 -- 172.21.15.107:0/114738758 shutdown_connections 2024-09-19T04:23:35.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555f1318bbf0 0x555f1318e010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555f13110020 0x555f13189df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f13105760 0x555f131145a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 --2- 172.21.15.107:0/114738758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13104e40 0x555f13114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 -- 172.21.15.107:0/114738758 >> 172.21.15.107:0/114738758 conn(0x555f131013d0 msgr2=0x555f13106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:35.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2132000640 0 client.25799 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:23:35.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 -- 172.21.15.107:0/114738758 shutdown_connections 2024-09-19T04:23:35.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 1 -- 172.21.15.107:0/114738758 wait complete. 2024-09-19T04:23:35.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.043+0000 7f2156086740 20 client.25799 trim_cache size 0 max 16384 2024-09-19T04:23:35.048 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:23:35.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 -- 172.21.15.107:0/1471906713 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f21340a33b0 msgr2=0x7f21340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:35.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f21340a33b0 0x7f21340a5870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2144005fd0 tx=0x7f2144009500 comp rx=0 tx=0).stop 2024-09-19T04:23:35.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 -- 172.21.15.107:0/1471906713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214c1019c0 msgr2=0x7f214c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:35.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214c1019c0 0x7f214c0744f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f214000e9f0 tx=0x7f214000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 -- 172.21.15.107:0/1471906713 shutdown_connections 2024-09-19T04:23:35.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f21340a33b0 0x7f21340a5870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f214c1079c0 0x7f214c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f214c102360 0x7f214c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 --2- 172.21.15.107:0/1471906713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214c1019c0 0x7f214c0744f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:35.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 -- 172.21.15.107:0/1471906713 >> 172.21.15.107:0/1471906713 conn(0x7f214c0fd6e0 msgr2=0x7f214c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:35.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 -- 172.21.15.107:0/1471906713 shutdown_connections 2024-09-19T04:23:35.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:35.044+0000 7f2152000640 1 -- 172.21.15.107:0/1471906713 wait complete. 2024-09-19T04:23:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:34 smithi115 ceph-mon[35477]: pgmap v100: 97 pgs: 97 active+clean; 81 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-09-19T04:23:35.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 124413 6.10 MB/sec warmup 113 sec latency 59.217 ms 2024-09-19T04:23:36.052 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:23:36.446 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:36.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 125876 6.10 MB/sec warmup 114 sec latency 12.003 ms 2024-09-19T04:23:36.921 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:36 smithi107 ceph-mon[30659]: pgmap v101: 97 pgs: 97 active+clean; 80 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-09-19T04:23:36.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:36 smithi145 ceph-mon[35096]: pgmap v101: 97 pgs: 97 active+clean; 80 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-09-19T04:23:37.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.039+0000 7f108a600640 1 -- 172.21.15.107:0/1173685106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10841019d0 msgr2=0x7f1084101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:37.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.039+0000 7f108a600640 1 --2- 172.21.15.107:0/1173685106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10841019d0 0x7f1084101e30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1070009930 tx=0x7f107002a440 comp rx=0 tx=0).stop 2024-09-19T04:23:37.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.039+0000 7f108a600640 1 -- 172.21.15.107:0/1173685106 shutdown_connections 2024-09-19T04:23:37.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.039+0000 7f108a600640 1 --2- 172.21.15.107:0/1173685106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1084102370 0x7f108410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.039+0000 7f108a600640 1 --2- 172.21.15.107:0/1173685106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10841019d0 0x7f1084101e30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.039+0000 7f108a600640 1 --2- 172.21.15.107:0/1173685106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10841079d0 0x7f1084107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.039+0000 7f108a600640 1 -- 172.21.15.107:0/1173685106 >> 172.21.15.107:0/1173685106 conn(0x7f10840fd710 msgr2=0x7f10840ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:37.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.039+0000 7f108a600640 1 -- 172.21.15.107:0/1173685106 shutdown_connections 2024-09-19T04:23:37.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.039+0000 7f108a600640 1 -- 172.21.15.107:0/1173685106 wait complete. 2024-09-19T04:23:37.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.040+0000 7f108a600640 1 Processor -- start 2024-09-19T04:23:37.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.040+0000 7f108a600640 1 -- start start 2024-09-19T04:23:37.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.040+0000 7f108a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10841019d0 0x7f10841982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.040+0000 7f108a600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102370 0x7f1084198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.040+0000 7f108a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10841079d0 0x7f108419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.040+0000 7f108a600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1084111de0 con 0x7f10841019d0 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.040+0000 7f108a600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1084111c60 con 0x7f1084102370 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.040+0000 7f108a600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1084111f60 con 0x7f10841079d0 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1089200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10841019d0 0x7f10841982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1083e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102370 0x7f1084198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1083e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102370 0x7f1084198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59258/0 (socket says 172.21.15.107:59258) 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1083e00640 1 -- 172.21.15.107:0/3976651250 learned_addr learned my addr 172.21.15.107:0/3976651250 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1089c00640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10841079d0 0x7f108419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1083e00640 1 -- 172.21.15.107:0/3976651250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10841079d0 msgr2=0x7f108419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1083e00640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10841079d0 0x7f108419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1083e00640 1 -- 172.21.15.107:0/3976651250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10841019d0 msgr2=0x7f10841982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1083e00640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10841019d0 0x7f10841982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1083e00640 1 -- 172.21.15.107:0/3976651250 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108419d280 con 0x7f1084102370 2024-09-19T04:23:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.041+0000 7f1089200640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10841019d0 0x7f10841982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.042+0000 7f1083e00640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102370 0x7f1084198810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f107002a960 tx=0x7f1070003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.042+0000 7f1081600640 1 -- 172.21.15.107:0/3976651250 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107003f070 con 0x7f1084102370 2024-09-19T04:23:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.042+0000 7f1081600640 1 -- 172.21.15.107:0/3976651250 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1070031c90 con 0x7f1084102370 2024-09-19T04:23:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.042+0000 7f108a600640 1 -- 172.21.15.107:0/3976651250 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108419d510 con 0x7f1084102370 2024-09-19T04:23:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.042+0000 7f1081600640 1 -- 172.21.15.107:0/3976651250 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070035030 con 0x7f1084102370 2024-09-19T04:23:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.042+0000 7f108a600640 1 -- 172.21.15.107:0/3976651250 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1084074420 con 0x7f1084102370 2024-09-19T04:23:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.045+0000 7f1081600640 1 -- 172.21.15.107:0/3976651250 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f1070042070 con 0x7f1084102370 2024-09-19T04:23:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.046+0000 7f108e80a740 1 Processor -- start 2024-09-19T04:23:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.046+0000 7f108e80a740 1 -- start start 2024-09-19T04:23:37.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.046+0000 7f108e80a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b536a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.046+0000 7f108e80a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b536a760 0x55a3b5374ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.046+0000 7f108e80a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5375020 0x55a3b5377410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.046+0000 7f108e80a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a3b52d03e0 con 0x55a3b5375020 2024-09-19T04:23:37.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.046+0000 7f108e80a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a3b5277d80 con 0x55a3b536a760 2024-09-19T04:23:37.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.046+0000 7f108e80a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a3b52cc3a0 con 0x55a3b5369e40 2024-09-19T04:23:37.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1081600640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f106c0a77c0 0x7f106c0a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1089c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b536a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1089c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b536a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:47590/0 (socket says 172.21.15.107:47590) 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1089c00640 1 -- 172.21.15.107:0/4176520156 learned_addr learned my addr 172.21.15.107:0/4176520156 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1089200640 1 --2- 172.21.15.107:0/4176520156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b536a760 0x55a3b5374ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1083e00640 1 --2- 172.21.15.107:0/4176520156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5375020 0x55a3b5377410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1081600640 1 -- 172.21.15.107:0/3976651250 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f10700f0360 con 0x7f1084102370 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1089200640 1 -- 172.21.15.107:0/4176520156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 msgr2=0x55a3b536a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1089200640 1 --2- 172.21.15.107:0/4176520156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b536a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1089200640 1 -- 172.21.15.107:0/4176520156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5375020 msgr2=0x55a3b5377410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1089200640 1 --2- 172.21.15.107:0/4176520156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5375020 0x55a3b5377410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1089200640 1 -- 172.21.15.107:0/4176520156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3b5296ae0 con 0x55a3b536a760 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1089c00640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f106c0a77c0 0x7f106c0a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.047+0000 7f1083e00640 1 --2- 172.21.15.107:0/4176520156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5375020 0x55a3b5377410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.048+0000 7f1089200640 1 --2- 172.21.15.107:0/4176520156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b536a760 0x55a3b5374ae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1078009f50 tx=0x7f10780118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae8d454b3f4e28ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.048+0000 7f1076000640 1 -- 172.21.15.107:0/4176520156 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078004020 con 0x55a3b536a760 2024-09-19T04:23:37.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.048+0000 7f1076000640 1 -- 172.21.15.107:0/4176520156 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10780041c0 con 0x55a3b536a760 2024-09-19T04:23:37.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.048+0000 7f1076000640 1 -- 172.21.15.107:0/4176520156 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107800f900 con 0x55a3b536a760 2024-09-19T04:23:37.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.048+0000 7f108e80a740 1 -- 172.21.15.107:0/4176520156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b536a760 msgr2=0x55a3b5374ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:37.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.048+0000 7f1089c00640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f106c0a77c0 0x7f106c0a9c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f107c005c40 tx=0x7f107c014040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:37.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.048+0000 7f108e80a740 1 --2- 172.21.15.107:0/4176520156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b536a760 0x55a3b5374ae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1078009f50 tx=0x7f10780118d0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.049+0000 7f108e80a740 1 -- 172.21.15.107:0/4176520156 shutdown_connections 2024-09-19T04:23:37.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.049+0000 7f108e80a740 1 --2- 172.21.15.107:0/4176520156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5375020 0x55a3b5377410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.049+0000 7f108e80a740 1 --2- 172.21.15.107:0/4176520156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b536a760 0x55a3b5374ae0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.049+0000 7f108e80a740 1 --2- 172.21.15.107:0/4176520156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b536a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.049+0000 7f108e80a740 1 -- 172.21.15.107:0/4176520156 >> 172.21.15.107:0/4176520156 conn(0x55a3b53663d0 msgr2=0x55a3b53667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.049+0000 7f108e80a740 1 -- 172.21.15.107:0/4176520156 shutdown_connections 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.049+0000 7f108e80a740 1 -- 172.21.15.107:0/4176520156 wait complete. 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.049+0000 7f108e80a740 1 -- start start 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f108e80a740 1 Processor -- start 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f108e80a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b5371f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f108e80a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b536a760 0x55a3b5372480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f108e80a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b53729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f108e80a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a3b5377bf0 con 0x55a3b536a760 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f108e80a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a3b528b0f0 con 0x55a3b5375020 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f108e80a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a3b5377d70 con 0x55a3b5369e40 2024-09-19T04:23:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f108e80a740 10 client.0 _ll_register_callbacks cb 0x55a3b5272bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:37.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f1089200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b5371f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f1089200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b5371f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:47594/0 (socket says 172.21.15.107:47594) 2024-09-19T04:23:37.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f1089200640 1 -- 172.21.15.107:0/2230688512 learned_addr learned my addr 172.21.15.107:0/2230688512 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:37.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f1089c00640 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b53729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.050+0000 7f1083e00640 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b536a760 0x55a3b5372480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.051+0000 7f1089200640 1 -- 172.21.15.107:0/2230688512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 msgr2=0x55a3b53729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:37.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.051+0000 7f1089200640 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b53729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.051+0000 7f1089200640 1 -- 172.21.15.107:0/2230688512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b536a760 msgr2=0x55a3b5372480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:37.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.051+0000 7f1089200640 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b536a760 0x55a3b5372480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.051+0000 7f1089200640 1 -- 172.21.15.107:0/2230688512 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3b50413c0 con 0x55a3b5369e40 2024-09-19T04:23:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.051+0000 7f1089200640 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b5371f40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1078012fd0 tx=0x7f10780045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7c4cc4d33712e89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.051+0000 7f106aa00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:23:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.051+0000 7f106aa00640 1 -- 172.21.15.107:0/2230688512 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078028020 con 0x55a3b5369e40 2024-09-19T04:23:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.051+0000 7f108e80a740 1 -- 172.21.15.107:0/2230688512 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10780041c0 con 0x55a3b5369e40 2024-09-19T04:23:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.051+0000 7f106aa00640 1 -- 172.21.15.107:0/2230688512 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f1078016040 con 0x55a3b5369e40 2024-09-19T04:23:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f106aa00640 1 -- 172.21.15.107:0/2230688512 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1078003fc0 con 0x55a3b5369e40 2024-09-19T04:23:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f108e80a740 10 client.25786 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f108e80a740 1 -- 172.21.15.107:0/2230688512 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a3b5383c60 con 0x55a3b5369e40 2024-09-19T04:23:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f106aa00640 1 -- 172.21.15.107:0/2230688512 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10780041c0 con 0x55a3b5369e40 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f106aa00640 1 -- 172.21.15.107:0/2230688512 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f107800f640 con 0x55a3b5369e40 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f108e80a740 10 client.25786 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f108e80a740 10 client.25786 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f108e80a740 20 client.25786 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f108e80a740 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3b5384fb0 0x55a3b53873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f108e80a740 4 client.25786 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.052+0000 7f108e80a740 1 -- 172.21.15.107:0/2230688512 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3b52dd950 con 0x55a3b5384fb0 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.053+0000 7f1083e00640 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3b5384fb0 0x55a3b53873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.053+0000 7f1083e00640 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3b5384fb0 0x55a3b53873a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a869000a6979728 server_cookie=4737f6b6396dc25 in_seq=0 out_seq=0 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.053+0000 7f106aa00640 10 client.25786 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f106aa00640 1 -- 172.21.15.107:0/2230688512 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3b52dd950 con 0x55a3b5384fb0 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f106aa00640 10 client.25786 handle_command_reply: tid=0 2024-09-19T04:23:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 client.25786 shutdown 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 -- 172.21.15.107:0/2230688512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 msgr2=0x55a3b5371f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b5371f40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1078012fd0 tx=0x7f10780045a0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 -- 172.21.15.107:0/2230688512 shutdown_connections 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3b5384fb0 0x55a3b53873a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b53729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b536a760 0x55a3b5372480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 --2- 172.21.15.107:0/2230688512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b5369e40 0x55a3b5371f40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 -- 172.21.15.107:0/2230688512 >> 172.21.15.107:0/2230688512 conn(0x55a3b53663d0 msgr2=0x55a3b536c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f106aa00640 0 client.25786 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 -- 172.21.15.107:0/2230688512 shutdown_connections 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 1 -- 172.21.15.107:0/2230688512 wait complete. 2024-09-19T04:23:37.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.054+0000 7f108e80a740 20 client.25786 trim_cache size 0 max 16384 2024-09-19T04:23:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.069+0000 7f108e80a740 1 Processor -- start 2024-09-19T04:23:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.069+0000 7f108e80a740 1 -- start start 2024-09-19T04:23:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.069+0000 7f108e80a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 0x55a3b536c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.069+0000 7f108e80a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 0x55a3b538cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.069+0000 7f108e80a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b538d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f108e80a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10841ab480 con 0x55a3b5369e40 2024-09-19T04:23:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f108e80a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f10841ad0b0 con 0x55a3b5375020 2024-09-19T04:23:37.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f108e80a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10841abb50 con 0x55a3b536a760 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1089200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 0x55a3b536c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1083e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 0x55a3b538cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1083e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 0x55a3b538cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:47598/0 (socket says 172.21.15.107:47598) 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1083e00640 1 -- 172.21.15.107:0/1153804297 learned_addr learned my addr 172.21.15.107:0/1153804297 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1089c00640 1 --2- 172.21.15.107:0/1153804297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b538d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1083e00640 1 -- 172.21.15.107:0/1153804297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 msgr2=0x55a3b538d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1083e00640 1 --2- 172.21.15.107:0/1153804297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b538d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1083e00640 1 -- 172.21.15.107:0/1153804297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 msgr2=0x55a3b536c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1083e00640 1 --2- 172.21.15.107:0/1153804297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 0x55a3b536c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1083e00640 1 -- 172.21.15.107:0/1153804297 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3b5369bc0 con 0x55a3b536a760 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.070+0000 7f1083e00640 1 --2- 172.21.15.107:0/1153804297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 0x55a3b538cb30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f10700bc3e0 tx=0x7f107003d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b2834366c5965f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.071+0000 7f1075600640 1 -- 172.21.15.107:0/1153804297 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10700a3070 con 0x55a3b536a760 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.071+0000 7f1075600640 1 -- 172.21.15.107:0/1153804297 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1070031e40 con 0x55a3b536a760 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.071+0000 7f1075600640 1 -- 172.21.15.107:0/1153804297 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070049040 con 0x55a3b536a760 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.071+0000 7f108e80a740 1 -- 172.21.15.107:0/1153804297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 msgr2=0x55a3b538cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.071+0000 7f108e80a740 1 --2- 172.21.15.107:0/1153804297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 0x55a3b538cb30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f10700bc3e0 tx=0x7f107003d040 comp rx=0 tx=0).stop 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.071+0000 7f108e80a740 1 -- 172.21.15.107:0/1153804297 shutdown_connections 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.071+0000 7f108e80a740 1 --2- 172.21.15.107:0/1153804297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b538d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.071+0000 7f108e80a740 1 --2- 172.21.15.107:0/1153804297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 0x55a3b538cb30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.071+0000 7f108e80a740 1 --2- 172.21.15.107:0/1153804297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 0x55a3b536c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.071+0000 7f108e80a740 1 -- 172.21.15.107:0/1153804297 >> 172.21.15.107:0/1153804297 conn(0x55a3b53663d0 msgr2=0x55a3b5373fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.072+0000 7f108e80a740 1 -- 172.21.15.107:0/1153804297 shutdown_connections 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.072+0000 7f108e80a740 1 -- 172.21.15.107:0/1153804297 wait complete. 2024-09-19T04:23:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.072+0000 7f108e80a740 1 -- start start 2024-09-19T04:23:37.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.072+0000 7f108e80a740 1 Processor -- start 2024-09-19T04:23:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f108e80a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 0x55a3b537d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f108e80a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 0x55a3b53787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f108e80a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b5378d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f108e80a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10841ac920 con 0x55a3b5369e40 2024-09-19T04:23:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f108e80a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f10841ac220 con 0x55a3b5375020 2024-09-19T04:23:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f108e80a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10841a9460 con 0x55a3b536a760 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f108e80a740 10 client.0 _ll_register_callbacks cb 0x7f1078009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f1089200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 0x55a3b537d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f1089200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 0x55a3b537d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36084/0 (socket says 172.21.15.107:36084) 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f1089200640 1 -- 172.21.15.107:0/790786555 learned_addr learned my addr 172.21.15.107:0/790786555 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f1089c00640 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b5378d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f1083e00640 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 0x55a3b53787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.073+0000 7f1089200640 1 -- 172.21.15.107:0/790786555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 msgr2=0x55a3b53787f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.074+0000 7f1089200640 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 0x55a3b53787f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.074+0000 7f1089200640 1 -- 172.21.15.107:0/790786555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 msgr2=0x55a3b5378d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.074+0000 7f1089200640 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b5378d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.074+0000 7f1089200640 1 -- 172.21.15.107:0/790786555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3b53792f0 con 0x55a3b5369e40 2024-09-19T04:23:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.074+0000 7f1089c00640 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b5378d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.074+0000 7f1089200640 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 0x55a3b537d4c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f107800fa60 tx=0x7f1078005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6765c1cef5cd372 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.074+0000 7f106aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.074+0000 7f106aa00640 1 -- 172.21.15.107:0/790786555 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078028070 con 0x55a3b5369e40 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.074+0000 7f106aa00640 1 -- 172.21.15.107:0/790786555 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f1078003b00 con 0x55a3b5369e40 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.074+0000 7f108e80a740 1 -- 172.21.15.107:0/790786555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a3b52d7e60 con 0x55a3b5369e40 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f106aa00640 1 -- 172.21.15.107:0/790786555 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1078045050 con 0x55a3b5369e40 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f108e80a740 10 client.15768 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f108e80a740 1 -- 172.21.15.107:0/790786555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a3b53ef8b0 con 0x55a3b5369e40 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f106aa00640 1 -- 172.21.15.107:0/790786555 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a3b52d7e60 con 0x55a3b5369e40 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f106aa00640 1 -- 172.21.15.107:0/790786555 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f1078004710 con 0x55a3b5369e40 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f108e80a740 10 client.15768 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f108e80a740 10 client.15768 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f108e80a740 20 client.15768 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f108e80a740 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3b53eff30 0x55a3b53f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f108e80a740 4 client.15768 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.075+0000 7f108e80a740 1 -- 172.21.15.107:0/790786555 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3b5254010 con 0x55a3b53eff30 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.076+0000 7f1083e00640 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3b53eff30 0x55a3b53f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.076+0000 7f1083e00640 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3b53eff30 0x55a3b53f2350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27374bd121c888f0 server_cookie=48b8bed08f0d2af6 in_seq=0 out_seq=0 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.076+0000 7f106aa00640 10 client.15768 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.076+0000 7f106aa00640 1 -- 172.21.15.107:0/790786555 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3b5254010 con 0x55a3b53eff30 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.076+0000 7f106aa00640 10 client.15768 handle_command_reply: tid=0 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.076+0000 7f108e80a740 1 client.15768 shutdown 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 1 -- 172.21.15.107:0/790786555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 msgr2=0x55a3b537d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 0x55a3b537d4c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f107800fa60 tx=0x7f1078005d30 comp rx=0 tx=0).stop 2024-09-19T04:23:37.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 1 -- 172.21.15.107:0/790786555 shutdown_connections 2024-09-19T04:23:37.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3b53eff30 0x55a3b53f2350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3b5375020 0x55a3b5378d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3b536a760 0x55a3b53787f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 1 --2- 172.21.15.107:0/790786555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3b5369e40 0x55a3b537d4c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 1 -- 172.21.15.107:0/790786555 >> 172.21.15.107:0/790786555 conn(0x55a3b53663d0 msgr2=0x55a3b53706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:37.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f106aa00640 0 client.15768 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:23:37.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 1 -- 172.21.15.107:0/790786555 shutdown_connections 2024-09-19T04:23:37.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 1 -- 172.21.15.107:0/790786555 wait complete. 2024-09-19T04:23:37.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.077+0000 7f108e80a740 20 client.15768 trim_cache size 0 max 16384 2024-09-19T04:23:37.083 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:23:37.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 -- 172.21.15.107:0/3976651250 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f106c0a77c0 msgr2=0x7f106c0a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:37.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f106c0a77c0 0x7f106c0a9c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f107c005c40 tx=0x7f107c014040 comp rx=0 tx=0).stop 2024-09-19T04:23:37.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 -- 172.21.15.107:0/3976651250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102370 msgr2=0x7f1084198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:37.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102370 0x7f1084198810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f107002a960 tx=0x7f1070003b20 comp rx=0 tx=0).stop 2024-09-19T04:23:37.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 -- 172.21.15.107:0/3976651250 shutdown_connections 2024-09-19T04:23:37.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f106c0a77c0 0x7f106c0a9c80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10841079d0 0x7f108419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102370 0x7f1084198810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 --2- 172.21.15.107:0/3976651250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10841019d0 0x7f10841982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 -- 172.21.15.107:0/3976651250 >> 172.21.15.107:0/3976651250 conn(0x7f10840fd710 msgr2=0x7f10840fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 -- 172.21.15.107:0/3976651250 shutdown_connections 2024-09-19T04:23:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:37.078+0000 7f108a600640 1 -- 172.21.15.107:0/3976651250 wait complete. 2024-09-19T04:23:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:36 smithi115 ceph-mon[35477]: pgmap v101: 97 pgs: 97 active+clean; 80 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-09-19T04:23:37.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 128035 6.16 MB/sec warmup 115 sec latency 32.570 ms 2024-09-19T04:23:37.935 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:23:38.309 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:38.641 INFO:tasks.workunit.client.0.smithi107.stdout: 1 128393 6.12 MB/sec warmup 116 sec latency 610.698 ms 2024-09-19T04:23:38.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:38 smithi107 ceph-mon[30659]: pgmap v102: 97 pgs: 97 active+clean; 80 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 776 KiB/s wr, 114 op/s 2024-09-19T04:23:38.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:38 smithi145 ceph-mon[35096]: pgmap v102: 97 pgs: 97 active+clean; 80 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 776 KiB/s wr, 114 op/s 2024-09-19T04:23:38.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.975+0000 7fe26b800640 1 -- 172.21.15.107:0/2585668548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264101de0 msgr2=0x7fe2641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:38.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.975+0000 7fe26b800640 1 --2- 172.21.15.107:0/2585668548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264101de0 0x7fe2641021c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe254009900 tx=0x7fe25402a410 comp rx=0 tx=0).stop 2024-09-19T04:23:38.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.975+0000 7fe26b800640 1 -- 172.21.15.107:0/2585668548 shutdown_connections 2024-09-19T04:23:38.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.975+0000 7fe26b800640 1 --2- 172.21.15.107:0/2585668548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe264106f10 0x7fe26410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.975+0000 7fe26b800640 1 --2- 172.21.15.107:0/2585668548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe264102790 0x7fe2641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.975+0000 7fe26b800640 1 --2- 172.21.15.107:0/2585668548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264101de0 0x7fe2641021c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.975+0000 7fe26b800640 1 -- 172.21.15.107:0/2585668548 >> 172.21.15.107:0/2585668548 conn(0x7fe2640fd8c0 msgr2=0x7fe2640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:38.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.975+0000 7fe26b800640 1 -- 172.21.15.107:0/2585668548 shutdown_connections 2024-09-19T04:23:38.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.975+0000 7fe26b800640 1 -- 172.21.15.107:0/2585668548 wait complete. 2024-09-19T04:23:38.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.976+0000 7fe26b800640 1 Processor -- start 2024-09-19T04:23:38.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.976+0000 7fe26b800640 1 -- start start 2024-09-19T04:23:38.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe26b800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe264101de0 0x7fe2641984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe26b800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe264102790 0x7fe264198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe26b800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264106f10 0x7fe26419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe26b800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe264111fc0 con 0x7fe264106f10 2024-09-19T04:23:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe26b800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe264111e40 con 0x7fe264101de0 2024-09-19T04:23:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe26b800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe264112140 con 0x7fe264102790 2024-09-19T04:23:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe268e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe264101de0 0x7fe2641984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe269800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264106f10 0x7fe26419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe268e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe264101de0 0x7fe2641984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44890/0 (socket says 172.21.15.107:44890) 2024-09-19T04:23:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe25be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe264102790 0x7fe264198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe269800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264106f10 0x7fe26419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46788/0 (socket says 172.21.15.107:46788) 2024-09-19T04:23:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.977+0000 7fe269800640 1 -- 172.21.15.107:0/4224151675 learned_addr learned my addr 172.21.15.107:0/4224151675 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe25be00640 1 -- 172.21.15.107:0/4224151675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe264101de0 msgr2=0x7fe2641984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe25be00640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe264101de0 0x7fe2641984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe25be00640 1 -- 172.21.15.107:0/4224151675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264106f10 msgr2=0x7fe26419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe25be00640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264106f10 0x7fe26419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe25be00640 1 -- 172.21.15.107:0/4224151675 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26419d470 con 0x7fe264102790 2024-09-19T04:23:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe25be00640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe264102790 0x7fe264198a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00e9f0 tx=0x7fe24c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe262000640 1 -- 172.21.15.107:0/4224151675 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00ccc0 con 0x7fe264102790 2024-09-19T04:23:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe262000640 1 -- 172.21.15.107:0/4224151675 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe24c005020 con 0x7fe264102790 2024-09-19T04:23:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe26b800640 1 -- 172.21.15.107:0/4224151675 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26419d760 con 0x7fe264102790 2024-09-19T04:23:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe262000640 1 -- 172.21.15.107:0/4224151675 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c0049e0 con 0x7fe264102790 2024-09-19T04:23:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.978+0000 7fe26b800640 1 -- 172.21.15.107:0/4224151675 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2641a5040 con 0x7fe264102790 2024-09-19T04:23:38.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.981+0000 7fe262000640 1 -- 172.21.15.107:0/4224151675 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fe24c005560 con 0x7fe264102790 2024-09-19T04:23:38.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.981+0000 7fe26e46a740 1 Processor -- start 2024-09-19T04:23:38.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe26e46a740 1 -- start start 2024-09-19T04:23:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eaeef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 0x56090eafdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eafe260 0x56090eb00650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe26e46a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56090ea553e0 con 0x56090eafe260 2024-09-19T04:23:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe26e46a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56090e9fcd80 con 0x56090eaef490 2024-09-19T04:23:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe26e46a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56090ea513a0 con 0x56090eaeeb70 2024-09-19T04:23:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe269800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eaeef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe25be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eafe260 0x56090eb00650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe269800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eaeef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:48988/0 (socket says 172.21.15.107:48988) 2024-09-19T04:23:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.982+0000 7fe269800640 1 -- 172.21.15.107:0/2555129461 learned_addr learned my addr 172.21.15.107:0/2555129461 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.983+0000 7fe268e00640 1 --2- 172.21.15.107:0/2555129461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 0x56090eafdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.983+0000 7fe269800640 1 -- 172.21.15.107:0/2555129461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 msgr2=0x56090eafdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.983+0000 7fe269800640 1 --2- 172.21.15.107:0/2555129461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 0x56090eafdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.983+0000 7fe269800640 1 -- 172.21.15.107:0/2555129461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eafe260 msgr2=0x56090eb00650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.983+0000 7fe269800640 1 --2- 172.21.15.107:0/2555129461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eafe260 0x56090eb00650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.983+0000 7fe269800640 1 -- 172.21.15.107:0/2555129461 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56090ea1bae0 con 0x56090eaeeb70 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.983+0000 7fe262000640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe2480a3430 0x7fe2480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.983+0000 7fe269800640 1 --2- 172.21.15.107:0/2555129461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eaeef50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009fb0 tx=0x7fe25c011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e120dc85645b8681 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.983+0000 7fe262000640 1 -- 172.21.15.107:0/4224151675 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fe24c0cfc00 con 0x7fe264102790 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.983+0000 7fe268e00640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe2480a3430 0x7fe2480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe25a000640 1 -- 172.21.15.107:0/2555129461 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0127d0 con 0x56090eaeeb70 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe25a000640 1 -- 172.21.15.107:0/2555129461 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe25c012e50 con 0x56090eaeeb70 2024-09-19T04:23:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe25a000640 1 -- 172.21.15.107:0/2555129461 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0193a0 con 0x56090eaeeb70 2024-09-19T04:23:38.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe26e46a740 1 -- 172.21.15.107:0/2555129461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 msgr2=0x56090eaeef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:38.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe268e00640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe2480a3430 0x7fe2480a58f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe254000c00 tx=0x7fe254005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:38.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe26e46a740 1 --2- 172.21.15.107:0/2555129461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eaeef50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009fb0 tx=0x7fe25c011870 comp rx=0 tx=0).stop 2024-09-19T04:23:38.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe26e46a740 1 -- 172.21.15.107:0/2555129461 shutdown_connections 2024-09-19T04:23:38.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe26e46a740 1 --2- 172.21.15.107:0/2555129461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eafe260 0x56090eb00650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe26e46a740 1 --2- 172.21.15.107:0/2555129461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 0x56090eafdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe26e46a740 1 --2- 172.21.15.107:0/2555129461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eaeef50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.984+0000 7fe26e46a740 1 -- 172.21.15.107:0/2555129461 >> 172.21.15.107:0/2555129461 conn(0x56090eaeb200 msgr2=0x56090eaeb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.985+0000 7fe26e46a740 1 -- 172.21.15.107:0/2555129461 shutdown_connections 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.985+0000 7fe26e46a740 1 -- 172.21.15.107:0/2555129461 wait complete. 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.985+0000 7fe26e46a740 1 -- start start 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.985+0000 7fe26e46a740 1 Processor -- start 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eafb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaef490 0x56090eafb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eafe260 0x56090eafbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe26e46a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56090eb00e90 con 0x56090eaef490 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe26e46a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56090ea100f0 con 0x56090eafe260 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe26e46a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56090eb01010 con 0x56090eaeeb70 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe26e46a740 10 client.0 _ll_register_callbacks cb 0x56090e9f7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:38.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe268e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eafb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe268e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eafb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:48998/0 (socket says 172.21.15.107:48998) 2024-09-19T04:23:38.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe268e00640 1 -- 172.21.15.107:0/633178595 learned_addr learned my addr 172.21.15.107:0/633178595 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:38.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe25be00640 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaef490 0x56090eafb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe269800640 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eafe260 0x56090eafbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe268e00640 1 -- 172.21.15.107:0/633178595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eafe260 msgr2=0x56090eafbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:38.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe268e00640 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eafe260 0x56090eafbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe268e00640 1 -- 172.21.15.107:0/633178595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaef490 msgr2=0x56090eafb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:38.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe268e00640 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaef490 0x56090eafb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.986+0000 7fe268e00640 1 -- 172.21.15.107:0/633178595 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56090e7c63c0 con 0x56090eaeeb70 2024-09-19T04:23:38.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.987+0000 7fe268e00640 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eafb160 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe254033000 tx=0x7fe254009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd1c6b6c75330628 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:38.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.987+0000 7fe252a00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:23:38.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.987+0000 7fe252a00640 1 -- 172.21.15.107:0/633178595 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25404e070 con 0x56090eaeeb70 2024-09-19T04:23:38.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.987+0000 7fe26e46a740 1 -- 172.21.15.107:0/633178595 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe25c012e50 con 0x56090eaeeb70 2024-09-19T04:23:38.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.987+0000 7fe252a00640 1 -- 172.21.15.107:0/633178595 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fe2540048c0 con 0x56090eaeeb70 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.987+0000 7fe252a00640 1 -- 172.21.15.107:0/633178595 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe25405a050 con 0x56090eaeeb70 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.987+0000 7fe26e46a740 10 client.25804 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.987+0000 7fe26e46a740 1 -- 172.21.15.107:0/633178595 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56090eb0cf70 con 0x56090eaeeb70 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.988+0000 7fe252a00640 1 -- 172.21.15.107:0/633178595 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe25c012e50 con 0x56090eaeeb70 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.988+0000 7fe252a00640 1 -- 172.21.15.107:0/633178595 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fe2540035f0 con 0x56090eaeeb70 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.988+0000 7fe26e46a740 10 client.25804 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.988+0000 7fe26e46a740 10 client.25804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.988+0000 7fe26e46a740 20 client.25804 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.988+0000 7fe26e46a740 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x56090eb0e380 0x56090eb10790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.988+0000 7fe26e46a740 4 client.25804 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.988+0000 7fe26e46a740 1 -- 172.21.15.107:0/633178595 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56090ea62950 con 0x56090eb0e380 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.988+0000 7fe25be00640 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x56090eb0e380 0x56090eb10790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.989+0000 7fe25be00640 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x56090eb0e380 0x56090eb10790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b65a7b8f8364ee38 server_cookie=2bb67b5583dd175a in_seq=0 out_seq=0 2024-09-19T04:23:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.989+0000 7fe252a00640 10 client.25804 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe252a00640 1 -- 172.21.15.107:0/633178595 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56090ea62950 con 0x56090eb0e380 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe252a00640 10 client.25804 handle_command_reply: tid=0 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 client.25804 shutdown 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 -- 172.21.15.107:0/633178595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 msgr2=0x56090eafb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eafb160 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe254033000 tx=0x7fe254009260 comp rx=0 tx=0).stop 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 -- 172.21.15.107:0/633178595 shutdown_connections 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x56090eb0e380 0x56090eb10790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eafe260 0x56090eafbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaef490 0x56090eafb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe252a00640 0 client.25804 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 --2- 172.21.15.107:0/633178595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eafb160 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 -- 172.21.15.107:0/633178595 >> 172.21.15.107:0/633178595 conn(0x56090eaeb200 msgr2=0x56090eaf0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 -- 172.21.15.107:0/633178595 shutdown_connections 2024-09-19T04:23:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 1 -- 172.21.15.107:0/633178595 wait complete. 2024-09-19T04:23:38.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:38.990+0000 7fe26e46a740 20 client.25804 trim_cache size 0 max 16384 2024-09-19T04:23:39.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe26e46a740 1 Processor -- start 2024-09-19T04:23:39.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe26e46a740 1 -- start start 2024-09-19T04:23:39.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaeeb70 0x56090eb16060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:39.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 0x56090eb165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:39.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eafe260 0x56090eaf83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:39.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe26e46a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2641ab530 con 0x56090eaeeb70 2024-09-19T04:23:39.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe26e46a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe2641ad160 con 0x56090eaef490 2024-09-19T04:23:39.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe26e46a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe2641abc00 con 0x56090eafe260 2024-09-19T04:23:39.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe268e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaeeb70 0x56090eb16060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:39.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe268e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaeeb70 0x56090eb16060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46824/0 (socket says 172.21.15.107:46824) 2024-09-19T04:23:39.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe268e00640 1 -- 172.21.15.107:0/3133402559 learned_addr learned my addr 172.21.15.107:0/3133402559 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:39.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe269800640 1 --2- 172.21.15.107:0/3133402559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eafe260 0x56090eaf83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:39.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.004+0000 7fe25be00640 1 --2- 172.21.15.107:0/3133402559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 0x56090eb165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:39.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.005+0000 7fe25be00640 1 -- 172.21.15.107:0/3133402559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eafe260 msgr2=0x56090eaf83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:39.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.005+0000 7fe25be00640 1 --2- 172.21.15.107:0/3133402559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eafe260 0x56090eaf83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.005+0000 7fe25be00640 1 -- 172.21.15.107:0/3133402559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaeeb70 msgr2=0x56090eb16060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:39.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.005+0000 7fe25be00640 1 --2- 172.21.15.107:0/3133402559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaeeb70 0x56090eb16060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.005+0000 7fe25be00640 1 -- 172.21.15.107:0/3133402559 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56090eae6770 con 0x56090eaef490 2024-09-19T04:23:39.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.005+0000 7fe268e00640 1 --2- 172.21.15.107:0/3133402559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaeeb70 0x56090eb16060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:39.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.005+0000 7fe25be00640 1 --2- 172.21.15.107:0/3133402559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 0x56090eb165a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0cd340 tx=0x7fe24c0d36b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e89542499c148674 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:39.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.005+0000 7fe259600640 1 -- 172.21.15.107:0/3133402559 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c01c040 con 0x56090eaef490 2024-09-19T04:23:39.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.005+0000 7fe259600640 1 -- 172.21.15.107:0/3133402559 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe24c0051d0 con 0x56090eaef490 2024-09-19T04:23:39.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.005+0000 7fe259600640 1 -- 172.21.15.107:0/3133402559 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c09a1d0 con 0x56090eaef490 2024-09-19T04:23:39.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.006+0000 7fe26e46a740 1 -- 172.21.15.107:0/3133402559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 msgr2=0x56090eb165a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:39.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.006+0000 7fe26e46a740 1 --2- 172.21.15.107:0/3133402559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 0x56090eb165a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0cd340 tx=0x7fe24c0d36b0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.006+0000 7fe26e46a740 1 -- 172.21.15.107:0/3133402559 shutdown_connections 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.006+0000 7fe26e46a740 1 --2- 172.21.15.107:0/3133402559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eafe260 0x56090eaf83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.006+0000 7fe26e46a740 1 --2- 172.21.15.107:0/3133402559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eaef490 0x56090eb165a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.006+0000 7fe26e46a740 1 --2- 172.21.15.107:0/3133402559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaeeb70 0x56090eb16060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.006+0000 7fe26e46a740 1 -- 172.21.15.107:0/3133402559 >> 172.21.15.107:0/3133402559 conn(0x56090eaeb200 msgr2=0x56090eaea380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.006+0000 7fe26e46a740 1 -- 172.21.15.107:0/3133402559 shutdown_connections 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.006+0000 7fe26e46a740 1 -- 172.21.15.107:0/3133402559 wait complete. 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe26e46a740 1 -- start start 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe26e46a740 1 Processor -- start 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eb06a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaef490 0x56090eaf7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe26e46a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eafe260 0x56090eaf8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:39.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe26e46a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2641ac9d0 con 0x56090eaef490 2024-09-19T04:23:39.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe26e46a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe2641ac2d0 con 0x56090eafe260 2024-09-19T04:23:39.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe26e46a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe2641a9510 con 0x56090eaeeb70 2024-09-19T04:23:39.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe26e46a740 10 client.0 _ll_register_callbacks cb 0x7fe2540042f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:39.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe268e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eb06a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:39.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe268e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eb06a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:49020/0 (socket says 172.21.15.107:49020) 2024-09-19T04:23:39.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe268e00640 1 -- 172.21.15.107:0/247162568 learned_addr learned my addr 172.21.15.107:0/247162568 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:39.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.007+0000 7fe25be00640 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaef490 0x56090eaf7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:39.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.008+0000 7fe269800640 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eafe260 0x56090eaf8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:39.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.008+0000 7fe268e00640 1 -- 172.21.15.107:0/247162568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eafe260 msgr2=0x56090eaf8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:39.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.008+0000 7fe268e00640 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eafe260 0x56090eaf8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.008+0000 7fe268e00640 1 -- 172.21.15.107:0/247162568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaef490 msgr2=0x56090eaf7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.008+0000 7fe268e00640 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaef490 0x56090eaf7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.008+0000 7fe268e00640 1 -- 172.21.15.107:0/247162568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56090eaf8c10 con 0x56090eaeeb70 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.008+0000 7fe268e00640 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eb06a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe254009230 tx=0x7fe254003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72b818fd5e02afb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.008+0000 7fe252a00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe252a00640 1 -- 172.21.15.107:0/247162568 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254062070 con 0x56090eaeeb70 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe252a00640 1 -- 172.21.15.107:0/247162568 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fe2540048c0 con 0x56090eaeeb70 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe26e46a740 1 -- 172.21.15.107:0/247162568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe24c0051d0 con 0x56090eaeeb70 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe252a00640 1 -- 172.21.15.107:0/247162568 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe25404e040 con 0x56090eaeeb70 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe26e46a740 10 client.25810 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:39.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe26e46a740 1 -- 172.21.15.107:0/247162568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56090eb79030 con 0x56090eaeeb70 2024-09-19T04:23:39.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe252a00640 1 -- 172.21.15.107:0/247162568 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe24c0051d0 con 0x56090eaeeb70 2024-09-19T04:23:39.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe252a00640 1 -- 172.21.15.107:0/247162568 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fe254032c90 con 0x56090eaeeb70 2024-09-19T04:23:39.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe26e46a740 10 client.25810 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:39.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe26e46a740 10 client.25810 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:23:39.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe26e46a740 20 client.25810 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:39.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe26e46a740 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x56090eb797d0 0x56090eb7bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe26e46a740 4 client.25810 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.009+0000 7fe26e46a740 1 -- 172.21.15.107:0/247162568 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56090ea5ce60 con 0x56090eb797d0 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.010+0000 7fe25be00640 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x56090eb797d0 0x56090eb7bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.010+0000 7fe25be00640 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x56090eb797d0 0x56090eb7bbf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96eef872ea39109b server_cookie=2469c0be24339992 in_seq=0 out_seq=0 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.010+0000 7fe252a00640 10 client.25810 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe252a00640 1 -- 172.21.15.107:0/247162568 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56090ea5ce60 con 0x56090eb797d0 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe252a00640 10 client.25810 handle_command_reply: tid=0 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 client.25810 shutdown 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 -- 172.21.15.107:0/247162568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 msgr2=0x56090eb06a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eb06a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe254009230 tx=0x7fe254003820 comp rx=0 tx=0).stop 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 -- 172.21.15.107:0/247162568 shutdown_connections 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x56090eb797d0 0x56090eb7bbf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56090eafe260 0x56090eaf8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56090eaef490 0x56090eaf7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 --2- 172.21.15.107:0/247162568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56090eaeeb70 0x56090eb06a80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe252a00640 0 client.25810 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:23:39.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 -- 172.21.15.107:0/247162568 >> 172.21.15.107:0/247162568 conn(0x56090eaeb200 msgr2=0x56090eae9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:39.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 -- 172.21.15.107:0/247162568 shutdown_connections 2024-09-19T04:23:39.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 1 -- 172.21.15.107:0/247162568 wait complete. 2024-09-19T04:23:39.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.011+0000 7fe26e46a740 20 client.25810 trim_cache size 0 max 16384 2024-09-19T04:23:39.017 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:23:39.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 -- 172.21.15.107:0/4224151675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe2480a3430 msgr2=0x7fe2480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:39.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe2480a3430 0x7fe2480a58f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe254000c00 tx=0x7fe254005b50 comp rx=0 tx=0).stop 2024-09-19T04:23:39.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 -- 172.21.15.107:0/4224151675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe264102790 msgr2=0x7fe264198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:39.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe264102790 0x7fe264198a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00e9f0 tx=0x7fe24c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 -- 172.21.15.107:0/4224151675 shutdown_connections 2024-09-19T04:23:39.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe2480a3430 0x7fe2480a58f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264106f10 0x7fe26419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe264102790 0x7fe264198a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 --2- 172.21.15.107:0/4224151675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe264101de0 0x7fe2641984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:39.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 -- 172.21.15.107:0/4224151675 >> 172.21.15.107:0/4224151675 conn(0x7fe2640fd8c0 msgr2=0x7fe2640fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:39.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 -- 172.21.15.107:0/4224151675 shutdown_connections 2024-09-19T04:23:39.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:39.012+0000 7fe26b800640 1 -- 172.21.15.107:0/4224151675 wait complete. 2024-09-19T04:23:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:38 smithi115 ceph-mon[35477]: pgmap v102: 97 pgs: 97 active+clean; 80 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 776 KiB/s wr, 114 op/s 2024-09-19T04:23:39.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 128449 6.06 MB/sec warmup 117 sec latency 1182.804 ms 2024-09-19T04:23:39.826 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:23:40.193 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:40.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 128449 6.01 MB/sec warmup 118 sec latency 1388.119 ms 2024-09-19T04:23:40.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:40 smithi107 ceph-mon[30659]: pgmap v103: 97 pgs: 97 active+clean; 80 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 776 KiB/s wr, 114 op/s 2024-09-19T04:23:40.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:40 smithi145 ceph-mon[35096]: pgmap v103: 97 pgs: 97 active+clean; 80 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 776 KiB/s wr, 114 op/s 2024-09-19T04:23:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.123+0000 7f5129a00640 1 -- 172.21.15.107:0/3820100656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5124074510 msgr2=0x7f51240748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.123+0000 7f5129a00640 1 --2- 172.21.15.107:0/3820100656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5124074510 0x7f51240748f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5114009900 tx=0x7f511402a4b0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.123+0000 7f5129a00640 1 -- 172.21.15.107:0/3820100656 shutdown_connections 2024-09-19T04:23:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.123+0000 7f5129a00640 1 --2- 172.21.15.107:0/3820100656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f512410e280 0x7f5124110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.123+0000 7f5129a00640 1 --2- 172.21.15.107:0/3820100656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5124074e30 0x7f512410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.123+0000 7f5129a00640 1 --2- 172.21.15.107:0/3820100656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5124074510 0x7f51240748f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.123+0000 7f5129a00640 1 -- 172.21.15.107:0/3820100656 >> 172.21.15.107:0/3820100656 conn(0x7f51240fd740 msgr2=0x7f51240ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.123+0000 7f5129a00640 1 -- 172.21.15.107:0/3820100656 shutdown_connections 2024-09-19T04:23:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.123+0000 7f5129a00640 1 -- 172.21.15.107:0/3820100656 wait complete. 2024-09-19T04:23:41.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.124+0000 7f5129a00640 1 Processor -- start 2024-09-19T04:23:41.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.124+0000 7f5129a00640 1 -- start start 2024-09-19T04:23:41.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.124+0000 7f5129a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5124074510 0x7f512419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.124+0000 7f5129a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5124074e30 0x7f512419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.124+0000 7f5129a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512410e280 0x7f512419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.124+0000 7f5129a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5124114060 con 0x7f512410e280 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.124+0000 7f5129a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5124113ee0 con 0x7f5124074510 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.124+0000 7f5129a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f51241141e0 con 0x7f5124074e30 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5129000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512410e280 0x7f512419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5123e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5124074510 0x7f512419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5123400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5124074e30 0x7f512419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5129000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512410e280 0x7f512419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46868/0 (socket says 172.21.15.107:46868) 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5129000640 1 -- 172.21.15.107:0/1942726753 learned_addr learned my addr 172.21.15.107:0/1942726753 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5123e00640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5124074510 0x7f512419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44944/0 (socket says 172.21.15.107:44944) 2024-09-19T04:23:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5129000640 1 -- 172.21.15.107:0/1942726753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5124074e30 msgr2=0x7f512419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5129000640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5124074e30 0x7f512419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5129000640 1 -- 172.21.15.107:0/1942726753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5124074510 msgr2=0x7f512419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5129000640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5124074510 0x7f512419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.125+0000 7f5129000640 1 -- 172.21.15.107:0/1942726753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512419f4d0 con 0x7f512410e280 2024-09-19T04:23:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.126+0000 7f5129000640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512410e280 0x7f512419edf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f511c00b9e0 tx=0x7f511c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:41.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.126+0000 7f5120c00640 1 -- 172.21.15.107:0/1942726753 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c019070 con 0x7f512410e280 2024-09-19T04:23:41.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.126+0000 7f5120c00640 1 -- 172.21.15.107:0/1942726753 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f511c014440 con 0x7f512410e280 2024-09-19T04:23:41.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.126+0000 7f5129a00640 1 -- 172.21.15.107:0/1942726753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512419f7c0 con 0x7f512410e280 2024-09-19T04:23:41.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.126+0000 7f5120c00640 1 -- 172.21.15.107:0/1942726753 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0125c0 con 0x7f512410e280 2024-09-19T04:23:41.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.126+0000 7f5129a00640 1 -- 172.21.15.107:0/1942726753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51241a70a0 con 0x7f512410e280 2024-09-19T04:23:41.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.129+0000 7f5120c00640 1 -- 172.21.15.107:0/1942726753 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f511c01c070 con 0x7f512410e280 2024-09-19T04:23:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.130+0000 7f512da42740 1 Processor -- start 2024-09-19T04:23:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.130+0000 7f512da42740 1 -- start start 2024-09-19T04:23:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.130+0000 7f5120c00640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f51080a33b0 0x7f51080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.130+0000 7f512da42740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 0x56387c8de220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.130+0000 7f512da42740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.130+0000 7f512da42740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c8eb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.130+0000 7f512da42740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56387c8443e0 con 0x56387c8de760 2024-09-19T04:23:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.130+0000 7f512da42740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56387c7ebd80 con 0x56387c8dde40 2024-09-19T04:23:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.130+0000 7f512da42740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56387c8403a0 con 0x56387c8e9020 2024-09-19T04:23:41.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.130+0000 7f5120c00640 1 -- 172.21.15.107:0/1942726753 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f511c0cf110 con 0x7f512410e280 2024-09-19T04:23:41.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.131+0000 7f5123400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.131+0000 7f5123400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8e8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46880/0 (socket says 172.21.15.107:46880) 2024-09-19T04:23:41.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.131+0000 7f5123400640 1 -- 172.21.15.107:0/329704388 learned_addr learned my addr 172.21.15.107:0/329704388 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:41.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.131+0000 7f5123e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 0x56387c8de220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.131+0000 7f5129000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c8eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.131+0000 7f5123400640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f51080a33b0 0x7f51080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.131+0000 7f5123400640 1 -- 172.21.15.107:0/329704388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 msgr2=0x56387c8eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:41.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.131+0000 7f5123400640 1 --2- 172.21.15.107:0/329704388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c8eb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.131+0000 7f5123400640 1 -- 172.21.15.107:0/329704388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 msgr2=0x56387c8de220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:41.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.131+0000 7f5123400640 1 --2- 172.21.15.107:0/329704388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 0x56387c8de220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.132+0000 7f5123400640 1 -- 172.21.15.107:0/329704388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387c80aae0 con 0x56387c8de760 2024-09-19T04:23:41.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.132+0000 7f5123400640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f51080a33b0 0x7f51080a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f511000f850 tx=0x7f5110018000 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:41.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.132+0000 7f5123400640 1 --2- 172.21.15.107:0/329704388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8e8ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5110011ab0 tx=0x7f5110011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfee14bcef721b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:41.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.132+0000 7f5119600640 1 -- 172.21.15.107:0/329704388 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110011050 con 0x56387c8de760 2024-09-19T04:23:41.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.132+0000 7f5119600640 1 -- 172.21.15.107:0/329704388 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f511000ed60 con 0x56387c8de760 2024-09-19T04:23:41.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.132+0000 7f5119600640 1 -- 172.21.15.107:0/329704388 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100055d0 con 0x56387c8de760 2024-09-19T04:23:41.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.133+0000 7f512da42740 1 -- 172.21.15.107:0/329704388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 msgr2=0x56387c8e8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:41.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.133+0000 7f512da42740 1 --2- 172.21.15.107:0/329704388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8e8ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5110011ab0 tx=0x7f5110011f70 comp rx=0 tx=0).stop 2024-09-19T04:23:41.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.133+0000 7f512da42740 1 -- 172.21.15.107:0/329704388 shutdown_connections 2024-09-19T04:23:41.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.133+0000 7f512da42740 1 --2- 172.21.15.107:0/329704388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c8eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.133+0000 7f512da42740 1 --2- 172.21.15.107:0/329704388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8e8ae0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.133+0000 7f512da42740 1 --2- 172.21.15.107:0/329704388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 0x56387c8de220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.133+0000 7f512da42740 1 -- 172.21.15.107:0/329704388 >> 172.21.15.107:0/329704388 conn(0x56387c8da3d0 msgr2=0x56387c8da7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:41.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.133+0000 7f512da42740 1 -- 172.21.15.107:0/329704388 shutdown_connections 2024-09-19T04:23:41.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.133+0000 7f512da42740 1 -- 172.21.15.107:0/329704388 wait complete. 2024-09-19T04:23:41.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.134+0000 7f512da42740 1 -- start start 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.134+0000 7f512da42740 1 Processor -- start 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.134+0000 7f512da42740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8dde40 0x56387c8e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f512da42740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8de760 0x56387c8e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f512da42740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8e9020 0x56387c8e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f512da42740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56387c8ebbf0 con 0x56387c8e9020 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f512da42740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56387c7ff0f0 con 0x56387c8de760 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f512da42740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56387c8ebd70 con 0x56387c8dde40 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f512da42740 10 client.0 _ll_register_callbacks cb 0x56387c7e6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5129000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8e9020 0x56387c8e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5129000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8e9020 0x56387c8e6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46894/0 (socket says 172.21.15.107:46894) 2024-09-19T04:23:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5123e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8dde40 0x56387c8e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5129000640 1 -- 172.21.15.107:0/1172160219 learned_addr learned my addr 172.21.15.107:0/1172160219 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5123400640 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8de760 0x56387c8e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5123e00640 1 -- 172.21.15.107:0/1172160219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8de760 msgr2=0x56387c8e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5123e00640 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8de760 0x56387c8e64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5123e00640 1 -- 172.21.15.107:0/1172160219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8e9020 msgr2=0x56387c8e6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5123e00640 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8e9020 0x56387c8e6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5123e00640 1 -- 172.21.15.107:0/1172160219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387c5b53c0 con 0x56387c8dde40 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.135+0000 7f5129000640 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8e9020 0x56387c8e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.136+0000 7f5123e00640 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8dde40 0x56387c8e5f90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f511402a9d0 tx=0x7f51140043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f90c7344dc99728b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.136+0000 7f510e000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.136+0000 7f510e000640 1 -- 172.21.15.107:0/1172160219 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114035040 con 0x56387c8dde40 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.136+0000 7f512da42740 1 -- 172.21.15.107:0/1172160219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f511000ed60 con 0x56387c8dde40 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.136+0000 7f510e000640 1 -- 172.21.15.107:0/1172160219 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f5114005220 con 0x56387c8dde40 2024-09-19T04:23:41.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.136+0000 7f510e000640 1 -- 172.21.15.107:0/1172160219 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5114005420 con 0x56387c8dde40 2024-09-19T04:23:41.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.136+0000 7f512da42740 10 client.25819 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:41.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.136+0000 7f512da42740 1 -- 172.21.15.107:0/1172160219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56387c8f7d00 con 0x56387c8dde40 2024-09-19T04:23:41.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.137+0000 7f510e000640 1 -- 172.21.15.107:0/1172160219 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f511000ed60 con 0x56387c8dde40 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.137+0000 7f510e000640 1 -- 172.21.15.107:0/1172160219 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f51140319c0 con 0x56387c8dde40 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.137+0000 7f512da42740 10 client.25819 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.137+0000 7f512da42740 10 client.25819 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.137+0000 7f512da42740 20 client.25819 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.137+0000 7f512da42740 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56387c8f9110 0x56387c8fb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.137+0000 7f512da42740 4 client.25819 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.137+0000 7f512da42740 1 -- 172.21.15.107:0/1172160219 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56387c851950 con 0x56387c8f9110 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.138+0000 7f5123400640 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56387c8f9110 0x56387c8fb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.138+0000 7f5123400640 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56387c8f9110 0x56387c8fb500 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f910dab19ee812c server_cookie=2378a19efb93bca5 in_seq=0 out_seq=0 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.138+0000 7f510e000640 10 client.25819 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f510e000640 1 -- 172.21.15.107:0/1172160219 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56387c851950 con 0x56387c8f9110 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f510e000640 10 client.25819 handle_command_reply: tid=0 2024-09-19T04:23:41.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f512da42740 1 client.25819 shutdown 2024-09-19T04:23:41.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f512da42740 1 -- 172.21.15.107:0/1172160219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8dde40 msgr2=0x56387c8e5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:41.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f512da42740 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8dde40 0x56387c8e5f90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f511402a9d0 tx=0x7f51140043b0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f512da42740 1 -- 172.21.15.107:0/1172160219 shutdown_connections 2024-09-19T04:23:41.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f512da42740 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56387c8f9110 0x56387c8fb500 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f512da42740 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8e9020 0x56387c8e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f512da42740 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8de760 0x56387c8e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f512da42740 1 --2- 172.21.15.107:0/1172160219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8dde40 0x56387c8e5f90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.139+0000 7f512da42740 1 -- 172.21.15.107:0/1172160219 >> 172.21.15.107:0/1172160219 conn(0x56387c8da3d0 msgr2=0x56387c8e00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:41.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.140+0000 7f510e000640 0 client.25819 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:23:41.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.140+0000 7f512da42740 1 -- 172.21.15.107:0/1172160219 shutdown_connections 2024-09-19T04:23:41.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.140+0000 7f512da42740 1 -- 172.21.15.107:0/1172160219 wait complete. 2024-09-19T04:23:41.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.140+0000 7f512da42740 20 client.25819 trim_cache size 0 max 16384 2024-09-19T04:23:41.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.153+0000 7f512da42740 1 Processor -- start 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.153+0000 7f512da42740 1 -- start start 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f512da42740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8dde40 0x56387c8d5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f512da42740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8de760 0x56387c900d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f512da42740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c9012a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f512da42740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51241af7a0 con 0x56387c8dde40 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f512da42740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f51241b13d0 con 0x56387c8de760 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f512da42740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f51241afe70 con 0x56387c8e9020 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f5123e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8dde40 0x56387c8d5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f5123e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8dde40 0x56387c8d5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46906/0 (socket says 172.21.15.107:46906) 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f5123e00640 1 -- 172.21.15.107:0/3071943285 learned_addr learned my addr 172.21.15.107:0/3071943285 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f5129000640 1 --2- 172.21.15.107:0/3071943285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c9012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f5123400640 1 --2- 172.21.15.107:0/3071943285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8de760 0x56387c900d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f5123e00640 1 -- 172.21.15.107:0/3071943285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 msgr2=0x56387c9012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f5123e00640 1 --2- 172.21.15.107:0/3071943285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c9012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f5123e00640 1 -- 172.21.15.107:0/3071943285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8de760 msgr2=0x56387c900d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f5123e00640 1 --2- 172.21.15.107:0/3071943285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8de760 0x56387c900d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.154+0000 7f5123e00640 1 -- 172.21.15.107:0/3071943285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387c8ddc70 con 0x56387c8dde40 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.155+0000 7f5123e00640 1 --2- 172.21.15.107:0/3071943285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8dde40 0x56387c8d5300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5114009900 tx=0x7f5114009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1e8396be29b3bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.155+0000 7f5118c00640 1 -- 172.21.15.107:0/3071943285 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114035030 con 0x56387c8dde40 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.155+0000 7f5118c00640 1 -- 172.21.15.107:0/3071943285 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5114031a90 con 0x56387c8dde40 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.155+0000 7f5118c00640 1 -- 172.21.15.107:0/3071943285 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511404f6e0 con 0x56387c8dde40 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 -- 172.21.15.107:0/3071943285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8dde40 msgr2=0x56387c8d5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:41.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 --2- 172.21.15.107:0/3071943285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8dde40 0x56387c8d5300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5114009900 tx=0x7f5114009830 comp rx=0 tx=0).stop 2024-09-19T04:23:41.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 -- 172.21.15.107:0/3071943285 shutdown_connections 2024-09-19T04:23:41.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 --2- 172.21.15.107:0/3071943285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c9012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 --2- 172.21.15.107:0/3071943285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8de760 0x56387c900d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 --2- 172.21.15.107:0/3071943285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8dde40 0x56387c8d5300 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 -- 172.21.15.107:0/3071943285 >> 172.21.15.107:0/3071943285 conn(0x56387c8da3d0 msgr2=0x56387c8e4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:41.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 -- 172.21.15.107:0/3071943285 shutdown_connections 2024-09-19T04:23:41.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 -- 172.21.15.107:0/3071943285 wait complete. 2024-09-19T04:23:41.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 -- start start 2024-09-19T04:23:41.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.156+0000 7f512da42740 1 Processor -- start 2024-09-19T04:23:41.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f512da42740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 0x56387c8ec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f512da42740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8ecc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f512da42740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c95bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f512da42740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51241b0c40 con 0x56387c8de760 2024-09-19T04:23:41.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f512da42740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f51241b0540 con 0x56387c8dde40 2024-09-19T04:23:41.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f512da42740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f51241ad780 con 0x56387c8e9020 2024-09-19T04:23:41.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f512da42740 10 client.0 _ll_register_callbacks cb 0x7f5114005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:41.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f5123400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8ecc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f5123400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8ecc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46920/0 (socket says 172.21.15.107:46920) 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f5123400640 1 -- 172.21.15.107:0/2921603869 learned_addr learned my addr 172.21.15.107:0/2921603869 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f5123e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 0x56387c8ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.157+0000 7f5129000640 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c95bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f5123e00640 1 -- 172.21.15.107:0/2921603869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 msgr2=0x56387c95bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f5123e00640 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c95bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f5123e00640 1 -- 172.21.15.107:0/2921603869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 msgr2=0x56387c8ecc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f5123e00640 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8ecc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f5123e00640 1 -- 172.21.15.107:0/2921603869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387c95c390 con 0x56387c8dde40 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f5129000640 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c95bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:23:41.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f5123e00640 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 0x56387c8ec700 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f511402a9d0 tx=0x7f5114003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a8b52ebf7dd5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f510e000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f510e000640 1 -- 172.21.15.107:0/2921603869 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511405c020 con 0x56387c8dde40 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f512da42740 1 -- 172.21.15.107:0/2921603869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5114031a90 con 0x56387c8dde40 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.158+0000 7f510e000640 1 -- 172.21.15.107:0/2921603869 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f511404f680 con 0x56387c8dde40 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f510e000640 1 -- 172.21.15.107:0/2921603869 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5114067050 con 0x56387c8dde40 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f512da42740 10 client.25853 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f512da42740 1 -- 172.21.15.107:0/2921603869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56387c963700 con 0x56387c8dde40 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f510e000640 1 -- 172.21.15.107:0/2921603869 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5114031a90 con 0x56387c8dde40 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f510e000640 1 -- 172.21.15.107:0/2921603869 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f511403d450 con 0x56387c8dde40 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f512da42740 10 client.25853 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f512da42740 10 client.25853 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:23:41.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f512da42740 20 client.25853 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:41.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f512da42740 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56387c963d00 0x56387c9660f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:41.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f512da42740 4 client.25853 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:23:41.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.159+0000 7f512da42740 1 -- 172.21.15.107:0/2921603869 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56387c84be60 con 0x56387c963d00 2024-09-19T04:23:41.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.160+0000 7f5123400640 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56387c963d00 0x56387c9660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:41.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.160+0000 7f5123400640 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56387c963d00 0x56387c9660f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb4b582f51eaeb9a server_cookie=4205e033176e3210 in_seq=0 out_seq=0 2024-09-19T04:23:41.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.160+0000 7f510e000640 10 client.25853 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:23:41.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.160+0000 7f510e000640 1 -- 172.21.15.107:0/2921603869 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56387c84be60 con 0x56387c963d00 2024-09-19T04:23:41.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.160+0000 7f510e000640 10 client.25853 handle_command_reply: tid=0 2024-09-19T04:23:41.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.160+0000 7f512da42740 1 client.25853 shutdown 2024-09-19T04:23:41.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 1 -- 172.21.15.107:0/2921603869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 msgr2=0x56387c8ec700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:41.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 0x56387c8ec700 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f511402a9d0 tx=0x7f5114003820 comp rx=0 tx=0).stop 2024-09-19T04:23:41.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 1 -- 172.21.15.107:0/2921603869 shutdown_connections 2024-09-19T04:23:41.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x56387c963d00 0x56387c9660f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56387c8e9020 0x56387c95bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56387c8de760 0x56387c8ecc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 1 --2- 172.21.15.107:0/2921603869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56387c8dde40 0x56387c8ec700 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f510e000640 0 client.25853 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:23:41.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 1 -- 172.21.15.107:0/2921603869 >> 172.21.15.107:0/2921603869 conn(0x56387c8da3d0 msgr2=0x56387c8e00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:41.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 1 -- 172.21.15.107:0/2921603869 shutdown_connections 2024-09-19T04:23:41.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 1 -- 172.21.15.107:0/2921603869 wait complete. 2024-09-19T04:23:41.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f512da42740 20 client.25853 trim_cache size 0 max 16384 2024-09-19T04:23:41.166 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:23:41.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f5129a00640 1 -- 172.21.15.107:0/1942726753 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f51080a33b0 msgr2=0x7f51080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:41.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f5129a00640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f51080a33b0 0x7f51080a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f511000f850 tx=0x7f5110018000 comp rx=0 tx=0).stop 2024-09-19T04:23:41.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f5129a00640 1 -- 172.21.15.107:0/1942726753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512410e280 msgr2=0x7f512419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:41.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.161+0000 7f5129a00640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512410e280 0x7f512419edf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f511c00b9e0 tx=0x7f511c00bea0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.162+0000 7f5129a00640 1 -- 172.21.15.107:0/1942726753 shutdown_connections 2024-09-19T04:23:41.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.162+0000 7f5129a00640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f51080a33b0 0x7f51080a5870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.162+0000 7f5129a00640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512410e280 0x7f512419edf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.162+0000 7f5129a00640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5124074e30 0x7f512419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.162+0000 7f5129a00640 1 --2- 172.21.15.107:0/1942726753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5124074510 0x7f512419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:41.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.162+0000 7f5129a00640 1 -- 172.21.15.107:0/1942726753 >> 172.21.15.107:0/1942726753 conn(0x7f51240fd740 msgr2=0x7f51240fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:41.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.162+0000 7f5129a00640 1 -- 172.21.15.107:0/1942726753 shutdown_connections 2024-09-19T04:23:41.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:41.162+0000 7f5129a00640 1 -- 172.21.15.107:0/1942726753 wait complete. 2024-09-19T04:23:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:40 smithi115 ceph-mon[35477]: pgmap v103: 97 pgs: 97 active+clean; 80 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 776 KiB/s wr, 114 op/s 2024-09-19T04:23:41.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 128449 5.96 MB/sec warmup 119 sec latency 2388.206 ms 2024-09-19T04:23:42.118 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:23:42.486 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:42.902 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:42 smithi107 ceph-mon[30659]: pgmap v104: 97 pgs: 97 active+clean; 83 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 177 op/s 2024-09-19T04:23:42.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:42 smithi145 ceph-mon[35096]: pgmap v104: 97 pgs: 97 active+clean; 83 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 177 op/s 2024-09-19T04:23:43.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.012+0000 7f975b200640 1 -- 172.21.15.107:0/665951186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f975c104800 msgr2=0x7f975c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:43.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.012+0000 7f975b200640 1 --2- 172.21.15.107:0/665951186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f975c104800 0x7f975c108960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f974402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.012+0000 7f975b200640 1 -- 172.21.15.107:0/665951186 shutdown_connections 2024-09-19T04:23:43.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.012+0000 7f975b200640 1 --2- 172.21.15.107:0/665951186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975c109090 0x7f975c110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.012+0000 7f975b200640 1 --2- 172.21.15.107:0/665951186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f975c104800 0x7f975c108960 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.012+0000 7f975b200640 1 --2- 172.21.15.107:0/665951186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f975c103e50 0x7f975c104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.012+0000 7f975b200640 1 -- 172.21.15.107:0/665951186 >> 172.21.15.107:0/665951186 conn(0x7f975c0ff8b0 msgr2=0x7f975c101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:43.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.012+0000 7f975b200640 1 -- 172.21.15.107:0/665951186 shutdown_connections 2024-09-19T04:23:43.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.012+0000 7f975b200640 1 -- 172.21.15.107:0/665951186 wait complete. 2024-09-19T04:23:43.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.013+0000 7f975b200640 1 Processor -- start 2024-09-19T04:23:43.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.013+0000 7f975b200640 1 -- start start 2024-09-19T04:23:43.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f975b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f975c103e50 0x7f975c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f975b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975c104800 0x7f975c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f975b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f975c109090 0x7f975c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f975b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f975c114100 con 0x7f975c103e50 2024-09-19T04:23:43.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f975b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f975c113f80 con 0x7f975c109090 2024-09-19T04:23:43.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f975b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f975c114280 con 0x7f975c104800 2024-09-19T04:23:43.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f9759e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f975c103e50 0x7f975c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f9759e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f975c103e50 0x7f975c19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46924/0 (socket says 172.21.15.107:46924) 2024-09-19T04:23:43.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f9759e00640 1 -- 172.21.15.107:0/4096628462 learned_addr learned my addr 172.21.15.107:0/4096628462 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:43.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f9753e00640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975c104800 0x7f975c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.014+0000 7f975a800640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f975c109090 0x7f975c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.015+0000 7f9759e00640 1 -- 172.21.15.107:0/4096628462 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975c104800 msgr2=0x7f975c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.015+0000 7f9759e00640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975c104800 0x7f975c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.015+0000 7f9759e00640 1 -- 172.21.15.107:0/4096628462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f975c109090 msgr2=0x7f975c19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.015+0000 7f9759e00640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f975c109090 0x7f975c19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.015+0000 7f9759e00640 1 -- 172.21.15.107:0/4096628462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c19f510 con 0x7f975c103e50 2024-09-19T04:23:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.015+0000 7f9759e00640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f975c103e50 0x7f975c19a560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f974c00d950 tx=0x7f974c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.015+0000 7f9752a00640 1 -- 172.21.15.107:0/4096628462 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c017030 con 0x7f975c103e50 2024-09-19T04:23:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.015+0000 7f975b200640 1 -- 172.21.15.107:0/4096628462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c19f800 con 0x7f975c103e50 2024-09-19T04:23:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.015+0000 7f9752a00640 1 -- 172.21.15.107:0/4096628462 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f974c004020 con 0x7f975c103e50 2024-09-19T04:23:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.015+0000 7f9752a00640 1 -- 172.21.15.107:0/4096628462 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c0047a0 con 0x7f975c103e50 2024-09-19T04:23:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.016+0000 7f975b200640 1 -- 172.21.15.107:0/4096628462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c1a70e0 con 0x7f975c103e50 2024-09-19T04:23:43.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.020+0000 7f9763aa2740 1 Processor -- start 2024-09-19T04:23:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.020+0000 7f9763aa2740 1 -- start start 2024-09-19T04:23:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.020+0000 7f9752a00640 1 -- 172.21.15.107:0/4096628462 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f974c0041d0 con 0x7f975c103e50 2024-09-19T04:23:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.020+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 0x55f94574f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.020+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 0x55f945759ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.020+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94575a020 0x55f94575c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.020+0000 7f9763aa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f9456b53e0 con 0x55f94575a020 2024-09-19T04:23:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.020+0000 7f9763aa2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f94565cd80 con 0x55f94574f760 2024-09-19T04:23:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.020+0000 7f9763aa2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f9456b13a0 con 0x55f94574ee40 2024-09-19T04:23:43.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.021+0000 7f975a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 0x55f94574f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.021+0000 7f9759e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 0x55f945759ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.021+0000 7f975a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 0x55f94574f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:49084/0 (socket says 172.21.15.107:49084) 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.021+0000 7f975a800640 1 -- 172.21.15.107:0/1443274295 learned_addr learned my addr 172.21.15.107:0/1443274295 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.021+0000 7f9753e00640 1 --2- 172.21.15.107:0/1443274295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94575a020 0x55f94575c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.021+0000 7f975a800640 1 -- 172.21.15.107:0/1443274295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 msgr2=0x55f945759ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.021+0000 7f975a800640 1 --2- 172.21.15.107:0/1443274295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 0x55f945759ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.021+0000 7f975a800640 1 -- 172.21.15.107:0/1443274295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94575a020 msgr2=0x55f94575c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.021+0000 7f975a800640 1 --2- 172.21.15.107:0/1443274295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94575a020 0x55f94575c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.021+0000 7f975a800640 1 -- 172.21.15.107:0/1443274295 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f94567bae0 con 0x55f94574ee40 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.022+0000 7f975a800640 1 --2- 172.21.15.107:0/1443274295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 0x55f94574f220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9754009f70 tx=0x7f9754010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=704c4a207ffdca54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:43.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.022+0000 7f974b400640 1 -- 172.21.15.107:0/1443274295 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754016070 con 0x55f94574ee40 2024-09-19T04:23:43.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.022+0000 7f9752a00640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97400a33b0 0x7f97400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.022+0000 7f974b400640 1 -- 172.21.15.107:0/1443274295 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9754004d20 con 0x55f94574ee40 2024-09-19T04:23:43.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.022+0000 7f974b400640 1 -- 172.21.15.107:0/1443274295 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754012910 con 0x55f94574ee40 2024-09-19T04:23:43.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.022+0000 7f9752a00640 1 -- 172.21.15.107:0/4096628462 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f974c0cf620 con 0x7f975c103e50 2024-09-19T04:23:43.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.022+0000 7f9759e00640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97400a33b0 0x7f97400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 -- 172.21.15.107:0/1443274295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 msgr2=0x55f94574f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1443274295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 0x55f94574f220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9754009f70 tx=0x7f9754010800 comp rx=0 tx=0).stop 2024-09-19T04:23:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 -- 172.21.15.107:0/1443274295 shutdown_connections 2024-09-19T04:23:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1443274295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94575a020 0x55f94575c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1443274295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 0x55f945759ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1443274295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 0x55f94574f220 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 -- 172.21.15.107:0/1443274295 >> 172.21.15.107:0/1443274295 conn(0x55f94574b3d0 msgr2=0x55f94574b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9759e00640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97400a33b0 0x7f97400a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f974c0cce70 tx=0x7f974c0d5550 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 -- 172.21.15.107:0/1443274295 shutdown_connections 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 -- 172.21.15.107:0/1443274295 wait complete. 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 -- start start 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.023+0000 7f9763aa2740 1 Processor -- start 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574ee40 0x55f945756fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 0x55f9457574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94575a020 0x55f945757a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f9763aa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f94575cbf0 con 0x55f94574ee40 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f9763aa2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f9456700f0 con 0x55f94574f760 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f9763aa2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f94575cd70 con 0x55f94575a020 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f9763aa2740 10 client.0 _ll_register_callbacks cb 0x55f945657bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f975a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574ee40 0x55f945756fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f9753e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 0x55f9457574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f975a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574ee40 0x55f945756fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46948/0 (socket says 172.21.15.107:46948) 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.024+0000 7f975a800640 1 -- 172.21.15.107:0/1681516749 learned_addr learned my addr 172.21.15.107:0/1681516749 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.025+0000 7f975a800640 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94575a020 0x55f945757a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.025+0000 7f9753e00640 1 -- 172.21.15.107:0/1681516749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94575a020 msgr2=0x55f945757a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.025+0000 7f9753e00640 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94575a020 0x55f945757a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.025+0000 7f9753e00640 1 -- 172.21.15.107:0/1681516749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574ee40 msgr2=0x55f945756fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.025+0000 7f9753e00640 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574ee40 0x55f945756fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.025+0000 7f9753e00640 1 -- 172.21.15.107:0/1681516749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9454263c0 con 0x55f94574f760 2024-09-19T04:23:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.025+0000 7f975a800640 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574ee40 0x55f945756fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:23:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.025+0000 7f9753e00640 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 0x55f9457574e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f974402a9e0 tx=0x7f97440044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21d5c9502e0cc8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.025+0000 7f973fe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:23:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.025+0000 7f973fe00640 1 -- 172.21.15.107:0/1681516749 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744035040 con 0x55f94574f760 2024-09-19T04:23:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.026+0000 7f973fe00640 1 -- 172.21.15.107:0/1681516749 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f9744005510 con 0x55f94574f760 2024-09-19T04:23:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.026+0000 7f9763aa2740 1 -- 172.21.15.107:0/1681516749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9754004d20 con 0x55f94574f760 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.026+0000 7f973fe00640 1 -- 172.21.15.107:0/1681516749 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f97440046c0 con 0x55f94574f760 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.026+0000 7f9763aa2740 10 client.25865 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.026+0000 7f9763aa2740 1 -- 172.21.15.107:0/1681516749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f945768cb0 con 0x55f94574f760 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.026+0000 7f973fe00640 1 -- 172.21.15.107:0/1681516749 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9754004d20 con 0x55f94574f760 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.027+0000 7f973fe00640 1 -- 172.21.15.107:0/1681516749 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f9744003bc0 con 0x55f94574f760 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.027+0000 7f9763aa2740 10 client.25865 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.027+0000 7f9763aa2740 10 client.25865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.027+0000 7f9763aa2740 20 client.25865 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.027+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f94576a1c0 0x55f94576c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.027+0000 7f9763aa2740 4 client.25865 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.027+0000 7f9763aa2740 1 -- 172.21.15.107:0/1681516749 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9456c2950 con 0x55f94576a1c0 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.027+0000 7f975a800640 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f94576a1c0 0x55f94576c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.028+0000 7f975a800640 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f94576a1c0 0x55f94576c610 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=843d05c173998b9 server_cookie=b72cc04059535d3a in_seq=0 out_seq=0 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.028+0000 7f973fe00640 10 client.25865 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.028+0000 7f973fe00640 1 -- 172.21.15.107:0/1681516749 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9456c2950 con 0x55f94576a1c0 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.028+0000 7f973fe00640 10 client.25865 handle_command_reply: tid=0 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.028+0000 7f9763aa2740 1 client.25865 shutdown 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 1 -- 172.21.15.107:0/1681516749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 msgr2=0x55f9457574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 0x55f9457574e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f974402a9e0 tx=0x7f97440044f0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 1 -- 172.21.15.107:0/1681516749 shutdown_connections 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f94576a1c0 0x55f94576c610 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94575a020 0x55f945757a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574f760 0x55f9457574e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f973fe00640 0 client.25865 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1681516749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574ee40 0x55f945756fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 1 -- 172.21.15.107:0/1681516749 >> 172.21.15.107:0/1681516749 conn(0x55f94574b3d0 msgr2=0x55f9457510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:43.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 1 -- 172.21.15.107:0/1681516749 shutdown_connections 2024-09-19T04:23:43.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 1 -- 172.21.15.107:0/1681516749 wait complete. 2024-09-19T04:23:43.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.029+0000 7f9763aa2740 20 client.25865 trim_cache size 0 max 16384 2024-09-19T04:23:43.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.042+0000 7f9763aa2740 1 Processor -- start 2024-09-19T04:23:43.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f9763aa2740 1 -- start start 2024-09-19T04:23:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 0x55f945758f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574f760 0x55f945771f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94575a020 0x55f945772460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f9763aa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1af7d0 con 0x55f94574f760 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f9763aa2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1b1400 con 0x55f94575a020 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f9763aa2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1afea0 con 0x55f94574ee40 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f9753e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574f760 0x55f945771f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f9753e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574f760 0x55f945771f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46960/0 (socket says 172.21.15.107:46960) 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f9753e00640 1 -- 172.21.15.107:0/145794343 learned_addr learned my addr 172.21.15.107:0/145794343 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f975a800640 1 --2- 172.21.15.107:0/145794343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 0x55f945758f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.043+0000 7f975a800640 1 --2- 172.21.15.107:0/145794343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94575a020 0x55f945772460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.044+0000 7f9753e00640 1 -- 172.21.15.107:0/145794343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 msgr2=0x55f945758f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.044+0000 7f9753e00640 1 --2- 172.21.15.107:0/145794343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 0x55f945758f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.044+0000 7f9753e00640 1 -- 172.21.15.107:0/145794343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94575a020 msgr2=0x55f945772460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.044+0000 7f9753e00640 1 --2- 172.21.15.107:0/145794343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94575a020 0x55f945772460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.044+0000 7f9753e00640 1 -- 172.21.15.107:0/145794343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f945766990 con 0x55f94574f760 2024-09-19T04:23:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.044+0000 7f975a800640 1 --2- 172.21.15.107:0/145794343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94575a020 0x55f945772460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:23:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.044+0000 7f9753e00640 1 --2- 172.21.15.107:0/145794343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574f760 0x55f945771f20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f97440319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f4c5715bca8cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.044+0000 7f974aa00640 1 -- 172.21.15.107:0/145794343 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744035030 con 0x55f94574f760 2024-09-19T04:23:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.044+0000 7f974aa00640 1 -- 172.21.15.107:0/145794343 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9744041070 con 0x55f94574f760 2024-09-19T04:23:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.044+0000 7f974aa00640 1 -- 172.21.15.107:0/145794343 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744004be0 con 0x55f94574f760 2024-09-19T04:23:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.045+0000 7f9763aa2740 1 -- 172.21.15.107:0/145794343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574f760 msgr2=0x55f945771f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.045+0000 7f9763aa2740 1 --2- 172.21.15.107:0/145794343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574f760 0x55f945771f20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f97440319e0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.045+0000 7f9763aa2740 1 -- 172.21.15.107:0/145794343 shutdown_connections 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.045+0000 7f9763aa2740 1 --2- 172.21.15.107:0/145794343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94575a020 0x55f945772460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.045+0000 7f9763aa2740 1 --2- 172.21.15.107:0/145794343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94574f760 0x55f945771f20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.045+0000 7f9763aa2740 1 --2- 172.21.15.107:0/145794343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574ee40 0x55f945758f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.045+0000 7f9763aa2740 1 -- 172.21.15.107:0/145794343 >> 172.21.15.107:0/145794343 conn(0x55f94574b3d0 msgr2=0x55f945758800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.045+0000 7f9763aa2740 1 -- 172.21.15.107:0/145794343 shutdown_connections 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.045+0000 7f9763aa2740 1 -- 172.21.15.107:0/145794343 wait complete. 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.045+0000 7f9763aa2740 1 -- start start 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.046+0000 7f9763aa2740 1 Processor -- start 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.046+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574ee40 0x55f9457628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.046+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574f760 0x55f94575dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.046+0000 7f9763aa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94575a020 0x55f94575e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.046+0000 7f9763aa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1b0c70 con 0x55f94575a020 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.046+0000 7f9763aa2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1b0570 con 0x55f94574ee40 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.046+0000 7f9763aa2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ad7b0 con 0x55f94574f760 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.046+0000 7f9763aa2740 10 client.0 _ll_register_callbacks cb 0x7f9744009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.046+0000 7f9753e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574f760 0x55f94575dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f9753e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574f760 0x55f94575dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:49114/0 (socket says 172.21.15.107:49114) 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f9753e00640 1 -- 172.21.15.107:0/1634498916 learned_addr learned my addr 172.21.15.107:0/1634498916 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f975a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94575a020 0x55f94575e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f975a800640 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574ee40 0x55f9457628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f9753e00640 1 -- 172.21.15.107:0/1634498916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574ee40 msgr2=0x55f9457628e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f9753e00640 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574ee40 0x55f9457628e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f9753e00640 1 -- 172.21.15.107:0/1634498916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94575a020 msgr2=0x55f94575e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f9753e00640 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94575a020 0x55f94575e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f9753e00640 1 -- 172.21.15.107:0/1634498916 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9457d3ef0 con 0x55f94574f760 2024-09-19T04:23:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f9753e00640 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574f760 0x55f94575dc10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9744031620 tx=0x7f974402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fb0bf749b12a62b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f973fe00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:23:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f973fe00640 1 -- 172.21.15.107:0/1634498916 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744035040 con 0x55f94574f760 2024-09-19T04:23:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.047+0000 7f9763aa2740 1 -- 172.21.15.107:0/1634498916 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9744041070 con 0x55f94574f760 2024-09-19T04:23:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f973fe00640 1 -- 172.21.15.107:0/1634498916 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f9744004c70 con 0x55f94574f760 2024-09-19T04:23:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f973fe00640 1 -- 172.21.15.107:0/1634498916 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9744069050 con 0x55f94574f760 2024-09-19T04:23:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f9763aa2740 10 client.25843 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f9763aa2740 1 -- 172.21.15.107:0/1634498916 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f9457d50d0 con 0x55f94574f760 2024-09-19T04:23:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f973fe00640 1 -- 172.21.15.107:0/1634498916 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9744041070 con 0x55f94574f760 2024-09-19T04:23:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f973fe00640 1 -- 172.21.15.107:0/1634498916 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f9744058020 con 0x55f94574f760 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f9763aa2740 10 client.25843 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f9763aa2740 10 client.25843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f9763aa2740 20 client.25843 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9457d57c0 0x55f9457d7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f9763aa2740 4 client.25843 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.048+0000 7f9763aa2740 1 -- 172.21.15.107:0/1634498916 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9456bce60 con 0x55f9457d57c0 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.049+0000 7f975a800640 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9457d57c0 0x55f9457d7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.049+0000 7f975a800640 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9457d57c0 0x55f9457d7be0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d72d9bbe89ed643 server_cookie=c17ce99459a597bd in_seq=0 out_seq=0 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.049+0000 7f973fe00640 10 client.25843 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.049+0000 7f973fe00640 1 -- 172.21.15.107:0/1634498916 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9456bce60 con 0x55f9457d57c0 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.049+0000 7f973fe00640 10 client.25843 handle_command_reply: tid=0 2024-09-19T04:23:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.049+0000 7f9763aa2740 1 client.25843 shutdown 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 1 -- 172.21.15.107:0/1634498916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574f760 msgr2=0x55f94575dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574f760 0x55f94575dc10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9744031620 tx=0x7f974402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 1 -- 172.21.15.107:0/1634498916 shutdown_connections 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f9457d57c0 0x55f9457d7be0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94575a020 0x55f94575e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f94574f760 0x55f94575dc10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f973fe00640 0 client.25843 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 1 --2- 172.21.15.107:0/1634498916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f94574ee40 0x55f9457628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 1 -- 172.21.15.107:0/1634498916 >> 172.21.15.107:0/1634498916 conn(0x55f94574b3d0 msgr2=0x55f94574a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 1 -- 172.21.15.107:0/1634498916 shutdown_connections 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 1 -- 172.21.15.107:0/1634498916 wait complete. 2024-09-19T04:23:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.050+0000 7f9763aa2740 20 client.25843 trim_cache size 0 max 16384 2024-09-19T04:23:43.056 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:23:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 -- 172.21.15.107:0/4096628462 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97400a33b0 msgr2=0x7f97400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97400a33b0 0x7f97400a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f974c0cce70 tx=0x7f974c0d5550 comp rx=0 tx=0).stop 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 -- 172.21.15.107:0/4096628462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f975c103e50 msgr2=0x7f975c19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f975c103e50 0x7f975c19a560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f974c00d950 tx=0x7f974c00de10 comp rx=0 tx=0).stop 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 -- 172.21.15.107:0/4096628462 shutdown_connections 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97400a33b0 0x7f97400a5870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f975c109090 0x7f975c19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975c104800 0x7f975c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 --2- 172.21.15.107:0/4096628462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f975c103e50 0x7f975c19a560 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 -- 172.21.15.107:0/4096628462 >> 172.21.15.107:0/4096628462 conn(0x7f975c0ff8b0 msgr2=0x7f975c100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 -- 172.21.15.107:0/4096628462 shutdown_connections 2024-09-19T04:23:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:43.051+0000 7f975b200640 1 -- 172.21.15.107:0/4096628462 wait complete. 2024-09-19T04:23:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:42 smithi115 ceph-mon[35477]: pgmap v104: 97 pgs: 97 active+clean; 83 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 177 op/s 2024-09-19T04:23:43.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 128449 0.00 MB/sec execute 1 sec latency 4388.335 ms 2024-09-19T04:23:44.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 128518 0.08 MB/sec execute 2 sec latency 4958.666 ms 2024-09-19T04:23:44.965 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:23:44.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:23:44.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:44 smithi145 ceph-mon[35096]: pgmap v105: 97 pgs: 97 active+clean; 83 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 590 KiB/s wr, 118 op/s 2024-09-19T04:23:44.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:44 smithi107 ceph-mon[30659]: pgmap v105: 97 pgs: 97 active+clean; 83 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 590 KiB/s wr, 118 op/s 2024-09-19T04:23:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:44 smithi115 ceph-mon[35477]: pgmap v105: 97 pgs: 97 active+clean; 83 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 590 KiB/s wr, 118 op/s 2024-09-19T04:23:45.336 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:45.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 128518 0.05 MB/sec execute 3 sec latency 1380.711 ms 2024-09-19T04:23:46.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 128518 0.04 MB/sec execute 4 sec latency 2380.777 ms 2024-09-19T04:23:46.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:46 smithi107 ceph-mon[30659]: pgmap v106: 97 pgs: 97 active+clean; 82 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 880 KiB/s wr, 144 op/s 2024-09-19T04:23:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.141+0000 7fe91a800640 1 -- 172.21.15.107:0/733078527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe91410e3c0 msgr2=0x7fe914110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.141+0000 7fe91a800640 1 --2- 172.21.15.107:0/733078527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe91410e3c0 0x7fe914110810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009ca0 tx=0x7fe90c02a950 comp rx=0 tx=0).stop 2024-09-19T04:23:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.141+0000 7fe91a800640 1 -- 172.21.15.107:0/733078527 shutdown_connections 2024-09-19T04:23:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.141+0000 7fe91a800640 1 --2- 172.21.15.107:0/733078527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe91410e3c0 0x7fe914110810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.141+0000 7fe91a800640 1 --2- 172.21.15.107:0/733078527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe914074f40 0x7fe91410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.141+0000 7fe91a800640 1 --2- 172.21.15.107:0/733078527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe914074620 0x7fe914074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.141+0000 7fe91a800640 1 -- 172.21.15.107:0/733078527 >> 172.21.15.107:0/733078527 conn(0x7fe9140fd8c0 msgr2=0x7fe9140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.142+0000 7fe91a800640 1 -- 172.21.15.107:0/733078527 shutdown_connections 2024-09-19T04:23:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.142+0000 7fe91a800640 1 -- 172.21.15.107:0/733078527 wait complete. 2024-09-19T04:23:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.142+0000 7fe91a800640 1 Processor -- start 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe91a800640 1 -- start start 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe91a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe914074620 0x7fe91419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe91a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe914074f40 0x7fe91419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe91a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe91410e3c0 0x7fe91419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe91a800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9141141c0 con 0x7fe91410e3c0 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe91a800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe914114040 con 0x7fe914074f40 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe91a800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe914114340 con 0x7fe914074620 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe912a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe914074f40 0x7fe91419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe912a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe914074f40 0x7fe91419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45054/0 (socket says 172.21.15.107:45054) 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe913e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe91410e3c0 0x7fe91419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe912a00640 1 -- 172.21.15.107:0/2216293631 learned_addr learned my addr 172.21.15.107:0/2216293631 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.143+0000 7fe913400640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe914074620 0x7fe91419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe913400640 1 -- 172.21.15.107:0/2216293631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe914074f40 msgr2=0x7fe91419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe913400640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe914074f40 0x7fe91419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe913400640 1 -- 172.21.15.107:0/2216293631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe91410e3c0 msgr2=0x7fe91419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe913400640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe91410e3c0 0x7fe91419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe913400640 1 -- 172.21.15.107:0/2216293631 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91419f6b0 con 0x7fe914074620 2024-09-19T04:23:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe913e00640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe91410e3c0 0x7fe91419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe912a00640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe914074f40 0x7fe91419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe913400640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe914074620 0x7fe91419a700 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe90400e9f0 tx=0x7fe90400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe90be00640 1 -- 172.21.15.107:0/2216293631 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90400ccc0 con 0x7fe914074620 2024-09-19T04:23:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe91a800640 1 -- 172.21.15.107:0/2216293631 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91419f9a0 con 0x7fe914074620 2024-09-19T04:23:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe90be00640 1 -- 172.21.15.107:0/2216293631 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe904010910 con 0x7fe914074620 2024-09-19T04:23:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.144+0000 7fe90be00640 1 -- 172.21.15.107:0/2216293631 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904018960 con 0x7fe914074620 2024-09-19T04:23:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.145+0000 7fe91a800640 1 -- 172.21.15.107:0/2216293631 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9141a7280 con 0x7fe914074620 2024-09-19T04:23:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.147+0000 7fe91d3e0740 1 Processor -- start 2024-09-19T04:23:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe91d3e0740 1 -- start start 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324afac0 0x55ed324afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324b03e0 0x55ed324ba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324baca0 0x55ed324bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe90be00640 1 -- 172.21.15.107:0/2216293631 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fe904010ac0 con 0x7fe914074620 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ed32416290 con 0x55ed324afac0 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ed323bdb40 con 0x55ed324b03e0 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ed32412250 con 0x55ed324baca0 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe913e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324afac0 0x55ed324afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe913e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324afac0 0x55ed324afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46984/0 (socket says 172.21.15.107:46984) 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe913e00640 1 -- 172.21.15.107:0/2629019125 learned_addr learned my addr 172.21.15.107:0/2629019125 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.148+0000 7fe913400640 1 --2- 172.21.15.107:0/2629019125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324b03e0 0x55ed324ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.149+0000 7fe913e00640 1 -- 172.21.15.107:0/2629019125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324baca0 msgr2=0x55ed324bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.149+0000 7fe913e00640 1 --2- 172.21.15.107:0/2629019125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324baca0 0x55ed324bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.149+0000 7fe913e00640 1 -- 172.21.15.107:0/2629019125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324b03e0 msgr2=0x55ed324ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.149+0000 7fe913e00640 1 --2- 172.21.15.107:0/2629019125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324b03e0 0x55ed324ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.149+0000 7fe913e00640 1 -- 172.21.15.107:0/2629019125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed321876a0 con 0x55ed324afac0 2024-09-19T04:23:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.149+0000 7fe913400640 1 --2- 172.21.15.107:0/2629019125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324b03e0 0x55ed324ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:23:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.149+0000 7fe913e00640 1 --2- 172.21.15.107:0/2629019125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324afac0 0x55ed324afea0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009ca0 tx=0x7fe90c005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702de25ae819ba9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.149+0000 7fe908c00640 1 -- 172.21.15.107:0/2629019125 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c048070 con 0x55ed324afac0 2024-09-19T04:23:47.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe90be00640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8f80a33b0 0x7fe8f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe908c00640 1 -- 172.21.15.107:0/2629019125 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe90c003e50 con 0x55ed324afac0 2024-09-19T04:23:47.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe908c00640 1 -- 172.21.15.107:0/2629019125 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c034040 con 0x55ed324afac0 2024-09-19T04:23:47.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe90be00640 1 -- 172.21.15.107:0/2216293631 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fe9040d00c0 con 0x7fe914074620 2024-09-19T04:23:47.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe913400640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8f80a33b0 0x7fe8f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe91d3e0740 1 -- 172.21.15.107:0/2629019125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324afac0 msgr2=0x55ed324afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/2629019125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324afac0 0x55ed324afea0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009ca0 tx=0x7fe90c005f80 comp rx=0 tx=0).stop 2024-09-19T04:23:47.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe91d3e0740 1 -- 172.21.15.107:0/2629019125 shutdown_connections 2024-09-19T04:23:47.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/2629019125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324baca0 0x55ed324bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/2629019125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324b03e0 0x55ed324ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/2629019125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324afac0 0x55ed324afea0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.150+0000 7fe91d3e0740 1 -- 172.21.15.107:0/2629019125 >> 172.21.15.107:0/2629019125 conn(0x55ed324ac050 msgr2=0x55ed324ac430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.151+0000 7fe913400640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8f80a33b0 0x7fe8f80a5870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55ed324aceb0 tx=0x7fe904009700 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.151+0000 7fe91d3e0740 1 -- 172.21.15.107:0/2629019125 shutdown_connections 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.151+0000 7fe91d3e0740 1 -- 172.21.15.107:0/2629019125 wait complete. 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.151+0000 7fe91d3e0740 1 -- start start 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.151+0000 7fe91d3e0740 1 Processor -- start 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 0x55ed324ba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324baca0 0x55ed324b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ed324bd910 con 0x55ed324b03e0 2024-09-19T04:23:47.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ed323d0fa0 con 0x55ed324afac0 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ed324bda90 con 0x55ed324baca0 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe91d3e0740 10 client.0 _ll_register_callbacks cb 0x55ed32124ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe912a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe912a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324b78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47000/0 (socket says 172.21.15.107:47000) 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe912a00640 1 -- 172.21.15.107:0/3569289409 learned_addr learned my addr 172.21.15.107:0/3569289409 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe913e00640 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 0x55ed324ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe913e00640 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324baca0 0x55ed324b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe912a00640 1 -- 172.21.15.107:0/3569289409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324baca0 msgr2=0x55ed324b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe912a00640 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324baca0 0x55ed324b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe912a00640 1 -- 172.21.15.107:0/3569289409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 msgr2=0x55ed324ba610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe912a00640 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 0x55ed324ba610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.152+0000 7fe912a00640 1 -- 172.21.15.107:0/3569289409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed324b8650 con 0x55ed324b03e0 2024-09-19T04:23:47.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.153+0000 7fe912a00640 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324b78c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe9000119e0 tx=0x7fe900011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637cb3d59213ea82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.153+0000 7fe8fd600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:23:47.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.153+0000 7fe8fd600640 1 -- 172.21.15.107:0/3569289409 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90001e060 con 0x55ed324b03e0 2024-09-19T04:23:47.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.153+0000 7fe8fd600640 1 -- 172.21.15.107:0/3569289409 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fe90000fce0 con 0x55ed324b03e0 2024-09-19T04:23:47.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.153+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3569289409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe90c003e50 con 0x55ed324b03e0 2024-09-19T04:23:47.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.153+0000 7fe8fd600640 1 -- 172.21.15.107:0/3569289409 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe900023460 con 0x55ed324b03e0 2024-09-19T04:23:47.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.153+0000 7fe91d3e0740 10 client.15834 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:47.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.153+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3569289409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ed324c98e0 con 0x55ed324b03e0 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.154+0000 7fe8fd600640 1 -- 172.21.15.107:0/3569289409 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe90c003e50 con 0x55ed324b03e0 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.154+0000 7fe8fd600640 1 -- 172.21.15.107:0/3569289409 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fe90000e4d0 con 0x55ed324b03e0 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.154+0000 7fe91d3e0740 10 client.15834 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.154+0000 7fe91d3e0740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.154+0000 7fe91d3e0740 20 client.15834 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.154+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ed324cae90 0x55ed324cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.154+0000 7fe91d3e0740 4 client.15834 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.154+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3569289409 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed32423800 con 0x55ed324cae90 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.154+0000 7fe913e00640 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ed324cae90 0x55ed324cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.155+0000 7fe913e00640 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ed324cae90 0x55ed324cd2b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c85dd923f166ff2 server_cookie=9e6723bd911cfcf3 in_seq=0 out_seq=0 2024-09-19T04:23:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.155+0000 7fe8fd600640 10 client.15834 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe8fd600640 1 -- 172.21.15.107:0/3569289409 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed32423800 con 0x55ed324cae90 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe8fd600640 10 client.15834 handle_command_reply: tid=0 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 client.15834 shutdown 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3569289409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 msgr2=0x55ed324b78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324b78c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe9000119e0 tx=0x7fe900011ea0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3569289409 shutdown_connections 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ed324cae90 0x55ed324cd2b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324baca0 0x55ed324b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324b78c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3569289409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 0x55ed324ba610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe8fd600640 0 client.15834 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3569289409 >> 172.21.15.107:0/3569289409 conn(0x55ed324ac050 msgr2=0x55ed324b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3569289409 shutdown_connections 2024-09-19T04:23:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.156+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3569289409 wait complete. 2024-09-19T04:23:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.157+0000 7fe91d3e0740 20 client.15834 trim_cache size 0 max 16384 2024-09-19T04:23:47.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.174+0000 7fe91d3e0740 1 Processor -- start 2024-09-19T04:23:47.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe91d3e0740 1 -- start start 2024-09-19T04:23:47.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324ac050 0x55ed324d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324afac0 0x55ed324d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9141af9c0 con 0x55ed324b03e0 2024-09-19T04:23:47.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe9141b15f0 con 0x55ed324ac050 2024-09-19T04:23:47.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe9141b0090 con 0x55ed324afac0 2024-09-19T04:23:47.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe913e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324ac050 0x55ed324d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe913e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324ac050 0x55ed324d2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45080/0 (socket says 172.21.15.107:45080) 2024-09-19T04:23:47.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe913e00640 1 -- 172.21.15.107:0/3351545303 learned_addr learned my addr 172.21.15.107:0/3351545303 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:47.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe913e00640 1 --2- 172.21.15.107:0/3351545303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.175+0000 7fe912a00640 1 --2- 172.21.15.107:0/3351545303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324afac0 0x55ed324d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.176+0000 7fe913e00640 1 -- 172.21.15.107:0/3351545303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324afac0 msgr2=0x55ed324d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.176+0000 7fe913e00640 1 --2- 172.21.15.107:0/3351545303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324afac0 0x55ed324d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.176+0000 7fe913e00640 1 -- 172.21.15.107:0/3351545303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 msgr2=0x55ed324b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.176+0000 7fe913e00640 1 --2- 172.21.15.107:0/3351545303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.176+0000 7fe913e00640 1 -- 172.21.15.107:0/3351545303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed324ad100 con 0x55ed324ac050 2024-09-19T04:23:47.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.176+0000 7fe913e00640 1 --2- 172.21.15.107:0/3351545303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324ac050 0x55ed324d2b60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe90c004af0 tx=0x7fe90c005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e6cd9101e8aa18b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.176+0000 7fe8ffe00640 1 -- 172.21.15.107:0/3351545303 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c048040 con 0x55ed324ac050 2024-09-19T04:23:47.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.176+0000 7fe8ffe00640 1 -- 172.21.15.107:0/3351545303 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe90c004720 con 0x55ed324ac050 2024-09-19T04:23:47.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.176+0000 7fe8ffe00640 1 -- 172.21.15.107:0/3351545303 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c030aa0 con 0x55ed324ac050 2024-09-19T04:23:47.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3351545303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324ac050 msgr2=0x55ed324d2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3351545303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324ac050 0x55ed324d2b60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe90c004af0 tx=0x7fe90c005d80 comp rx=0 tx=0).stop 2024-09-19T04:23:47.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3351545303 shutdown_connections 2024-09-19T04:23:47.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3351545303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3351545303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324afac0 0x55ed324d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3351545303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324ac050 0x55ed324d2b60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3351545303 >> 172.21.15.107:0/3351545303 conn(0x55ed324acad0 msgr2=0x55ed324ab1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3351545303 shutdown_connections 2024-09-19T04:23:47.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3351545303 wait complete. 2024-09-19T04:23:47.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 -- start start 2024-09-19T04:23:47.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.177+0000 7fe91d3e0740 1 Processor -- start 2024-09-19T04:23:47.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324ac050 0x55ed324c3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 0x55ed324c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe91d3e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9141b0e60 con 0x55ed324b03e0 2024-09-19T04:23:47.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe9141b0760 con 0x55ed324afac0 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe91d3e0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe9141ad9a0 con 0x55ed324ac050 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe91d3e0740 10 client.0 _ll_register_callbacks cb 0x55ed324b8820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe913e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324ac050 0x55ed324c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe913e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe913e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324ac050 0x55ed324c3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:49162/0 (socket says 172.21.15.107:49162) 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe913e00640 1 -- 172.21.15.107:0/3177680270 learned_addr learned my addr 172.21.15.107:0/3177680270 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.178+0000 7fe912a00640 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 0x55ed324c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe913e00640 1 -- 172.21.15.107:0/3177680270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 msgr2=0x55ed324c35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe913e00640 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 0x55ed324c35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe913e00640 1 -- 172.21.15.107:0/3177680270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 msgr2=0x55ed324bfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe913e00640 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324bfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe913e00640 1 -- 172.21.15.107:0/3177680270 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed324b8190 con 0x55ed324ac050 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe912a00640 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 0x55ed324c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe913e00640 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324ac050 0x55ed324c3080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe90c02fd70 tx=0x7fe90c030680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8303440e8ac7699f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe8fd600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe8fd600640 1 -- 172.21.15.107:0/3177680270 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c048040 con 0x55ed324ac050 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe8fd600640 1 -- 172.21.15.107:0/3177680270 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fe90c003bb0 con 0x55ed324ac050 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.179+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3177680270 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe90c004720 con 0x55ed324ac050 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe8fd600640 1 -- 172.21.15.107:0/3177680270 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe90c0304b0 con 0x55ed324ac050 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe91d3e0740 10 client.25858 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3177680270 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ed32536600 con 0x55ed324ac050 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe8fd600640 1 -- 172.21.15.107:0/3177680270 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe90c004720 con 0x55ed324ac050 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe8fd600640 1 -- 172.21.15.107:0/3177680270 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fe90c04e070 con 0x55ed324ac050 2024-09-19T04:23:47.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe91d3e0740 10 client.25858 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:47.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe91d3e0740 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:47.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe91d3e0740 20 client.25858 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:47.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ed324baca0 0x55ed32538930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:47.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe91d3e0740 4 client.25858 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:23:47.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3177680270 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ed3241dd10 con 0x55ed324baca0 2024-09-19T04:23:47.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.180+0000 7fe912a00640 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ed324baca0 0x55ed32538930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:47.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.181+0000 7fe912a00640 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ed324baca0 0x55ed32538930 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=384485d5544bad3b server_cookie=68767930d2448cea in_seq=0 out_seq=0 2024-09-19T04:23:47.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.181+0000 7fe8fd600640 10 client.25858 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:47.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.182+0000 7fe8fd600640 1 -- 172.21.15.107:0/3177680270 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ed3241dd10 con 0x55ed324baca0 2024-09-19T04:23:47.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.182+0000 7fe8fd600640 10 client.25858 handle_command_reply: tid=0 2024-09-19T04:23:47.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.182+0000 7fe91d3e0740 1 client.25858 shutdown 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.182+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3177680270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324ac050 msgr2=0x55ed324c3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324ac050 0x55ed324c3080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe90c02fd70 tx=0x7fe90c030680 comp rx=0 tx=0).stop 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3177680270 shutdown_connections 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ed324baca0 0x55ed32538930 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed324b03e0 0x55ed324bfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed324afac0 0x55ed324c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91d3e0740 1 --2- 172.21.15.107:0/3177680270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ed324ac050 0x55ed324c3080 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3177680270 >> 172.21.15.107:0/3177680270 conn(0x55ed324acad0 msgr2=0x55ed324b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe8fd600640 0 client.25858 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3177680270 shutdown_connections 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91d3e0740 1 -- 172.21.15.107:0/3177680270 wait complete. 2024-09-19T04:23:47.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91d3e0740 20 client.25858 trim_cache size 0 max 16384 2024-09-19T04:23:47.189 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:23:47.189 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:23:47.189 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "762763bc-1b39-4dec-afe1-7bfcb0bdd217", 2024-09-19T04:23:47.189 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:23:47.189 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91a800640 1 -- 172.21.15.107:0/2216293631 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8f80a33b0 msgr2=0x7fe8f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.183+0000 7fe91a800640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8f80a33b0 0x7fe8f80a5870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55ed324aceb0 tx=0x7fe904009700 comp rx=0 tx=0).stop 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.184+0000 7fe91a800640 1 -- 172.21.15.107:0/2216293631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe914074620 msgr2=0x7fe91419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.184+0000 7fe91a800640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe914074620 0x7fe91419a700 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe90400e9f0 tx=0x7fe90400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.184+0000 7fe91a800640 1 -- 172.21.15.107:0/2216293631 shutdown_connections 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.184+0000 7fe91a800640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe8f80a33b0 0x7fe8f80a5870 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.184+0000 7fe91a800640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe91410e3c0 0x7fe91419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.184+0000 7fe91a800640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe914074f40 0x7fe91419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.184+0000 7fe91a800640 1 --2- 172.21.15.107:0/2216293631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe914074620 0x7fe91419a700 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.184+0000 7fe91a800640 1 -- 172.21.15.107:0/2216293631 >> 172.21.15.107:0/2216293631 conn(0x7fe9140fd8c0 msgr2=0x7fe9140fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:47.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.184+0000 7fe91a800640 1 -- 172.21.15.107:0/2216293631 shutdown_connections 2024-09-19T04:23:47.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:47.184+0000 7fe91a800640 1 -- 172.21.15.107:0/2216293631 wait complete. 2024-09-19T04:23:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:46 smithi115 ceph-mon[35477]: pgmap v106: 97 pgs: 97 active+clean; 82 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 880 KiB/s wr, 144 op/s 2024-09-19T04:23:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:46 smithi145 ceph-mon[35096]: pgmap v106: 97 pgs: 97 active+clean; 82 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 880 KiB/s wr, 144 op/s 2024-09-19T04:23:47.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 129340 0.99 MB/sec execute 5 sec latency 2946.935 ms 2024-09-19T04:23:47.881 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:47 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:23:47.881 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:47 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:23:47.881 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:47 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:47.881 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:47 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:47.881 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:47 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:23:47.881 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:47 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.061 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:23:48.096 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:47 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:23:48.097 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:47 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:23:48.097 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:47 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.097 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:47 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.097 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:47 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:23:48.097 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:47 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:47 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:23:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:47 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:23:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:47 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:47 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:47 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:23:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:47 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.495 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:23:48.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 131524 3.09 MB/sec execute 6 sec latency 11.466 ms 2024-09-19T04:23:48.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:23:48.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:48.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:48 smithi107 ceph-mon[30659]: pgmap v107: 97 pgs: 97 active+clean; 82 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 614 KiB/s wr, 88 op/s 2024-09-19T04:23:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.046+0000 7f0640200640 1 -- 172.21.15.107:0/3824464380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638109120 msgr2=0x7f0638110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.046+0000 7f0640200640 1 --2- 172.21.15.107:0/3824464380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638109120 0x7f0638110ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0630009ca0 tx=0x7f063002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.046+0000 7f0640200640 1 -- 172.21.15.107:0/3824464380 shutdown_connections 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.046+0000 7f0640200640 1 --2- 172.21.15.107:0/3824464380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638109120 0x7f0638110ca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.046+0000 7f0640200640 1 --2- 172.21.15.107:0/3824464380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06381049a0 0x7f06381089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.046+0000 7f0640200640 1 --2- 172.21.15.107:0/3824464380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0638103ff0 0x7f06381043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.046+0000 7f0640200640 1 -- 172.21.15.107:0/3824464380 >> 172.21.15.107:0/3824464380 conn(0x7f06380ffa90 msgr2=0x7f0638101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.046+0000 7f0640200640 1 -- 172.21.15.107:0/3824464380 shutdown_connections 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.046+0000 7f0640200640 1 -- 172.21.15.107:0/3824464380 wait complete. 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.047+0000 7f0640200640 1 Processor -- start 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.047+0000 7f0640200640 1 -- start start 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.047+0000 7f0640200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638103ff0 0x7f063819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.047+0000 7f0640200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06381049a0 0x7f063819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.047+0000 7f0640200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0638109120 0x7f063819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.047+0000 7f0640200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0638114180 con 0x7f0638109120 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.047+0000 7f0640200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0638114000 con 0x7f0638103ff0 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.047+0000 7f0640200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0638114300 con 0x7f06381049a0 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063d800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638103ff0 0x7f063819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063d800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638103ff0 0x7f063819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:55958/0 (socket says 172.21.15.107:55958) 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06381049a0 0x7f063819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063d800640 1 -- 172.21.15.107:0/1062689765 learned_addr learned my addr 172.21.15.107:0/1062689765 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063e200640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0638109120 0x7f063819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063d800640 1 -- 172.21.15.107:0/1062689765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06381049a0 msgr2=0x7f063819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063d800640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06381049a0 0x7f063819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063d800640 1 -- 172.21.15.107:0/1062689765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0638109120 msgr2=0x7f063819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063d800640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0638109120 0x7f063819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063d800640 1 -- 172.21.15.107:0/1062689765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063819f540 con 0x7f0638103ff0 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.048+0000 7f063d800640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638103ff0 0x7f063819a590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f062c00d950 tx=0x7f062c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.049+0000 7f0636000640 1 -- 172.21.15.107:0/1062689765 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c017070 con 0x7f0638103ff0 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.049+0000 7f0636000640 1 -- 172.21.15.107:0/1062689765 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f062c009c30 con 0x7f0638103ff0 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.049+0000 7f0636000640 1 -- 172.21.15.107:0/1062689765 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c00f6b0 con 0x7f0638103ff0 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.049+0000 7f0640200640 1 -- 172.21.15.107:0/1062689765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063819f830 con 0x7f0638103ff0 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.049+0000 7f0640200640 1 -- 172.21.15.107:0/1062689765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06381a7110 con 0x7f0638103ff0 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.052+0000 7f0642dfc740 1 Processor -- start 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.052+0000 7f0636000640 1 -- 172.21.15.107:0/1062689765 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f062c0109d0 con 0x7f0638103ff0 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.052+0000 7f0642dfc740 1 -- start start 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.052+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.052+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 0x5580835f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.052+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.052+0000 7f0642dfc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5580835503e0 con 0x5580835e9b90 2024-09-19T04:23:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.052+0000 7f0642dfc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5580834f7d80 con 0x5580835f4d70 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.052+0000 7f0642dfc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55808354c3a0 con 0x5580835ea4b0 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063ce00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063e200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 0x5580835f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063ce00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:55972/0 (socket says 172.21.15.107:55972) 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063ce00640 1 -- 172.21.15.107:0/569599 learned_addr learned my addr 172.21.15.107:0/569599 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063ce00640 1 -- 172.21.15.107:0/569599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 msgr2=0x5580835f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063ce00640 1 --2- 172.21.15.107:0/569599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 0x5580835f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063ce00640 1 -- 172.21.15.107:0/569599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 msgr2=0x5580835e9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063ce00640 1 --2- 172.21.15.107:0/569599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835e9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063ce00640 1 -- 172.21.15.107:0/569599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558083516ae0 con 0x5580835f4d70 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063e200640 1 --2- 172.21.15.107:0/569599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835e9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f063ce00640 1 --2- 172.21.15.107:0/569599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f7160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f062400bc30 tx=0x7f06240107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8209556a5d6a3f91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.053+0000 7f0636000640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06200a3430 0x7f06200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.054+0000 7f062aa00640 1 -- 172.21.15.107:0/569599 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062401c480 con 0x5580835f4d70 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.054+0000 7f062aa00640 1 -- 172.21.15.107:0/569599 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f062401cb00 con 0x5580835f4d70 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.054+0000 7f062aa00640 1 -- 172.21.15.107:0/569599 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624011c00 con 0x5580835f4d70 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.054+0000 7f0636000640 1 -- 172.21.15.107:0/1062689765 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f062c0cf870 con 0x7f0638103ff0 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.054+0000 7f063e200640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06200a3430 0x7f06200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.055+0000 7f0642dfc740 1 -- 172.21.15.107:0/569599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 msgr2=0x5580835f7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.055+0000 7f0642dfc740 1 --2- 172.21.15.107:0/569599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f7160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f062400bc30 tx=0x7f06240107a0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.055+0000 7f0642dfc740 1 -- 172.21.15.107:0/569599 shutdown_connections 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.055+0000 7f0642dfc740 1 --2- 172.21.15.107:0/569599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f7160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.055+0000 7f0642dfc740 1 --2- 172.21.15.107:0/569599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 0x5580835f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.055+0000 7f0642dfc740 1 --2- 172.21.15.107:0/569599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835e9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.055+0000 7f0642dfc740 1 -- 172.21.15.107:0/569599 >> 172.21.15.107:0/569599 conn(0x5580835e6120 msgr2=0x5580835e6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.055+0000 7f063e200640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06200a3430 0x7f06200a58f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0630006010 tx=0x7f0630005d80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.055+0000 7f0642dfc740 1 -- 172.21.15.107:0/569599 shutdown_connections 2024-09-19T04:23:49.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.055+0000 7f0642dfc740 1 -- 172.21.15.107:0/569599 wait complete. 2024-09-19T04:23:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f0642dfc740 1 -- start start 2024-09-19T04:23:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f0642dfc740 1 Processor -- start 2024-09-19T04:23:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 0x5580835f2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f0642dfc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5580835f79e0 con 0x5580835e9b90 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f0642dfc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55808350b0f0 con 0x5580835f4d70 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f0642dfc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5580835f7b60 con 0x5580835ea4b0 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f0642dfc740 10 client.0 _ll_register_callbacks cb 0x5580834fc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f063e200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.056+0000 7f063ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 0x5580835f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 0x5580835f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:49990/0 (socket says 172.21.15.107:49990) 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063e200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:55980/0 (socket says 172.21.15.107:55980) 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063ce00640 1 -- 172.21.15.107:0/2653149629 learned_addr learned my addr 172.21.15.107:0/2653149629 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063d800640 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063ce00640 1 -- 172.21.15.107:0/2653149629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 msgr2=0x5580835f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063ce00640 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063ce00640 1 -- 172.21.15.107:0/2653149629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 msgr2=0x5580835f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063ce00640 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063ce00640 1 -- 172.21.15.107:0/2653149629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580832c13c0 con 0x5580835ea4b0 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063d800640 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f063ce00640 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 0x5580835f2180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f062400b830 tx=0x7f062400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e113fde84ab8d612 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.057+0000 7f061f400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f061f400640 1 -- 172.21.15.107:0/2653149629 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062402b060 con 0x5580835ea4b0 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f061f400640 1 -- 172.21.15.107:0/2653149629 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f062401c690 con 0x5580835ea4b0 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f0642dfc740 1 -- 172.21.15.107:0/2653149629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f062401cb00 con 0x5580835ea4b0 2024-09-19T04:23:49.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f061f400640 1 -- 172.21.15.107:0/2653149629 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f062400be60 con 0x5580835ea4b0 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f0642dfc740 10 client.25870 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f0642dfc740 1 -- 172.21.15.107:0/2653149629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5580836036a0 con 0x5580835ea4b0 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f061f400640 1 -- 172.21.15.107:0/2653149629 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f062401cb00 con 0x5580835ea4b0 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f061f400640 1 -- 172.21.15.107:0/2653149629 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0624005490 con 0x5580835ea4b0 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f0642dfc740 10 client.25870 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f0642dfc740 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f0642dfc740 20 client.25870 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f0642dfc740 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558083604b60 0x558083606f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f0642dfc740 4 client.25870 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.058+0000 7f0642dfc740 1 -- 172.21.15.107:0/2653149629 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55808355d950 con 0x558083604b60 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.059+0000 7f063d800640 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558083604b60 0x558083606f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.059+0000 7f063d800640 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558083604b60 0x558083606f50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea65d6f45e7302bf server_cookie=b3378461b6cd2f5a in_seq=0 out_seq=0 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.059+0000 7f061f400640 10 client.25870 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f061f400640 1 -- 172.21.15.107:0/2653149629 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55808355d950 con 0x558083604b60 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f061f400640 10 client.25870 handle_command_reply: tid=0 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 client.25870 shutdown 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 -- 172.21.15.107:0/2653149629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 msgr2=0x5580835f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 0x5580835f2180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f062400b830 tx=0x7f062400b860 comp rx=0 tx=0).stop 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 -- 172.21.15.107:0/2653149629 shutdown_connections 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558083604b60 0x558083606f50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x5580835f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835ea4b0 0x5580835f2180 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f061f400640 0 client.25870 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 --2- 172.21.15.107:0/2653149629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 -- 172.21.15.107:0/2653149629 >> 172.21.15.107:0/2653149629 conn(0x5580835e6120 msgr2=0x5580835ebe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 -- 172.21.15.107:0/2653149629 shutdown_connections 2024-09-19T04:23:49.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 1 -- 172.21.15.107:0/2653149629 wait complete. 2024-09-19T04:23:49.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.060+0000 7f0642dfc740 20 client.25870 trim_cache size 0 max 16384 2024-09-19T04:23:49.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.078+0000 7f0642dfc740 1 Processor -- start 2024-09-19T04:23:49.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f0642dfc740 1 -- start start 2024-09-19T04:23:49.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835e9b90 0x5580835f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835ea4b0 0x55808360c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x55808360caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f0642dfc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06381b3db0 con 0x5580835ea4b0 2024-09-19T04:23:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f0642dfc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f06381b59e0 con 0x5580835f4d70 2024-09-19T04:23:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f0642dfc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f06381b4480 con 0x5580835e9b90 2024-09-19T04:23:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f063d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835e9b90 0x5580835f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f063ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835ea4b0 0x55808360c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f063d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835e9b90 0x5580835f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:50000/0 (socket says 172.21.15.107:50000) 2024-09-19T04:23:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f063d800640 1 -- 172.21.15.107:0/2067899238 learned_addr learned my addr 172.21.15.107:0/2067899238 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.079+0000 7f063e200640 1 --2- 172.21.15.107:0/2067899238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x55808360caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.080+0000 7f063ce00640 1 -- 172.21.15.107:0/2067899238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835e9b90 msgr2=0x5580835f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.080+0000 7f063ce00640 1 --2- 172.21.15.107:0/2067899238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835e9b90 0x5580835f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.080+0000 7f063ce00640 1 -- 172.21.15.107:0/2067899238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 msgr2=0x55808360caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.080+0000 7f063ce00640 1 --2- 172.21.15.107:0/2067899238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x55808360caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.080+0000 7f063ce00640 1 -- 172.21.15.107:0/2067899238 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580835f0190 con 0x5580835ea4b0 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.080+0000 7f063d800640 1 --2- 172.21.15.107:0/2067899238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835e9b90 0x5580835f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.080+0000 7f063ce00640 1 --2- 172.21.15.107:0/2067899238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835ea4b0 0x55808360c5b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0624011730 tx=0x7f0624004eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d02f915c853564e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.080+0000 7f062a000640 1 -- 172.21.15.107:0/2067899238 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062401c690 con 0x5580835ea4b0 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.080+0000 7f062a000640 1 -- 172.21.15.107:0/2067899238 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0624011cb0 con 0x5580835ea4b0 2024-09-19T04:23:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.080+0000 7f062a000640 1 -- 172.21.15.107:0/2067899238 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624005040 con 0x5580835ea4b0 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.081+0000 7f0642dfc740 1 -- 172.21.15.107:0/2067899238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835ea4b0 msgr2=0x55808360c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.081+0000 7f0642dfc740 1 --2- 172.21.15.107:0/2067899238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835ea4b0 0x55808360c5b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0624011730 tx=0x7f0624004eb0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.081+0000 7f0642dfc740 1 -- 172.21.15.107:0/2067899238 shutdown_connections 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.081+0000 7f0642dfc740 1 --2- 172.21.15.107:0/2067899238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835f4d70 0x55808360caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.081+0000 7f0642dfc740 1 --2- 172.21.15.107:0/2067899238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835ea4b0 0x55808360c5b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.081+0000 7f0642dfc740 1 --2- 172.21.15.107:0/2067899238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835e9b90 0x5580835f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.081+0000 7f0642dfc740 1 -- 172.21.15.107:0/2067899238 >> 172.21.15.107:0/2067899238 conn(0x5580835e6120 msgr2=0x5580835f34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.081+0000 7f0642dfc740 1 -- 172.21.15.107:0/2067899238 shutdown_connections 2024-09-19T04:23:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.081+0000 7f0642dfc740 1 -- 172.21.15.107:0/2067899238 wait complete. 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f0642dfc740 1 -- start start 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f0642dfc740 1 Processor -- start 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835ea4b0 0x5580835f8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f0642dfc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835f4d70 0x5580835f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f0642dfc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06381b5250 con 0x5580835e9b90 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f0642dfc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f06381b4b50 con 0x5580835ea4b0 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f0642dfc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f06381b1dd0 con 0x5580835f4d70 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f0642dfc740 10 client.0 _ll_register_callbacks cb 0x7f062400ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f063e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835f4d70 0x5580835f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f063d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.082+0000 7f063e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835f4d70 0x5580835f8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:50008/0 (socket says 172.21.15.107:50008) 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f063e200640 1 -- 172.21.15.107:0/3091947984 learned_addr learned my addr 172.21.15.107:0/3091947984 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f063ce00640 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835ea4b0 0x5580835f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f063e200640 1 -- 172.21.15.107:0/3091947984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835ea4b0 msgr2=0x5580835f8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f063e200640 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835ea4b0 0x5580835f8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f063e200640 1 -- 172.21.15.107:0/3091947984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 msgr2=0x5580835fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f063e200640 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835fd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f063e200640 1 -- 172.21.15.107:0/3091947984 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580835f94f0 con 0x5580835f4d70 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f063ce00640 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835ea4b0 0x5580835f8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f063d800640 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f063e200640 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835f4d70 0x5580835f8cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f063002aeb0 tx=0x7f063002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4a8eb1d692779a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:23:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.083+0000 7f061f400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:23:49.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f061f400640 1 -- 172.21.15.107:0/3091947984 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630031660 con 0x5580835f4d70 2024-09-19T04:23:49.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f0642dfc740 1 -- 172.21.15.107:0/3091947984 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0624011cb0 con 0x5580835f4d70 2024-09-19T04:23:49.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f061f400640 1 -- 172.21.15.107:0/3091947984 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0630042030 con 0x5580835f4d70 2024-09-19T04:23:49.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f061f400640 1 -- 172.21.15.107:0/3091947984 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0630031800 con 0x5580835f4d70 2024-09-19T04:23:49.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f0642dfc740 10 client.25876 fetch_fsmap learned FSMap version 31 2024-09-19T04:23:49.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f0642dfc740 1 -- 172.21.15.107:0/3091947984 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55808366f4c0 con 0x5580835f4d70 2024-09-19T04:23:49.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f061f400640 1 -- 172.21.15.107:0/3091947984 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0624011cb0 con 0x5580835f4d70 2024-09-19T04:23:49.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f061f400640 1 -- 172.21.15.107:0/3091947984 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f063002f850 con 0x5580835f4d70 2024-09-19T04:23:49.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f0642dfc740 10 client.25876 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f0642dfc740 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.084+0000 7f0642dfc740 20 client.25876 populate_metadata read hostname 'smithi107' 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.085+0000 7f0642dfc740 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55808366fcd0 0x5580836720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.085+0000 7f0642dfc740 4 client.25876 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.085+0000 7f0642dfc740 1 -- 172.21.15.107:0/3091947984 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558083557e60 con 0x55808366fcd0 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.085+0000 7f063d800640 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55808366fcd0 0x5580836720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.085+0000 7f063d800640 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55808366fcd0 0x5580836720f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a6e9bdebdd8bd9 server_cookie=e358911e09e00802 in_seq=0 out_seq=0 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.085+0000 7f061f400640 10 client.25876 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f061f400640 1 -- 172.21.15.107:0/3091947984 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558083557e60 con 0x55808366fcd0 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f061f400640 10 client.25876 handle_command_reply: tid=0 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 client.25876 shutdown 2024-09-19T04:23:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 -- 172.21.15.107:0/3091947984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835f4d70 msgr2=0x5580835f8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835f4d70 0x5580835f8cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f063002aeb0 tx=0x7f063002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 -- 172.21.15.107:0/3091947984 shutdown_connections 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55808366fcd0 0x5580836720f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580835f4d70 0x5580835f8cc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580835ea4b0 0x5580835f8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 --2- 172.21.15.107:0/3091947984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580835e9b90 0x5580835fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 -- 172.21.15.107:0/3091947984 >> 172.21.15.107:0/3091947984 conn(0x5580835e6120 msgr2=0x5580835ebe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f061f400640 0 client.25876 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 -- 172.21.15.107:0/3091947984 shutdown_connections 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.086+0000 7f0642dfc740 1 -- 172.21.15.107:0/3091947984 wait complete. 2024-09-19T04:23:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0642dfc740 20 client.25876 trim_cache size 0 max 16384 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "762763bc-1b39-4dec-afe1-7bfcb0bdd217": { 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "762763bc-1b39-4dec-afe1-7bfcb0bdd217", 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:23:49.093 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0640200640 1 -- 172.21.15.107:0/1062689765 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06200a3430 msgr2=0x7f06200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0640200640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06200a3430 0x7f06200a58f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0630006010 tx=0x7f0630005d80 comp rx=0 tx=0).stop 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0640200640 1 -- 172.21.15.107:0/1062689765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638103ff0 msgr2=0x7f063819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:23:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0640200640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638103ff0 0x7f063819a590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f062c00d950 tx=0x7f062c00de10 comp rx=0 tx=0).stop 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0640200640 1 -- 172.21.15.107:0/1062689765 shutdown_connections 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0640200640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06200a3430 0x7f06200a58f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0640200640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0638109120 0x7f063819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0640200640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06381049a0 0x7f063819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0640200640 1 --2- 172.21.15.107:0/1062689765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638103ff0 0x7f063819a590 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.087+0000 7f0640200640 1 -- 172.21.15.107:0/1062689765 >> 172.21.15.107:0/1062689765 conn(0x7f06380ffa90 msgr2=0x7f0638101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.088+0000 7f0640200640 1 -- 172.21.15.107:0/1062689765 shutdown_connections 2024-09-19T04:23:49.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:23:49.088+0000 7f0640200640 1 -- 172.21.15.107:0/1062689765 wait complete. 2024-09-19T04:23:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:48 smithi115 ceph-mon[35477]: pgmap v107: 97 pgs: 97 active+clean; 82 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 614 KiB/s wr, 88 op/s 2024-09-19T04:23:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:23:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:23:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:23:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:23:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:23:49.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:23:49.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:23:49.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:48 smithi145 ceph-mon[35096]: pgmap v107: 97 pgs: 97 active+clean; 82 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 614 KiB/s wr, 88 op/s 2024-09-19T04:23:49.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 132728 3.42 MB/sec execute 7 sec latency 86.575 ms 2024-09-19T04:23:49.966 INFO:tasks.cephfs.filesystem:scrub status for tag:762763bc-1b39-4dec-afe1-7bfcb0bdd217 - {'path': '/', 'tag': '762763bc-1b39-4dec-afe1-7bfcb0bdd217', 'options': 'recursive,force'} 2024-09-19T04:23:50.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 135074 4.71 MB/sec execute 8 sec latency 20.079 ms 2024-09-19T04:23:50.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:50 smithi107 ceph-mon[30659]: pgmap v108: 97 pgs: 97 active+clean; 82 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 614 KiB/s wr, 88 op/s 2024-09-19T04:23:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:50 smithi115 ceph-mon[35477]: pgmap v108: 97 pgs: 97 active+clean; 82 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 614 KiB/s wr, 88 op/s 2024-09-19T04:23:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:50 smithi145 ceph-mon[35096]: pgmap v108: 97 pgs: 97 active+clean; 82 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 614 KiB/s wr, 88 op/s 2024-09-19T04:23:51.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 135487 4.24 MB/sec execute 9 sec latency 420.905 ms 2024-09-19T04:23:52.642 INFO:tasks.workunit.client.0.smithi107.stdout: 1 135487 3.81 MB/sec execute 10 sec latency 1421.008 ms 2024-09-19T04:23:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:52 smithi115 ceph-mon[35477]: pgmap v109: 97 pgs: 97 active+clean; 85 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-19T04:23:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:52 smithi145 ceph-mon[35096]: pgmap v109: 97 pgs: 97 active+clean; 85 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-19T04:23:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:52 smithi107 ceph-mon[30659]: pgmap v109: 97 pgs: 97 active+clean; 85 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-19T04:23:53.643 INFO:tasks.workunit.client.0.smithi107.stdout: 1 135487 3.47 MB/sec execute 11 sec latency 2421.133 ms 2024-09-19T04:23:54.643 INFO:tasks.workunit.client.0.smithi107.stdout: 1 136349 3.60 MB/sec execute 12 sec latency 2990.752 ms 2024-09-19T04:23:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:54 smithi115 ceph-mon[35477]: pgmap v110: 97 pgs: 97 active+clean; 85 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-09-19T04:23:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:54 smithi145 ceph-mon[35096]: pgmap v110: 97 pgs: 97 active+clean; 85 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-09-19T04:23:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:54 smithi107 ceph-mon[30659]: pgmap v110: 97 pgs: 97 active+clean; 85 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-09-19T04:23:55.643 INFO:tasks.workunit.client.0.smithi107.stdout: 1 138693 4.37 MB/sec execute 13 sec latency 9.645 ms 2024-09-19T04:23:56.643 INFO:tasks.workunit.client.0.smithi107.stdout: 1 139034 4.09 MB/sec execute 14 sec latency 387.747 ms 2024-09-19T04:23:57.160 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:56 smithi107 ceph-mon[30659]: pgmap v111: 97 pgs: 97 active+clean; 87 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-09-19T04:23:57.160 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:56 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:23:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:56 smithi115 ceph-mon[35477]: pgmap v111: 97 pgs: 97 active+clean; 87 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-09-19T04:23:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:56 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:23:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:56 smithi145 ceph-mon[35096]: pgmap v111: 97 pgs: 97 active+clean; 87 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-09-19T04:23:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:56 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:23:57.643 INFO:tasks.workunit.client.0.smithi107.stdout: 1 139034 3.82 MB/sec execute 15 sec latency 1387.835 ms 2024-09-19T04:23:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:57 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:23:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:57 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:23:58.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:57 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:23:58.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:57 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:23:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:57 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:23:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:57 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:23:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:57 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:23:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:57 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:23:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:57 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:23:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:57 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:23:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:57 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:23:58.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:57 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:23:58.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 139034 3.58 MB/sec execute 16 sec latency 2387.997 ms 2024-09-19T04:23:58.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:23:58 smithi107 ceph-mon[30659]: pgmap v112: 97 pgs: 97 active+clean; 87 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-19T04:23:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:23:58 smithi115 ceph-mon[35477]: pgmap v112: 97 pgs: 97 active+clean; 87 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-19T04:23:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:23:58 smithi145 ceph-mon[35096]: pgmap v112: 97 pgs: 97 active+clean; 87 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-19T04:23:59.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 139932 3.66 MB/sec execute 17 sec latency 2959.915 ms 2024-09-19T04:24:00.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 142210 4.22 MB/sec execute 18 sec latency 9.841 ms 2024-09-19T04:24:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:00 smithi115 ceph-mon[35477]: pgmap v113: 97 pgs: 97 active+clean; 87 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-19T04:24:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:00 smithi145 ceph-mon[35096]: pgmap v113: 97 pgs: 97 active+clean; 87 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-19T04:24:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:00 smithi107 ceph-mon[30659]: pgmap v113: 97 pgs: 97 active+clean; 87 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-19T04:24:01.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 142577 4.02 MB/sec execute 19 sec latency 522.301 ms 2024-09-19T04:24:02.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 142577 3.82 MB/sec execute 20 sec latency 1045.965 ms 2024-09-19T04:24:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:02 smithi115 ceph-mon[35477]: pgmap v114: 97 pgs: 97 active+clean; 89 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-09-19T04:24:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:02 smithi145 ceph-mon[35096]: pgmap v114: 97 pgs: 97 active+clean; 89 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-09-19T04:24:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:02 smithi107 ceph-mon[30659]: pgmap v114: 97 pgs: 97 active+clean; 89 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-09-19T04:24:03.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 142577 3.64 MB/sec execute 21 sec latency 2046.086 ms 2024-09-19T04:24:04.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 142942 3.54 MB/sec execute 22 sec latency 2616.599 ms 2024-09-19T04:24:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:04 smithi115 ceph-mon[35477]: pgmap v115: 97 pgs: 97 active+clean; 89 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 701 KiB/s wr, 105 op/s 2024-09-19T04:24:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:04 smithi145 ceph-mon[35096]: pgmap v115: 97 pgs: 97 active+clean; 89 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 701 KiB/s wr, 105 op/s 2024-09-19T04:24:05.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:04 smithi107 ceph-mon[30659]: pgmap v115: 97 pgs: 97 active+clean; 89 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 701 KiB/s wr, 105 op/s 2024-09-19T04:24:05.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 145285 3.95 MB/sec execute 23 sec latency 22.872 ms 2024-09-19T04:24:06.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 145782 3.96 MB/sec execute 24 sec latency 588.251 ms 2024-09-19T04:24:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:06 smithi115 ceph-mon[35477]: pgmap v116: 97 pgs: 97 active+clean; 91 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-09-19T04:24:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:06 smithi145 ceph-mon[35096]: pgmap v116: 97 pgs: 97 active+clean; 91 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-09-19T04:24:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:06 smithi107 ceph-mon[30659]: pgmap v116: 97 pgs: 97 active+clean; 91 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-09-19T04:24:07.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 146126 3.82 MB/sec execute 25 sec latency 656.730 ms 2024-09-19T04:24:08.644 INFO:tasks.workunit.client.0.smithi107.stdout: 1 146126 3.67 MB/sec execute 26 sec latency 1537.543 ms 2024-09-19T04:24:08.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:08 smithi107 ceph-mon[30659]: pgmap v117: 97 pgs: 97 active+clean; 91 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-09-19T04:24:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:08 smithi115 ceph-mon[35477]: pgmap v117: 97 pgs: 97 active+clean; 91 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-09-19T04:24:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:08 smithi145 ceph-mon[35096]: pgmap v117: 97 pgs: 97 active+clean; 91 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-09-19T04:24:09.645 INFO:tasks.workunit.client.0.smithi107.stdout: 1 146250 3.55 MB/sec execute 27 sec latency 2107.131 ms 2024-09-19T04:24:10.645 INFO:tasks.workunit.client.0.smithi107.stdout: 1 146250 3.42 MB/sec execute 28 sec latency 1341.289 ms 2024-09-19T04:24:11.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:10 smithi107 ceph-mon[30659]: pgmap v118: 97 pgs: 97 active+clean; 91 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-09-19T04:24:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:10 smithi115 ceph-mon[35477]: pgmap v118: 97 pgs: 97 active+clean; 91 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-09-19T04:24:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:10 smithi145 ceph-mon[35096]: pgmap v118: 97 pgs: 97 active+clean; 91 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-09-19T04:24:11.648 INFO:tasks.workunit.client.0.smithi107.stdout: 1 146250 3.30 MB/sec execute 29 sec latency 2344.289 ms 2024-09-19T04:24:12.648 INFO:tasks.workunit.client.0.smithi107.stdout: 1 146250 3.19 MB/sec execute 30 sec latency 3344.372 ms 2024-09-19T04:24:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:12 smithi115 ceph-mon[35477]: pgmap v119: 97 pgs: 97 active+clean; 92 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-19T04:24:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:12 smithi145 ceph-mon[35096]: pgmap v119: 97 pgs: 97 active+clean; 92 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-19T04:24:13.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:12 smithi107 ceph-mon[30659]: pgmap v119: 97 pgs: 97 active+clean; 92 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-19T04:24:13.648 INFO:tasks.workunit.client.0.smithi107.stdout: 1 146250 3.09 MB/sec execute 31 sec latency 4344.494 ms 2024-09-19T04:24:14.648 INFO:tasks.workunit.client.0.smithi107.stdout: 1 147271 3.15 MB/sec execute 32 sec latency 4915.260 ms 2024-09-19T04:24:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:14 smithi115 ceph-mon[35477]: pgmap v120: 97 pgs: 97 active+clean; 92 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-09-19T04:24:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:14 smithi145 ceph-mon[35096]: pgmap v120: 97 pgs: 97 active+clean; 92 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-09-19T04:24:15.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:14 smithi107 ceph-mon[30659]: pgmap v120: 97 pgs: 97 active+clean; 92 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-09-19T04:24:15.648 INFO:tasks.workunit.client.0.smithi107.stdout: 1 149360 3.47 MB/sec execute 33 sec latency 11.423 ms 2024-09-19T04:24:16.648 INFO:tasks.workunit.client.0.smithi107.stdout: 1 149667 3.37 MB/sec execute 34 sec latency 644.152 ms 2024-09-19T04:24:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:16 smithi115 ceph-mon[35477]: pgmap v121: 97 pgs: 97 active+clean; 95 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-19T04:24:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:16 smithi145 ceph-mon[35096]: pgmap v121: 97 pgs: 97 active+clean; 95 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-19T04:24:17.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:16 smithi107 ceph-mon[30659]: pgmap v121: 97 pgs: 97 active+clean; 95 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-19T04:24:17.648 INFO:tasks.workunit.client.0.smithi107.stdout: 1 149667 3.27 MB/sec execute 35 sec latency 1644.288 ms 2024-09-19T04:24:18.648 INFO:tasks.workunit.client.0.smithi107.stdout: 1 149667 3.18 MB/sec execute 36 sec latency 2644.419 ms 2024-09-19T04:24:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:18 smithi115 ceph-mon[35477]: pgmap v122: 97 pgs: 97 active+clean; 95 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.0 MiB/s wr, 88 op/s 2024-09-19T04:24:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:18 smithi145 ceph-mon[35096]: pgmap v122: 97 pgs: 97 active+clean; 95 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.0 MiB/s wr, 88 op/s 2024-09-19T04:24:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:18 smithi107 ceph-mon[30659]: pgmap v122: 97 pgs: 97 active+clean; 95 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.0 MiB/s wr, 88 op/s 2024-09-19T04:24:19.648 INFO:tasks.workunit.client.0.smithi107.stdout: 1 149791 3.10 MB/sec execute 37 sec latency 3211.593 ms 2024-09-19T04:24:19.967 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:24:20.353 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:20.649 INFO:tasks.workunit.client.0.smithi107.stdout: 1 149791 3.02 MB/sec execute 38 sec latency 1331.391 ms 2024-09-19T04:24:20.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.902+0000 7f5983800640 1 -- 172.21.15.107:0/1639921343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c106f10 msgr2=0x7f597c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.902+0000 7f5983800640 1 --2- 172.21.15.107:0/1639921343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c106f10 0x7f597c10ea90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5974009ca0 tx=0x7f597402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.903+0000 7f5983800640 1 -- 172.21.15.107:0/1639921343 shutdown_connections 2024-09-19T04:24:20.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.903+0000 7f5983800640 1 --2- 172.21.15.107:0/1639921343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c106f10 0x7f597c10ea90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.903+0000 7f5983800640 1 --2- 172.21.15.107:0/1639921343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f597c102790 0x7f597c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.903+0000 7f5983800640 1 --2- 172.21.15.107:0/1639921343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f597c101de0 0x7f597c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.903+0000 7f5983800640 1 -- 172.21.15.107:0/1639921343 >> 172.21.15.107:0/1639921343 conn(0x7f597c0fd8c0 msgr2=0x7f597c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.903+0000 7f5983800640 1 -- 172.21.15.107:0/1639921343 shutdown_connections 2024-09-19T04:24:20.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.903+0000 7f5983800640 1 -- 172.21.15.107:0/1639921343 wait complete. 2024-09-19T04:24:20.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5983800640 1 Processor -- start 2024-09-19T04:24:20.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5983800640 1 -- start start 2024-09-19T04:24:20.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5983800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f597c101de0 0x7f597c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5983800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f597c102790 0x7f597c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5983800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c106f10 0x7f597c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5983800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f597c111fc0 con 0x7f597c106f10 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5983800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f597c111e40 con 0x7f597c101de0 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5983800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f597c112140 con 0x7f597c102790 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5981800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c106f10 0x7f597c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5973e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f597c102790 0x7f597c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.904+0000 7f5973e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f597c102790 0x7f597c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:56624/0 (socket says 172.21.15.107:56624) 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f5973e00640 1 -- 172.21.15.107:0/3045017505 learned_addr learned my addr 172.21.15.107:0/3045017505 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f5980e00640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f597c101de0 0x7f597c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f5973e00640 1 -- 172.21.15.107:0/3045017505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f597c101de0 msgr2=0x7f597c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f5973e00640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f597c101de0 0x7f597c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f5973e00640 1 -- 172.21.15.107:0/3045017505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c106f10 msgr2=0x7f597c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f5973e00640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c106f10 0x7f597c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f5973e00640 1 -- 172.21.15.107:0/3045017505 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c19f5f0 con 0x7f597c102790 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f5973e00640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f597c102790 0x7f597c19ab80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f596400e9f0 tx=0x7f596400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f597a000640 1 -- 172.21.15.107:0/3045017505 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596400cd80 con 0x7f597c102790 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f597a000640 1 -- 172.21.15.107:0/3045017505 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5964010910 con 0x7f597c102790 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.905+0000 7f5983800640 1 -- 172.21.15.107:0/3045017505 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c19f8e0 con 0x7f597c102790 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.906+0000 7f597a000640 1 -- 172.21.15.107:0/3045017505 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964018990 con 0x7f597c102790 2024-09-19T04:24:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.906+0000 7f5983800640 1 -- 172.21.15.107:0/3045017505 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c1a71c0 con 0x7f597c102790 2024-09-19T04:24:20.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.909+0000 7f59861f5740 1 Processor -- start 2024-09-19T04:24:20.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.909+0000 7f597a000640 1 -- 172.21.15.107:0/3045017505 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f5964010ac0 con 0x7f597c102790 2024-09-19T04:24:20.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.909+0000 7f59861f5740 1 -- start start 2024-09-19T04:24:20.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.909+0000 7f59861f5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 0x55ac94ba5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.909+0000 7f59861f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.909+0000 7f59861f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 0x55ac94bb3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.909+0000 7f59861f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ac94b0c3e0 con 0x55ac94ba64b0 2024-09-19T04:24:20.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.909+0000 7f59861f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ac94ab3d80 con 0x55ac94bb0d70 2024-09-19T04:24:20.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.909+0000 7f59861f5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ac94b083a0 con 0x55ac94ba5b90 2024-09-19T04:24:20.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5980e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5973e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 0x55ac94bb3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5981800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 0x55ac94ba5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5981800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 0x55ac94ba5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:56638/0 (socket says 172.21.15.107:56638) 2024-09-19T04:24:20.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5981800640 1 -- 172.21.15.107:0/2919722527 learned_addr learned my addr 172.21.15.107:0/2919722527 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5973e00640 1 -- 172.21.15.107:0/2919722527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 msgr2=0x55ac94ba5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5973e00640 1 --2- 172.21.15.107:0/2919722527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 0x55ac94ba5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5973e00640 1 -- 172.21.15.107:0/2919722527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 msgr2=0x55ac94bb0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5973e00640 1 --2- 172.21.15.107:0/2919722527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5973e00640 1 -- 172.21.15.107:0/2919722527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac94ad2ae0 con 0x55ac94bb0d70 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5981800640 1 --2- 172.21.15.107:0/2919722527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 0x55ac94ba5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5980e00640 1 --2- 172.21.15.107:0/2919722527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f597a000640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59600a33b0 0x7f59600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5973e00640 1 --2- 172.21.15.107:0/2919722527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 0x55ac94bb3160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f59640cd700 tx=0x7f59640d86f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed9fcbcd7581685 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f597a000640 1 -- 172.21.15.107:0/3045017505 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f59640cff60 con 0x7f597c102790 2024-09-19T04:24:20.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.910+0000 7f5981800640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59600a33b0 0x7f59600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.911+0000 7f5972000640 1 -- 172.21.15.107:0/2919722527 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59640985e0 con 0x55ac94bb0d70 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.911+0000 7f5972000640 1 -- 172.21.15.107:0/2919722527 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59640a0640 con 0x55ac94bb0d70 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.911+0000 7f5972000640 1 -- 172.21.15.107:0/2919722527 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596409bca0 con 0x55ac94bb0d70 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.911+0000 7f59861f5740 1 -- 172.21.15.107:0/2919722527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 msgr2=0x55ac94bb3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.911+0000 7f59861f5740 1 --2- 172.21.15.107:0/2919722527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 0x55ac94bb3160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f59640cd700 tx=0x7f59640d86f0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.911+0000 7f5981800640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59600a33b0 0x7f59600a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f597402aeb0 tx=0x7f5974005d80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.911+0000 7f59861f5740 1 -- 172.21.15.107:0/2919722527 shutdown_connections 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.911+0000 7f59861f5740 1 --2- 172.21.15.107:0/2919722527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 0x55ac94bb3160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.911+0000 7f59861f5740 1 --2- 172.21.15.107:0/2919722527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.911+0000 7f59861f5740 1 --2- 172.21.15.107:0/2919722527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 0x55ac94ba5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.912+0000 7f59861f5740 1 -- 172.21.15.107:0/2919722527 >> 172.21.15.107:0/2919722527 conn(0x55ac94ba2120 msgr2=0x55ac94ba2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.912+0000 7f59861f5740 1 -- 172.21.15.107:0/2919722527 shutdown_connections 2024-09-19T04:24:20.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.912+0000 7f59861f5740 1 -- 172.21.15.107:0/2919722527 wait complete. 2024-09-19T04:24:20.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.912+0000 7f59861f5740 1 -- start start 2024-09-19T04:24:20.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.912+0000 7f59861f5740 1 Processor -- start 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f59861f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba5b90 0x55ac94badcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f59861f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba64b0 0x55ac94bae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f59861f5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94bb0d70 0x55ac94bae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f59861f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ac94bb39e0 con 0x55ac94ba5b90 2024-09-19T04:24:20.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f59861f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ac94ac70f0 con 0x55ac94ba64b0 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f59861f5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ac94bb3b60 con 0x55ac94bb0d70 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f59861f5740 10 client.0 _ll_register_callbacks cb 0x55ac94ab89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f5973e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba64b0 0x55ac94bae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f5973e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba64b0 0x55ac94bae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36994/0 (socket says 172.21.15.107:36994) 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f5973e00640 1 -- 172.21.15.107:0/3006158790 learned_addr learned my addr 172.21.15.107:0/3006158790 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f5980e00640 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba5b90 0x55ac94badcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f5981800640 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94bb0d70 0x55ac94bae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f5973e00640 1 -- 172.21.15.107:0/3006158790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94bb0d70 msgr2=0x55ac94bae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f5973e00640 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94bb0d70 0x55ac94bae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f5973e00640 1 -- 172.21.15.107:0/3006158790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba5b90 msgr2=0x55ac94badcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f5973e00640 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba5b90 0x55ac94badcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.913+0000 7f5973e00640 1 -- 172.21.15.107:0/3006158790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9487d3c0 con 0x55ac94ba64b0 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.914+0000 7f5973e00640 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba64b0 0x55ac94bae200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f596409c040 tx=0x7f5964000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec3f677cbc6608d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.914+0000 7f596aa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.914+0000 7f596aa00640 1 -- 172.21.15.107:0/3006158790 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596409be80 con 0x55ac94ba64b0 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.914+0000 7f596aa00640 1 -- 172.21.15.107:0/3006158790 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f59640bc110 con 0x55ac94ba64b0 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.914+0000 7f59861f5740 1 -- 172.21.15.107:0/3006158790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59640a0640 con 0x55ac94ba64b0 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.914+0000 7f596aa00640 1 -- 172.21.15.107:0/3006158790 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f59640b4020 con 0x55ac94ba64b0 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.914+0000 7f59861f5740 10 client.25919 fetch_fsmap learned FSMap version 31 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.914+0000 7f59861f5740 1 -- 172.21.15.107:0/3006158790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ac94bb39e0 con 0x55ac94ba64b0 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.914+0000 7f596aa00640 1 -- 172.21.15.107:0/3006158790 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59640a0640 con 0x55ac94ba64b0 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.915+0000 7f596aa00640 1 -- 172.21.15.107:0/3006158790 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f596409c5a0 con 0x55ac94ba64b0 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.915+0000 7f59861f5740 10 client.25919 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:24:20.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.915+0000 7f59861f5740 10 client.25919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:20.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.915+0000 7f59861f5740 20 client.25919 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:20.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.915+0000 7f59861f5740 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ac94bc0870 0x55ac94bc2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.915+0000 7f59861f5740 4 client.25919 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:20.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.915+0000 7f59861f5740 1 -- 172.21.15.107:0/3006158790 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac94b19950 con 0x55ac94bc0870 2024-09-19T04:24:20.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.915+0000 7f5980e00640 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ac94bc0870 0x55ac94bc2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.916+0000 7f5980e00640 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ac94bc0870 0x55ac94bc2c60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa3cec8782bc828 server_cookie=33c00991ba693296 in_seq=0 out_seq=0 2024-09-19T04:24:20.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.916+0000 7f596aa00640 10 client.25919 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:20.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.916+0000 7f596aa00640 1 -- 172.21.15.107:0/3006158790 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac94b19950 con 0x55ac94bc0870 2024-09-19T04:24:20.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.916+0000 7f596aa00640 10 client.25919 handle_command_reply: tid=0 2024-09-19T04:24:20.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 client.25919 shutdown 2024-09-19T04:24:20.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 -- 172.21.15.107:0/3006158790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba64b0 msgr2=0x55ac94bae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba64b0 0x55ac94bae200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f596409c040 tx=0x7f5964000bc0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 -- 172.21.15.107:0/3006158790 shutdown_connections 2024-09-19T04:24:20.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ac94bc0870 0x55ac94bc2c60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94bb0d70 0x55ac94bae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba64b0 0x55ac94bae200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 --2- 172.21.15.107:0/3006158790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba5b90 0x55ac94badcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f596aa00640 0 client.25919 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 -- 172.21.15.107:0/3006158790 >> 172.21.15.107:0/3006158790 conn(0x55ac94ba2120 msgr2=0x55ac94ba7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 -- 172.21.15.107:0/3006158790 shutdown_connections 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 1 -- 172.21.15.107:0/3006158790 wait complete. 2024-09-19T04:24:20.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.917+0000 7f59861f5740 20 client.25919 trim_cache size 0 max 16384 2024-09-19T04:24:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f59861f5740 1 Processor -- start 2024-09-19T04:24:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f59861f5740 1 -- start start 2024-09-19T04:24:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f59861f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba5b90 0x55ac94bafc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f59861f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bc8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f59861f5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94bb0d70 0x55ac94bc8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f59861f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1af8b0 con 0x55ac94ba64b0 2024-09-19T04:24:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f59861f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1b14e0 con 0x55ac94ba5b90 2024-09-19T04:24:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f59861f5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1aff80 con 0x55ac94bb0d70 2024-09-19T04:24:20.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f5980e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba5b90 0x55ac94bafc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f5980e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba5b90 0x55ac94bafc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37002/0 (socket says 172.21.15.107:37002) 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f5980e00640 1 -- 172.21.15.107:0/3288501863 learned_addr learned my addr 172.21.15.107:0/3288501863 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.935+0000 7f5973e00640 1 --2- 172.21.15.107:0/3288501863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bc8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.936+0000 7f5981800640 1 --2- 172.21.15.107:0/3288501863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94bb0d70 0x55ac94bc8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.936+0000 7f5980e00640 1 -- 172.21.15.107:0/3288501863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94bb0d70 msgr2=0x55ac94bc8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.936+0000 7f5980e00640 1 --2- 172.21.15.107:0/3288501863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94bb0d70 0x55ac94bc8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.936+0000 7f5980e00640 1 -- 172.21.15.107:0/3288501863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 msgr2=0x55ac94bc8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.936+0000 7f5980e00640 1 --2- 172.21.15.107:0/3288501863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bc8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.936+0000 7f5980e00640 1 -- 172.21.15.107:0/3288501863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac94bac210 con 0x55ac94ba5b90 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.936+0000 7f5980e00640 1 --2- 172.21.15.107:0/3288501863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba5b90 0x55ac94bafc30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f596c005d00 tx=0x7f596c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9ddfcddb2932bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.936+0000 7f5971600640 1 -- 172.21.15.107:0/3288501863 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c004250 con 0x55ac94ba5b90 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.936+0000 7f5971600640 1 -- 172.21.15.107:0/3288501863 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f596c0043f0 con 0x55ac94ba5b90 2024-09-19T04:24:20.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.936+0000 7f5971600640 1 -- 172.21.15.107:0/3288501863 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c0035f0 con 0x55ac94ba5b90 2024-09-19T04:24:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.937+0000 7f59861f5740 1 -- 172.21.15.107:0/3288501863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba5b90 msgr2=0x55ac94bafc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.937+0000 7f59861f5740 1 --2- 172.21.15.107:0/3288501863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba5b90 0x55ac94bafc30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f596c005d00 tx=0x7f596c005f90 comp rx=0 tx=0).stop 2024-09-19T04:24:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.937+0000 7f59861f5740 1 -- 172.21.15.107:0/3288501863 shutdown_connections 2024-09-19T04:24:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.937+0000 7f59861f5740 1 --2- 172.21.15.107:0/3288501863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94bb0d70 0x55ac94bc8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.937+0000 7f59861f5740 1 --2- 172.21.15.107:0/3288501863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bc8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.937+0000 7f59861f5740 1 --2- 172.21.15.107:0/3288501863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94ba5b90 0x55ac94bafc30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.937+0000 7f59861f5740 1 -- 172.21.15.107:0/3288501863 >> 172.21.15.107:0/3288501863 conn(0x55ac94ba2120 msgr2=0x55ac94baf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.937+0000 7f59861f5740 1 -- 172.21.15.107:0/3288501863 shutdown_connections 2024-09-19T04:24:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.937+0000 7f59861f5740 1 -- 172.21.15.107:0/3288501863 wait complete. 2024-09-19T04:24:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.937+0000 7f59861f5740 1 -- start start 2024-09-19T04:24:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f59861f5740 1 Processor -- start 2024-09-19T04:24:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f59861f5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 0x55ac94bb4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f59861f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f59861f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 0x55ac94bb5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f59861f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1b0d50 con 0x55ac94ba64b0 2024-09-19T04:24:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f59861f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1b0650 con 0x55ac94bb0d70 2024-09-19T04:24:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f59861f5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1ad890 con 0x55ac94ba5b90 2024-09-19T04:24:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f59861f5740 10 client.0 _ll_register_callbacks cb 0x7f59640d80e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f5973e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f5973e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37636/0 (socket says 172.21.15.107:37636) 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f5973e00640 1 -- 172.21.15.107:0/948854402 learned_addr learned my addr 172.21.15.107:0/948854402 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.938+0000 7f5980e00640 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 0x55ac94bb4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f5981800640 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 0x55ac94bb5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f5973e00640 1 -- 172.21.15.107:0/948854402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 msgr2=0x55ac94bb4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f5973e00640 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 0x55ac94bb4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f5973e00640 1 -- 172.21.15.107:0/948854402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 msgr2=0x55ac94bb5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f5973e00640 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 0x55ac94bb5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f5973e00640 1 -- 172.21.15.107:0/948854402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac94c2a3f0 con 0x55ac94ba64b0 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f5981800640 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 0x55ac94bb5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f5973e00640 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb5000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f59640cdd00 tx=0x7f59640a4f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa34a6cca4959864 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f596aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f596aa00640 1 -- 172.21.15.107:0/948854402 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596409c110 con 0x55ac94ba64b0 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f596aa00640 1 -- 172.21.15.107:0/948854402 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f596409c5a0 con 0x55ac94ba64b0 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.939+0000 7f59861f5740 1 -- 172.21.15.107:0/948854402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f596c0043f0 con 0x55ac94ba64b0 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f596aa00640 1 -- 172.21.15.107:0/948854402 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f596409be80 con 0x55ac94ba64b0 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f59861f5740 10 client.15873 fetch_fsmap learned FSMap version 31 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f59861f5740 1 -- 172.21.15.107:0/948854402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ac94c2a3f0 con 0x55ac94ba64b0 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f596aa00640 1 -- 172.21.15.107:0/948854402 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f596c0043f0 con 0x55ac94ba64b0 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f596aa00640 1 -- 172.21.15.107:0/948854402 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f59640a0120 con 0x55ac94ba64b0 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f59861f5740 10 client.15873 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f59861f5740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f59861f5740 20 client.15873 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f59861f5740 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ac94c2baf0 0x55ac94c2df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f59861f5740 4 client.15873 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:24:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.940+0000 7f59861f5740 1 -- 172.21.15.107:0/948854402 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac94b13e60 con 0x55ac94c2baf0 2024-09-19T04:24:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.941+0000 7f5980e00640 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ac94c2baf0 0x55ac94c2df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.941+0000 7f5980e00640 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ac94c2baf0 0x55ac94c2df10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb403f3780ee79ce server_cookie=c3662b32c3a6ad04 in_seq=0 out_seq=0 2024-09-19T04:24:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.941+0000 7f596aa00640 10 client.15873 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.941+0000 7f596aa00640 1 -- 172.21.15.107:0/948854402 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ac94b13e60 con 0x55ac94c2baf0 2024-09-19T04:24:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.941+0000 7f596aa00640 10 client.15873 handle_command_reply: tid=0 2024-09-19T04:24:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.941+0000 7f59861f5740 1 client.15873 shutdown 2024-09-19T04:24:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.941+0000 7f59861f5740 1 -- 172.21.15.107:0/948854402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 msgr2=0x55ac94bb5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.941+0000 7f59861f5740 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb5000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f59640cdd00 tx=0x7f59640a4f70 comp rx=0 tx=0).stop 2024-09-19T04:24:20.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f59861f5740 1 -- 172.21.15.107:0/948854402 shutdown_connections 2024-09-19T04:24:20.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f59861f5740 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ac94c2baf0 0x55ac94c2df10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f59861f5740 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac94bb0d70 0x55ac94bb5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f59861f5740 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac94ba64b0 0x55ac94bb5000 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f59861f5740 1 --2- 172.21.15.107:0/948854402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ac94ba5b90 0x55ac94bb4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f59861f5740 1 -- 172.21.15.107:0/948854402 >> 172.21.15.107:0/948854402 conn(0x55ac94ba2120 msgr2=0x55ac94ba7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f596aa00640 0 client.15873 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:20.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f59861f5740 1 -- 172.21.15.107:0/948854402 shutdown_connections 2024-09-19T04:24:20.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f59861f5740 1 -- 172.21.15.107:0/948854402 wait complete. 2024-09-19T04:24:20.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f59861f5740 20 client.15873 trim_cache size 0 max 16384 2024-09-19T04:24:20.948 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:24:20.948 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:24:20.948 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:24:20.948 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:24:20.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f5983800640 1 -- 172.21.15.107:0/3045017505 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59600a33b0 msgr2=0x7f59600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f5983800640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59600a33b0 0x7f59600a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f597402aeb0 tx=0x7f5974005d80 comp rx=0 tx=0).stop 2024-09-19T04:24:20.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f5983800640 1 -- 172.21.15.107:0/3045017505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f597c102790 msgr2=0x7f597c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:20.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.942+0000 7f5983800640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f597c102790 0x7f597c19ab80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f596400e9f0 tx=0x7f596400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.943+0000 7f5983800640 1 -- 172.21.15.107:0/3045017505 shutdown_connections 2024-09-19T04:24:20.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.943+0000 7f5983800640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f59600a33b0 0x7f59600a5870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.943+0000 7f5983800640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c106f10 0x7f597c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.943+0000 7f5983800640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f597c102790 0x7f597c19ab80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.943+0000 7f5983800640 1 --2- 172.21.15.107:0/3045017505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f597c101de0 0x7f597c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:20.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.943+0000 7f5983800640 1 -- 172.21.15.107:0/3045017505 >> 172.21.15.107:0/3045017505 conn(0x7f597c0fd8c0 msgr2=0x7f597c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:20.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.943+0000 7f5983800640 1 -- 172.21.15.107:0/3045017505 shutdown_connections 2024-09-19T04:24:20.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:20.943+0000 7f5983800640 1 -- 172.21.15.107:0/3045017505 wait complete. 2024-09-19T04:24:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:20 smithi115 ceph-mon[35477]: pgmap v123: 97 pgs: 97 active+clean; 95 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.0 MiB/s wr, 88 op/s 2024-09-19T04:24:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:20 smithi145 ceph-mon[35096]: pgmap v123: 97 pgs: 97 active+clean; 95 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.0 MiB/s wr, 88 op/s 2024-09-19T04:24:21.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:20 smithi107 ceph-mon[30659]: pgmap v123: 97 pgs: 97 active+clean; 95 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.0 MiB/s wr, 88 op/s 2024-09-19T04:24:21.658 INFO:tasks.workunit.client.0.smithi107.stdout: 1 149791 2.94 MB/sec execute 39 sec latency 2340.435 ms 2024-09-19T04:24:22.658 INFO:tasks.workunit.client.0.smithi107.stdout: 1 149791 2.87 MB/sec execute 40 sec latency 3340.503 ms 2024-09-19T04:24:22.904 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:22 smithi107 ceph-mon[30659]: pgmap v124: 97 pgs: 97 active+clean; 96 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-09-19T04:24:22.940 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:24:22.941 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:24:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:22 smithi115 ceph-mon[35477]: pgmap v124: 97 pgs: 97 active+clean; 96 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-09-19T04:24:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:22 smithi145 ceph-mon[35096]: pgmap v124: 97 pgs: 97 active+clean; 96 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-09-19T04:24:23.308 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:23.658 INFO:tasks.workunit.client.0.smithi107.stdout: 1 149791 2.80 MB/sec execute 41 sec latency 4340.563 ms 2024-09-19T04:24:23.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.865+0000 7f777a200640 1 -- 172.21.15.107:0/1978971814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7774102790 msgr2=0x7f77741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:23.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.865+0000 7f777a200640 1 --2- 172.21.15.107:0/1978971814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7774102790 0x7f77741067e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7760009900 tx=0x7f776002a410 comp rx=0 tx=0).stop 2024-09-19T04:24:23.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.865+0000 7f777a200640 1 -- 172.21.15.107:0/1978971814 shutdown_connections 2024-09-19T04:24:23.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.865+0000 7f777a200640 1 --2- 172.21.15.107:0/1978971814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7774106f10 0x7f777410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.865+0000 7f777a200640 1 --2- 172.21.15.107:0/1978971814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7774102790 0x7f77741067e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.865+0000 7f777a200640 1 --2- 172.21.15.107:0/1978971814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7774101de0 0x7f77741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.865+0000 7f777a200640 1 -- 172.21.15.107:0/1978971814 >> 172.21.15.107:0/1978971814 conn(0x7f77740fd8c0 msgr2=0x7f77740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.866+0000 7f777a200640 1 -- 172.21.15.107:0/1978971814 shutdown_connections 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.866+0000 7f777a200640 1 -- 172.21.15.107:0/1978971814 wait complete. 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f777a200640 1 Processor -- start 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f777a200640 1 -- start start 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f777a200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7774101de0 0x7f7774198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f777a200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7774102790 0x7f7774198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f777a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7774106f10 0x7f777419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f777a200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7774111f70 con 0x7f7774106f10 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f777a200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7774111df0 con 0x7f7774101de0 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f777a200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77741120f0 con 0x7f7774102790 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f7773400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7774101de0 0x7f7774198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f7773e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7774106f10 0x7f777419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f7773400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7774101de0 0x7f7774198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37026/0 (socket says 172.21.15.107:37026) 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.867+0000 7f7773400640 1 -- 172.21.15.107:0/3861775908 learned_addr learned my addr 172.21.15.107:0/3861775908 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f7772a00640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7774102790 0x7f7774198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f7773e00640 1 -- 172.21.15.107:0/3861775908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7774102790 msgr2=0x7f7774198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f7773e00640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7774102790 0x7f7774198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f7773e00640 1 -- 172.21.15.107:0/3861775908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7774101de0 msgr2=0x7f7774198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:23.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f7773e00640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7774101de0 0x7f7774198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f7773e00640 1 -- 172.21.15.107:0/3861775908 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777419d400 con 0x7f7774106f10 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f7773400640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7774101de0 0x7f7774198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f7773e00640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7774106f10 0x7f777419cd20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f776c009fd0 tx=0x7f776c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f776be00640 1 -- 172.21.15.107:0/3861775908 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c00ce20 con 0x7f7774106f10 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f776be00640 1 -- 172.21.15.107:0/3861775908 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c005020 con 0x7f7774106f10 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f777a200640 1 -- 172.21.15.107:0/3861775908 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777419d6f0 con 0x7f7774106f10 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.868+0000 7f776be00640 1 -- 172.21.15.107:0/3861775908 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c010720 con 0x7f7774106f10 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.869+0000 7f777a200640 1 -- 172.21.15.107:0/3861775908 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77741a4fd0 con 0x7f7774106f10 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.870+0000 7f777a200640 1 -- 172.21.15.107:0/3861775908 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77741021c0 con 0x7f7774106f10 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.871+0000 7f776be00640 1 -- 172.21.15.107:0/3861775908 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f776c005540 con 0x7f7774106f10 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.873+0000 7f776be00640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77580a3430 0x7f77580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:23.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.873+0000 7f776be00640 1 -- 172.21.15.107:0/3861775908 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f776c0d02f0 con 0x7f7774106f10 2024-09-19T04:24:23.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.873+0000 7f7773400640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77580a3430 0x7f77580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:23.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.874+0000 7f7773400640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77580a3430 0x7f77580a58f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7764011500 tx=0x7f7764005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:23.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:23.877+0000 7f776be00640 1 -- 172.21.15.107:0/3861775908 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f776c09ccc0 con 0x7f7774106f10 2024-09-19T04:24:24.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.090+0000 7f777a200640 1 -- 172.21.15.107:0/3861775908 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f777402d7c0 con 0x7f7774106f10 2024-09-19T04:24:24.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.091+0000 7f776be00640 1 -- 172.21.15.107:0/3861775908 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12000 (secure 0 0 0) 0x7f776c09c5c0 con 0x7f7774106f10 2024-09-19T04:24:24.093 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:24:24.094 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":31,"btime":"2024-09-19T04:22:42:592499+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:22:41.847132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:24:24.094 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 31 2024-09-19T04:24:24.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 -- 172.21.15.107:0/3861775908 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77580a3430 msgr2=0x7f77580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:24.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77580a3430 0x7f77580a58f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7764011500 tx=0x7f7764005f90 comp rx=0 tx=0).stop 2024-09-19T04:24:24.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 -- 172.21.15.107:0/3861775908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7774106f10 msgr2=0x7f777419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:24.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7774106f10 0x7f777419cd20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f776c009fd0 tx=0x7f776c00ef10 comp rx=0 tx=0).stop 2024-09-19T04:24:24.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 -- 172.21.15.107:0/3861775908 shutdown_connections 2024-09-19T04:24:24.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77580a3430 0x7f77580a58f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:24.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7774106f10 0x7f777419cd20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:24.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7774102790 0x7f7774198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:24.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 --2- 172.21.15.107:0/3861775908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7774101de0 0x7f7774198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:24.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 -- 172.21.15.107:0/3861775908 >> 172.21.15.107:0/3861775908 conn(0x7f77740fd8c0 msgr2=0x7f77740febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:24.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 -- 172.21.15.107:0/3861775908 shutdown_connections 2024-09-19T04:24:24.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:24.094+0000 7f777a200640 1 -- 172.21.15.107:0/3861775908 wait complete. 2024-09-19T04:24:24.658 INFO:tasks.workunit.client.0.smithi107.stdout: 1 151035 2.86 MB/sec execute 42 sec latency 4896.511 ms 2024-09-19T04:24:24.912 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:24:25.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:24 smithi115 ceph-mon[35477]: pgmap v125: 97 pgs: 97 active+clean; 96 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 756 KiB/s wr, 68 op/s 2024-09-19T04:24:25.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:24 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3861775908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:24:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:24 smithi145 ceph-mon[35096]: pgmap v125: 97 pgs: 97 active+clean; 96 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 756 KiB/s wr, 68 op/s 2024-09-19T04:24:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:24 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3861775908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:24:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:24 smithi107 ceph-mon[30659]: pgmap v125: 97 pgs: 97 active+clean; 96 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 756 KiB/s wr, 68 op/s 2024-09-19T04:24:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:24 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3861775908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:24:25.315 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:25.658 INFO:tasks.workunit.client.0.smithi107.stdout: 1 152928 3.11 MB/sec execute 43 sec latency 13.695 ms 2024-09-19T04:24:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.984+0000 7f6f39200640 1 -- 172.21.15.107:0/4200582815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f341049c0 msgr2=0x7f6f34108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.984+0000 7f6f39200640 1 --2- 172.21.15.107:0/4200582815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f341049c0 0x7f6f34108a10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009900 tx=0x7f6f2002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.984+0000 7f6f39200640 1 -- 172.21.15.107:0/4200582815 shutdown_connections 2024-09-19T04:24:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.984+0000 7f6f39200640 1 --2- 172.21.15.107:0/4200582815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f34109140 0x7f6f34110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.984+0000 7f6f39200640 1 --2- 172.21.15.107:0/4200582815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f341049c0 0x7f6f34108a10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.984+0000 7f6f39200640 1 --2- 172.21.15.107:0/4200582815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f34104010 0x7f6f341043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.984+0000 7f6f39200640 1 -- 172.21.15.107:0/4200582815 >> 172.21.15.107:0/4200582815 conn(0x7f6f340ffa50 msgr2=0x7f6f34101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:25.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.984+0000 7f6f39200640 1 -- 172.21.15.107:0/4200582815 shutdown_connections 2024-09-19T04:24:25.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.984+0000 7f6f39200640 1 -- 172.21.15.107:0/4200582815 wait complete. 2024-09-19T04:24:25.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.985+0000 7f6f39200640 1 Processor -- start 2024-09-19T04:24:25.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.985+0000 7f6f39200640 1 -- start start 2024-09-19T04:24:25.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.985+0000 7f6f39200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f34104010 0x7f6f3419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.985+0000 7f6f39200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f341049c0 0x7f6f3419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.985+0000 7f6f39200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f34109140 0x7f6f3419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.985+0000 7f6f39200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34114140 con 0x7f6f34109140 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.985+0000 7f6f39200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34113fc0 con 0x7f6f34104010 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.985+0000 7f6f39200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341142c0 con 0x7f6f341049c0 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f32800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f34109140 0x7f6f3419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f31e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f34104010 0x7f6f3419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f31e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f34104010 0x7f6f3419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37036/0 (socket says 172.21.15.107:37036) 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f31e00640 1 -- 172.21.15.107:0/623096842 learned_addr learned my addr 172.21.15.107:0/623096842 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f31400640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f341049c0 0x7f6f3419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f31e00640 1 -- 172.21.15.107:0/623096842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f341049c0 msgr2=0x7f6f3419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f31e00640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f341049c0 0x7f6f3419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f31e00640 1 -- 172.21.15.107:0/623096842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f34109140 msgr2=0x7f6f3419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f31e00640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f34109140 0x7f6f3419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f31e00640 1 -- 172.21.15.107:0/623096842 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3419f690 con 0x7f6f34104010 2024-09-19T04:24:25.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.986+0000 7f6f31e00640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f34104010 0x7f6f3419a6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400ea60 tx=0x7f6f2400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:25.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.987+0000 7f6f2aa00640 1 -- 172.21.15.107:0/623096842 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400cde0 con 0x7f6f34104010 2024-09-19T04:24:25.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.987+0000 7f6f39200640 1 -- 172.21.15.107:0/623096842 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3419f980 con 0x7f6f34104010 2024-09-19T04:24:25.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.987+0000 7f6f39200640 1 -- 172.21.15.107:0/623096842 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f341a7260 con 0x7f6f34104010 2024-09-19T04:24:25.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.987+0000 7f6f2aa00640 1 -- 172.21.15.107:0/623096842 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f240035f0 con 0x7f6f34104010 2024-09-19T04:24:25.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.987+0000 7f6f2aa00640 1 -- 172.21.15.107:0/623096842 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24018690 con 0x7f6f34104010 2024-09-19T04:24:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.990+0000 7f6f3bdd5740 1 Processor -- start 2024-09-19T04:24:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.990+0000 7f6f2aa00640 1 -- 172.21.15.107:0/623096842 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f6f24003a80 con 0x7f6f34104010 2024-09-19T04:24:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.990+0000 7f6f3bdd5740 1 -- start start 2024-09-19T04:24:25.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.990+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03dc70 0x557e6a03e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.990+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03e590 0x557e6a048910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.990+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 0x557e6a04b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.990+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557e69fa43e0 con 0x557e6a03dc70 2024-09-19T04:24:25.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.990+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557e69f4bd80 con 0x557e6a03e590 2024-09-19T04:24:25.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.990+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557e69fa03a0 con 0x557e6a048e50 2024-09-19T04:24:25.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f31400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 0x557e6a04b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f32800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03dc70 0x557e6a03e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f31400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 0x557e6a04b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:56686/0 (socket says 172.21.15.107:56686) 2024-09-19T04:24:25.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f31400640 1 -- 172.21.15.107:0/733589630 learned_addr learned my addr 172.21.15.107:0/733589630 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:25.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f31e00640 1 --2- 172.21.15.107:0/733589630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03e590 0x557e6a048910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f32800640 1 -- 172.21.15.107:0/733589630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 msgr2=0x557e6a04b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f32800640 1 --2- 172.21.15.107:0/733589630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 0x557e6a04b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f32800640 1 -- 172.21.15.107:0/733589630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03e590 msgr2=0x557e6a048910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f32800640 1 --2- 172.21.15.107:0/733589630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03e590 0x557e6a048910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f32800640 1 -- 172.21.15.107:0/733589630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e69f6aae0 con 0x557e6a03dc70 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f31e00640 1 --2- 172.21.15.107:0/733589630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03e590 0x557e6a048910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f2aa00640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6f180a3430 0x7f6f180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.991+0000 7f6f32800640 1 --2- 172.21.15.107:0/733589630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03dc70 0x557e6a03e050 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009f00 tx=0x7f6f2c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aa0b335ae3f7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.992+0000 7f6f1f400640 1 -- 172.21.15.107:0/733589630 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c004d20 con 0x557e6a03dc70 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.992+0000 7f6f31e00640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6f180a3430 0x7f6f180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.992+0000 7f6f2aa00640 1 -- 172.21.15.107:0/623096842 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f6f240cfff0 con 0x7f6f34104010 2024-09-19T04:24:25.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.992+0000 7f6f1f400640 1 -- 172.21.15.107:0/733589630 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f2c004050 con 0x557e6a03dc70 2024-09-19T04:24:25.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.992+0000 7f6f1f400640 1 -- 172.21.15.107:0/733589630 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c005280 con 0x557e6a03dc70 2024-09-19T04:24:25.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.992+0000 7f6f31e00640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6f180a3430 0x7f6f180a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x557e6a03b060 tx=0x7f6f240d3ac0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:25.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.992+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/733589630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03dc70 msgr2=0x557e6a03e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:25.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.993+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/733589630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03dc70 0x557e6a03e050 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009f00 tx=0x7f6f2c011810 comp rx=0 tx=0).stop 2024-09-19T04:24:25.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.993+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/733589630 shutdown_connections 2024-09-19T04:24:25.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.993+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/733589630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 0x557e6a04b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.993+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/733589630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03e590 0x557e6a048910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.993+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/733589630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03dc70 0x557e6a03e050 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.993+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/733589630 >> 172.21.15.107:0/733589630 conn(0x557e6a03a200 msgr2=0x557e6a03a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:25.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.993+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/733589630 shutdown_connections 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.993+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/733589630 wait complete. 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.993+0000 7f6f3bdd5740 1 -- start start 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.993+0000 7f6f3bdd5740 1 Processor -- start 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03dc70 0x557e6a045d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a0462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a048e50 0x557e6a046810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557e6a04ba20 con 0x557e6a03e590 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557e69f5f0f0 con 0x557e6a048e50 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557e6a04bba0 con 0x557e6a03dc70 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f3bdd5740 10 client.0 _ll_register_callbacks cb 0x557e69f46bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:25.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f31400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a0462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f31400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a0462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37708/0 (socket says 172.21.15.107:37708) 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f31400640 1 -- 172.21.15.107:0/2430587968 learned_addr learned my addr 172.21.15.107:0/2430587968 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.994+0000 7f6f32800640 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03dc70 0x557e6a045d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f32800640 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a048e50 0x557e6a046810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f31400640 1 -- 172.21.15.107:0/2430587968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03dc70 msgr2=0x557e6a045d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f31400640 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03dc70 0x557e6a045d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f31400640 1 -- 172.21.15.107:0/2430587968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a048e50 msgr2=0x557e6a046810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f31400640 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a048e50 0x557e6a046810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f31400640 1 -- 172.21.15.107:0/2430587968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e69d153c0 con 0x557e6a03e590 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f31400640 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a0462d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009230 tx=0x7f6f200317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290ed323f45521d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f13e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f13e00640 1 -- 172.21.15.107:0/2430587968 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f20035040 con 0x557e6a03e590 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f13e00640 1 -- 172.21.15.107:0/2430587968 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f6f20004460 con 0x557e6a03e590 2024-09-19T04:24:25.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.995+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/2430587968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f2c004050 con 0x557e6a03e590 2024-09-19T04:24:26.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f13e00640 1 -- 172.21.15.107:0/2430587968 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6f2004a020 con 0x557e6a03e590 2024-09-19T04:24:26.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f3bdd5740 10 client.15891 fetch_fsmap learned FSMap version 31 2024-09-19T04:24:26.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/2430587968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557e6a057b10 con 0x557e6a03e590 2024-09-19T04:24:26.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f13e00640 1 -- 172.21.15.107:0/2430587968 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f2c004050 con 0x557e6a03e590 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f13e00640 1 -- 172.21.15.107:0/2430587968 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f6f20004c10 con 0x557e6a03e590 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f3bdd5740 10 client.15891 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f3bdd5740 10 client.15891 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f3bdd5740 20 client.15891 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x557e6a058f70 0x557e6a05b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f3bdd5740 4 client.15891 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.996+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/2430587968 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e69fb1950 con 0x557e6a058f70 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.997+0000 7f6f32800640 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x557e6a058f70 0x557e6a05b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.997+0000 7f6f32800640 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x557e6a058f70 0x557e6a05b390 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6427463c0c4cb052 server_cookie=f7c908057f3af264 in_seq=0 out_seq=0 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.997+0000 7f6f13e00640 10 client.15891 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f13e00640 1 -- 172.21.15.107:0/2430587968 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e69fb1950 con 0x557e6a058f70 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f13e00640 10 client.15891 handle_command_reply: tid=0 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 client.15891 shutdown 2024-09-19T04:24:26.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/2430587968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 msgr2=0x557e6a0462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a0462d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009230 tx=0x7f6f200317e0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/2430587968 shutdown_connections 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x557e6a058f70 0x557e6a05b390 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a048e50 0x557e6a046810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a0462d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/2430587968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03dc70 0x557e6a045d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/2430587968 >> 172.21.15.107:0/2430587968 conn(0x557e6a03a200 msgr2=0x557e6a039380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f13e00640 0 client.15891 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/2430587968 shutdown_connections 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/2430587968 wait complete. 2024-09-19T04:24:26.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:25.998+0000 7f6f3bdd5740 20 client.15891 trim_cache size 0 max 16384 2024-09-19T04:24:26.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f3bdd5740 1 Processor -- start 2024-09-19T04:24:26.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f3bdd5740 1 -- start start 2024-09-19T04:24:26.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 0x557e6a060c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:26.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a061180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:26.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 0x557e6a038b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:26.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341af990 con 0x557e6a03e590 2024-09-19T04:24:26.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b15c0 con 0x557e6a03dc70 2024-09-19T04:24:26.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b0060 con 0x557e6a048e50 2024-09-19T04:24:26.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f31400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a061180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:26.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f31400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a061180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37718/0 (socket says 172.21.15.107:37718) 2024-09-19T04:24:26.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.012+0000 7f6f31400640 1 -- 172.21.15.107:0/751471704 learned_addr learned my addr 172.21.15.107:0/751471704 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:26.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f32800640 1 --2- 172.21.15.107:0/751471704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 0x557e6a060c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:26.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f32800640 1 --2- 172.21.15.107:0/751471704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 0x557e6a038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:26.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f31400640 1 -- 172.21.15.107:0/751471704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 msgr2=0x557e6a038b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:26.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f31400640 1 --2- 172.21.15.107:0/751471704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 0x557e6a038b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f31400640 1 -- 172.21.15.107:0/751471704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 msgr2=0x557e6a060c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:26.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f31400640 1 --2- 172.21.15.107:0/751471704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 0x557e6a060c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f31400640 1 -- 172.21.15.107:0/751471704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6a035770 con 0x557e6a03e590 2024-09-19T04:24:26.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f32800640 1 --2- 172.21.15.107:0/751471704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 0x557e6a060c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:26.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f31400640 1 --2- 172.21.15.107:0/751471704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a061180 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f20005940 tx=0x7f6f20009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3de7f5cd98af2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:26.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f1ea00640 1 -- 172.21.15.107:0/751471704 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f20035070 con 0x557e6a03e590 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f1ea00640 1 -- 172.21.15.107:0/751471704 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f20041030 con 0x557e6a03e590 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.013+0000 7f6f1ea00640 1 -- 172.21.15.107:0/751471704 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f20000e60 con 0x557e6a03e590 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.014+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/751471704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 msgr2=0x557e6a061180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.014+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/751471704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a061180 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f20005940 tx=0x7f6f20009830 comp rx=0 tx=0).stop 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.014+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/751471704 shutdown_connections 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.014+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/751471704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a048e50 0x557e6a038b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.014+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/751471704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a03e590 0x557e6a061180 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.014+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/751471704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 0x557e6a060c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.014+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/751471704 >> 172.21.15.107:0/751471704 conn(0x557e6a03a200 msgr2=0x557e6a041090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.014+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/751471704 shutdown_connections 2024-09-19T04:24:26.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.014+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/751471704 wait complete. 2024-09-19T04:24:26.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.015+0000 7f6f3bdd5740 1 -- start start 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.015+0000 7f6f3bdd5740 1 Processor -- start 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.015+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 0x557e6a051720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03e590 0x557e6a04ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f3bdd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a048e50 0x557e6a04d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b0e30 con 0x557e6a048e50 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b0730 con 0x557e6a03dc70 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f3bdd5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341ad970 con 0x557e6a03e590 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f3bdd5740 10 client.0 _ll_register_callbacks cb 0x7f6f20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f32800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 0x557e6a051720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f32800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a048e50 0x557e6a04d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f32800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a048e50 0x557e6a04d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37730/0 (socket says 172.21.15.107:37730) 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f32800640 1 -- 172.21.15.107:0/3300450303 learned_addr learned my addr 172.21.15.107:0/3300450303 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:26.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f31400640 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03e590 0x557e6a04ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f31400640 1 -- 172.21.15.107:0/3300450303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 msgr2=0x557e6a051720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f31400640 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 0x557e6a051720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f31400640 1 -- 172.21.15.107:0/3300450303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a048e50 msgr2=0x557e6a04d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f31400640 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a048e50 0x557e6a04d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f31400640 1 -- 172.21.15.107:0/3300450303 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6a0c2c10 con 0x557e6a03e590 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.016+0000 7f6f32800640 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a048e50 0x557e6a04d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.017+0000 7f6f31400640 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03e590 0x557e6a04ca50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6f20031920 tx=0x7f6f20005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6be6f468600aaa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.017+0000 7f6f13e00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.017+0000 7f6f13e00640 1 -- 172.21.15.107:0/3300450303 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2005c060 con 0x557e6a03e590 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.017+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/3300450303 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f20041030 con 0x557e6a03e590 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.017+0000 7f6f13e00640 1 -- 172.21.15.107:0/3300450303 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f6f20005070 con 0x557e6a03e590 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.017+0000 7f6f13e00640 1 -- 172.21.15.107:0/3300450303 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6f20035030 con 0x557e6a03e590 2024-09-19T04:24:26.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.017+0000 7f6f3bdd5740 10 client.25918 fetch_fsmap learned FSMap version 31 2024-09-19T04:24:26.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.017+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/3300450303 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557e6a0c3da0 con 0x557e6a03e590 2024-09-19T04:24:26.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.017+0000 7f6f13e00640 1 -- 172.21.15.107:0/3300450303 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f20041030 con 0x557e6a03e590 2024-09-19T04:24:26.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.017+0000 7f6f13e00640 1 -- 172.21.15.107:0/3300450303 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f6f20004b50 con 0x557e6a03e590 2024-09-19T04:24:26.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.018+0000 7f6f3bdd5740 10 client.25918 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:24:26.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.018+0000 7f6f3bdd5740 10 client.25918 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:24:26.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.018+0000 7f6f3bdd5740 20 client.25918 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:26.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.018+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x557e6a0c4420 0x557e6a0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:26.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.018+0000 7f6f3bdd5740 4 client.25918 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:24:26.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.018+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/3300450303 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e69fabe60 con 0x557e6a0c4420 2024-09-19T04:24:26.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.018+0000 7f6f32800640 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x557e6a0c4420 0x557e6a0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:26.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.018+0000 7f6f32800640 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x557e6a0c4420 0x557e6a0c6840 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=887a175eed42b5ff server_cookie=b5e68b5b0df6a114 in_seq=0 out_seq=0 2024-09-19T04:24:26.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.018+0000 7f6f13e00640 10 client.25918 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:24:26.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f13e00640 1 -- 172.21.15.107:0/3300450303 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e69fabe60 con 0x557e6a0c4420 2024-09-19T04:24:26.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f13e00640 10 client.25918 handle_command_reply: tid=0 2024-09-19T04:24:26.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 client.25918 shutdown 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/3300450303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03e590 msgr2=0x557e6a04ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03e590 0x557e6a04ca50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6f20031920 tx=0x7f6f20005a20 comp rx=0 tx=0).stop 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/3300450303 shutdown_connections 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x557e6a0c4420 0x557e6a0c6840 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e6a048e50 0x557e6a04d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f13e00640 0 client.25918 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e6a03e590 0x557e6a04ca50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 --2- 172.21.15.107:0/3300450303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e6a03dc70 0x557e6a051720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/3300450303 >> 172.21.15.107:0/3300450303 conn(0x557e6a03a200 msgr2=0x557e6a03f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/3300450303 shutdown_connections 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 1 -- 172.21.15.107:0/3300450303 wait complete. 2024-09-19T04:24:26.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.019+0000 7f6f3bdd5740 20 client.25918 trim_cache size 0 max 16384 2024-09-19T04:24:26.025 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:24:26.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 -- 172.21.15.107:0/623096842 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6f180a3430 msgr2=0x7f6f180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:26.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6f180a3430 0x7f6f180a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x557e6a03b060 tx=0x7f6f240d3ac0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 -- 172.21.15.107:0/623096842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f34104010 msgr2=0x7f6f3419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:26.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f34104010 0x7f6f3419a6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400ea60 tx=0x7f6f2400ef20 comp rx=0 tx=0).stop 2024-09-19T04:24:26.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 -- 172.21.15.107:0/623096842 shutdown_connections 2024-09-19T04:24:26.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6f180a3430 0x7f6f180a58f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f34109140 0x7f6f3419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f341049c0 0x7f6f3419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 --2- 172.21.15.107:0/623096842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f34104010 0x7f6f3419a6e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:26.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 -- 172.21.15.107:0/623096842 >> 172.21.15.107:0/623096842 conn(0x7f6f340ffa50 msgr2=0x7f6f34100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:26.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 -- 172.21.15.107:0/623096842 shutdown_connections 2024-09-19T04:24:26.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:26.020+0000 7f6f39200640 1 -- 172.21.15.107:0/623096842 wait complete. 2024-09-19T04:24:26.658 INFO:tasks.workunit.client.0.smithi107.stdout: 1 153217 3.04 MB/sec execute 44 sec latency 674.327 ms 2024-09-19T04:24:27.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:24:27.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:26 smithi107 ceph-mon[30659]: pgmap v126: 97 pgs: 97 active+clean; 97 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-19T04:24:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:26 smithi115 ceph-mon[35477]: pgmap v126: 97 pgs: 97 active+clean; 97 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-19T04:24:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:26 smithi145 ceph-mon[35096]: pgmap v126: 97 pgs: 97 active+clean; 97 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-19T04:24:27.403 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:27.658 INFO:tasks.workunit.client.0.smithi107.stdout: 1 153217 2.97 MB/sec execute 45 sec latency 1674.388 ms 2024-09-19T04:24:27.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.989+0000 7f08a0a00640 1 -- 172.21.15.107:0/3245924633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c108f60 msgr2=0x7f089c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.989+0000 7f08a0a00640 1 --2- 172.21.15.107:0/3245924633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c108f60 0x7f089c110ae0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0894009ca0 tx=0x7f089402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.990+0000 7f08a0a00640 1 -- 172.21.15.107:0/3245924633 shutdown_connections 2024-09-19T04:24:27.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.990+0000 7f08a0a00640 1 --2- 172.21.15.107:0/3245924633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c108f60 0x7f089c110ae0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.990+0000 7f08a0a00640 1 --2- 172.21.15.107:0/3245924633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c1047e0 0x7f089c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.990+0000 7f08a0a00640 1 --2- 172.21.15.107:0/3245924633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f089c103e30 0x7f089c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.990+0000 7f08a0a00640 1 -- 172.21.15.107:0/3245924633 >> 172.21.15.107:0/3245924633 conn(0x7f089c0ff8d0 msgr2=0x7f089c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:27.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.990+0000 7f08a0a00640 1 -- 172.21.15.107:0/3245924633 shutdown_connections 2024-09-19T04:24:27.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.990+0000 7f08a0a00640 1 -- 172.21.15.107:0/3245924633 wait complete. 2024-09-19T04:24:27.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f08a0a00640 1 Processor -- start 2024-09-19T04:24:27.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f08a0a00640 1 -- start start 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f08a0a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103e30 0x7f089c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f08a0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c1047e0 0x7f089c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f08a0a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f089c108f60 0x7f089c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f08a0a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f089c113fc0 con 0x7f089c1047e0 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f08a0a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f089c113e40 con 0x7f089c103e30 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f08a0a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f089c114140 con 0x7f089c108f60 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f0891e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c1047e0 0x7f089c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f089a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103e30 0x7f089c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f089b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f089c108f60 0x7f089c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f0891e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c1047e0 0x7f089c19a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37750/0 (socket says 172.21.15.107:37750) 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f0891e00640 1 -- 172.21.15.107:0/3875521113 learned_addr learned my addr 172.21.15.107:0/3875521113 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f089b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f089c108f60 0x7f089c19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:56728/0 (socket says 172.21.15.107:56728) 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.991+0000 7f089a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103e30 0x7f089c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37080/0 (socket says 172.21.15.107:37080) 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f089a800640 1 -- 172.21.15.107:0/3875521113 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f089c108f60 msgr2=0x7f089c19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f089a800640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f089c108f60 0x7f089c19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f089a800640 1 -- 172.21.15.107:0/3875521113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c1047e0 msgr2=0x7f089c19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f089a800640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c1047e0 0x7f089c19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f089a800640 1 -- 172.21.15.107:0/3875521113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c19f3b0 con 0x7f089c103e30 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f0891e00640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c1047e0 0x7f089c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f089a800640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103e30 0x7f089c19a400 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f088c00e9f0 tx=0x7f088c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f0893e00640 1 -- 172.21.15.107:0/3875521113 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c00ccc0 con 0x7f089c103e30 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f0893e00640 1 -- 172.21.15.107:0/3875521113 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f088c005020 con 0x7f089c103e30 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f0893e00640 1 -- 172.21.15.107:0/3875521113 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c0049e0 con 0x7f089c103e30 2024-09-19T04:24:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.992+0000 7f08a0a00640 1 -- 172.21.15.107:0/3875521113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c19f6a0 con 0x7f089c103e30 2024-09-19T04:24:27.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.993+0000 7f08a0a00640 1 -- 172.21.15.107:0/3875521113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c1a6f80 con 0x7f089c103e30 2024-09-19T04:24:27.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.995+0000 7f08a3ec7740 1 Processor -- start 2024-09-19T04:24:27.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f08a3ec7740 1 -- start start 2024-09-19T04:24:27.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f0893e00640 1 -- 172.21.15.107:0/3875521113 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f088c005560 con 0x7f089c103e30 2024-09-19T04:24:27.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6ae40 0x563f13c6b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6b760 0x563f13c75ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c76020 0x563f13c78410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f13bd13e0 con 0x563f13c6b760 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563f13b78d80 con 0x563f13c6ae40 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563f13bcd3a0 con 0x563f13c76020 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f089b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6ae40 0x563f13c6b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f089a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6b760 0x563f13c75ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f089b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6ae40 0x563f13c6b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37084/0 (socket says 172.21.15.107:37084) 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.996+0000 7f089b200640 1 -- 172.21.15.107:0/2113437244 learned_addr learned my addr 172.21.15.107:0/2113437244 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.997+0000 7f0891e00640 1 --2- 172.21.15.107:0/2113437244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c76020 0x563f13c78410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.997+0000 7f089b200640 1 -- 172.21.15.107:0/2113437244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c76020 msgr2=0x563f13c78410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.997+0000 7f089b200640 1 --2- 172.21.15.107:0/2113437244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c76020 0x563f13c78410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.997+0000 7f089b200640 1 -- 172.21.15.107:0/2113437244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6b760 msgr2=0x563f13c75ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.997+0000 7f089b200640 1 --2- 172.21.15.107:0/2113437244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6b760 0x563f13c75ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.997+0000 7f089b200640 1 -- 172.21.15.107:0/2113437244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f13b97ae0 con 0x563f13c6ae40 2024-09-19T04:24:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.997+0000 7f089b200640 1 --2- 172.21.15.107:0/2113437244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6ae40 0x563f13c6b220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0894009ca0 tx=0x7f0894005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f63f1eaafaf42d27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.997+0000 7f088b400640 1 -- 172.21.15.107:0/2113437244 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894031db0 con 0x563f13c6ae40 2024-09-19T04:24:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f0893e00640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f08800a3430 0x7f08800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f088b400640 1 -- 172.21.15.107:0/2113437244 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0894036040 con 0x563f13c6ae40 2024-09-19T04:24:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f088b400640 1 -- 172.21.15.107:0/2113437244 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894004330 con 0x563f13c6ae40 2024-09-19T04:24:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f0893e00640 1 -- 172.21.15.107:0/3875521113 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f088c0cfc40 con 0x7f089c103e30 2024-09-19T04:24:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f089a800640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f08800a3430 0x7f08800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2113437244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6ae40 msgr2=0x563f13c6b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/2113437244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6ae40 0x563f13c6b220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0894009ca0 tx=0x7f0894005f10 comp rx=0 tx=0).stop 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2113437244 shutdown_connections 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/2113437244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c76020 0x563f13c78410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/2113437244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6b760 0x563f13c75ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/2113437244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6ae40 0x563f13c6b220 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.998+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2113437244 >> 172.21.15.107:0/2113437244 conn(0x563f13c673d0 msgr2=0x563f13c677b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.999+0000 7f089a800640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f08800a3430 0x7f08800a58f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x563f13c68230 tx=0x7f088c0d36f0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.999+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2113437244 shutdown_connections 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.999+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2113437244 wait complete. 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.999+0000 7f08a3ec7740 1 -- start start 2024-09-19T04:24:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:27.999+0000 7f08a3ec7740 1 Processor -- start 2024-09-19T04:24:28.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6ae40 0x563f13c72f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c73450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c76020 0x563f13c73990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f13c78a70 con 0x563f13c6ae40 2024-09-19T04:24:28.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563f13b8c0f0 con 0x563f13c6b760 2024-09-19T04:24:28.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563f13c78bf0 con 0x563f13c76020 2024-09-19T04:24:28.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f08a3ec7740 10 client.0 _ll_register_callbacks cb 0x563f13b73bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:28.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f0891e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c73450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f0891e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c73450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37090/0 (socket says 172.21.15.107:37090) 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f0891e00640 1 -- 172.21.15.107:0/864073978 learned_addr learned my addr 172.21.15.107:0/864073978 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f089b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6ae40 0x563f13c72f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f089b200640 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c76020 0x563f13c73990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f0891e00640 1 -- 172.21.15.107:0/864073978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c76020 msgr2=0x563f13c73990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f0891e00640 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c76020 0x563f13c73990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f0891e00640 1 -- 172.21.15.107:0/864073978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6ae40 msgr2=0x563f13c72f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f0891e00640 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6ae40 0x563f13c72f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f0891e00640 1 -- 172.21.15.107:0/864073978 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f139423c0 con 0x563f13c6b760 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.000+0000 7f089b200640 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6ae40 0x563f13c72f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.001+0000 7f0891e00640 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c73450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0884012d20 tx=0x7f088400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212df6869c70bbb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.001+0000 7f087fe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.001+0000 7f087fe00640 1 -- 172.21.15.107:0/864073978 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884012110 con 0x563f13c6b760 2024-09-19T04:24:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.001+0000 7f08a3ec7740 1 -- 172.21.15.107:0/864073978 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0894036040 con 0x563f13c6b760 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.001+0000 7f087fe00640 1 -- 172.21.15.107:0/864073978 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0884004030 con 0x563f13c6b760 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.001+0000 7f087fe00640 1 -- 172.21.15.107:0/864073978 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f08840122b0 con 0x563f13c6b760 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.001+0000 7f08a3ec7740 10 client.25961 fetch_fsmap learned FSMap version 31 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.001+0000 7f08a3ec7740 1 -- 172.21.15.107:0/864073978 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563f13c84b10 con 0x563f13c6b760 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.002+0000 7f087fe00640 1 -- 172.21.15.107:0/864073978 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0894036040 con 0x563f13c6b760 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.002+0000 7f087fe00640 1 -- 172.21.15.107:0/864073978 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f08840256b0 con 0x563f13c6b760 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.002+0000 7f08a3ec7740 10 client.25961 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.002+0000 7f08a3ec7740 10 client.25961 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.002+0000 7f08a3ec7740 20 client.25961 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.002+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x563f13c85f20 0x563f13c88330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.002+0000 7f08a3ec7740 4 client.25961 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.002+0000 7f08a3ec7740 1 -- 172.21.15.107:0/864073978 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f13bde950 con 0x563f13c85f20 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.002+0000 7f089b200640 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x563f13c85f20 0x563f13c88330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.003+0000 7f089b200640 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x563f13c85f20 0x563f13c88330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fe5bcb5eff9e3d81 server_cookie=143b97e057f21ecd in_seq=0 out_seq=0 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.003+0000 7f087fe00640 10 client.25961 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f087fe00640 1 -- 172.21.15.107:0/864073978 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f13bde950 con 0x563f13c85f20 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f087fe00640 10 client.25961 handle_command_reply: tid=0 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 client.25961 shutdown 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 -- 172.21.15.107:0/864073978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 msgr2=0x563f13c73450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c73450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0884012d20 tx=0x7f088400f790 comp rx=0 tx=0).stop 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 -- 172.21.15.107:0/864073978 shutdown_connections 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x563f13c85f20 0x563f13c88330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c76020 0x563f13c73990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c73450 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f087fe00640 0 client.25961 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:24:28.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/864073978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6ae40 0x563f13c72f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 -- 172.21.15.107:0/864073978 >> 172.21.15.107:0/864073978 conn(0x563f13c673d0 msgr2=0x563f13c6d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:28.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 -- 172.21.15.107:0/864073978 shutdown_connections 2024-09-19T04:24:28.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.004+0000 7f08a3ec7740 1 -- 172.21.15.107:0/864073978 wait complete. 2024-09-19T04:24:28.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.005+0000 7f08a3ec7740 20 client.25961 trim_cache size 0 max 16384 2024-09-19T04:24:28.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.018+0000 7f08a3ec7740 1 Processor -- start 2024-09-19T04:24:28.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.018+0000 7f08a3ec7740 1 -- start start 2024-09-19T04:24:28.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.018+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 0x563f13c6d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.018+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6b760 0x563f13c74e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.018+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c76020 0x563f13c8dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.018+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1af720 con 0x563f13c6b760 2024-09-19T04:24:28.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.018+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1b1350 con 0x563f13c76020 2024-09-19T04:24:28.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.018+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1afdf0 con 0x563f13c6ae40 2024-09-19T04:24:28.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.018+0000 7f089b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 0x563f13c6d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f0891e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6b760 0x563f13c74e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f089b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 0x563f13c6d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:56756/0 (socket says 172.21.15.107:56756) 2024-09-19T04:24:28.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f089b200640 1 -- 172.21.15.107:0/1651430907 learned_addr learned my addr 172.21.15.107:0/1651430907 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:28.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f089b200640 1 --2- 172.21.15.107:0/1651430907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c76020 0x563f13c8dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f089b200640 1 -- 172.21.15.107:0/1651430907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c76020 msgr2=0x563f13c8dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:28.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f089b200640 1 --2- 172.21.15.107:0/1651430907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c76020 0x563f13c8dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f089b200640 1 -- 172.21.15.107:0/1651430907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6b760 msgr2=0x563f13c74e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:28.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f089b200640 1 --2- 172.21.15.107:0/1651430907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6b760 0x563f13c74e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f089b200640 1 -- 172.21.15.107:0/1651430907 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f13c68060 con 0x563f13c6ae40 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f089b200640 1 --2- 172.21.15.107:0/1651430907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 0x563f13c6d8a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0894000f80 tx=0x7f0894031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ff37a865416adfe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f088aa00640 1 -- 172.21.15.107:0/1651430907 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089404a040 con 0x563f13c6ae40 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f088aa00640 1 -- 172.21.15.107:0/1651430907 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0894036070 con 0x563f13c6ae40 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.019+0000 7f088aa00640 1 -- 172.21.15.107:0/1651430907 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894053650 con 0x563f13c6ae40 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.020+0000 7f08a3ec7740 1 -- 172.21.15.107:0/1651430907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 msgr2=0x563f13c6d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.020+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/1651430907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 0x563f13c6d8a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0894000f80 tx=0x7f0894031a20 comp rx=0 tx=0).stop 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.020+0000 7f08a3ec7740 1 -- 172.21.15.107:0/1651430907 shutdown_connections 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.020+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/1651430907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c76020 0x563f13c8dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.020+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/1651430907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c6b760 0x563f13c74e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.020+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/1651430907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 0x563f13c6d8a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.020+0000 7f08a3ec7740 1 -- 172.21.15.107:0/1651430907 >> 172.21.15.107:0/1651430907 conn(0x563f13c673d0 msgr2=0x563f13c74770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.020+0000 7f08a3ec7740 1 -- 172.21.15.107:0/1651430907 shutdown_connections 2024-09-19T04:24:28.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.020+0000 7f08a3ec7740 1 -- 172.21.15.107:0/1651430907 wait complete. 2024-09-19T04:24:28.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.021+0000 7f08a3ec7740 1 -- start start 2024-09-19T04:24:28.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.021+0000 7f08a3ec7740 1 Processor -- start 2024-09-19T04:24:28.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.021+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 0x563f13c7de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.021+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c7e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.021+0000 7f08a3ec7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c76020 0x563f13c7ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.021+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1b0bc0 con 0x563f13c76020 2024-09-19T04:24:28.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.021+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1b04c0 con 0x563f13c6b760 2024-09-19T04:24:28.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.021+0000 7f08a3ec7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1ad720 con 0x563f13c6ae40 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.021+0000 7f08a3ec7740 10 client.0 _ll_register_callbacks cb 0x7f0884009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f089b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 0x563f13c7de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f089b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c76020 0x563f13c7ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f089b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c76020 0x563f13c7ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37784/0 (socket says 172.21.15.107:37784) 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f0891e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c7e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f089b200640 1 -- 172.21.15.107:0/2266369008 learned_addr learned my addr 172.21.15.107:0/2266369008 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f0891e00640 1 -- 172.21.15.107:0/2266369008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 msgr2=0x563f13c7de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f0891e00640 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 0x563f13c7de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f0891e00640 1 -- 172.21.15.107:0/2266369008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c76020 msgr2=0x563f13c7ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f0891e00640 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c76020 0x563f13c7ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f0891e00640 1 -- 172.21.15.107:0/2266369008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f13c7b350 con 0x563f13c6b760 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f089b200640 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c76020 0x563f13c7ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f0891e00640 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c7e380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0884025a80 tx=0x7f0884005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3596097c443a2dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.022+0000 7f087fe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f087fe00640 1 -- 172.21.15.107:0/2266369008 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884025640 con 0x563f13c6b760 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f087fe00640 1 -- 172.21.15.107:0/2266369008 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f08840257e0 con 0x563f13c6b760 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2266369008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0894036070 con 0x563f13c6b760 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f087fe00640 1 -- 172.21.15.107:0/2266369008 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f08840104a0 con 0x563f13c6b760 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f08a3ec7740 10 client.25967 fetch_fsmap learned FSMap version 31 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2266369008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563f13cf13a0 con 0x563f13c6b760 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f087fe00640 1 -- 172.21.15.107:0/2266369008 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0894036070 con 0x563f13c6b760 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f087fe00640 1 -- 172.21.15.107:0/2266369008 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f088401c420 con 0x563f13c6b760 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f08a3ec7740 10 client.25967 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:24:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f08a3ec7740 10 client.25967 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f08a3ec7740 20 client.25967 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x563f13cf1b40 0x563f13cf3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f08a3ec7740 4 client.25967 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.023+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2266369008 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f13bd8e60 con 0x563f13cf1b40 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f089b200640 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x563f13cf1b40 0x563f13cf3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f089b200640 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x563f13cf1b40 0x563f13cf3f60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b886235bfae1170f server_cookie=71afa423bdc59b44 in_seq=0 out_seq=0 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f087fe00640 10 client.25967 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f087fe00640 1 -- 172.21.15.107:0/2266369008 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f13bd8e60 con 0x563f13cf1b40 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f087fe00640 10 client.25967 handle_command_reply: tid=0 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f08a3ec7740 1 client.25967 shutdown 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2266369008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 msgr2=0x563f13c7e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c7e380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0884025a80 tx=0x7f0884005700 comp rx=0 tx=0).stop 2024-09-19T04:24:28.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2266369008 shutdown_connections 2024-09-19T04:24:28.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x563f13cf1b40 0x563f13cf3f60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f13c76020 0x563f13c7ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f13c6b760 0x563f13c7e380 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f08a3ec7740 1 --2- 172.21.15.107:0/2266369008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f13c6ae40 0x563f13c7de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.024+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2266369008 >> 172.21.15.107:0/2266369008 conn(0x563f13c673d0 msgr2=0x563f13c74f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:28.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f087fe00640 0 client.25967 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:24:28.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2266369008 shutdown_connections 2024-09-19T04:24:28.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a3ec7740 1 -- 172.21.15.107:0/2266369008 wait complete. 2024-09-19T04:24:28.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a3ec7740 20 client.25967 trim_cache size 0 max 16384 2024-09-19T04:24:28.032 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:24:28.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a0a00640 1 -- 172.21.15.107:0/3875521113 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f08800a3430 msgr2=0x7f08800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:28.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a0a00640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f08800a3430 0x7f08800a58f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x563f13c68230 tx=0x7f088c0d36f0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a0a00640 1 -- 172.21.15.107:0/3875521113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103e30 msgr2=0x7f089c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:28.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a0a00640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103e30 0x7f089c19a400 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f088c00e9f0 tx=0x7f088c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a0a00640 1 -- 172.21.15.107:0/3875521113 shutdown_connections 2024-09-19T04:24:28.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a0a00640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f08800a3430 0x7f08800a58f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a0a00640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f089c108f60 0x7f089c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a0a00640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c1047e0 0x7f089c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a0a00640 1 --2- 172.21.15.107:0/3875521113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103e30 0x7f089c19a400 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:28.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.025+0000 7f08a0a00640 1 -- 172.21.15.107:0/3875521113 >> 172.21.15.107:0/3875521113 conn(0x7f089c0ff8d0 msgr2=0x7f089c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:28.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.026+0000 7f08a0a00640 1 -- 172.21.15.107:0/3875521113 shutdown_connections 2024-09-19T04:24:28.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:28.026+0000 7f08a0a00640 1 -- 172.21.15.107:0/3875521113 wait complete. 2024-09-19T04:24:28.658 INFO:tasks.workunit.client.0.smithi107.stdout: 1 153217 2.91 MB/sec execute 46 sec latency 2674.428 ms 2024-09-19T04:24:28.835 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:24:28.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:28 smithi107 ceph-mon[30659]: pgmap v127: 97 pgs: 97 active+clean; 97 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 828 KiB/s wr, 74 op/s 2024-09-19T04:24:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:28 smithi115 ceph-mon[35477]: pgmap v127: 97 pgs: 97 active+clean; 97 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 828 KiB/s wr, 74 op/s 2024-09-19T04:24:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:28 smithi145 ceph-mon[35096]: pgmap v127: 97 pgs: 97 active+clean; 97 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 828 KiB/s wr, 74 op/s 2024-09-19T04:24:29.265 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:29.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 153986 2.95 MB/sec execute 47 sec latency 3233.572 ms 2024-09-19T04:24:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.887+0000 7f61c7e00640 1 -- 172.21.15.107:0/2119140555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c810e210 msgr2=0x7f61c8110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.887+0000 7f61c7e00640 1 --2- 172.21.15.107:0/2119140555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c810e210 0x7f61c8110660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009ca0 tx=0x7f61c002a950 comp rx=0 tx=0).stop 2024-09-19T04:24:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.887+0000 7f61c7e00640 1 -- 172.21.15.107:0/2119140555 shutdown_connections 2024-09-19T04:24:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.887+0000 7f61c7e00640 1 --2- 172.21.15.107:0/2119140555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c810e210 0x7f61c8110660 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.887+0000 7f61c7e00640 1 --2- 172.21.15.107:0/2119140555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c8074f40 0x7f61c810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.887+0000 7f61c7e00640 1 --2- 172.21.15.107:0/2119140555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c8074620 0x7f61c8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.887+0000 7f61c7e00640 1 -- 172.21.15.107:0/2119140555 >> 172.21.15.107:0/2119140555 conn(0x7f61c80fd710 msgr2=0x7f61c80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.887+0000 7f61c7e00640 1 -- 172.21.15.107:0/2119140555 shutdown_connections 2024-09-19T04:24:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.887+0000 7f61c7e00640 1 -- 172.21.15.107:0/2119140555 wait complete. 2024-09-19T04:24:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.888+0000 7f61c7e00640 1 Processor -- start 2024-09-19T04:24:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.888+0000 7f61c7e00640 1 -- start start 2024-09-19T04:24:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c8074620 0x7f61c819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c8074f40 0x7f61c819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c810e210 0x7f61c819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c7e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61c8113f80 con 0x7f61c810e210 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c7e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61c8113e00 con 0x7f61c8074620 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c7e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f61c8114100 con 0x7f61c8074f40 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c8074620 0x7f61c819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c8074f40 0x7f61c819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c8074620 0x7f61c819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59664/0 (socket says 172.21.15.107:59664) 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c6a00640 1 -- 172.21.15.107:0/652200671 learned_addr learned my addr 172.21.15.107:0/652200671 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c7400640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c810e210 0x7f61c819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c6a00640 1 -- 172.21.15.107:0/652200671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c8074f40 msgr2=0x7f61c819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c6a00640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c8074f40 0x7f61c819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c6a00640 1 -- 172.21.15.107:0/652200671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c810e210 msgr2=0x7f61c819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c6a00640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c810e210 0x7f61c819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.889+0000 7f61c6a00640 1 -- 172.21.15.107:0/652200671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c819f470 con 0x7f61c8074620 2024-09-19T04:24:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.890+0000 7f61c6000640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c8074f40 0x7f61c819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.890+0000 7f61c6a00640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c8074620 0x7f61c819a4c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f61b800e9f0 tx=0x7f61b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:29.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.890+0000 7f61bf400640 1 -- 172.21.15.107:0/652200671 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b800cd80 con 0x7f61c8074620 2024-09-19T04:24:29.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.890+0000 7f61bf400640 1 -- 172.21.15.107:0/652200671 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b8010910 con 0x7f61c8074620 2024-09-19T04:24:29.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.890+0000 7f61bf400640 1 -- 172.21.15.107:0/652200671 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8018990 con 0x7f61c8074620 2024-09-19T04:24:29.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.890+0000 7f61c7e00640 1 -- 172.21.15.107:0/652200671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c819f760 con 0x7f61c8074620 2024-09-19T04:24:29.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.890+0000 7f61c7e00640 1 -- 172.21.15.107:0/652200671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c81a7040 con 0x7f61c8074620 2024-09-19T04:24:29.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.894+0000 7f61d05fc740 1 Processor -- start 2024-09-19T04:24:29.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.894+0000 7f61d05fc740 1 -- start start 2024-09-19T04:24:29.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.894+0000 7f61bf400640 1 -- 172.21.15.107:0/652200671 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f61b8010ac0 con 0x7f61c8074620 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.894+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf12b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.894+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf135ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.894+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 0x556aaf138410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.894+0000 7f61d05fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556aaf0913e0 con 0x556aaf12ae40 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.894+0000 7f61d05fc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556aaf038d80 con 0x556aaf12b760 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.894+0000 7f61d05fc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556aaf08d3a0 con 0x556aaf136020 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.895+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf135ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.895+0000 7f61c6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 0x556aaf138410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.895+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf135ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59666/0 (socket says 172.21.15.107:59666) 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.895+0000 7f61c6a00640 1 -- 172.21.15.107:0/1101165030 learned_addr learned my addr 172.21.15.107:0/1101165030 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.895+0000 7f61c7400640 1 --2- 172.21.15.107:0/1101165030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf12b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.895+0000 7f61c6000640 1 -- 172.21.15.107:0/1101165030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 msgr2=0x556aaf135ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:29.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.895+0000 7f61c6000640 1 --2- 172.21.15.107:0/1101165030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf135ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.895+0000 7f61c6000640 1 -- 172.21.15.107:0/1101165030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 msgr2=0x556aaf12b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:29.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.895+0000 7f61c6000640 1 --2- 172.21.15.107:0/1101165030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf12b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.895+0000 7f61c6000640 1 -- 172.21.15.107:0/1101165030 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aaf057ae0 con 0x556aaf136020 2024-09-19T04:24:29.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.896+0000 7f61c6000640 1 --2- 172.21.15.107:0/1101165030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 0x556aaf138410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009f20 tx=0x7f61b40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e258ecf6ef1e43e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:29.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.896+0000 7f61b3e00640 1 -- 172.21.15.107:0/1101165030 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4004030 con 0x556aaf136020 2024-09-19T04:24:29.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.896+0000 7f61b3e00640 1 -- 172.21.15.107:0/1101165030 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b4005500 con 0x556aaf136020 2024-09-19T04:24:29.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.896+0000 7f61bf400640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f61ac0a3430 0x7f61ac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.896+0000 7f61b3e00640 1 -- 172.21.15.107:0/1101165030 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4004e20 con 0x556aaf136020 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.896+0000 7f61bf400640 1 -- 172.21.15.107:0/652200671 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f61b80cffe0 con 0x7f61c8074620 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.896+0000 7f61c7400640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f61ac0a3430 0x7f61ac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61d05fc740 1 -- 172.21.15.107:0/1101165030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 msgr2=0x556aaf138410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61d05fc740 1 --2- 172.21.15.107:0/1101165030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 0x556aaf138410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009f20 tx=0x7f61b40118d0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61d05fc740 1 -- 172.21.15.107:0/1101165030 shutdown_connections 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61d05fc740 1 --2- 172.21.15.107:0/1101165030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 0x556aaf138410 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61d05fc740 1 --2- 172.21.15.107:0/1101165030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf135ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61d05fc740 1 --2- 172.21.15.107:0/1101165030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf12b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61d05fc740 1 -- 172.21.15.107:0/1101165030 >> 172.21.15.107:0/1101165030 conn(0x556aaf1273d0 msgr2=0x556aaf1277b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61c7400640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f61ac0a3430 0x7f61ac0a58f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61c0005e30 tx=0x7f61c0005c50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61d05fc740 1 -- 172.21.15.107:0/1101165030 shutdown_connections 2024-09-19T04:24:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61d05fc740 1 -- 172.21.15.107:0/1101165030 wait complete. 2024-09-19T04:24:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.897+0000 7f61d05fc740 1 -- start start 2024-09-19T04:24:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.898+0000 7f61d05fc740 1 Processor -- start 2024-09-19T04:24:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.898+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf132e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.898+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf12b760 0x556aaf1333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.898+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf133910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.898+0000 7f61d05fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556aaf138bf0 con 0x556aaf12ae40 2024-09-19T04:24:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.898+0000 7f61d05fc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556aaf04c0f0 con 0x556aaf136020 2024-09-19T04:24:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.898+0000 7f61d05fc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556aaf138d70 con 0x556aaf12b760 2024-09-19T04:24:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.898+0000 7f61d05fc740 10 client.0 _ll_register_callbacks cb 0x556aaf033bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.898+0000 7f61c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf133910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf133910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59678/0 (socket says 172.21.15.107:59678) 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61c7400640 1 -- 172.21.15.107:0/3660555357 learned_addr learned my addr 172.21.15.107:0/3660555357 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf132e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61c6000640 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf12b760 0x556aaf1333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61c7400640 1 -- 172.21.15.107:0/3660555357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf12b760 msgr2=0x556aaf1333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61c7400640 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf12b760 0x556aaf1333d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61c7400640 1 -- 172.21.15.107:0/3660555357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 msgr2=0x556aaf132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61c7400640 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf132e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61c7400640 1 -- 172.21.15.107:0/3660555357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aaee023c0 con 0x556aaf136020 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61c7400640 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf133910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f61c002f650 tx=0x7f61c00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a462b94a0964516 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.899+0000 7f61b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.900+0000 7f61b0c00640 1 -- 172.21.15.107:0/3660555357 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0034040 con 0x556aaf136020 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.900+0000 7f61b0c00640 1 -- 172.21.15.107:0/3660555357 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f61c0004450 con 0x556aaf136020 2024-09-19T04:24:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.900+0000 7f61d05fc740 1 -- 172.21.15.107:0/3660555357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b4005500 con 0x556aaf136020 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.900+0000 7f61b0c00640 1 -- 172.21.15.107:0/3660555357 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f61c002f740 con 0x556aaf136020 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.900+0000 7f61d05fc740 10 client.25979 fetch_fsmap learned FSMap version 31 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.900+0000 7f61d05fc740 1 -- 172.21.15.107:0/3660555357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556aaf144c10 con 0x556aaf136020 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.900+0000 7f61b0c00640 1 -- 172.21.15.107:0/3660555357 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b4005500 con 0x556aaf136020 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.901+0000 7f61b0c00640 1 -- 172.21.15.107:0/3660555357 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f61c0050020 con 0x556aaf136020 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.901+0000 7f61d05fc740 10 client.25979 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.901+0000 7f61d05fc740 10 client.25979 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.901+0000 7f61d05fc740 20 client.25979 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.901+0000 7f61d05fc740 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x556aaf146020 0x556aaf148470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.901+0000 7f61d05fc740 4 client.25979 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.901+0000 7f61d05fc740 1 -- 172.21.15.107:0/3660555357 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aaf09e950 con 0x556aaf146020 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.901+0000 7f61c6a00640 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x556aaf146020 0x556aaf148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.901+0000 7f61c6a00640 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x556aaf146020 0x556aaf148470 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f4c678cc514d02b server_cookie=6f3146d05b7ce412 in_seq=0 out_seq=0 2024-09-19T04:24:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.901+0000 7f61b0c00640 10 client.25979 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:24:29.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61b0c00640 1 -- 172.21.15.107:0/3660555357 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556aaf09e950 con 0x556aaf146020 2024-09-19T04:24:29.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61b0c00640 10 client.25979 handle_command_reply: tid=0 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61d05fc740 1 client.25979 shutdown 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61d05fc740 1 -- 172.21.15.107:0/3660555357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 msgr2=0x556aaf133910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61d05fc740 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf133910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f61c002f650 tx=0x7f61c00095c0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61d05fc740 1 -- 172.21.15.107:0/3660555357 shutdown_connections 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61d05fc740 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x556aaf146020 0x556aaf148470 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61d05fc740 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf133910 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61d05fc740 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf12b760 0x556aaf1333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61d05fc740 1 --2- 172.21.15.107:0/3660555357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf132e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.902+0000 7f61d05fc740 1 -- 172.21.15.107:0/3660555357 >> 172.21.15.107:0/3660555357 conn(0x556aaf1273d0 msgr2=0x556aaf12d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.903+0000 7f61b0c00640 0 client.25979 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.903+0000 7f61d05fc740 1 -- 172.21.15.107:0/3660555357 shutdown_connections 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.903+0000 7f61d05fc740 1 -- 172.21.15.107:0/3660555357 wait complete. 2024-09-19T04:24:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.903+0000 7f61d05fc740 20 client.25979 trim_cache size 0 max 16384 2024-09-19T04:24:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.917+0000 7f61d05fc740 1 Processor -- start 2024-09-19T04:24:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.917+0000 7f61d05fc740 1 -- start start 2024-09-19T04:24:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.917+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf12d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.917+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.917+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 0x556aaf14e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.917+0000 7f61d05fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81af880 con 0x556aaf12ae40 2024-09-19T04:24:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.917+0000 7f61d05fc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81b14b0 con 0x556aaf12b760 2024-09-19T04:24:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.917+0000 7f61d05fc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81aff50 con 0x556aaf136020 2024-09-19T04:24:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c6000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 0x556aaf14e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c6000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf14dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59692/0 (socket says 172.21.15.107:59692) 2024-09-19T04:24:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 0x556aaf14e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35016/0 (socket says 172.21.15.107:35016) 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c6000640 1 -- 172.21.15.107:0/4115262837 learned_addr learned my addr 172.21.15.107:0/4115262837 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c6a00640 1 --2- 172.21.15.107:0/4115262837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf12d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c6000640 1 -- 172.21.15.107:0/4115262837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 msgr2=0x556aaf14e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c6000640 1 --2- 172.21.15.107:0/4115262837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 0x556aaf14e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c6000640 1 -- 172.21.15.107:0/4115262837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 msgr2=0x556aaf12d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c6000640 1 --2- 172.21.15.107:0/4115262837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf12d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c6000640 1 -- 172.21.15.107:0/4115262837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aaf12ac70 con 0x556aaf12b760 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.918+0000 7f61c6000640 1 --2- 172.21.15.107:0/4115262837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf14dd40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f61b4004990 tx=0x7f61b40049c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c3cfdffbfa63dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.919+0000 7f61b3400640 1 -- 172.21.15.107:0/4115262837 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4004180 con 0x556aaf12b760 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.919+0000 7f61b3400640 1 -- 172.21.15.107:0/4115262837 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b40035f0 con 0x556aaf12b760 2024-09-19T04:24:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.919+0000 7f61b3400640 1 -- 172.21.15.107:0/4115262837 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b400f780 con 0x556aaf12b760 2024-09-19T04:24:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.919+0000 7f61d05fc740 1 -- 172.21.15.107:0/4115262837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 msgr2=0x556aaf14dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.919+0000 7f61d05fc740 1 --2- 172.21.15.107:0/4115262837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf14dd40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f61b4004990 tx=0x7f61b40049c0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.919+0000 7f61d05fc740 1 -- 172.21.15.107:0/4115262837 shutdown_connections 2024-09-19T04:24:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.919+0000 7f61d05fc740 1 --2- 172.21.15.107:0/4115262837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf136020 0x556aaf14e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.919+0000 7f61d05fc740 1 --2- 172.21.15.107:0/4115262837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf12b760 0x556aaf14dd40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.919+0000 7f61d05fc740 1 --2- 172.21.15.107:0/4115262837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf12d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.919+0000 7f61d05fc740 1 -- 172.21.15.107:0/4115262837 >> 172.21.15.107:0/4115262837 conn(0x556aaf1273d0 msgr2=0x556aaf134f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.920+0000 7f61d05fc740 1 -- 172.21.15.107:0/4115262837 shutdown_connections 2024-09-19T04:24:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.920+0000 7f61d05fc740 1 -- 172.21.15.107:0/4115262837 wait complete. 2024-09-19T04:24:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.920+0000 7f61d05fc740 1 -- start start 2024-09-19T04:24:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.920+0000 7f61d05fc740 1 Processor -- start 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.920+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf139d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf12b760 0x556aaf13a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61d05fc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf13a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61d05fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81b0d20 con 0x556aaf12ae40 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61d05fc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81b0620 con 0x556aaf136020 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61d05fc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81ad860 con 0x556aaf12b760 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61d05fc740 10 client.0 _ll_register_callbacks cb 0x7f61b8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf13a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf13a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59704/0 (socket says 172.21.15.107:59704) 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf12b760 0x556aaf13a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c7400640 1 -- 172.21.15.107:0/1292389040 learned_addr learned my addr 172.21.15.107:0/1292389040 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c7400640 1 -- 172.21.15.107:0/1292389040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf12b760 msgr2=0x556aaf13a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c7400640 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf12b760 0x556aaf13a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c7400640 1 -- 172.21.15.107:0/1292389040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 msgr2=0x556aaf139d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c7400640 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf139d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c7400640 1 -- 172.21.15.107:0/1292389040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aaf1afd10 con 0x556aaf136020 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61c7400640 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf13a7d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f61c002f8e0 tx=0x7f61c0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f959adb0f9061a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.921+0000 7f61b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61b0c00640 1 -- 172.21.15.107:0/1292389040 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0034070 con 0x556aaf136020 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61d05fc740 1 -- 172.21.15.107:0/1292389040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b40035f0 con 0x556aaf136020 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61b0c00640 1 -- 172.21.15.107:0/1292389040 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f61c0052650 con 0x556aaf136020 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61b0c00640 1 -- 172.21.15.107:0/1292389040 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f61c0003d40 con 0x556aaf136020 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61d05fc740 10 client.25985 fetch_fsmap learned FSMap version 31 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61d05fc740 1 -- 172.21.15.107:0/1292389040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556aaf1b0f70 con 0x556aaf136020 2024-09-19T04:24:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61b0c00640 1 -- 172.21.15.107:0/1292389040 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b40035f0 con 0x556aaf136020 2024-09-19T04:24:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61b0c00640 1 -- 172.21.15.107:0/1292389040 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f61c0050040 con 0x556aaf136020 2024-09-19T04:24:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61d05fc740 10 client.25985 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:24:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61d05fc740 10 client.25985 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:24:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61d05fc740 20 client.25985 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.922+0000 7f61d05fc740 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x556aaf1b16c0 0x556aaf1b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.923+0000 7f61d05fc740 4 client.25985 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:24:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.923+0000 7f61d05fc740 1 -- 172.21.15.107:0/1292389040 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556aaf098e60 con 0x556aaf1b16c0 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.923+0000 7f61c6a00640 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x556aaf1b16c0 0x556aaf1b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.923+0000 7f61c6a00640 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x556aaf1b16c0 0x556aaf1b3ae0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35d3bcc6e8cf2109 server_cookie=313450927e5af79e in_seq=0 out_seq=0 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.923+0000 7f61b0c00640 10 client.25985 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.923+0000 7f61b0c00640 1 -- 172.21.15.107:0/1292389040 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556aaf098e60 con 0x556aaf1b16c0 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.923+0000 7f61b0c00640 10 client.25985 handle_command_reply: tid=0 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.923+0000 7f61d05fc740 1 client.25985 shutdown 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 1 -- 172.21.15.107:0/1292389040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 msgr2=0x556aaf13a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf13a7d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f61c002f8e0 tx=0x7f61c0004a40 comp rx=0 tx=0).stop 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 1 -- 172.21.15.107:0/1292389040 shutdown_connections 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x556aaf1b16c0 0x556aaf1b3ae0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aaf136020 0x556aaf13a7d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aaf12b760 0x556aaf13a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 1 --2- 172.21.15.107:0/1292389040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556aaf12ae40 0x556aaf139d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 1 -- 172.21.15.107:0/1292389040 >> 172.21.15.107:0/1292389040 conn(0x556aaf1273d0 msgr2=0x556aaf132e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61b0c00640 0 client.25985 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:24:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 1 -- 172.21.15.107:0/1292389040 shutdown_connections 2024-09-19T04:24:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 1 -- 172.21.15.107:0/1292389040 wait complete. 2024-09-19T04:24:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.924+0000 7f61d05fc740 20 client.25985 trim_cache size 0 max 16384 2024-09-19T04:24:29.930 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:24:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 -- 172.21.15.107:0/652200671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f61ac0a3430 msgr2=0x7f61ac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f61ac0a3430 0x7f61ac0a58f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61c0005e30 tx=0x7f61c0005c50 comp rx=0 tx=0).stop 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 -- 172.21.15.107:0/652200671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c8074620 msgr2=0x7f61c819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c8074620 0x7f61c819a4c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f61b800e9f0 tx=0x7f61b800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 -- 172.21.15.107:0/652200671 shutdown_connections 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f61ac0a3430 0x7f61ac0a58f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c810e210 0x7f61c819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c8074f40 0x7f61c819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 --2- 172.21.15.107:0/652200671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c8074620 0x7f61c819a4c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 -- 172.21.15.107:0/652200671 >> 172.21.15.107:0/652200671 conn(0x7f61c80fd710 msgr2=0x7f61c80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 -- 172.21.15.107:0/652200671 shutdown_connections 2024-09-19T04:24:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:29.925+0000 7f61c7e00640 1 -- 172.21.15.107:0/652200671 wait complete. 2024-09-19T04:24:30.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 156353 3.17 MB/sec execute 48 sec latency 10.909 ms 2024-09-19T04:24:30.772 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:24:31.163 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:30 smithi115 ceph-mon[35477]: pgmap v128: 97 pgs: 97 active+clean; 97 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 828 KiB/s wr, 74 op/s 2024-09-19T04:24:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:30 smithi145 ceph-mon[35096]: pgmap v128: 97 pgs: 97 active+clean; 97 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 828 KiB/s wr, 74 op/s 2024-09-19T04:24:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:30 smithi107 ceph-mon[30659]: pgmap v128: 97 pgs: 97 active+clean; 97 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 828 KiB/s wr, 74 op/s 2024-09-19T04:24:31.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 156815 3.12 MB/sec execute 49 sec latency 508.253 ms 2024-09-19T04:24:31.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.967+0000 7f5d75e00640 1 -- 172.21.15.107:0/1861002100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d70101de0 msgr2=0x7f5d701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.967+0000 7f5d75e00640 1 --2- 172.21.15.107:0/1861002100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d70101de0 0x7f5d701021c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009900 tx=0x7f5d6002a410 comp rx=0 tx=0).stop 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.967+0000 7f5d75e00640 1 -- 172.21.15.107:0/1861002100 shutdown_connections 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.967+0000 7f5d75e00640 1 --2- 172.21.15.107:0/1861002100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d70106f10 0x7f5d7010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.967+0000 7f5d75e00640 1 --2- 172.21.15.107:0/1861002100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d70102790 0x7f5d701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.967+0000 7f5d75e00640 1 --2- 172.21.15.107:0/1861002100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d70101de0 0x7f5d701021c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.967+0000 7f5d75e00640 1 -- 172.21.15.107:0/1861002100 >> 172.21.15.107:0/1861002100 conn(0x7f5d700fd8c0 msgr2=0x7f5d700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.967+0000 7f5d75e00640 1 -- 172.21.15.107:0/1861002100 shutdown_connections 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.967+0000 7f5d75e00640 1 -- 172.21.15.107:0/1861002100 wait complete. 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.968+0000 7f5d75e00640 1 Processor -- start 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.968+0000 7f5d75e00640 1 -- start start 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.968+0000 7f5d75e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d70101de0 0x7f5d7019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.968+0000 7f5d75e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d70102790 0x7f5d7019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.968+0000 7f5d75e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d70106f10 0x7f5d7019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.968+0000 7f5d75e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70111f70 con 0x7f5d70106f10 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.968+0000 7f5d75e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70111df0 con 0x7f5d70101de0 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.968+0000 7f5d75e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701120f0 con 0x7f5d70102790 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d6ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d70101de0 0x7f5d7019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d6ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d70101de0 0x7f5d7019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59724/0 (socket says 172.21.15.107:59724) 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d6ea00640 1 -- 172.21.15.107:0/3908592527 learned_addr learned my addr 172.21.15.107:0/3908592527 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d66000640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d70102790 0x7f5d7019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d6f400640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d70106f10 0x7f5d7019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d6ea00640 1 -- 172.21.15.107:0/3908592527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d70102790 msgr2=0x7f5d7019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d70102790 0x7f5d7019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d6ea00640 1 -- 172.21.15.107:0/3908592527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d70106f10 msgr2=0x7f5d7019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d70106f10 0x7f5d7019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d6ea00640 1 -- 172.21.15.107:0/3908592527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7019f5f0 con 0x7f5d70101de0 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d66000640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d70102790 0x7f5d7019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.969+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d70101de0 0x7f5d7019a640 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5d60006010 tx=0x7f5d60004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.970+0000 7f5d74a00640 1 -- 172.21.15.107:0/3908592527 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6003f030 con 0x7f5d70101de0 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.970+0000 7f5d74a00640 1 -- 172.21.15.107:0/3908592527 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d60004440 con 0x7f5d70101de0 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.970+0000 7f5d74a00640 1 -- 172.21.15.107:0/3908592527 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60035070 con 0x7f5d70101de0 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.970+0000 7f5d75e00640 1 -- 172.21.15.107:0/3908592527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7019f880 con 0x7f5d70101de0 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.970+0000 7f5d75e00640 1 -- 172.21.15.107:0/3908592527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7019fd10 con 0x7f5d70101de0 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d78890740 1 Processor -- start 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d78890740 1 -- start start 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d78890740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03717c70 0x556e03718050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d78890740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03718590 0x556e03722910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d78890740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03725240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d78890740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556e0367e3e0 con 0x556e03717c70 2024-09-19T04:24:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d78890740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556e03625d80 con 0x556e03718590 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d78890740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556e0367a3a0 con 0x556e03722e50 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d74a00640 1 -- 172.21.15.107:0/3908592527 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f5d60042030 con 0x7f5d70101de0 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d6f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03717c70 0x556e03718050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d6f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03717c70 0x556e03718050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46472/0 (socket says 172.21.15.107:46472) 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d6f400640 1 -- 172.21.15.107:0/3765328902 learned_addr learned my addr 172.21.15.107:0/3765328902 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.973+0000 7f5d66000640 1 --2- 172.21.15.107:0/3765328902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03725240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.974+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/3765328902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03718590 0x556e03722910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.974+0000 7f5d66000640 1 -- 172.21.15.107:0/3765328902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03718590 msgr2=0x556e03722910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.974+0000 7f5d66000640 1 --2- 172.21.15.107:0/3765328902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03718590 0x556e03722910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.974+0000 7f5d66000640 1 -- 172.21.15.107:0/3765328902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03717c70 msgr2=0x556e03718050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.974+0000 7f5d66000640 1 --2- 172.21.15.107:0/3765328902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03717c70 0x556e03718050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.974+0000 7f5d66000640 1 -- 172.21.15.107:0/3765328902 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e03644ae0 con 0x556e03722e50 2024-09-19T04:24:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.974+0000 7f5d66000640 1 --2- 172.21.15.107:0/3765328902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03725240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009fa0 tx=0x7f5d580118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9380779560872dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.974+0000 7f5d5fe00640 1 -- 172.21.15.107:0/3765328902 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58004030 con 0x556e03722e50 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d5fe00640 1 -- 172.21.15.107:0/3765328902 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d58005500 con 0x556e03722e50 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d5fe00640 1 -- 172.21.15.107:0/3765328902 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58004e10 con 0x556e03722e50 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d74a00640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d540a3430 0x7f5d540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d74a00640 1 -- 172.21.15.107:0/3908592527 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f5d600eff90 con 0x7f5d70101de0 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d6f400640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d540a3430 0x7f5d540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d78890740 1 -- 172.21.15.107:0/3765328902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 msgr2=0x556e03725240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d78890740 1 --2- 172.21.15.107:0/3765328902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03725240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009fa0 tx=0x7f5d580118d0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d78890740 1 -- 172.21.15.107:0/3765328902 shutdown_connections 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d78890740 1 --2- 172.21.15.107:0/3765328902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03725240 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d78890740 1 --2- 172.21.15.107:0/3765328902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03718590 0x556e03722910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d78890740 1 --2- 172.21.15.107:0/3765328902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03717c70 0x556e03718050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.975+0000 7f5d78890740 1 -- 172.21.15.107:0/3765328902 >> 172.21.15.107:0/3765328902 conn(0x556e03714200 msgr2=0x556e037145e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.976+0000 7f5d78890740 1 -- 172.21.15.107:0/3765328902 shutdown_connections 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.976+0000 7f5d78890740 1 -- 172.21.15.107:0/3765328902 wait complete. 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.976+0000 7f5d6f400640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d540a3430 0x7f5d540a58f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d68011630 tx=0x7f5d68009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.976+0000 7f5d78890740 1 -- start start 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.976+0000 7f5d78890740 1 Processor -- start 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d78890740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d78890740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e03720220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d78890740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03720760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d78890740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556e03725a20 con 0x556e03718590 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d78890740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556e036390f0 con 0x556e03717c70 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d78890740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556e03725ba0 con 0x556e03722e50 2024-09-19T04:24:31.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d78890740 10 client.0 _ll_register_callbacks cb 0x556e03620bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e03720220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d6f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03720760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d6f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03720760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35070/0 (socket says 172.21.15.107:35070) 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d6f400640 1 -- 172.21.15.107:0/3457292928 learned_addr learned my addr 172.21.15.107:0/3457292928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.977+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d6f400640 1 -- 172.21.15.107:0/3457292928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 msgr2=0x556e0371fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d6f400640 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d6f400640 1 -- 172.21.15.107:0/3457292928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 msgr2=0x556e03720220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d6f400640 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e03720220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d6f400640 1 -- 172.21.15.107:0/3457292928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e033ef3c0 con 0x556e03722e50 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d6f400640 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03720760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5d68003f40 tx=0x7f5d68004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3b4654281f9f81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:24:31.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d5cc00640 1 -- 172.21.15.107:0/3457292928 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6800f910 con 0x556e03722e50 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d78890740 1 -- 172.21.15.107:0/3457292928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d58005500 con 0x556e03722e50 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d5cc00640 1 -- 172.21.15.107:0/3457292928 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f5d6800fab0 con 0x556e03722e50 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d5cc00640 1 -- 172.21.15.107:0/3457292928 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5d6800fcb0 con 0x556e03722e50 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d78890740 10 client.25966 fetch_fsmap learned FSMap version 31 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.978+0000 7f5d78890740 1 -- 172.21.15.107:0/3457292928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556e03731aa0 con 0x556e03722e50 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.979+0000 7f5d5cc00640 1 -- 172.21.15.107:0/3457292928 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d58005500 con 0x556e03722e50 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.979+0000 7f5d5cc00640 1 -- 172.21.15.107:0/3457292928 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5d68026070 con 0x556e03722e50 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.979+0000 7f5d78890740 10 client.25966 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.979+0000 7f5d78890740 10 client.25966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.979+0000 7f5d78890740 20 client.25966 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:31.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.979+0000 7f5d78890740 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556e03732f00 0x556e03735350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.979+0000 7f5d78890740 4 client.25966 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.979+0000 7f5d78890740 1 -- 172.21.15.107:0/3457292928 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e0368b950 con 0x556e03732f00 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.979+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556e03732f00 0x556e03735350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.980+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556e03732f00 0x556e03735350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2afac0ddbd2dc7e9 server_cookie=164cea2fce16733 in_seq=0 out_seq=0 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.980+0000 7f5d5cc00640 10 client.25966 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.980+0000 7f5d5cc00640 1 -- 172.21.15.107:0/3457292928 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5d600cbe80 con 0x556e03732f00 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.980+0000 7f5d5cc00640 10 client.25966 handle_command_reply: tid=0 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.980+0000 7f5d78890740 1 client.25966 shutdown 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 1 -- 172.21.15.107:0/3457292928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 msgr2=0x556e03720760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03720760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5d68003f40 tx=0x7f5d68004a40 comp rx=0 tx=0).stop 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 1 -- 172.21.15.107:0/3457292928 shutdown_connections 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556e03732f00 0x556e03735350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e03720760 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e03720220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 1 --2- 172.21.15.107:0/3457292928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:31.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d5cc00640 0 client.25966 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:24:31.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 1 -- 172.21.15.107:0/3457292928 >> 172.21.15.107:0/3457292928 conn(0x556e03714200 msgr2=0x556e03719f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:31.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 1 -- 172.21.15.107:0/3457292928 shutdown_connections 2024-09-19T04:24:31.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 1 -- 172.21.15.107:0/3457292928 wait complete. 2024-09-19T04:24:31.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.981+0000 7f5d78890740 20 client.25966 trim_cache size 0 max 16384 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d78890740 1 Processor -- start 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d78890740 1 -- start start 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d78890740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d78890740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0373ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d78890740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e0373b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d78890740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701af9d0 con 0x556e03718590 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d78890740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b1600 con 0x556e03717c70 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d78890740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b00a0 con 0x556e03722e50 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0373ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d6ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0373ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46492/0 (socket says 172.21.15.107:46492) 2024-09-19T04:24:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d66000640 1 -- 172.21.15.107:0/966303184 learned_addr learned my addr 172.21.15.107:0/966303184 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d6ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59768/0 (socket says 172.21.15.107:59768) 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d6f400640 1 --2- 172.21.15.107:0/966303184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e0373b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d6ea00640 1 -- 172.21.15.107:0/966303184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 msgr2=0x556e0373b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/966303184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e0373b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d6ea00640 1 -- 172.21.15.107:0/966303184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 msgr2=0x556e0373ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/966303184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0373ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d6ea00640 1 -- 172.21.15.107:0/966303184 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e0370f770 con 0x556e03717c70 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/966303184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371a720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5d600ed730 tx=0x7f5d600f24a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0c44e35acc7522f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d5f400640 1 -- 172.21.15.107:0/966303184 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60049040 con 0x556e03717c70 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d5f400640 1 -- 172.21.15.107:0/966303184 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556e0368b950 con 0x556e03717c70 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.996+0000 7f5d5f400640 1 -- 172.21.15.107:0/966303184 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d600ba700 con 0x556e03717c70 2024-09-19T04:24:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.997+0000 7f5d78890740 1 -- 172.21.15.107:0/966303184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 msgr2=0x556e0371a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.997+0000 7f5d78890740 1 --2- 172.21.15.107:0/966303184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371a720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5d600ed730 tx=0x7f5d600f24a0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.997+0000 7f5d78890740 1 -- 172.21.15.107:0/966303184 shutdown_connections 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.997+0000 7f5d78890740 1 --2- 172.21.15.107:0/966303184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e0373b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.997+0000 7f5d78890740 1 --2- 172.21.15.107:0/966303184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0373ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.997+0000 7f5d78890740 1 --2- 172.21.15.107:0/966303184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371a720 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.997+0000 7f5d78890740 1 -- 172.21.15.107:0/966303184 >> 172.21.15.107:0/966303184 conn(0x556e03714200 msgr2=0x556e03721d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.997+0000 7f5d78890740 1 -- 172.21.15.107:0/966303184 shutdown_connections 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.997+0000 7f5d78890740 1 -- 172.21.15.107:0/966303184 wait complete. 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.998+0000 7f5d78890740 1 -- start start 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.998+0000 7f5d78890740 1 Processor -- start 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d78890740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d78890740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0371d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d78890740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e0371d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d78890740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b0e70 con 0x556e03718590 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d78890740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b0770 con 0x556e03717c70 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d78890740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701ad9b0 con 0x556e03722e50 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d78890740 10 client.0 _ll_register_callbacks cb 0x7f5d68009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0371d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0371d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46508/0 (socket says 172.21.15.107:46508) 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d66000640 1 -- 172.21.15.107:0/2423120177 learned_addr learned my addr 172.21.15.107:0/2423120177 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d6f400640 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e0371d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d66000640 1 -- 172.21.15.107:0/2423120177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 msgr2=0x556e0371d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d66000640 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e0371d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d66000640 1 -- 172.21.15.107:0/2423120177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 msgr2=0x556e0371cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d66000640 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d66000640 1 -- 172.21.15.107:0/2423120177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e0379cca0 con 0x556e03718590 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:31.999+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.000+0000 7f5d66000640 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0371d310 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800a010 tx=0x7f5d58004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e835cf32aa7eb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.000+0000 7f5d5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.000+0000 7f5d5cc00640 1 -- 172.21.15.107:0/2423120177 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5801d040 con 0x556e03718590 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.000+0000 7f5d5cc00640 1 -- 172.21.15.107:0/2423120177 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f5d580035f0 con 0x556e03718590 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.000+0000 7f5d78890740 1 -- 172.21.15.107:0/2423120177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e0368b950 con 0x556e03718590 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.000+0000 7f5d5cc00640 1 -- 172.21.15.107:0/2423120177 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5d580037f0 con 0x556e03718590 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.000+0000 7f5d78890740 10 client.15951 fetch_fsmap learned FSMap version 31 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.000+0000 7f5d78890740 1 -- 172.21.15.107:0/2423120177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556e0379df80 con 0x556e03718590 2024-09-19T04:24:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.001+0000 7f5d5cc00640 1 -- 172.21.15.107:0/2423120177 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556e0368b950 con 0x556e03718590 2024-09-19T04:24:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.001+0000 7f5d5cc00640 1 -- 172.21.15.107:0/2423120177 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5d5802c030 con 0x556e03718590 2024-09-19T04:24:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.001+0000 7f5d78890740 10 client.15951 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T04:24:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.001+0000 7f5d78890740 10 client.15951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:24:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.001+0000 7f5d78890740 20 client.15951 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.001+0000 7f5d78890740 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556e0379e490 0x556e037a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.001+0000 7f5d78890740 4 client.15951 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:24:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.001+0000 7f5d78890740 1 -- 172.21.15.107:0/2423120177 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e03685e60 con 0x556e0379e490 2024-09-19T04:24:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.001+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556e0379e490 0x556e037a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:32.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d6ea00640 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556e0379e490 0x556e037a08b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a98f327b4a3c43ab server_cookie=b1371d8dee10f804 in_seq=0 out_seq=0 2024-09-19T04:24:32.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d5cc00640 10 client.15951 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:24:32.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d5cc00640 1 -- 172.21.15.107:0/2423120177 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e03685e60 con 0x556e0379e490 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d5cc00640 10 client.15951 handle_command_reply: tid=0 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 client.15951 shutdown 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 -- 172.21.15.107:0/2423120177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 msgr2=0x556e0371d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0371d310 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800a010 tx=0x7f5d58004610 comp rx=0 tx=0).stop 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 -- 172.21.15.107:0/2423120177 shutdown_connections 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556e0379e490 0x556e037a08b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556e03722e50 0x556e0371d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d5cc00640 0 client.15951 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e03718590 0x556e0371d310 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 --2- 172.21.15.107:0/2423120177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e03717c70 0x556e0371cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 -- 172.21.15.107:0/2423120177 >> 172.21.15.107:0/2423120177 conn(0x556e03714200 msgr2=0x556e03712b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 -- 172.21.15.107:0/2423120177 shutdown_connections 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.002+0000 7f5d78890740 1 -- 172.21.15.107:0/2423120177 wait complete. 2024-09-19T04:24:32.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d78890740 20 client.15951 trim_cache size 0 max 16384 2024-09-19T04:24:32.011 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:24:32.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d75e00640 1 -- 172.21.15.107:0/3908592527 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d540a3430 msgr2=0x7f5d540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d75e00640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d540a3430 0x7f5d540a58f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d68011630 tx=0x7f5d68009500 comp rx=0 tx=0).stop 2024-09-19T04:24:32.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d75e00640 1 -- 172.21.15.107:0/3908592527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d70101de0 msgr2=0x7f5d7019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:32.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d75e00640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d70101de0 0x7f5d7019a640 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5d60006010 tx=0x7f5d60004040 comp rx=0 tx=0).stop 2024-09-19T04:24:32.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d75e00640 1 -- 172.21.15.107:0/3908592527 shutdown_connections 2024-09-19T04:24:32.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d75e00640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d540a3430 0x7f5d540a58f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d75e00640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d70106f10 0x7f5d7019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d75e00640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d70102790 0x7f5d7019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d75e00640 1 --2- 172.21.15.107:0/3908592527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d70101de0 0x7f5d7019a640 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:32.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.003+0000 7f5d75e00640 1 -- 172.21.15.107:0/3908592527 >> 172.21.15.107:0/3908592527 conn(0x7f5d700fd8c0 msgr2=0x7f5d700ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:32.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.004+0000 7f5d75e00640 1 -- 172.21.15.107:0/3908592527 shutdown_connections 2024-09-19T04:24:32.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:32.004+0000 7f5d75e00640 1 -- 172.21.15.107:0/3908592527 wait complete. 2024-09-19T04:24:32.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 156815 3.06 MB/sec execute 50 sec latency 1508.356 ms 2024-09-19T04:24:32.789 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:24:33.158 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:32 smithi115 ceph-mon[35477]: pgmap v129: 97 pgs: 97 active+clean; 99 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T04:24:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:32 smithi145 ceph-mon[35096]: pgmap v129: 97 pgs: 97 active+clean; 99 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T04:24:33.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:32 smithi107 ceph-mon[30659]: pgmap v129: 97 pgs: 97 active+clean; 99 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-19T04:24:33.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 156815 3.00 MB/sec execute 51 sec latency 2508.427 ms 2024-09-19T04:24:33.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.711+0000 7f0d79c00640 1 -- 172.21.15.107:0/2710602544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d741049c0 msgr2=0x7f0d74108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:33.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.711+0000 7f0d79c00640 1 --2- 172.21.15.107:0/2710602544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d741049c0 0x7f0d74108a10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009860 tx=0x7f0d6002a340 comp rx=0 tx=0).stop 2024-09-19T04:24:33.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.712+0000 7f0d79c00640 1 -- 172.21.15.107:0/2710602544 shutdown_connections 2024-09-19T04:24:33.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.712+0000 7f0d79c00640 1 --2- 172.21.15.107:0/2710602544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d74109140 0x7f0d74110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.712+0000 7f0d79c00640 1 --2- 172.21.15.107:0/2710602544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d741049c0 0x7f0d74108a10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.712+0000 7f0d79c00640 1 --2- 172.21.15.107:0/2710602544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d74104010 0x7f0d741043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.712+0000 7f0d79c00640 1 -- 172.21.15.107:0/2710602544 >> 172.21.15.107:0/2710602544 conn(0x7f0d740ffa50 msgr2=0x7f0d74101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:33.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.712+0000 7f0d79c00640 1 -- 172.21.15.107:0/2710602544 shutdown_connections 2024-09-19T04:24:33.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.712+0000 7f0d79c00640 1 -- 172.21.15.107:0/2710602544 wait complete. 2024-09-19T04:24:33.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d79c00640 1 Processor -- start 2024-09-19T04:24:33.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d79c00640 1 -- start start 2024-09-19T04:24:33.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d79c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d74104010 0x7f0d7419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d79c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d741049c0 0x7f0d7419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d79c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d74109140 0x7f0d7419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d79c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74114140 con 0x7f0d74109140 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d79c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74113fc0 con 0x7f0d74104010 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d79c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741142c0 con 0x7f0d741049c0 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d72000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d741049c0 0x7f0d7419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d73400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d74109140 0x7f0d7419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d72a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d74104010 0x7f0d7419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d72000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d741049c0 0x7f0d7419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35118/0 (socket says 172.21.15.107:35118) 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.713+0000 7f0d72000640 1 -- 172.21.15.107:0/574917660 learned_addr learned my addr 172.21.15.107:0/574917660 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.714+0000 7f0d72000640 1 -- 172.21.15.107:0/574917660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d74104010 msgr2=0x7f0d7419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.714+0000 7f0d72000640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d74104010 0x7f0d7419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.714+0000 7f0d72000640 1 -- 172.21.15.107:0/574917660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d74109140 msgr2=0x7f0d7419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.714+0000 7f0d72000640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d74109140 0x7f0d7419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.714+0000 7f0d72000640 1 -- 172.21.15.107:0/574917660 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7419f660 con 0x7f0d741049c0 2024-09-19T04:24:33.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.714+0000 7f0d72a00640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d74104010 0x7f0d7419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:33.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.714+0000 7f0d72000640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d741049c0 0x7f0d7419abf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0d60035040 tx=0x7f0d60033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:33.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.714+0000 7f0d6b400640 1 -- 172.21.15.107:0/574917660 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6003f020 con 0x7f0d741049c0 2024-09-19T04:24:33.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.714+0000 7f0d6b400640 1 -- 172.21.15.107:0/574917660 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d60032770 con 0x7f0d741049c0 2024-09-19T04:24:33.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.714+0000 7f0d79c00640 1 -- 172.21.15.107:0/574917660 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7419f8f0 con 0x7f0d741049c0 2024-09-19T04:24:33.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.715+0000 7f0d6b400640 1 -- 172.21.15.107:0/574917660 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60037040 con 0x7f0d741049c0 2024-09-19T04:24:33.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.715+0000 7f0d79c00640 1 -- 172.21.15.107:0/574917660 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d741a7230 con 0x7f0d741049c0 2024-09-19T04:24:33.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.717+0000 7f0d6b400640 1 -- 172.21.15.107:0/574917660 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f0d60042020 con 0x7f0d741049c0 2024-09-19T04:24:33.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.718+0000 7f0d7c6d3740 1 Processor -- start 2024-09-19T04:24:33.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.718+0000 7f0d7c6d3740 1 -- start start 2024-09-19T04:24:33.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb6fe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 0x5605bb708910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb70b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5605bb6643e0 con 0x5605bb6fe590 2024-09-19T04:24:33.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5605bb60bd80 con 0x5605bb708e50 2024-09-19T04:24:33.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5605bb6603a0 con 0x5605bb6fdc70 2024-09-19T04:24:33.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d6b400640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d580a33b0 0x7f0d580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d72a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 0x5605bb708910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d6b400640 1 -- 172.21.15.107:0/574917660 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0d600ef760 con 0x7f0d741049c0 2024-09-19T04:24:33.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d72a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 0x5605bb708910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46540/0 (socket says 172.21.15.107:46540) 2024-09-19T04:24:33.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d73400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb6fe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d72a00640 1 -- 172.21.15.107:0/253951635 learned_addr learned my addr 172.21.15.107:0/253951635 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:33.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.719+0000 7f0d72000640 1 --2- 172.21.15.107:0/253951635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb70b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.720+0000 7f0d73400640 1 -- 172.21.15.107:0/253951635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 msgr2=0x5605bb70b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:33.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.720+0000 7f0d73400640 1 --2- 172.21.15.107:0/253951635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb70b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.720+0000 7f0d73400640 1 -- 172.21.15.107:0/253951635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 msgr2=0x5605bb708910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:33.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.720+0000 7f0d73400640 1 --2- 172.21.15.107:0/253951635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 0x5605bb708910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.720+0000 7f0d73400640 1 -- 172.21.15.107:0/253951635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605bb62aae0 con 0x5605bb6fdc70 2024-09-19T04:24:33.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.720+0000 7f0d73400640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d580a33b0 0x7f0d580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.720+0000 7f0d73400640 1 --2- 172.21.15.107:0/253951635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb6fe050 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c011490 tx=0x7f0d6c011950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d99d55d96a6c8d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:33.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.720+0000 7f0d5fe00640 1 -- 172.21.15.107:0/253951635 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c01b3f0 con 0x5605bb6fdc70 2024-09-19T04:24:33.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.721+0000 7f0d5fe00640 1 -- 172.21.15.107:0/253951635 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6c01ba70 con 0x5605bb6fdc70 2024-09-19T04:24:33.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.721+0000 7f0d5fe00640 1 -- 172.21.15.107:0/253951635 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c00e930 con 0x5605bb6fdc70 2024-09-19T04:24:33.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.721+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/253951635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 msgr2=0x5605bb6fe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:33.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.721+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/253951635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb6fe050 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c011490 tx=0x7f0d6c011950 comp rx=0 tx=0).stop 2024-09-19T04:24:33.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.721+0000 7f0d73400640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d580a33b0 0x7f0d580a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c023000 tx=0x7f0d6c020000 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:33.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.721+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/253951635 shutdown_connections 2024-09-19T04:24:33.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.721+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/253951635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb70b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.721+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/253951635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 0x5605bb708910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.721+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/253951635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb6fe050 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.721+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/253951635 >> 172.21.15.107:0/253951635 conn(0x5605bb6fa200 msgr2=0x5605bb6fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:33.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.722+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/253951635 shutdown_connections 2024-09-19T04:24:33.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.722+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/253951635 wait complete. 2024-09-19T04:24:33.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.722+0000 7f0d7c6d3740 1 -- start start 2024-09-19T04:24:33.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.723+0000 7f0d7c6d3740 1 Processor -- start 2024-09-19T04:24:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.723+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb705d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.723+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 0x5605bb706290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.723+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb7067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.723+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5605bb70ba20 con 0x5605bb6fe590 2024-09-19T04:24:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.723+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5605bb61f0f0 con 0x5605bb708e50 2024-09-19T04:24:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.723+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5605bb70bba0 con 0x5605bb6fdc70 2024-09-19T04:24:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.723+0000 7f0d7c6d3740 10 client.0 _ll_register_callbacks cb 0x5605bb606bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.723+0000 7f0d72a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb705d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d72a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb705d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35128/0 (socket says 172.21.15.107:35128) 2024-09-19T04:24:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d72a00640 1 -- 172.21.15.107:0/3022918171 learned_addr learned my addr 172.21.15.107:0/3022918171 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d73400640 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb7067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d72000640 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 0x5605bb706290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d72a00640 1 -- 172.21.15.107:0/3022918171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 msgr2=0x5605bb7067d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d72a00640 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb7067d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d72a00640 1 -- 172.21.15.107:0/3022918171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 msgr2=0x5605bb706290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d72a00640 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 0x5605bb706290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d72a00640 1 -- 172.21.15.107:0/3022918171 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605bb3d53c0 con 0x5605bb6fdc70 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d72a00640 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb705d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0d64012d50 tx=0x7f0d6400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1018f7dd6e3651e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3022918171 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64012110 con 0x5605bb6fdc70 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3022918171 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6c01ba70 con 0x5605bb6fdc70 2024-09-19T04:24:33.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.724+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3022918171 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0d64004050 con 0x5605bb6fdc70 2024-09-19T04:24:33.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3022918171 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0d640122b0 con 0x5605bb6fdc70 2024-09-19T04:24:33.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d7c6d3740 10 client.25984 fetch_fsmap learned FSMap version 32 2024-09-19T04:24:33.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3022918171 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5605bb717b20 con 0x5605bb6fdc70 2024-09-19T04:24:33.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3022918171 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6c01ba70 con 0x5605bb6fdc70 2024-09-19T04:24:33.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3022918171 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f0d64025430 con 0x5605bb6fdc70 2024-09-19T04:24:33.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d7c6d3740 10 client.25984 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T04:24:33.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d7c6d3740 10 client.25984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:33.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d7c6d3740 20 client.25984 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5605bb718f30 0x5605bb71b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d7c6d3740 4 client.25984 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.725+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3022918171 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605bb671950 con 0x5605bb718f30 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.726+0000 7f0d72000640 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5605bb718f30 0x5605bb71b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.726+0000 7f0d72000640 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5605bb718f30 0x5605bb71b3f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27cac9bc4602b4f server_cookie=ef57063eb75f9e7 in_seq=0 out_seq=0 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.726+0000 7f0d5cc00640 10 client.25984 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3022918171 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605bb671950 con 0x5605bb718f30 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d5cc00640 10 client.25984 handle_command_reply: tid=0 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d7c6d3740 1 client.25984 shutdown 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3022918171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 msgr2=0x5605bb705d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb705d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0d64012d50 tx=0x7f0d6400f790 comp rx=0 tx=0).stop 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3022918171 shutdown_connections 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5605bb718f30 0x5605bb71b3f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb7067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fe590 0x5605bb706290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3022918171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fdc70 0x5605bb705d50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d5cc00640 0 client.25984 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:33.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.727+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3022918171 >> 172.21.15.107:0/3022918171 conn(0x5605bb6fa200 msgr2=0x5605bb6fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:33.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.728+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3022918171 shutdown_connections 2024-09-19T04:24:33.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.728+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3022918171 wait complete. 2024-09-19T04:24:33.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.728+0000 7f0d7c6d3740 20 client.25984 trim_cache size 0 max 16384 2024-09-19T04:24:33.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.741+0000 7f0d7c6d3740 1 Processor -- start 2024-09-19T04:24:33.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.741+0000 7f0d7c6d3740 1 -- start start 2024-09-19T04:24:33.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 0x5605bb720cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb6fe590 0x5605bb721200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb708e50 0x5605bb702fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741ab500 con 0x5605bb6fdc70 2024-09-19T04:24:33.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741ad100 con 0x5605bb6fe590 2024-09-19T04:24:33.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741abbd0 con 0x5605bb708e50 2024-09-19T04:24:33.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d72000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb6fe590 0x5605bb721200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d73400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb708e50 0x5605bb702fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d72a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 0x5605bb720cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d73400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb708e50 0x5605bb702fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35142/0 (socket says 172.21.15.107:35142) 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d73400640 1 -- 172.21.15.107:0/2793574695 learned_addr learned my addr 172.21.15.107:0/2793574695 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d72000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb6fe590 0x5605bb721200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59832/0 (socket says 172.21.15.107:59832) 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d73400640 1 -- 172.21.15.107:0/2793574695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb6fe590 msgr2=0x5605bb721200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d73400640 1 --2- 172.21.15.107:0/2793574695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb6fe590 0x5605bb721200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d73400640 1 -- 172.21.15.107:0/2793574695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 msgr2=0x5605bb720cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d73400640 1 --2- 172.21.15.107:0/2793574695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 0x5605bb720cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.742+0000 7f0d73400640 1 -- 172.21.15.107:0/2793574695 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605bb6fa030 con 0x5605bb708e50 2024-09-19T04:24:33.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.743+0000 7f0d73400640 1 --2- 172.21.15.107:0/2793574695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb708e50 0x5605bb702fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c005c10 tx=0x7f0d6c029000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab7abd6e606d0a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:33.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.743+0000 7f0d5f400640 1 -- 172.21.15.107:0/2793574695 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c034070 con 0x5605bb708e50 2024-09-19T04:24:33.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.743+0000 7f0d5f400640 1 -- 172.21.15.107:0/2793574695 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6c009c50 con 0x5605bb708e50 2024-09-19T04:24:33.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.743+0000 7f0d5f400640 1 -- 172.21.15.107:0/2793574695 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c02a040 con 0x5605bb708e50 2024-09-19T04:24:33.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.744+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/2793574695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb708e50 msgr2=0x5605bb702fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:33.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.744+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/2793574695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb708e50 0x5605bb702fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c005c10 tx=0x7f0d6c029000 comp rx=0 tx=0).stop 2024-09-19T04:24:33.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.744+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/2793574695 shutdown_connections 2024-09-19T04:24:33.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.744+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/2793574695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb708e50 0x5605bb702fb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.744+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/2793574695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb6fe590 0x5605bb721200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.744+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/2793574695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 0x5605bb720cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.744+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/2793574695 >> 172.21.15.107:0/2793574695 conn(0x5605bb6fa200 msgr2=0x5605bb6f9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.744+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/2793574695 shutdown_connections 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.744+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/2793574695 wait complete. 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.744+0000 7f0d7c6d3740 1 -- start start 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d7c6d3740 1 Processor -- start 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 0x5605bb70cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fe590 0x5605bb70d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d7c6d3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb70d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741ac970 con 0x5605bb6fdc70 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741ac2a0 con 0x5605bb708e50 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d7c6d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741a9520 con 0x5605bb6fe590 2024-09-19T04:24:33.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d7c6d3740 10 client.0 _ll_register_callbacks cb 0x7f0d64009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:33.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d72a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 0x5605bb70cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d73400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb70d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d72000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fe590 0x5605bb70d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d72a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 0x5605bb70cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46548/0 (socket says 172.21.15.107:46548) 2024-09-19T04:24:33.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.745+0000 7f0d72a00640 1 -- 172.21.15.107:0/3010088165 learned_addr learned my addr 172.21.15.107:0/3010088165 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d72a00640 1 -- 172.21.15.107:0/3010088165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fe590 msgr2=0x5605bb70d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d72a00640 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fe590 0x5605bb70d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d72a00640 1 -- 172.21.15.107:0/3010088165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 msgr2=0x5605bb70d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d72a00640 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb70d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d72a00640 1 -- 172.21.15.107:0/3010088165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605bb6f5770 con 0x5605bb6fdc70 2024-09-19T04:24:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d73400640 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb70d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d72000640 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fe590 0x5605bb70d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d72a00640 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 0x5605bb70cdc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0d64012d50 tx=0x7f0d64025d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930cc6350b20ccb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3010088165 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64034060 con 0x5605bb6fdc70 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3010088165 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0d640052d0 con 0x5605bb6fdc70 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.746+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3010088165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605bb66be60 con 0x5605bb6fdc70 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3010088165 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0d6402b070 con 0x5605bb6fdc70 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d7c6d3740 10 client.15969 fetch_fsmap learned FSMap version 32 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3010088165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5605bb784010 con 0x5605bb6fdc70 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3010088165 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5605bb66be60 con 0x5605bb6fdc70 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3010088165 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f0d64003ad0 con 0x5605bb6fdc70 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d7c6d3740 10 client.15969 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d7c6d3740 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d7c6d3740 20 client.15969 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5605bb784700 0x5605bb786b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d7c6d3740 4 client.15969 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.747+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3010088165 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605bb5e8010 con 0x5605bb784700 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.748+0000 7f0d72000640 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5605bb784700 0x5605bb786b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.748+0000 7f0d5cc00640 10 client.15969 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.748+0000 7f0d72000640 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5605bb784700 0x5605bb786b20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9196ee8a51b53b1 server_cookie=e8b4d68ece14b186 in_seq=0 out_seq=0 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.748+0000 7f0d5cc00640 1 -- 172.21.15.107:0/3010088165 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605bb5e8010 con 0x5605bb784700 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.748+0000 7f0d5cc00640 10 client.15969 handle_command_reply: tid=0 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.748+0000 7f0d7c6d3740 1 client.15969 shutdown 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3010088165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 msgr2=0x5605bb70cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 0x5605bb70cdc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0d64012d50 tx=0x7f0d64025d20 comp rx=0 tx=0).stop 2024-09-19T04:24:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3010088165 shutdown_connections 2024-09-19T04:24:33.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5605bb784700 0x5605bb786b20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605bb708e50 0x5605bb70d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605bb6fe590 0x5605bb70d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 1 --2- 172.21.15.107:0/3010088165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605bb6fdc70 0x5605bb70cdc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3010088165 >> 172.21.15.107:0/3010088165 conn(0x5605bb6fa200 msgr2=0x5605bb6ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:33.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d5cc00640 0 client.15969 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:33.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3010088165 shutdown_connections 2024-09-19T04:24:33.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 1 -- 172.21.15.107:0/3010088165 wait complete. 2024-09-19T04:24:33.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.749+0000 7f0d7c6d3740 20 client.15969 trim_cache size 0 max 16384 2024-09-19T04:24:33.753 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:24:33.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 -- 172.21.15.107:0/574917660 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d580a33b0 msgr2=0x7f0d580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:33.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d580a33b0 0x7f0d580a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c023000 tx=0x7f0d6c020000 comp rx=0 tx=0).stop 2024-09-19T04:24:33.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 -- 172.21.15.107:0/574917660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d741049c0 msgr2=0x7f0d7419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:33.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d741049c0 0x7f0d7419abf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0d60035040 tx=0x7f0d60033d90 comp rx=0 tx=0).stop 2024-09-19T04:24:33.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 -- 172.21.15.107:0/574917660 shutdown_connections 2024-09-19T04:24:33.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d580a33b0 0x7f0d580a5870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d74109140 0x7f0d7419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d741049c0 0x7f0d7419abf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 --2- 172.21.15.107:0/574917660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d74104010 0x7f0d7419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:33.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 -- 172.21.15.107:0/574917660 >> 172.21.15.107:0/574917660 conn(0x7f0d740ffa50 msgr2=0x7f0d74100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:33.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 -- 172.21.15.107:0/574917660 shutdown_connections 2024-09-19T04:24:33.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:33.750+0000 7f0d79c00640 1 -- 172.21.15.107:0/574917660 wait complete. 2024-09-19T04:24:33.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:33 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:24:33.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:33 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:24:34.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:33 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:24:34.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:33 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:24:34.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:33 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:24:34.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:33 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:24:34.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 157761 3.04 MB/sec execute 52 sec latency 3061.841 ms 2024-09-19T04:24:34.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:34 smithi107 ceph-mon[30659]: pgmap v130: 97 pgs: 97 active+clean; 99 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-19T04:24:34.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:34 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:24:34.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:34 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:24:34.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:34 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:24:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:34 smithi115 ceph-mon[35477]: pgmap v130: 97 pgs: 97 active+clean; 99 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-19T04:24:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:34 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:24:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:34 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:24:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:34 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:24:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:34 smithi145 ceph-mon[35096]: pgmap v130: 97 pgs: 97 active+clean; 99 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-19T04:24:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:34 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:24:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:34 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:24:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:34 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:24:35.629 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:24:35.630 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:24:35.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 159961 3.24 MB/sec execute 53 sec latency 11.514 ms 2024-09-19T04:24:36.016 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:36.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.634+0000 7f9865800640 1 -- 172.21.15.107:0/2172867000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9860103ff0 msgr2=0x7f98601043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.634+0000 7f9865800640 1 --2- 172.21.15.107:0/2172867000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9860103ff0 0x7f98601043d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9850009900 tx=0x7f985002a400 comp rx=0 tx=0).stop 2024-09-19T04:24:36.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.634+0000 7f9865800640 1 -- 172.21.15.107:0/2172867000 shutdown_connections 2024-09-19T04:24:36.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.634+0000 7f9865800640 1 --2- 172.21.15.107:0/2172867000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9860109120 0x7f9860110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.634+0000 7f9865800640 1 --2- 172.21.15.107:0/2172867000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98601049a0 0x7f98601089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.634+0000 7f9865800640 1 --2- 172.21.15.107:0/2172867000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9860103ff0 0x7f98601043d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.634+0000 7f9865800640 1 -- 172.21.15.107:0/2172867000 >> 172.21.15.107:0/2172867000 conn(0x7f98600ffa90 msgr2=0x7f9860101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.634+0000 7f9865800640 1 -- 172.21.15.107:0/2172867000 shutdown_connections 2024-09-19T04:24:36.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.634+0000 7f9865800640 1 -- 172.21.15.107:0/2172867000 wait complete. 2024-09-19T04:24:36.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.635+0000 7f9865800640 1 Processor -- start 2024-09-19T04:24:36.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.635+0000 7f9865800640 1 -- start start 2024-09-19T04:24:36.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.635+0000 7f9865800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9860103ff0 0x7f986019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.635+0000 7f9865800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98601049a0 0x7f986019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.635+0000 7f9865800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9860109120 0x7f986019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.635+0000 7f9865800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9860114140 con 0x7f9860109120 2024-09-19T04:24:36.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.635+0000 7f9865800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9860113fc0 con 0x7f9860103ff0 2024-09-19T04:24:36.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.635+0000 7f9865800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98601142c0 con 0x7f98601049a0 2024-09-19T04:24:36.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.636+0000 7f985ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9860103ff0 0x7f986019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.636+0000 7f985ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9860103ff0 0x7f986019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59864/0 (socket says 172.21.15.107:59864) 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.636+0000 7f985ea00640 1 -- 172.21.15.107:0/762655845 learned_addr learned my addr 172.21.15.107:0/762655845 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.636+0000 7f985f400640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9860109120 0x7f986019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.636+0000 7f9856000640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98601049a0 0x7f986019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.636+0000 7f985ea00640 1 -- 172.21.15.107:0/762655845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98601049a0 msgr2=0x7f986019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.636+0000 7f985ea00640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98601049a0 0x7f986019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.636+0000 7f985ea00640 1 -- 172.21.15.107:0/762655845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9860109120 msgr2=0x7f986019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.636+0000 7f985ea00640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9860109120 0x7f986019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.636+0000 7f985ea00640 1 -- 172.21.15.107:0/762655845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986019f5b0 con 0x7f9860103ff0 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.637+0000 7f985ea00640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9860103ff0 0x7f986019a600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9850006010 tx=0x7f9850004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.637+0000 7f9857e00640 1 -- 172.21.15.107:0/762655845 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985003f030 con 0x7f9860103ff0 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.637+0000 7f9865800640 1 -- 172.21.15.107:0/762655845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986019f840 con 0x7f9860103ff0 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.637+0000 7f9857e00640 1 -- 172.21.15.107:0/762655845 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9850004440 con 0x7f9860103ff0 2024-09-19T04:24:36.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.637+0000 7f9857e00640 1 -- 172.21.15.107:0/762655845 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850035070 con 0x7f9860103ff0 2024-09-19T04:24:36.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.637+0000 7f9865800640 1 -- 172.21.15.107:0/762655845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98601a7180 con 0x7f9860103ff0 2024-09-19T04:24:36.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.640+0000 7f9857e00640 1 -- 172.21.15.107:0/762655845 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f9850042030 con 0x7f9860103ff0 2024-09-19T04:24:36.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.640+0000 7f98681cc740 1 Processor -- start 2024-09-19T04:24:36.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.640+0000 7f98681cc740 1 -- start start 2024-09-19T04:24:36.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.640+0000 7f98681cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daefea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f98681cc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9daf03e0 0x55fb9dafa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f98681cc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9dafaca0 0x55fb9dafd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f98681cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fb9da56290 con 0x55fb9daefac0 2024-09-19T04:24:36.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f98681cc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fb9d9fdb40 con 0x55fb9dafaca0 2024-09-19T04:24:36.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f98681cc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fb9da52250 con 0x55fb9daf03e0 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f985f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daefea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f985f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daefea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46560/0 (socket says 172.21.15.107:46560) 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f985f400640 1 -- 172.21.15.107:0/4182061009 learned_addr learned my addr 172.21.15.107:0/4182061009 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f985ea00640 1 --2- 172.21.15.107:0/4182061009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9daf03e0 0x55fb9dafa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f9856000640 1 --2- 172.21.15.107:0/4182061009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9dafaca0 0x55fb9dafd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f985f400640 1 -- 172.21.15.107:0/4182061009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9daf03e0 msgr2=0x55fb9dafa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f985f400640 1 --2- 172.21.15.107:0/4182061009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9daf03e0 0x55fb9dafa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f985f400640 1 -- 172.21.15.107:0/4182061009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9dafaca0 msgr2=0x55fb9dafd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f985f400640 1 --2- 172.21.15.107:0/4182061009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9dafaca0 0x55fb9dafd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f985f400640 1 -- 172.21.15.107:0/4182061009 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb9d7c76a0 con 0x55fb9daefac0 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.641+0000 7f9856000640 1 --2- 172.21.15.107:0/4182061009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9dafaca0 0x55fb9dafd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:24:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.642+0000 7f9857e00640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98440a33b0 0x7f98440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.642+0000 7f9857e00640 1 -- 172.21.15.107:0/762655845 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f98500efcd0 con 0x7f9860103ff0 2024-09-19T04:24:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.642+0000 7f985ea00640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98440a33b0 0x7f98440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.642+0000 7f985f400640 1 --2- 172.21.15.107:0/4182061009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daefea0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9858009f50 tx=0x7f98580118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc502889f404fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.642+0000 7f984fe00640 1 -- 172.21.15.107:0/4182061009 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858004d20 con 0x55fb9daefac0 2024-09-19T04:24:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.642+0000 7f984fe00640 1 -- 172.21.15.107:0/4182061009 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9858004050 con 0x55fb9daefac0 2024-09-19T04:24:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.642+0000 7f984fe00640 1 -- 172.21.15.107:0/4182061009 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858005280 con 0x55fb9daefac0 2024-09-19T04:24:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.643+0000 7f98681cc740 1 -- 172.21.15.107:0/4182061009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 msgr2=0x55fb9daefea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.643+0000 7f985ea00640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98440a33b0 0x7f98440a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55fb9daeceb0 tx=0x7f985002a400 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.643+0000 7f98681cc740 1 --2- 172.21.15.107:0/4182061009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daefea0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9858009f50 tx=0x7f98580118d0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.643+0000 7f98681cc740 1 -- 172.21.15.107:0/4182061009 shutdown_connections 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.643+0000 7f98681cc740 1 --2- 172.21.15.107:0/4182061009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9dafaca0 0x55fb9dafd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.643+0000 7f98681cc740 1 --2- 172.21.15.107:0/4182061009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9daf03e0 0x55fb9dafa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.643+0000 7f98681cc740 1 --2- 172.21.15.107:0/4182061009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daefea0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.643+0000 7f98681cc740 1 -- 172.21.15.107:0/4182061009 >> 172.21.15.107:0/4182061009 conn(0x55fb9daec050 msgr2=0x55fb9daec430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.643+0000 7f98681cc740 1 -- 172.21.15.107:0/4182061009 shutdown_connections 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.643+0000 7f98681cc740 1 -- 172.21.15.107:0/4182061009 wait complete. 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.644+0000 7f98681cc740 1 -- start start 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.644+0000 7f98681cc740 1 Processor -- start 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f98681cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9dafa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f98681cc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9daf78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f98681cc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9daf7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f98681cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fb9dafd910 con 0x55fb9daefac0 2024-09-19T04:24:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f98681cc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fb9da10fa0 con 0x55fb9daf03e0 2024-09-19T04:24:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f98681cc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fb9dafda90 con 0x55fb9dafaca0 2024-09-19T04:24:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f98681cc740 10 client.0 _ll_register_callbacks cb 0x55fb9d764ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f985f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9dafa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f985f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9dafa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46568/0 (socket says 172.21.15.107:46568) 2024-09-19T04:24:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f985f400640 1 -- 172.21.15.107:0/4032791298 learned_addr learned my addr 172.21.15.107:0/4032791298 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f9856000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9daf78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f985f400640 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9daf7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f985f400640 1 -- 172.21.15.107:0/4032791298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 msgr2=0x55fb9daf7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f985f400640 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9daf7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f985f400640 1 -- 172.21.15.107:0/4032791298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 msgr2=0x55fb9daf78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f985f400640 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9daf78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f985f400640 1 -- 172.21.15.107:0/4032791298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb9daf86c0 con 0x55fb9daefac0 2024-09-19T04:24:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.645+0000 7f9856000640 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9daf78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.646+0000 7f985f400640 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9dafa640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9858009250 tx=0x7f9858004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c6c5c94caeb58b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.646+0000 7f984cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.646+0000 7f984cc00640 1 -- 172.21.15.107:0/4032791298 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985802b070 con 0x55fb9daefac0 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.646+0000 7f984cc00640 1 -- 172.21.15.107:0/4032791298 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f9858025450 con 0x55fb9daefac0 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.646+0000 7f98681cc740 1 -- 172.21.15.107:0/4032791298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9858004050 con 0x55fb9daefac0 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.646+0000 7f984cc00640 1 -- 172.21.15.107:0/4032791298 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9858025650 con 0x55fb9daefac0 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.646+0000 7f98681cc740 10 client.15981 fetch_fsmap learned FSMap version 32 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.646+0000 7f98681cc740 1 -- 172.21.15.107:0/4032791298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fb9db09890 con 0x55fb9daefac0 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.647+0000 7f984cc00640 1 -- 172.21.15.107:0/4032791298 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9858004050 con 0x55fb9daefac0 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.647+0000 7f984cc00640 1 -- 172.21.15.107:0/4032791298 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f985800f640 con 0x55fb9daefac0 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.647+0000 7f98681cc740 10 client.15981 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T04:24:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.647+0000 7f98681cc740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.647+0000 7f98681cc740 20 client.15981 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.647+0000 7f98681cc740 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fb9db0acf0 0x55fb9db0d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.647+0000 7f98681cc740 4 client.15981 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.647+0000 7f98681cc740 1 -- 172.21.15.107:0/4032791298 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb9da63800 con 0x55fb9db0acf0 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.647+0000 7f9856000640 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fb9db0acf0 0x55fb9db0d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.648+0000 7f9856000640 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fb9db0acf0 0x55fb9db0d0e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a74ca87c9ab6c server_cookie=1516e12bcdd6f201 in_seq=0 out_seq=0 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.648+0000 7f984cc00640 10 client.15981 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f984cc00640 1 -- 172.21.15.107:0/4032791298 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb9da63800 con 0x55fb9db0acf0 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f984cc00640 10 client.15981 handle_command_reply: tid=0 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f98681cc740 1 client.15981 shutdown 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f98681cc740 1 -- 172.21.15.107:0/4032791298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 msgr2=0x55fb9dafa640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f98681cc740 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9dafa640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9858009250 tx=0x7f9858004820 comp rx=0 tx=0).stop 2024-09-19T04:24:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f98681cc740 1 -- 172.21.15.107:0/4032791298 shutdown_connections 2024-09-19T04:24:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f98681cc740 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fb9db0acf0 0x55fb9db0d0e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f98681cc740 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9daf7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f98681cc740 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9daf78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f984cc00640 0 client.15981 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f98681cc740 1 --2- 172.21.15.107:0/4032791298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9dafa640 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.649+0000 7f98681cc740 1 -- 172.21.15.107:0/4032791298 >> 172.21.15.107:0/4032791298 conn(0x55fb9daec050 msgr2=0x55fb9daeb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.650+0000 7f98681cc740 1 -- 172.21.15.107:0/4032791298 shutdown_connections 2024-09-19T04:24:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.650+0000 7f98681cc740 1 -- 172.21.15.107:0/4032791298 wait complete. 2024-09-19T04:24:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.650+0000 7f98681cc740 20 client.15981 trim_cache size 0 max 16384 2024-09-19T04:24:36.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 160362 3.19 MB/sec execute 54 sec latency 574.169 ms 2024-09-19T04:24:36.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f98681cc740 1 Processor -- start 2024-09-19T04:24:36.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f98681cc740 1 -- start start 2024-09-19T04:24:36.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f98681cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daeb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f98681cc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9daf9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f98681cc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9db12820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f98681cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98601af990 con 0x55fb9daefac0 2024-09-19T04:24:36.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f98681cc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f98601b1590 con 0x55fb9daf03e0 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f98681cc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98601b0060 con 0x55fb9dafaca0 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f985f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daeb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f985f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daeb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46570/0 (socket says 172.21.15.107:46570) 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f985f400640 1 -- 172.21.15.107:0/2250981565 learned_addr learned my addr 172.21.15.107:0/2250981565 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f9856000640 1 --2- 172.21.15.107:0/2250981565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9daf9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.668+0000 7f985f400640 1 --2- 172.21.15.107:0/2250981565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9db12820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.669+0000 7f985f400640 1 -- 172.21.15.107:0/2250981565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 msgr2=0x55fb9db12820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.669+0000 7f985f400640 1 --2- 172.21.15.107:0/2250981565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9db12820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.669+0000 7f985f400640 1 -- 172.21.15.107:0/2250981565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 msgr2=0x55fb9daf9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.669+0000 7f985f400640 1 --2- 172.21.15.107:0/2250981565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9daf9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.669+0000 7f985f400640 1 -- 172.21.15.107:0/2250981565 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb9daf5e80 con 0x55fb9daefac0 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.669+0000 7f985f400640 1 --2- 172.21.15.107:0/2250981565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daeb9b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9858009d00 tx=0x7f9858009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779a5e0190bbb3dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.669+0000 7f984f400640 1 -- 172.21.15.107:0/2250981565 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985802b070 con 0x55fb9daefac0 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.669+0000 7f984f400640 1 -- 172.21.15.107:0/2250981565 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985800fcd0 con 0x55fb9daefac0 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.669+0000 7f984f400640 1 -- 172.21.15.107:0/2250981565 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98580257f0 con 0x55fb9daefac0 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.670+0000 7f98681cc740 1 -- 172.21.15.107:0/2250981565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 msgr2=0x55fb9daeb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.670+0000 7f98681cc740 1 --2- 172.21.15.107:0/2250981565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daeb9b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9858009d00 tx=0x7f9858009b30 comp rx=0 tx=0).stop 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.670+0000 7f98681cc740 1 -- 172.21.15.107:0/2250981565 shutdown_connections 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.670+0000 7f98681cc740 1 --2- 172.21.15.107:0/2250981565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9db12820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.670+0000 7f98681cc740 1 --2- 172.21.15.107:0/2250981565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9daf9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.670+0000 7f98681cc740 1 --2- 172.21.15.107:0/2250981565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9daeb9b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.670+0000 7f98681cc740 1 -- 172.21.15.107:0/2250981565 >> 172.21.15.107:0/2250981565 conn(0x55fb9daec050 msgr2=0x55fb9daf9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.670+0000 7f98681cc740 1 -- 172.21.15.107:0/2250981565 shutdown_connections 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.670+0000 7f98681cc740 1 -- 172.21.15.107:0/2250981565 wait complete. 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.670+0000 7f98681cc740 1 -- start start 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f98681cc740 1 Processor -- start 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f98681cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9db033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f98681cc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9dafe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f98681cc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9dafec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f98681cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98601b0e00 con 0x55fb9daefac0 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f98681cc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f98601b0730 con 0x55fb9daf03e0 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f98681cc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98601ad970 con 0x55fb9dafaca0 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f98681cc740 10 client.0 _ll_register_callbacks cb 0x7f9858009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f985f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9db033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f985f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9db033a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46578/0 (socket says 172.21.15.107:46578) 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f985f400640 1 -- 172.21.15.107:0/784361073 learned_addr learned my addr 172.21.15.107:0/784361073 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.671+0000 7f9856000640 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9dafe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f985f400640 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9dafec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f9856000640 1 -- 172.21.15.107:0/784361073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 msgr2=0x55fb9dafec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f9856000640 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9dafec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f9856000640 1 -- 172.21.15.107:0/784361073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 msgr2=0x55fb9db033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f9856000640 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9db033a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f9856000640 1 -- 172.21.15.107:0/784361073 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb9daff470 con 0x55fb9daf03e0 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f9856000640 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9dafe720 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55fb9db0d7c0 tx=0x7f9848012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=387e303a67289da8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f984cc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f984cc00640 1 -- 172.21.15.107:0/784361073 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984802d060 con 0x55fb9daf03e0 2024-09-19T04:24:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f984cc00640 1 -- 172.21.15.107:0/784361073 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f98480208b0 con 0x55fb9daf03e0 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.672+0000 7f98681cc740 1 -- 172.21.15.107:0/784361073 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985800fcd0 con 0x55fb9daf03e0 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f984cc00640 1 -- 172.21.15.107:0/784361073 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9848020ab0 con 0x55fb9daf03e0 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f98681cc740 10 client.26039 fetch_fsmap learned FSMap version 32 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f98681cc740 1 -- 172.21.15.107:0/784361073 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fb9db75800 con 0x55fb9daf03e0 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f984cc00640 1 -- 172.21.15.107:0/784361073 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985800fcd0 con 0x55fb9daf03e0 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f984cc00640 1 -- 172.21.15.107:0/784361073 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f9848032ce0 con 0x55fb9daf03e0 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f98681cc740 10 client.26039 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f98681cc740 10 client.26039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f98681cc740 20 client.26039 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f98681cc740 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fb9db76010 0x55fb9db78430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f98681cc740 4 client.26039 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f98681cc740 1 -- 172.21.15.107:0/784361073 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fb9da5dd10 con 0x55fb9db76010 2024-09-19T04:24:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.673+0000 7f985f400640 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fb9db76010 0x55fb9db78430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.674+0000 7f985f400640 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fb9db76010 0x55fb9db78430 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29411c68be5c5d52 server_cookie=766db0c6dd2cb03f in_seq=0 out_seq=0 2024-09-19T04:24:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.674+0000 7f984cc00640 10 client.26039 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f984cc00640 1 -- 172.21.15.107:0/784361073 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fb9da5dd10 con 0x55fb9db76010 2024-09-19T04:24:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f984cc00640 10 client.26039 handle_command_reply: tid=0 2024-09-19T04:24:36.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 client.26039 shutdown 2024-09-19T04:24:36.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 -- 172.21.15.107:0/784361073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 msgr2=0x55fb9dafe720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9dafe720 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55fb9db0d7c0 tx=0x7f9848012bf0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 -- 172.21.15.107:0/784361073 shutdown_connections 2024-09-19T04:24:36.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fb9db76010 0x55fb9db78430 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb9dafaca0 0x55fb9dafec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb9daf03e0 0x55fb9dafe720 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f984cc00640 0 client.26039 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:36.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 --2- 172.21.15.107:0/784361073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb9daefac0 0x55fb9db033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 -- 172.21.15.107:0/784361073 >> 172.21.15.107:0/784361073 conn(0x55fb9daec050 msgr2=0x55fb9db06380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 -- 172.21.15.107:0/784361073 shutdown_connections 2024-09-19T04:24:36.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 1 -- 172.21.15.107:0/784361073 wait complete. 2024-09-19T04:24:36.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.675+0000 7f98681cc740 20 client.26039 trim_cache size 0 max 16384 2024-09-19T04:24:36.996 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:24:36.996 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:24:36.996 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "93a9cada-7d44-4565-ba6d-f0babffad31f", 2024-09-19T04:24:36.996 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:24:36.996 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:24:36.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 -- 172.21.15.107:0/762655845 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98440a33b0 msgr2=0x7f98440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98440a33b0 0x7f98440a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55fb9daeceb0 tx=0x7f985002a400 comp rx=0 tx=0).stop 2024-09-19T04:24:36.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 -- 172.21.15.107:0/762655845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9860103ff0 msgr2=0x7f986019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:36.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9860103ff0 0x7f986019a600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9850006010 tx=0x7f9850004040 comp rx=0 tx=0).stop 2024-09-19T04:24:36.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 -- 172.21.15.107:0/762655845 shutdown_connections 2024-09-19T04:24:36.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98440a33b0 0x7f98440a5870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9860109120 0x7f986019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98601049a0 0x7f986019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 --2- 172.21.15.107:0/762655845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9860103ff0 0x7f986019a600 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:36.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 -- 172.21.15.107:0/762655845 >> 172.21.15.107:0/762655845 conn(0x7f98600ffa90 msgr2=0x7f9860100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:36.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 -- 172.21.15.107:0/762655845 shutdown_connections 2024-09-19T04:24:36.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:36.676+0000 7f9865800640 1 -- 172.21.15.107:0/762655845 wait complete. 2024-09-19T04:24:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:36 smithi115 ceph-mon[35477]: pgmap v131: 97 pgs: 97 active+clean; 102 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-09-19T04:24:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:36 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:24:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:36 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:36 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:24:37.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:36 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:36 smithi145 ceph-mon[35096]: pgmap v131: 97 pgs: 97 active+clean; 102 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-09-19T04:24:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:36 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:24:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:36 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:36 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:24:37.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:36 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:36 smithi107 ceph-mon[30659]: pgmap v131: 97 pgs: 97 active+clean; 102 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-09-19T04:24:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:36 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:24:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:36 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:36 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:24:37.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:36 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:37.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 160430 3.13 MB/sec execute 55 sec latency 614.070 ms 2024-09-19T04:24:37.890 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:24:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:37 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:24:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:37 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:37 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:24:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:24:38.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 160430 3.08 MB/sec execute 56 sec latency 1312.680 ms 2024-09-19T04:24:38.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.841+0000 7f0389a00640 1 -- 172.21.15.107:0/749250361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03841079c0 msgr2=0x7f0384107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.841+0000 7f0389a00640 1 --2- 172.21.15.107:0/749250361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03841079c0 0x7f0384107da0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0374009cd0 tx=0x7f037402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.841+0000 7f0389a00640 1 -- 172.21.15.107:0/749250361 shutdown_connections 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.841+0000 7f0389a00640 1 --2- 172.21.15.107:0/749250361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0384102360 0x7f038410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.841+0000 7f0389a00640 1 --2- 172.21.15.107:0/749250361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03841019c0 0x7f0384101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.841+0000 7f0389a00640 1 --2- 172.21.15.107:0/749250361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03841079c0 0x7f0384107da0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.841+0000 7f0389a00640 1 -- 172.21.15.107:0/749250361 >> 172.21.15.107:0/749250361 conn(0x7f03840fd740 msgr2=0x7f03840ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.841+0000 7f0389a00640 1 -- 172.21.15.107:0/749250361 shutdown_connections 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.841+0000 7f0389a00640 1 -- 172.21.15.107:0/749250361 wait complete. 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f0389a00640 1 Processor -- start 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f0389a00640 1 -- start start 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f0389a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03841019c0 0x7f038419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f0389a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0384102360 0x7f038419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f0389a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03841079c0 0x7f038419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f0389a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0384111dc0 con 0x7f0384102360 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f0389a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0384111c40 con 0x7f03841079c0 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f0389a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0384111f40 con 0x7f03841019c0 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f037b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0384102360 0x7f038419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f037b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0384102360 0x7f038419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60986/0 (socket says 172.21.15.107:60986) 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.842+0000 7f037b400640 1 -- 172.21.15.107:0/2220640708 learned_addr learned my addr 172.21.15.107:0/2220640708 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.843+0000 7f0383e00640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03841019c0 0x7f038419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.843+0000 7f0389000640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03841079c0 0x7f038419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.843+0000 7f037b400640 1 -- 172.21.15.107:0/2220640708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03841019c0 msgr2=0x7f038419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.843+0000 7f037b400640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03841019c0 0x7f038419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.843+0000 7f037b400640 1 -- 172.21.15.107:0/2220640708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03841079c0 msgr2=0x7f038419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.843+0000 7f037b400640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03841079c0 0x7f038419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.843+0000 7f037b400640 1 -- 172.21.15.107:0/2220640708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038419f4b0 con 0x7f0384102360 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.843+0000 7f037b400640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0384102360 0x7f038419aa40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f036c00e9f0 tx=0x7f036c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.843+0000 7f0381600640 1 -- 172.21.15.107:0/2220640708 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c00cd80 con 0x7f0384102360 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.844+0000 7f0389a00640 1 -- 172.21.15.107:0/2220640708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038419f7a0 con 0x7f0384102360 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.844+0000 7f0381600640 1 -- 172.21.15.107:0/2220640708 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c005500 con 0x7f0384102360 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.844+0000 7f0389a00640 1 -- 172.21.15.107:0/2220640708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03841a7080 con 0x7f0384102360 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.844+0000 7f0381600640 1 -- 172.21.15.107:0/2220640708 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c018630 con 0x7f0384102360 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.847+0000 7f038dc64740 1 Processor -- start 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f038dc64740 1 -- start start 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f038dc64740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b498d50 0x55db9b499130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f038dc64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f0381600640 1 -- 172.21.15.107:0/2220640708 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f036c0046c0 con 0x7f0384102360 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f038dc64740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 0x55db9b4a6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f038dc64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55db9b3ff3e0 con 0x55db9b499670 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f038dc64740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55db9b3a6d80 con 0x55db9b498d50 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f038dc64740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55db9b3fb3a0 con 0x55db9b4a3f30 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f0383e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f0383e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60996/0 (socket says 172.21.15.107:60996) 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.848+0000 7f0383e00640 1 -- 172.21.15.107:0/4185540442 learned_addr learned my addr 172.21.15.107:0/4185540442 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.849+0000 7f037b400640 1 --2- 172.21.15.107:0/4185540442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 0x55db9b4a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.849+0000 7f0389000640 1 --2- 172.21.15.107:0/4185540442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b498d50 0x55db9b499130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.849+0000 7f0383e00640 1 -- 172.21.15.107:0/4185540442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 msgr2=0x55db9b4a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.849+0000 7f0383e00640 1 --2- 172.21.15.107:0/4185540442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 0x55db9b4a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.849+0000 7f0383e00640 1 -- 172.21.15.107:0/4185540442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b498d50 msgr2=0x55db9b499130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.849+0000 7f0383e00640 1 --2- 172.21.15.107:0/4185540442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b498d50 0x55db9b499130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.849+0000 7f0383e00640 1 -- 172.21.15.107:0/4185540442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9b3c5ae0 con 0x55db9b499670 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.849+0000 7f0383e00640 1 --2- 172.21.15.107:0/4185540442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a39f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0374009cd0 tx=0x7f0374009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9096b2c8690ea750 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.850+0000 7f0379600640 1 -- 172.21.15.107:0/4185540442 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374048070 con 0x55db9b499670 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.850+0000 7f0379600640 1 -- 172.21.15.107:0/4185540442 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0374036070 con 0x55db9b499670 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.850+0000 7f0379600640 1 -- 172.21.15.107:0/4185540442 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374004400 con 0x55db9b499670 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.850+0000 7f0381600640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f03680a33b0 0x7f03680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.850+0000 7f0381600640 1 -- 172.21.15.107:0/2220640708 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f036c0d0210 con 0x7f0384102360 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.850+0000 7f0389000640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f03680a33b0 0x7f03680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.850+0000 7f038dc64740 1 -- 172.21.15.107:0/4185540442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 msgr2=0x55db9b4a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.850+0000 7f038dc64740 1 --2- 172.21.15.107:0/4185540442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a39f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0374009cd0 tx=0x7f0374009a20 comp rx=0 tx=0).stop 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.851+0000 7f038dc64740 1 -- 172.21.15.107:0/4185540442 shutdown_connections 2024-09-19T04:24:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.851+0000 7f038dc64740 1 --2- 172.21.15.107:0/4185540442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 0x55db9b4a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.851+0000 7f038dc64740 1 --2- 172.21.15.107:0/4185540442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a39f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.851+0000 7f038dc64740 1 --2- 172.21.15.107:0/4185540442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b498d50 0x55db9b499130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.851+0000 7f038dc64740 1 -- 172.21.15.107:0/4185540442 >> 172.21.15.107:0/4185540442 conn(0x55db9b4952e0 msgr2=0x55db9b4956c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.851+0000 7f038dc64740 1 -- 172.21.15.107:0/4185540442 shutdown_connections 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.851+0000 7f0389000640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f03680a33b0 0x7f03680a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f037c009d20 tx=0x7f037c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.851+0000 7f038dc64740 1 -- 172.21.15.107:0/4185540442 wait complete. 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.851+0000 7f038dc64740 1 -- start start 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.851+0000 7f038dc64740 1 Processor -- start 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f038dc64740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b498d50 0x55db9b4a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f038dc64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f038dc64740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 0x55db9b4a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f038dc64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55db9b4a6ba0 con 0x55db9b499670 2024-09-19T04:24:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f038dc64740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55db9b3ba0f0 con 0x55db9b498d50 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f038dc64740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55db9b4a6d20 con 0x55db9b4a3f30 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f038dc64740 10 client.0 _ll_register_callbacks cb 0x55db9b3ab9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f037b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f037b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32768/0 (socket says 172.21.15.107:32768) 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f037b400640 1 -- 172.21.15.107:0/2513603677 learned_addr learned my addr 172.21.15.107:0/2513603677 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f0389000640 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 0x55db9b4a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.852+0000 7f0383e00640 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b498d50 0x55db9b4a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f037b400640 1 -- 172.21.15.107:0/2513603677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 msgr2=0x55db9b4a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f037b400640 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 0x55db9b4a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f037b400640 1 -- 172.21.15.107:0/2513603677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b498d50 msgr2=0x55db9b4a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f037b400640 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b498d50 0x55db9b4a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f037b400640 1 -- 172.21.15.107:0/2513603677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9b1703c0 con 0x55db9b499670 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f0389000640 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 0x55db9b4a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f037b400640 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a1390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f036c09c470 tx=0x7f036c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b1ae45bb8bf43b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f0372000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f0372000640 1 -- 172.21.15.107:0/2513603677 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c09c2b0 con 0x55db9b499670 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f0372000640 1 -- 172.21.15.107:0/2513603677 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f036c09bb30 con 0x55db9b499670 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f038dc64740 1 -- 172.21.15.107:0/2513603677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0374036070 con 0x55db9b499670 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f0372000640 1 -- 172.21.15.107:0/2513603677 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f036c0cdde0 con 0x55db9b499670 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f038dc64740 10 client.15999 fetch_fsmap learned FSMap version 32 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.853+0000 7f038dc64740 1 -- 172.21.15.107:0/2513603677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db9b4b2960 con 0x55db9b499670 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.854+0000 7f0372000640 1 -- 172.21.15.107:0/2513603677 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0374036070 con 0x55db9b499670 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.854+0000 7f0372000640 1 -- 172.21.15.107:0/2513603677 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f036c09b430 con 0x55db9b499670 2024-09-19T04:24:38.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.854+0000 7f038dc64740 10 client.15999 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.854+0000 7f038dc64740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.854+0000 7f038dc64740 20 client.15999 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.854+0000 7f038dc64740 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55db9b4b3e20 0x55db9b4b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.854+0000 7f038dc64740 4 client.15999 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.854+0000 7f038dc64740 1 -- 172.21.15.107:0/2513603677 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db9b40c950 con 0x55db9b4b3e20 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.855+0000 7f0383e00640 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55db9b4b3e20 0x55db9b4b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.855+0000 7f0383e00640 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55db9b4b3e20 0x55db9b4b6210 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa6e1caa9c8244cb server_cookie=1f48f2235efe8a81 in_seq=0 out_seq=0 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.855+0000 7f0372000640 10 client.15999 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f0372000640 1 -- 172.21.15.107:0/2513603677 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db9b40c950 con 0x55db9b4b3e20 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f0372000640 10 client.15999 handle_command_reply: tid=0 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 client.15999 shutdown 2024-09-19T04:24:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 -- 172.21.15.107:0/2513603677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 msgr2=0x55db9b4a1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a1390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f036c09c470 tx=0x7f036c00cf90 comp rx=0 tx=0).stop 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 -- 172.21.15.107:0/2513603677 shutdown_connections 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55db9b4b3e20 0x55db9b4b6210 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b4a3f30 0x55db9b4a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4a1390 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 --2- 172.21.15.107:0/2513603677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b498d50 0x55db9b4a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 -- 172.21.15.107:0/2513603677 >> 172.21.15.107:0/2513603677 conn(0x55db9b4952e0 msgr2=0x55db9b49b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f0372000640 0 client.15999 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 -- 172.21.15.107:0/2513603677 shutdown_connections 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 1 -- 172.21.15.107:0/2513603677 wait complete. 2024-09-19T04:24:38.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.856+0000 7f038dc64740 20 client.15999 trim_cache size 0 max 16384 2024-09-19T04:24:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.874+0000 7f038dc64740 1 Processor -- start 2024-09-19T04:24:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.874+0000 7f038dc64740 1 -- start start 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.874+0000 7f038dc64740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b498d50 0x55db9b4a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.874+0000 7f038dc64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.874+0000 7f038dc64740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b4a3f30 0x55db9b4bc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.874+0000 7f038dc64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03841af850 con 0x55db9b499670 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.874+0000 7f038dc64740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b1450 con 0x55db9b4a3f30 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.874+0000 7f038dc64740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03841aff20 con 0x55db9b498d50 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f037b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f0383e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b498d50 0x55db9b4a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f037b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4bbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32774/0 (socket says 172.21.15.107:32774) 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f037b400640 1 -- 172.21.15.107:0/4071898293 learned_addr learned my addr 172.21.15.107:0/4071898293 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f0389000640 1 --2- 172.21.15.107:0/4071898293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b4a3f30 0x55db9b4bc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f037b400640 1 -- 172.21.15.107:0/4071898293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b498d50 msgr2=0x55db9b4a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f037b400640 1 --2- 172.21.15.107:0/4071898293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b498d50 0x55db9b4a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f037b400640 1 -- 172.21.15.107:0/4071898293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b4a3f30 msgr2=0x55db9b4bc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f037b400640 1 --2- 172.21.15.107:0/4071898293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b4a3f30 0x55db9b4bc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f037b400640 1 -- 172.21.15.107:0/4071898293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9b4a6680 con 0x55db9b499670 2024-09-19T04:24:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f0383e00640 1 --2- 172.21.15.107:0/4071898293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b498d50 0x55db9b4a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f037b400640 1 --2- 172.21.15.107:0/4071898293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4bbad0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f036c09c1e0 tx=0x7f036c098180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ea1ef10d8603a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.875+0000 7f0378c00640 1 -- 172.21.15.107:0/4071898293 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c09be40 con 0x55db9b499670 2024-09-19T04:24:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f0378c00640 1 -- 172.21.15.107:0/4071898293 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c09c4e0 con 0x55db9b499670 2024-09-19T04:24:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f0378c00640 1 -- 172.21.15.107:0/4071898293 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c09c020 con 0x55db9b499670 2024-09-19T04:24:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f038dc64740 1 -- 172.21.15.107:0/4071898293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 msgr2=0x55db9b4bbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f038dc64740 1 --2- 172.21.15.107:0/4071898293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4bbad0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f036c09c1e0 tx=0x7f036c098180 comp rx=0 tx=0).stop 2024-09-19T04:24:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f038dc64740 1 -- 172.21.15.107:0/4071898293 shutdown_connections 2024-09-19T04:24:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f038dc64740 1 --2- 172.21.15.107:0/4071898293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b4a3f30 0x55db9b4bc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f038dc64740 1 --2- 172.21.15.107:0/4071898293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b499670 0x55db9b4bbad0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f038dc64740 1 --2- 172.21.15.107:0/4071898293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b498d50 0x55db9b4a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f038dc64740 1 -- 172.21.15.107:0/4071898293 >> 172.21.15.107:0/4071898293 conn(0x55db9b4952e0 msgr2=0x55db9b4a26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f038dc64740 1 -- 172.21.15.107:0/4071898293 shutdown_connections 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.876+0000 7f038dc64740 1 -- 172.21.15.107:0/4071898293 wait complete. 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.877+0000 7f038dc64740 1 -- start start 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.877+0000 7f038dc64740 1 Processor -- start 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.877+0000 7f038dc64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b498d50 0x55db9b4ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.877+0000 7f038dc64740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b499670 0x55db9b4ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.877+0000 7f038dc64740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b4a3f30 0x55db9b49d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.877+0000 7f038dc64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b0cc0 con 0x55db9b498d50 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.877+0000 7f038dc64740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b05f0 con 0x55db9b4a3f30 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.877+0000 7f038dc64740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03841ad830 con 0x55db9b499670 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.877+0000 7f038dc64740 10 client.0 _ll_register_callbacks cb 0x7f036c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f0383e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b498d50 0x55db9b4ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f0383e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b498d50 0x55db9b4ac210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32786/0 (socket says 172.21.15.107:32786) 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f0383e00640 1 -- 172.21.15.107:0/1251614114 learned_addr learned my addr 172.21.15.107:0/1251614114 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f037b400640 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b499670 0x55db9b4ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f0389000640 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b4a3f30 0x55db9b49d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f037b400640 1 -- 172.21.15.107:0/1251614114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b4a3f30 msgr2=0x55db9b49d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f037b400640 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b4a3f30 0x55db9b49d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f037b400640 1 -- 172.21.15.107:0/1251614114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b498d50 msgr2=0x55db9b4ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f037b400640 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b498d50 0x55db9b4ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f037b400640 1 -- 172.21.15.107:0/1251614114 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9b49e030 con 0x55db9b499670 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f0383e00640 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b498d50 0x55db9b4ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f037b400640 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b499670 0x55db9b4ac750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55db9b4b0710 tx=0x7f036c09bbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=279cae022055fe82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:24:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.878+0000 7f0372000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:24:38.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f0372000640 1 -- 172.21.15.107:0/1251614114 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c09c020 con 0x55db9b499670 2024-09-19T04:24:38.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f0372000640 1 -- 172.21.15.107:0/1251614114 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f036c098230 con 0x55db9b499670 2024-09-19T04:24:38.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f038dc64740 1 -- 172.21.15.107:0/1251614114 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036c09c4e0 con 0x55db9b499670 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f0372000640 1 -- 172.21.15.107:0/1251614114 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f036c09be40 con 0x55db9b499670 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f038dc64740 10 client.26026 fetch_fsmap learned FSMap version 32 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f038dc64740 1 -- 172.21.15.107:0/1251614114 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db9b51e650 con 0x55db9b499670 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f0372000640 1 -- 172.21.15.107:0/1251614114 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c09c4e0 con 0x55db9b499670 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f0372000640 1 -- 172.21.15.107:0/1251614114 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f036c005040 con 0x55db9b499670 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f038dc64740 10 client.26026 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f038dc64740 10 client.26026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f038dc64740 20 client.26026 populate_metadata read hostname 'smithi107' 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.879+0000 7f038dc64740 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55db9b51ec00 0x55db9b521020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.880+0000 7f038dc64740 4 client.26026 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:24:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.880+0000 7f038dc64740 1 -- 172.21.15.107:0/1251614114 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db9b406e60 con 0x55db9b51ec00 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.880+0000 7f0383e00640 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55db9b51ec00 0x55db9b521020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.880+0000 7f0383e00640 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55db9b51ec00 0x55db9b521020 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7143e0adf999b8b9 server_cookie=f6e50aa112a944ed in_seq=0 out_seq=0 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.880+0000 7f0372000640 10 client.26026 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f0372000640 1 -- 172.21.15.107:0/1251614114 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55db9b406e60 con 0x55db9b51ec00 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f0372000640 10 client.26026 handle_command_reply: tid=0 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 client.26026 shutdown 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 -- 172.21.15.107:0/1251614114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b499670 msgr2=0x55db9b4ac750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b499670 0x55db9b4ac750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55db9b4b0710 tx=0x7f036c09bbc0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 -- 172.21.15.107:0/1251614114 shutdown_connections 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55db9b51ec00 0x55db9b521020 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db9b4a3f30 0x55db9b49d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55db9b499670 0x55db9b4ac750 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 --2- 172.21.15.107:0/1251614114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db9b498d50 0x55db9b4ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f0372000640 0 client.26026 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:24:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 -- 172.21.15.107:0/1251614114 >> 172.21.15.107:0/1251614114 conn(0x55db9b4952e0 msgr2=0x55db9b49b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 -- 172.21.15.107:0/1251614114 shutdown_connections 2024-09-19T04:24:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 1 -- 172.21.15.107:0/1251614114 wait complete. 2024-09-19T04:24:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.881+0000 7f038dc64740 20 client.26026 trim_cache size 0 max 16384 2024-09-19T04:24:38.887 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:24:38.887 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:24:38.887 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:24:38.887 INFO:teuthology.orchestra.run.smithi107.stdout: "93a9cada-7d44-4565-ba6d-f0babffad31f": { 2024-09-19T04:24:38.887 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:24:38.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "93a9cada-7d44-4565-ba6d-f0babffad31f", 2024-09-19T04:24:38.888 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:24:38.888 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:24:38.888 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:24:38.888 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:24:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 -- 172.21.15.107:0/2220640708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f03680a33b0 msgr2=0x7f03680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f03680a33b0 0x7f03680a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f037c009d20 tx=0x7f037c005f90 comp rx=0 tx=0).stop 2024-09-19T04:24:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 -- 172.21.15.107:0/2220640708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0384102360 msgr2=0x7f038419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:24:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0384102360 0x7f038419aa40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f036c00e9f0 tx=0x7f036c00edf0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 -- 172.21.15.107:0/2220640708 shutdown_connections 2024-09-19T04:24:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f03680a33b0 0x7f03680a5870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03841079c0 0x7f038419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0384102360 0x7f038419aa40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 --2- 172.21.15.107:0/2220640708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03841019c0 0x7f038419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:24:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 -- 172.21.15.107:0/2220640708 >> 172.21.15.107:0/2220640708 conn(0x7f03840fd740 msgr2=0x7f038410cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:24:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 -- 172.21.15.107:0/2220640708 shutdown_connections 2024-09-19T04:24:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:24:38.882+0000 7f0389a00640 1 -- 172.21.15.107:0/2220640708 wait complete. 2024-09-19T04:24:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:38 smithi107 ceph-mon[30659]: pgmap v132: 97 pgs: 97 active+clean; 102 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 843 KiB/s wr, 104 op/s 2024-09-19T04:24:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:38 smithi115 ceph-mon[35477]: pgmap v132: 97 pgs: 97 active+clean; 102 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 843 KiB/s wr, 104 op/s 2024-09-19T04:24:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:38 smithi145 ceph-mon[35096]: pgmap v132: 97 pgs: 97 active+clean; 102 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 843 KiB/s wr, 104 op/s 2024-09-19T04:24:39.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 161533 3.11 MB/sec execute 57 sec latency 1868.766 ms 2024-09-19T04:24:40.031 INFO:tasks.cephfs.filesystem:scrub status for tag:93a9cada-7d44-4565-ba6d-f0babffad31f - {'path': '/', 'tag': '93a9cada-7d44-4565-ba6d-f0babffad31f', 'options': 'recursive,force'} 2024-09-19T04:24:40.659 INFO:tasks.workunit.client.0.smithi107.stdout: 1 163571 3.29 MB/sec execute 58 sec latency 13.031 ms 2024-09-19T04:24:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:40 smithi115 ceph-mon[35477]: pgmap v133: 97 pgs: 97 active+clean; 102 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 843 KiB/s wr, 104 op/s 2024-09-19T04:24:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:40 smithi145 ceph-mon[35096]: pgmap v133: 97 pgs: 97 active+clean; 102 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 843 KiB/s wr, 104 op/s 2024-09-19T04:24:41.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:40 smithi107 ceph-mon[30659]: pgmap v133: 97 pgs: 97 active+clean; 102 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 843 KiB/s wr, 104 op/s 2024-09-19T04:24:41.660 INFO:tasks.workunit.client.0.smithi107.stdout: 1 163974 3.24 MB/sec execute 59 sec latency 609.874 ms 2024-09-19T04:24:42.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:41 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:24:42.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:41 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:24:42.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:41 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:24:42.660 INFO:tasks.workunit.client.0.smithi107.stdout: 1 163974 3.19 MB/sec execute 60 sec latency 1610.019 ms 2024-09-19T04:24:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:42 smithi115 ceph-mon[35477]: pgmap v134: 97 pgs: 97 active+clean; 104 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T04:24:43.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:42 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:24:43.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:42 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:24:43.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:42 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:24:43.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:42 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:24:43.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:42 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:24:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:42 smithi145 ceph-mon[35096]: pgmap v134: 97 pgs: 97 active+clean; 104 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T04:24:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:42 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:24:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:42 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:24:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:42 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:24:43.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:42 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:24:43.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:42 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:24:43.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:42 smithi107 ceph-mon[30659]: pgmap v134: 97 pgs: 97 active+clean; 104 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-09-19T04:24:43.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:42 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:24:43.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:42 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:24:43.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:42 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:24:43.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:42 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:24:43.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:42 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:24:43.660 INFO:tasks.workunit.client.0.smithi107.stdout: 1 163974 3.14 MB/sec execute 61 sec latency 2610.160 ms 2024-09-19T04:24:44.660 INFO:tasks.workunit.client.0.smithi107.stdout: 1 165150 3.17 MB/sec execute 62 sec latency 3165.987 ms 2024-09-19T04:24:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:44 smithi115 ceph-mon[35477]: pgmap v135: 97 pgs: 97 active+clean; 104 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 849 KiB/s wr, 110 op/s 2024-09-19T04:24:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:44 smithi145 ceph-mon[35096]: pgmap v135: 97 pgs: 97 active+clean; 104 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 849 KiB/s wr, 110 op/s 2024-09-19T04:24:45.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:44 smithi107 ceph-mon[30659]: pgmap v135: 97 pgs: 97 active+clean; 104 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 849 KiB/s wr, 110 op/s 2024-09-19T04:24:45.660 INFO:tasks.workunit.client.0.smithi107.stdout: 1 167098 3.33 MB/sec execute 63 sec latency 11.683 ms 2024-09-19T04:24:46.660 INFO:tasks.workunit.client.0.smithi107.stdout: 1 167455 3.29 MB/sec execute 64 sec latency 628.984 ms 2024-09-19T04:24:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:46 smithi115 ceph-mon[35477]: pgmap v136: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-09-19T04:24:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:46 smithi145 ceph-mon[35096]: pgmap v136: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-09-19T04:24:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:46 smithi107 ceph-mon[30659]: pgmap v136: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-09-19T04:24:47.660 INFO:tasks.workunit.client.0.smithi107.stdout: 1 167455 3.24 MB/sec execute 65 sec latency 1629.079 ms 2024-09-19T04:24:48.660 INFO:tasks.workunit.client.0.smithi107.stdout: 1 167455 3.19 MB/sec execute 66 sec latency 2629.207 ms 2024-09-19T04:24:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:48 smithi115 ceph-mon[35477]: pgmap v137: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 753 KiB/s wr, 102 op/s 2024-09-19T04:24:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:48 smithi145 ceph-mon[35096]: pgmap v137: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 753 KiB/s wr, 102 op/s 2024-09-19T04:24:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:48 smithi107 ceph-mon[30659]: pgmap v137: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 753 KiB/s wr, 102 op/s 2024-09-19T04:24:49.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 167523 3.14 MB/sec execute 67 sec latency 3183.237 ms 2024-09-19T04:24:50.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 167523 3.09 MB/sec execute 68 sec latency 1399.298 ms 2024-09-19T04:24:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:50 smithi115 ceph-mon[35477]: pgmap v138: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 753 KiB/s wr, 102 op/s 2024-09-19T04:24:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:50 smithi145 ceph-mon[35096]: pgmap v138: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 753 KiB/s wr, 102 op/s 2024-09-19T04:24:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:50 smithi107 ceph-mon[30659]: pgmap v138: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 753 KiB/s wr, 102 op/s 2024-09-19T04:24:51.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 167523 3.05 MB/sec execute 69 sec latency 2399.435 ms 2024-09-19T04:24:52.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 167523 3.01 MB/sec execute 70 sec latency 3399.563 ms 2024-09-19T04:24:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:52 smithi115 ceph-mon[35477]: pgmap v139: 97 pgs: 97 active+clean; 108 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-19T04:24:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:52 smithi145 ceph-mon[35096]: pgmap v139: 97 pgs: 97 active+clean; 108 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-19T04:24:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:52 smithi107 ceph-mon[30659]: pgmap v139: 97 pgs: 97 active+clean; 108 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-19T04:24:53.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 167523 2.96 MB/sec execute 71 sec latency 4399.703 ms 2024-09-19T04:24:54.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 168436 2.99 MB/sec execute 72 sec latency 4954.014 ms 2024-09-19T04:24:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:54 smithi115 ceph-mon[35477]: pgmap v140: 97 pgs: 97 active+clean; 108 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 620 KiB/s wr, 79 op/s 2024-09-19T04:24:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:54 smithi145 ceph-mon[35096]: pgmap v140: 97 pgs: 97 active+clean; 108 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 620 KiB/s wr, 79 op/s 2024-09-19T04:24:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:54 smithi107 ceph-mon[30659]: pgmap v140: 97 pgs: 97 active+clean; 108 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 620 KiB/s wr, 79 op/s 2024-09-19T04:24:55.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 170637 3.14 MB/sec execute 73 sec latency 11.169 ms 2024-09-19T04:24:56.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 172671 3.19 MB/sec execute 74 sec latency 12.809 ms 2024-09-19T04:24:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:56 smithi115 ceph-mon[35477]: pgmap v141: 97 pgs: 97 active+clean; 109 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 787 KiB/s wr, 111 op/s 2024-09-19T04:24:57.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:56 smithi107 ceph-mon[30659]: pgmap v141: 97 pgs: 97 active+clean; 109 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 787 KiB/s wr, 111 op/s 2024-09-19T04:24:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:56 smithi145 ceph-mon[35096]: pgmap v141: 97 pgs: 97 active+clean; 109 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 787 KiB/s wr, 111 op/s 2024-09-19T04:24:57.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 173827 3.29 MB/sec execute 75 sec latency 481.420 ms 2024-09-19T04:24:58.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 174489 3.27 MB/sec execute 76 sec latency 584.117 ms 2024-09-19T04:24:58.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:24:58 smithi107 ceph-mon[30659]: pgmap v142: 97 pgs: 97 active+clean; 109 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 520 KiB/s wr, 66 op/s 2024-09-19T04:24:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:24:58 smithi115 ceph-mon[35477]: pgmap v142: 97 pgs: 97 active+clean; 109 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 520 KiB/s wr, 66 op/s 2024-09-19T04:24:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:24:58 smithi145 ceph-mon[35096]: pgmap v142: 97 pgs: 97 active+clean; 109 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 520 KiB/s wr, 66 op/s 2024-09-19T04:24:59.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 175471 3.29 MB/sec execute 77 sec latency 733.332 ms 2024-09-19T04:25:00.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 177348 3.41 MB/sec execute 78 sec latency 24.853 ms 2024-09-19T04:25:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:00 smithi115 ceph-mon[35477]: pgmap v143: 97 pgs: 97 active+clean; 109 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 520 KiB/s wr, 66 op/s 2024-09-19T04:25:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:00 smithi145 ceph-mon[35096]: pgmap v143: 97 pgs: 97 active+clean; 109 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 520 KiB/s wr, 66 op/s 2024-09-19T04:25:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:00 smithi107 ceph-mon[30659]: pgmap v143: 97 pgs: 97 active+clean; 109 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 520 KiB/s wr, 66 op/s 2024-09-19T04:25:01.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 178088 3.39 MB/sec execute 79 sec latency 278.447 ms 2024-09-19T04:25:02.661 INFO:tasks.workunit.client.0.smithi107.stdout: 1 178088 3.35 MB/sec execute 80 sec latency 1278.513 ms 2024-09-19T04:25:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:02 smithi115 ceph-mon[35477]: pgmap v144: 97 pgs: 97 active+clean; 113 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-09-19T04:25:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:02 smithi145 ceph-mon[35096]: pgmap v144: 97 pgs: 97 active+clean; 113 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-09-19T04:25:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:02 smithi107 ceph-mon[30659]: pgmap v144: 97 pgs: 97 active+clean; 113 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-09-19T04:25:03.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 178088 3.30 MB/sec execute 81 sec latency 2278.609 ms 2024-09-19T04:25:04.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 178975 3.32 MB/sec execute 82 sec latency 2831.158 ms 2024-09-19T04:25:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:04 smithi115 ceph-mon[35477]: pgmap v145: 97 pgs: 97 active+clean; 113 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 922 KiB/s wr, 127 op/s 2024-09-19T04:25:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:04 smithi145 ceph-mon[35096]: pgmap v145: 97 pgs: 97 active+clean; 113 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 922 KiB/s wr, 127 op/s 2024-09-19T04:25:05.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:04 smithi107 ceph-mon[30659]: pgmap v145: 97 pgs: 97 active+clean; 113 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 922 KiB/s wr, 127 op/s 2024-09-19T04:25:05.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 181146 3.45 MB/sec execute 83 sec latency 12.567 ms 2024-09-19T04:25:06.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 181703 3.42 MB/sec execute 84 sec latency 359.725 ms 2024-09-19T04:25:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:06 smithi115 ceph-mon[35477]: pgmap v146: 97 pgs: 97 active+clean; 116 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2024-09-19T04:25:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:06 smithi145 ceph-mon[35096]: pgmap v146: 97 pgs: 97 active+clean; 116 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2024-09-19T04:25:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:06 smithi107 ceph-mon[30659]: pgmap v146: 97 pgs: 97 active+clean; 116 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2024-09-19T04:25:07.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 181703 3.38 MB/sec execute 85 sec latency 1359.794 ms 2024-09-19T04:25:08.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 181703 3.34 MB/sec execute 86 sec latency 2359.890 ms 2024-09-19T04:25:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:08 smithi115 ceph-mon[35477]: pgmap v147: 97 pgs: 97 active+clean; 116 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-19T04:25:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:08 smithi145 ceph-mon[35096]: pgmap v147: 97 pgs: 97 active+clean; 116 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-19T04:25:09.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:08 smithi107 ceph-mon[30659]: pgmap v147: 97 pgs: 97 active+clean; 116 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-19T04:25:09.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 182804 3.36 MB/sec execute 87 sec latency 2912.261 ms 2024-09-19T04:25:10.032 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:25:10.441 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:10.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 184783 3.47 MB/sec execute 88 sec latency 13.974 ms 2024-09-19T04:25:11.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.085+0000 7f7162c00640 1 -- 172.21.15.107:0/3010820539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f715c1049a0 msgr2=0x7f715c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.085+0000 7f7162c00640 1 --2- 172.21.15.107:0/3010820539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f715c1049a0 0x7f715c1089f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7144009900 tx=0x7f714402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.085+0000 7f7162c00640 1 -- 172.21.15.107:0/3010820539 shutdown_connections 2024-09-19T04:25:11.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.085+0000 7f7162c00640 1 --2- 172.21.15.107:0/3010820539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f715c109120 0x7f715c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.085+0000 7f7162c00640 1 --2- 172.21.15.107:0/3010820539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f715c1049a0 0x7f715c1089f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.085+0000 7f7162c00640 1 --2- 172.21.15.107:0/3010820539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f715c103ff0 0x7f715c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.085+0000 7f7162c00640 1 -- 172.21.15.107:0/3010820539 >> 172.21.15.107:0/3010820539 conn(0x7f715c0ffa90 msgr2=0x7f715c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:11.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.085+0000 7f7162c00640 1 -- 172.21.15.107:0/3010820539 shutdown_connections 2024-09-19T04:25:11.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.085+0000 7f7162c00640 1 -- 172.21.15.107:0/3010820539 wait complete. 2024-09-19T04:25:11.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.086+0000 7f7162c00640 1 Processor -- start 2024-09-19T04:25:11.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.086+0000 7f7162c00640 1 -- start start 2024-09-19T04:25:11.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.086+0000 7f7162c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f715c103ff0 0x7f715c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.086+0000 7f7162c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f715c1049a0 0x7f715c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.086+0000 7f7162c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f715c109120 0x7f715c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.086+0000 7f7162c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1141a0 con 0x7f715c103ff0 2024-09-19T04:25:11.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.086+0000 7f7162c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f715c114020 con 0x7f715c109120 2024-09-19T04:25:11.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.086+0000 7f7162c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f715c114320 con 0x7f715c1049a0 2024-09-19T04:25:11.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.087+0000 7f715be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f715c103ff0 0x7f715c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.087+0000 7f715be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f715c103ff0 0x7f715c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50666/0 (socket says 172.21.15.107:50666) 2024-09-19T04:25:11.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.087+0000 7f715be00640 1 -- 172.21.15.107:0/2528613976 learned_addr learned my addr 172.21.15.107:0/2528613976 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:11.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.087+0000 7f7160c00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f715c109120 0x7f715c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.087+0000 7f715be00640 1 -- 172.21.15.107:0/2528613976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f715c1049a0 msgr2=0x7f715c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.087+0000 7f715be00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f715c1049a0 0x7f715c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.087+0000 7f715be00640 1 -- 172.21.15.107:0/2528613976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f715c109120 msgr2=0x7f715c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.087+0000 7f715be00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f715c109120 0x7f715c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.087+0000 7f715be00640 1 -- 172.21.15.107:0/2528613976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c19f630 con 0x7f715c103ff0 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.087+0000 7f715be00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f715c103ff0 0x7f715c19a680 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e9f0 tx=0x7f714c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.088+0000 7f7159600640 1 -- 172.21.15.107:0/2528613976 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c00cd80 con 0x7f715c103ff0 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.088+0000 7f7159600640 1 -- 172.21.15.107:0/2528613976 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714c010910 con 0x7f715c103ff0 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.088+0000 7f7162c00640 1 -- 172.21.15.107:0/2528613976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c19f920 con 0x7f715c103ff0 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.088+0000 7f7159600640 1 -- 172.21.15.107:0/2528613976 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c018990 con 0x7f715c103ff0 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.088+0000 7f7162c00640 1 -- 172.21.15.107:0/2528613976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c1a7200 con 0x7f715c103ff0 2024-09-19T04:25:11.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.091+0000 7f71657e4740 1 Processor -- start 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.092+0000 7f71657e4740 1 -- start start 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.092+0000 7f71657e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a76153f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.092+0000 7f71657e4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 0x563a7615e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.092+0000 7f71657e4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a7615ed70 0x563a76161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.092+0000 7f71657e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563a760ba3e0 con 0x563a76153b90 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.092+0000 7f71657e4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563a76061d80 con 0x563a761544b0 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.092+0000 7f71657e4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563a760b63a0 con 0x563a7615ed70 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.092+0000 7f7159600640 1 -- 172.21.15.107:0/2528613976 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f714c010ac0 con 0x7f715c103ff0 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.092+0000 7f7160c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a76153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f715be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 0x563a7615e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f7160c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a76153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50668/0 (socket says 172.21.15.107:50668) 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f7160c00640 1 -- 172.21.15.107:0/1255930409 learned_addr learned my addr 172.21.15.107:0/1255930409 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f7153400640 1 --2- 172.21.15.107:0/1255930409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a7615ed70 0x563a76161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f715be00640 1 -- 172.21.15.107:0/1255930409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a7615ed70 msgr2=0x563a76161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f715be00640 1 --2- 172.21.15.107:0/1255930409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a7615ed70 0x563a76161160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f715be00640 1 -- 172.21.15.107:0/1255930409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 msgr2=0x563a76153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f715be00640 1 --2- 172.21.15.107:0/1255930409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a76153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f715be00640 1 -- 172.21.15.107:0/1255930409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a76080ae0 con 0x563a761544b0 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f7160c00640 1 --2- 172.21.15.107:0/1255930409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a76153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f715be00640 1 --2- 172.21.15.107:0/1255930409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 0x563a7615e830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f714c0cd830 tx=0x7f714c0d8780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ca1c9ae111ff19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.093+0000 7f7151200640 1 -- 172.21.15.107:0/1255930409 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c0e3070 con 0x563a761544b0 2024-09-19T04:25:11.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f7151200640 1 -- 172.21.15.107:0/1255930409 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714c0d09c0 con 0x563a761544b0 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f7151200640 1 -- 172.21.15.107:0/1255930409 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c0de9f0 con 0x563a761544b0 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f7159600640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71400a33b0 0x7f71400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f7159600640 1 -- 172.21.15.107:0/2528613976 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f714c0d00f0 con 0x7f715c103ff0 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f7160c00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71400a33b0 0x7f71400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f71657e4740 1 -- 172.21.15.107:0/1255930409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 msgr2=0x563a7615e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f71657e4740 1 --2- 172.21.15.107:0/1255930409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 0x563a7615e830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f714c0cd830 tx=0x7f714c0d8780 comp rx=0 tx=0).stop 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f71657e4740 1 -- 172.21.15.107:0/1255930409 shutdown_connections 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f71657e4740 1 --2- 172.21.15.107:0/1255930409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a7615ed70 0x563a76161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f71657e4740 1 --2- 172.21.15.107:0/1255930409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 0x563a7615e830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f71657e4740 1 --2- 172.21.15.107:0/1255930409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a76153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f71657e4740 1 -- 172.21.15.107:0/1255930409 >> 172.21.15.107:0/1255930409 conn(0x563a76150120 msgr2=0x563a76150500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.094+0000 7f71657e4740 1 -- 172.21.15.107:0/1255930409 shutdown_connections 2024-09-19T04:25:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.095+0000 7f71657e4740 1 -- 172.21.15.107:0/1255930409 wait complete. 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.095+0000 7f7160c00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71400a33b0 0x7f71400a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7154009fd0 tx=0x7f715400d040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.095+0000 7f71657e4740 1 -- start start 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.095+0000 7f71657e4740 1 Processor -- start 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f71657e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a7615bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f71657e4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 0x563a7615c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f71657e4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 0x563a7615c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f71657e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563a761619e0 con 0x563a76153b90 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f71657e4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563a760750f0 con 0x563a7615ed70 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f71657e4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563a76161b60 con 0x563a761544b0 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f71657e4740 10 client.0 _ll_register_callbacks cb 0x563a760669e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f715be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a7615bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f7160c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 0x563a7615c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f7153400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 0x563a7615c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f715be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a7615bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50682/0 (socket says 172.21.15.107:50682) 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f715be00640 1 -- 172.21.15.107:0/4136320798 learned_addr learned my addr 172.21.15.107:0/4136320798 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f715be00640 1 -- 172.21.15.107:0/4136320798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 msgr2=0x563a7615c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f715be00640 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 0x563a7615c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f715be00640 1 -- 172.21.15.107:0/4136320798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 msgr2=0x563a7615c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f715be00640 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 0x563a7615c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.096+0000 7f715be00640 1 -- 172.21.15.107:0/4136320798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a75e2b3c0 con 0x563a76153b90 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.097+0000 7f715be00640 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a7615bc50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f714c098820 tx=0x7f714c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b9728f8f3e5717 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.097+0000 7f7149600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.097+0000 7f7149600640 1 -- 172.21.15.107:0/4136320798 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c098660 con 0x563a76153b90 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.097+0000 7f7149600640 1 -- 172.21.15.107:0/4136320798 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f714c0d3380 con 0x563a76153b90 2024-09-19T04:25:11.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.097+0000 7f71657e4740 1 -- 172.21.15.107:0/4136320798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f714c0d09c0 con 0x563a76153b90 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.097+0000 7f7149600640 1 -- 172.21.15.107:0/4136320798 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f714c09bd20 con 0x563a76153b90 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.097+0000 7f71657e4740 10 client.16017 fetch_fsmap learned FSMap version 33 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.097+0000 7f71657e4740 1 -- 172.21.15.107:0/4136320798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563a761619e0 con 0x563a76153b90 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.097+0000 7f7149600640 1 -- 172.21.15.107:0/4136320798 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714c0d09c0 con 0x563a76153b90 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.098+0000 7f7149600640 1 -- 172.21.15.107:0/4136320798 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f714c0cdab0 con 0x563a76153b90 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.098+0000 7f71657e4740 10 client.16017 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.098+0000 7f71657e4740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.098+0000 7f71657e4740 20 client.16017 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.098+0000 7f71657e4740 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563a7616e810 0x563a76170c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.098+0000 7f71657e4740 4 client.16017 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.098+0000 7f71657e4740 1 -- 172.21.15.107:0/4136320798 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a760c7950 con 0x563a7616e810 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.098+0000 7f7153400640 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563a7616e810 0x563a76170c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.099+0000 7f7153400640 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563a7616e810 0x563a76170c00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a2421346b88610 server_cookie=32e175fb148c430b in_seq=0 out_seq=0 2024-09-19T04:25:11.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.099+0000 7f7149600640 10 client.16017 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:11.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.099+0000 7f7149600640 1 -- 172.21.15.107:0/4136320798 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a760c7950 con 0x563a7616e810 2024-09-19T04:25:11.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.099+0000 7f7149600640 10 client.16017 handle_command_reply: tid=0 2024-09-19T04:25:11.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 client.16017 shutdown 2024-09-19T04:25:11.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 -- 172.21.15.107:0/4136320798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 msgr2=0x563a7615bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a7615bc50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f714c098820 tx=0x7f714c00cf40 comp rx=0 tx=0).stop 2024-09-19T04:25:11.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 -- 172.21.15.107:0/4136320798 shutdown_connections 2024-09-19T04:25:11.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563a7616e810 0x563a76170c00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 0x563a7615c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 0x563a7615c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 --2- 172.21.15.107:0/4136320798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a76153b90 0x563a7615bc50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 -- 172.21.15.107:0/4136320798 >> 172.21.15.107:0/4136320798 conn(0x563a76150120 msgr2=0x563a76155e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:11.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f7149600640 0 client.16017 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:11.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 -- 172.21.15.107:0/4136320798 shutdown_connections 2024-09-19T04:25:11.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.100+0000 7f71657e4740 1 -- 172.21.15.107:0/4136320798 wait complete. 2024-09-19T04:25:11.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.101+0000 7f71657e4740 20 client.16017 trim_cache size 0 max 16384 2024-09-19T04:25:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.118+0000 7f71657e4740 1 Processor -- start 2024-09-19T04:25:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.118+0000 7f71657e4740 1 -- start start 2024-09-19T04:25:11.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f71657e4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 0x563a761565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f71657e4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 0x563a7614f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f71657e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a761761a0 0x563a76176620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f71657e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1afa20 con 0x563a761761a0 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f71657e4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1b1560 con 0x563a7615ed70 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f71657e4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1b00f0 con 0x563a761544b0 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f7160c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a761761a0 0x563a76176620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f715be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 0x563a761565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f7160c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a761761a0 0x563a76176620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50694/0 (socket says 172.21.15.107:50694) 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f715be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 0x563a761565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:45608/0 (socket says 172.21.15.107:45608) 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f7160c00640 1 -- 172.21.15.107:0/2164644159 learned_addr learned my addr 172.21.15.107:0/2164644159 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f7153400640 1 --2- 172.21.15.107:0/2164644159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 0x563a7614f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f715be00640 1 -- 172.21.15.107:0/2164644159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 msgr2=0x563a7614f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f715be00640 1 --2- 172.21.15.107:0/2164644159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 0x563a7614f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f715be00640 1 -- 172.21.15.107:0/2164644159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a761761a0 msgr2=0x563a76176620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:11.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f715be00640 1 --2- 172.21.15.107:0/2164644159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a761761a0 0x563a76176620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f715be00640 1 -- 172.21.15.107:0/2164644159 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a7615e5c0 con 0x563a761544b0 2024-09-19T04:25:11.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.119+0000 7f7153400640 1 --2- 172.21.15.107:0/2164644159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 0x563a7614f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:11.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.120+0000 7f715be00640 1 --2- 172.21.15.107:0/2164644159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 0x563a761565f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c6a0 tx=0x7f714c09a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ffcd7882163969c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.120+0000 7f714be00640 1 -- 172.21.15.107:0/2164644159 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c09bf00 con 0x563a761544b0 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.120+0000 7f714be00640 1 -- 172.21.15.107:0/2164644159 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714c0d3970 con 0x563a761544b0 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.120+0000 7f714be00640 1 -- 172.21.15.107:0/2164644159 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c09c190 con 0x563a761544b0 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 -- 172.21.15.107:0/2164644159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 msgr2=0x563a761565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 --2- 172.21.15.107:0/2164644159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 0x563a761565f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c6a0 tx=0x7f714c09a8d0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 -- 172.21.15.107:0/2164644159 shutdown_connections 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 --2- 172.21.15.107:0/2164644159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a761761a0 0x563a76176620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 --2- 172.21.15.107:0/2164644159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a7615ed70 0x563a7614f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 --2- 172.21.15.107:0/2164644159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761544b0 0x563a761565f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 -- 172.21.15.107:0/2164644159 >> 172.21.15.107:0/2164644159 conn(0x563a76150120 msgr2=0x563a7614e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:11.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 -- 172.21.15.107:0/2164644159 shutdown_connections 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 -- 172.21.15.107:0/2164644159 wait complete. 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 -- start start 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.121+0000 7f71657e4740 1 Processor -- start 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f71657e4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 0x563a76162c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f71657e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a7615ed70 0x563a761631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f71657e4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761761a0 0x563a761636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f71657e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1b0e90 con 0x563a7615ed70 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f71657e4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1b07c0 con 0x563a761544b0 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f71657e4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1ada00 con 0x563a761761a0 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f71657e4740 10 client.0 _ll_register_callbacks cb 0x7f714c0d8170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:11.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f7153400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a7615ed70 0x563a761631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f7160c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761761a0 0x563a761636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f7160c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761761a0 0x563a761636f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:45622/0 (socket says 172.21.15.107:45622) 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f7160c00640 1 -- 172.21.15.107:0/1428419805 learned_addr learned my addr 172.21.15.107:0/1428419805 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f715be00640 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 0x563a76162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.122+0000 7f715be00640 1 -- 172.21.15.107:0/1428419805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761761a0 msgr2=0x563a761636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f715be00640 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761761a0 0x563a761636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f715be00640 1 -- 172.21.15.107:0/1428419805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a7615ed70 msgr2=0x563a761631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f715be00640 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a7615ed70 0x563a761631b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f715be00640 1 -- 172.21.15.107:0/1428419805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a761e0af0 con 0x563a761544b0 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f7160c00640 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761761a0 0x563a761636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:11.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f715be00640 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 0x563a76162c70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f714c0d8240 tx=0x7f714c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d657dd4cebaadf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f7149600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f7149600640 1 -- 172.21.15.107:0/1428419805 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c09c190 con 0x563a761544b0 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f7149600640 1 -- 172.21.15.107:0/1428419805 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f714c0b5170 con 0x563a761544b0 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f71657e4740 1 -- 172.21.15.107:0/1428419805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f714c0d3970 con 0x563a761544b0 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f7149600640 1 -- 172.21.15.107:0/1428419805 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f714c0b9030 con 0x563a761544b0 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f71657e4740 10 client.26075 fetch_fsmap learned FSMap version 33 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.123+0000 7f71657e4740 1 -- 172.21.15.107:0/1428419805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563a761e0af0 con 0x563a761544b0 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.124+0000 7f7149600640 1 -- 172.21.15.107:0/1428419805 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714c0d3970 con 0x563a761544b0 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.124+0000 7f7149600640 1 -- 172.21.15.107:0/1428419805 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f714c0a4ba0 con 0x563a761544b0 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.124+0000 7f71657e4740 10 client.26075 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.124+0000 7f71657e4740 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.124+0000 7f71657e4740 20 client.26075 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:11.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.124+0000 7f71657e4740 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563a761e1fd0 0x563a761e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:11.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.124+0000 7f71657e4740 4 client.26075 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:25:11.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.124+0000 7f71657e4740 1 -- 172.21.15.107:0/1428419805 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a760c1e60 con 0x563a761e1fd0 2024-09-19T04:25:11.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.124+0000 7f7153400640 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563a761e1fd0 0x563a761e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:11.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f7153400640 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563a761e1fd0 0x563a761e43c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21239a88cc5f2a3f server_cookie=99f86e20222ce060 in_seq=0 out_seq=0 2024-09-19T04:25:11.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f7149600640 10 client.26075 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:11.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f7149600640 1 -- 172.21.15.107:0/1428419805 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563a760c1e60 con 0x563a761e1fd0 2024-09-19T04:25:11.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f7149600640 10 client.26075 handle_command_reply: tid=0 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 client.26075 shutdown 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 -- 172.21.15.107:0/1428419805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 msgr2=0x563a76162c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 0x563a76162c70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f714c0d8240 tx=0x7f714c009770 comp rx=0 tx=0).stop 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 -- 172.21.15.107:0/1428419805 shutdown_connections 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563a761e1fd0 0x563a761e43c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563a761761a0 0x563a761636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a7615ed70 0x563a761631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 --2- 172.21.15.107:0/1428419805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a761544b0 0x563a76162c70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 -- 172.21.15.107:0/1428419805 >> 172.21.15.107:0/1428419805 conn(0x563a76150120 msgr2=0x563a7614e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f7149600640 0 client.26075 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 -- 172.21.15.107:0/1428419805 shutdown_connections 2024-09-19T04:25:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 1 -- 172.21.15.107:0/1428419805 wait complete. 2024-09-19T04:25:11.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.125+0000 7f71657e4740 20 client.26075 trim_cache size 0 max 16384 2024-09-19T04:25:11.133 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:25:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:25:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:25:11.133 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:25:11.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 -- 172.21.15.107:0/2528613976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71400a33b0 msgr2=0x7f71400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71400a33b0 0x7f71400a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7154009fd0 tx=0x7f715400d040 comp rx=0 tx=0).stop 2024-09-19T04:25:11.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 -- 172.21.15.107:0/2528613976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f715c103ff0 msgr2=0x7f715c19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:11.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f715c103ff0 0x7f715c19a680 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e9f0 tx=0x7f714c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 -- 172.21.15.107:0/2528613976 shutdown_connections 2024-09-19T04:25:11.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f71400a33b0 0x7f71400a5870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f715c109120 0x7f715c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f715c1049a0 0x7f715c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 --2- 172.21.15.107:0/2528613976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f715c103ff0 0x7f715c19a680 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:11.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 -- 172.21.15.107:0/2528613976 >> 172.21.15.107:0/2528613976 conn(0x7f715c0ffa90 msgr2=0x7f715c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:11.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 -- 172.21.15.107:0/2528613976 shutdown_connections 2024-09-19T04:25:11.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:11.126+0000 7f7162c00640 1 -- 172.21.15.107:0/2528613976 wait complete. 2024-09-19T04:25:11.140 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:10 smithi107 ceph-mon[30659]: pgmap v148: 97 pgs: 97 active+clean; 116 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-19T04:25:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:10 smithi115 ceph-mon[35477]: pgmap v148: 97 pgs: 97 active+clean; 116 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-19T04:25:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:10 smithi145 ceph-mon[35096]: pgmap v148: 97 pgs: 97 active+clean; 116 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-19T04:25:11.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 185122 3.44 MB/sec execute 89 sec latency 597.515 ms 2024-09-19T04:25:11.971 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:25:11.971 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:25:12.343 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:12.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 185122 3.40 MB/sec execute 90 sec latency 1597.561 ms 2024-09-19T04:25:13.069 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:12 smithi107 ceph-mon[30659]: pgmap v149: 97 pgs: 97 active+clean; 119 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-19T04:25:13.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.081+0000 7f80a3200640 1 -- 172.21.15.107:0/2074657272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a4103e30 msgr2=0x7f80a4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:13.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.081+0000 7f80a3200640 1 --2- 172.21.15.107:0/2074657272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a4103e30 0x7f80a4104210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8094009900 tx=0x7f809402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:25:13.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.081+0000 7f80a3200640 1 -- 172.21.15.107:0/2074657272 shutdown_connections 2024-09-19T04:25:13.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.081+0000 7f80a3200640 1 --2- 172.21.15.107:0/2074657272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a4108f60 0x7f80a4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:13.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.081+0000 7f80a3200640 1 --2- 172.21.15.107:0/2074657272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a41047e0 0x7f80a4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:13.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.081+0000 7f80a3200640 1 --2- 172.21.15.107:0/2074657272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a4103e30 0x7f80a4104210 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:13.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.081+0000 7f80a3200640 1 -- 172.21.15.107:0/2074657272 >> 172.21.15.107:0/2074657272 conn(0x7f80a40ff910 msgr2=0x7f80a4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:13.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.081+0000 7f80a3200640 1 -- 172.21.15.107:0/2074657272 shutdown_connections 2024-09-19T04:25:13.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.081+0000 7f80a3200640 1 -- 172.21.15.107:0/2074657272 wait complete. 2024-09-19T04:25:13.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.082+0000 7f80a3200640 1 Processor -- start 2024-09-19T04:25:13.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.082+0000 7f80a3200640 1 -- start start 2024-09-19T04:25:13.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.082+0000 7f80a3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a4103e30 0x7f80a419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:13.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.082+0000 7f80a3200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a41047e0 0x7f80a419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:13.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.082+0000 7f80a3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a4108f60 0x7f80a419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:13.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f80a4113fc0 con 0x7f80a4103e30 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a3200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80a4113e40 con 0x7f80a41047e0 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f80a4114140 con 0x7f80a4108f60 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a4103e30 0x7f80a419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a4103e30 0x7f80a419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50720/0 (socket says 172.21.15.107:50720) 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a1e00640 1 -- 172.21.15.107:0/4171740486 learned_addr learned my addr 172.21.15.107:0/4171740486 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a1400640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a41047e0 0x7f80a419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a2800640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a4108f60 0x7f80a419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a1e00640 1 -- 172.21.15.107:0/4171740486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a4108f60 msgr2=0x7f80a419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a1e00640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a4108f60 0x7f80a419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a1e00640 1 -- 172.21.15.107:0/4171740486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a41047e0 msgr2=0x7f80a419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a1e00640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a41047e0 0x7f80a419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.083+0000 7f80a1e00640 1 -- 172.21.15.107:0/4171740486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a419f470 con 0x7f80a4103e30 2024-09-19T04:25:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.084+0000 7f80a1e00640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a4103e30 0x7f80a419a4c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8094006010 tx=0x7f8094004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:13.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.084+0000 7f809aa00640 1 -- 172.21.15.107:0/4171740486 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809403f030 con 0x7f80a4103e30 2024-09-19T04:25:13.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.084+0000 7f809aa00640 1 -- 172.21.15.107:0/4171740486 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8094004440 con 0x7f80a4103e30 2024-09-19T04:25:13.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.084+0000 7f809aa00640 1 -- 172.21.15.107:0/4171740486 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094035070 con 0x7f80a4103e30 2024-09-19T04:25:13.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.084+0000 7f80a3200640 1 -- 172.21.15.107:0/4171740486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a419f700 con 0x7f80a4103e30 2024-09-19T04:25:13.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.084+0000 7f80a3200640 1 -- 172.21.15.107:0/4171740486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a419fb90 con 0x7f80a4103e30 2024-09-19T04:25:13.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.085+0000 7f80a3200640 1 -- 172.21.15.107:0/4171740486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a4105310 con 0x7f80a4103e30 2024-09-19T04:25:13.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.087+0000 7f809aa00640 1 -- 172.21.15.107:0/4171740486 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f8094042030 con 0x7f80a4103e30 2024-09-19T04:25:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.088+0000 7f809aa00640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f80880a3430 0x7f80880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.088+0000 7f809aa00640 1 -- 172.21.15.107:0/4171740486 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f80940efff0 con 0x7f80a4103e30 2024-09-19T04:25:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.089+0000 7f80a1400640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f80880a3430 0x7f80880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.089+0000 7f80a1400640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f80880a3430 0x7f80880a58f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80a410a820 tx=0x7f8090005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:13.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.093+0000 7f809aa00640 1 -- 172.21.15.107:0/4171740486 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f80940bc370 con 0x7f80a4103e30 2024-09-19T04:25:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:12 smithi115 ceph-mon[35477]: pgmap v149: 97 pgs: 97 active+clean; 119 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-19T04:25:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:12 smithi145 ceph-mon[35096]: pgmap v149: 97 pgs: 97 active+clean; 119 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-19T04:25:13.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.307+0000 7f80a3200640 1 -- 172.21.15.107:0/4171740486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80a4104210 con 0x7f80a4103e30 2024-09-19T04:25:13.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.308+0000 7f809aa00640 1 -- 172.21.15.107:0/4171740486 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11992 (secure 0 0 0) 0x7f80940c0670 con 0x7f80a4103e30 2024-09-19T04:25:13.310 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:25:13.310 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":33,"btime":"2024-09-19T04:24:41:825507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:24:41.704121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:25:13.310 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 33 2024-09-19T04:25:13.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 -- 172.21.15.107:0/4171740486 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f80880a3430 msgr2=0x7f80880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:13.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f80880a3430 0x7f80880a58f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80a410a820 tx=0x7f8090005f90 comp rx=0 tx=0).stop 2024-09-19T04:25:13.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 -- 172.21.15.107:0/4171740486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a4103e30 msgr2=0x7f80a419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:13.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a4103e30 0x7f80a419a4c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8094006010 tx=0x7f8094004040 comp rx=0 tx=0).stop 2024-09-19T04:25:13.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 -- 172.21.15.107:0/4171740486 shutdown_connections 2024-09-19T04:25:13.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f80880a3430 0x7f80880a58f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:13.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a4108f60 0x7f80a419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:13.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a41047e0 0x7f80a419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:13.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 --2- 172.21.15.107:0/4171740486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a4103e30 0x7f80a419a4c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:13.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 -- 172.21.15.107:0/4171740486 >> 172.21.15.107:0/4171740486 conn(0x7f80a40ff910 msgr2=0x7f80a4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:13.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 -- 172.21.15.107:0/4171740486 shutdown_connections 2024-09-19T04:25:13.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:13.310+0000 7f80a3200640 1 -- 172.21.15.107:0/4171740486 wait complete. 2024-09-19T04:25:13.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 185122 3.36 MB/sec execute 91 sec latency 2597.636 ms 2024-09-19T04:25:14.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:13 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/4171740486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:25:14.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:25:14.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:13 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/4171740486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:25:14.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:13 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/4171740486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:25:14.558 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:14.662 INFO:tasks.workunit.client.0.smithi107.stdout: 1 185866 3.38 MB/sec execute 92 sec latency 3153.498 ms 2024-09-19T04:25:14.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:14 smithi107 ceph-mon[30659]: pgmap v150: 97 pgs: 97 active+clean; 119 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-19T04:25:15.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.132+0000 7fc0b3c00640 1 -- 172.21.15.107:0/79024794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac104010 msgr2=0x7fc0ac1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.132+0000 7fc0b3c00640 1 --2- 172.21.15.107:0/79024794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac104010 0x7fc0ac1043f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009900 tx=0x7fc0a002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.133+0000 7fc0b3c00640 1 -- 172.21.15.107:0/79024794 shutdown_connections 2024-09-19T04:25:15.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.133+0000 7fc0b3c00640 1 --2- 172.21.15.107:0/79024794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ac109140 0x7fc0ac110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.133+0000 7fc0b3c00640 1 --2- 172.21.15.107:0/79024794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ac1049c0 0x7fc0ac108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.133+0000 7fc0b3c00640 1 --2- 172.21.15.107:0/79024794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac104010 0x7fc0ac1043f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.133+0000 7fc0b3c00640 1 -- 172.21.15.107:0/79024794 >> 172.21.15.107:0/79024794 conn(0x7fc0ac0ffa50 msgr2=0x7fc0ac101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.133+0000 7fc0b3c00640 1 -- 172.21.15.107:0/79024794 shutdown_connections 2024-09-19T04:25:15.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.133+0000 7fc0b3c00640 1 -- 172.21.15.107:0/79024794 wait complete. 2024-09-19T04:25:15.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b3c00640 1 Processor -- start 2024-09-19T04:25:15.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b3c00640 1 -- start start 2024-09-19T04:25:15.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b3c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ac104010 0x7fc0ac19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b3c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ac1049c0 0x7fc0ac19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b3c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac109140 0x7fc0ac19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b3c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1141c0 con 0x7fc0ac1049c0 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b3c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac114040 con 0x7fc0ac109140 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b3c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac114340 con 0x7fc0ac104010 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ac104010 0x7fc0ac19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ac104010 0x7fc0ac19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:45660/0 (socket says 172.21.15.107:45660) 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b1200640 1 -- 172.21.15.107:0/3991685474 learned_addr learned my addr 172.21.15.107:0/3991685474 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0abe00640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ac1049c0 0x7fc0ac19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.134+0000 7fc0b1c00640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac109140 0x7fc0ac19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.135+0000 7fc0b1200640 1 -- 172.21.15.107:0/3991685474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac109140 msgr2=0x7fc0ac19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.135+0000 7fc0b1200640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac109140 0x7fc0ac19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.135+0000 7fc0b1200640 1 -- 172.21.15.107:0/3991685474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ac1049c0 msgr2=0x7fc0ac19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.135+0000 7fc0b1200640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ac1049c0 0x7fc0ac19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.135+0000 7fc0b1200640 1 -- 172.21.15.107:0/3991685474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac19f690 con 0x7fc0ac104010 2024-09-19T04:25:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.135+0000 7fc0b1200640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ac104010 0x7fc0ac19a6e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0004770 tx=0x7fc0a0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.135+0000 7fc0a9600640 1 -- 172.21.15.107:0/3991685474 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a003f070 con 0x7fc0ac104010 2024-09-19T04:25:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.135+0000 7fc0a9600640 1 -- 172.21.15.107:0/3991685474 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a0031ca0 con 0x7fc0ac104010 2024-09-19T04:25:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.135+0000 7fc0b3c00640 1 -- 172.21.15.107:0/3991685474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac19f920 con 0x7fc0ac104010 2024-09-19T04:25:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.136+0000 7fc0a9600640 1 -- 172.21.15.107:0/3991685474 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0035030 con 0x7fc0ac104010 2024-09-19T04:25:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.136+0000 7fc0b3c00640 1 -- 172.21.15.107:0/3991685474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac1a7260 con 0x7fc0ac104010 2024-09-19T04:25:15.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.139+0000 7fc0b6681740 1 Processor -- start 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.139+0000 7fc0b6681740 1 -- start start 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0a9600640 1 -- 172.21.15.107:0/3991685474 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fc0a0042070 con 0x7fc0ac104010 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4724a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a4724a590 0x558a47254910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47254e50 0x558a47257240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b6681740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558a471b03e0 con 0x558a4724a590 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b6681740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558a47157d80 con 0x558a47254e50 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b6681740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558a471ac3a0 con 0x558a47249c70 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b1c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4724a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b1c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4724a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:45674/0 (socket says 172.21.15.107:45674) 2024-09-19T04:25:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b1c00640 1 -- 172.21.15.107:0/615893603 learned_addr learned my addr 172.21.15.107:0/615893603 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:15.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0abe00640 1 --2- 172.21.15.107:0/615893603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47254e50 0x558a47257240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b1200640 1 --2- 172.21.15.107:0/615893603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a4724a590 0x558a47254910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0abe00640 1 -- 172.21.15.107:0/615893603 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 msgr2=0x558a4724a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0abe00640 1 --2- 172.21.15.107:0/615893603 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4724a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0abe00640 1 -- 172.21.15.107:0/615893603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a4724a590 msgr2=0x558a47254910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0abe00640 1 --2- 172.21.15.107:0/615893603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a4724a590 0x558a47254910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0abe00640 1 -- 172.21.15.107:0/615893603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a47176ae0 con 0x558a47254e50 2024-09-19T04:25:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.140+0000 7fc0b1c00640 1 --2- 172.21.15.107:0/615893603 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4724a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.141+0000 7fc0abe00640 1 --2- 172.21.15.107:0/615893603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47254e50 0x558a47257240 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc098009bf0 tx=0x7fc098011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552f8a23f36fb862 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.141+0000 7fc09e000640 1 -- 172.21.15.107:0/615893603 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098009da0 con 0x558a47254e50 2024-09-19T04:25:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.141+0000 7fc09e000640 1 -- 172.21.15.107:0/615893603 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc098005020 con 0x558a47254e50 2024-09-19T04:25:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.141+0000 7fc09e000640 1 -- 172.21.15.107:0/615893603 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098004c60 con 0x558a47254e50 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.141+0000 7fc0a9600640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc0940a3430 0x7fc0940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0a9600640 1 -- 172.21.15.107:0/3991685474 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fc0a00f0100 con 0x7fc0ac104010 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0b1c00640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc0940a3430 0x7fc0940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0b6681740 1 -- 172.21.15.107:0/615893603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47254e50 msgr2=0x558a47257240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0b6681740 1 --2- 172.21.15.107:0/615893603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47254e50 0x558a47257240 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc098009bf0 tx=0x7fc098011810 comp rx=0 tx=0).stop 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0b6681740 1 -- 172.21.15.107:0/615893603 shutdown_connections 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0b6681740 1 --2- 172.21.15.107:0/615893603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47254e50 0x558a47257240 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0b6681740 1 --2- 172.21.15.107:0/615893603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a4724a590 0x558a47254910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0b6681740 1 --2- 172.21.15.107:0/615893603 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4724a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0b6681740 1 -- 172.21.15.107:0/615893603 >> 172.21.15.107:0/615893603 conn(0x558a47246200 msgr2=0x558a472465e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0b6681740 1 -- 172.21.15.107:0/615893603 shutdown_connections 2024-09-19T04:25:15.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.142+0000 7fc0b6681740 1 -- 172.21.15.107:0/615893603 wait complete. 2024-09-19T04:25:15.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b1c00640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc0940a3430 0x7fc0940a58f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009d60 tx=0x7fc0a4009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b6681740 1 -- start start 2024-09-19T04:25:15.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b6681740 1 Processor -- start 2024-09-19T04:25:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47249c70 0x558a47251c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a4724a590 0x558a472521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a472526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b6681740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558a47257a20 con 0x558a47254e50 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b6681740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558a4716b0f0 con 0x558a47249c70 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b6681740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558a47257ba0 con 0x558a4724a590 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b6681740 10 client.0 _ll_register_callbacks cb 0x558a47152bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.143+0000 7fc0b1200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47249c70 0x558a47251c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0b1200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47249c70 0x558a47251c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35110/0 (socket says 172.21.15.107:35110) 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0abe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a4724a590 0x558a472521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0b1200640 1 -- 172.21.15.107:0/3290432393 learned_addr learned my addr 172.21.15.107:0/3290432393 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0b1200640 1 -- 172.21.15.107:0/3290432393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a4724a590 msgr2=0x558a472521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0b1200640 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a4724a590 0x558a472521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0b1200640 1 -- 172.21.15.107:0/3290432393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 msgr2=0x558a472526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0b1200640 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a472526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0b1200640 1 -- 172.21.15.107:0/3290432393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a46f213c0 con 0x558a47249c70 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0abe00640 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a4724a590 0x558a472521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0b1200640 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47249c70 0x558a47251c70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0000c00 tx=0x7fc0a00b8860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e1a2e6d598c603 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc092a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc092a00640 1 -- 172.21.15.107:0/3290432393 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0049070 con 0x558a47249c70 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc092a00640 1 -- 172.21.15.107:0/3290432393 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fc0a00edb40 con 0x558a47249c70 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc0b6681740 1 -- 172.21.15.107:0/3290432393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc098005020 con 0x558a47249c70 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.144+0000 7fc092a00640 1 -- 172.21.15.107:0/3290432393 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc0a00bbf20 con 0x558a47249c70 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc0b6681740 10 client.26093 fetch_fsmap learned FSMap version 33 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc0b6681740 1 -- 172.21.15.107:0/3290432393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558a472639f0 con 0x558a47249c70 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc092a00640 1 -- 172.21.15.107:0/3290432393 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc098005020 con 0x558a47249c70 2024-09-19T04:25:15.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc092a00640 1 -- 172.21.15.107:0/3290432393 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc0a00c4cf0 con 0x558a47249c70 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc0b6681740 10 client.26093 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc0b6681740 10 client.26093 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc0b6681740 20 client.26093 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc0b6681740 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x558a47264d40 0x558a472671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc0b6681740 4 client.26093 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc0b6681740 1 -- 172.21.15.107:0/3290432393 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a471bd950 con 0x558a47264d40 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.145+0000 7fc0abe00640 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x558a47264d40 0x558a472671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.146+0000 7fc0abe00640 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x558a47264d40 0x558a472671d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a7a586cda9d6d90e server_cookie=206ae54b923c8ee0 in_seq=0 out_seq=0 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.146+0000 7fc092a00640 10 client.26093 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc092a00640 1 -- 172.21.15.107:0/3290432393 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a471bd950 con 0x558a47264d40 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc092a00640 10 client.26093 handle_command_reply: tid=0 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 client.26093 shutdown 2024-09-19T04:25:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 -- 172.21.15.107:0/3290432393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47249c70 msgr2=0x558a47251c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47249c70 0x558a47251c70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0000c00 tx=0x7fc0a00b8860 comp rx=0 tx=0).stop 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 -- 172.21.15.107:0/3290432393 shutdown_connections 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x558a47264d40 0x558a472671d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a472526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a4724a590 0x558a472521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 --2- 172.21.15.107:0/3290432393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a47249c70 0x558a47251c70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc092a00640 0 client.26093 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 -- 172.21.15.107:0/3290432393 >> 172.21.15.107:0/3290432393 conn(0x558a47246200 msgr2=0x558a4724bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 -- 172.21.15.107:0/3290432393 shutdown_connections 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 1 -- 172.21.15.107:0/3290432393 wait complete. 2024-09-19T04:25:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.147+0000 7fc0b6681740 20 client.26093 trim_cache size 0 max 16384 2024-09-19T04:25:15.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.162+0000 7fc0b6681740 1 Processor -- start 2024-09-19T04:25:15.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.162+0000 7fc0b6681740 1 -- start start 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4726caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4726d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a4724eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0b6681740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c00e600 con 0x558a47254e50 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0b6681740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c010140 con 0x558a4724a590 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0b6681740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c00ecd0 con 0x558a47249c70 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0abe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4726d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0abe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4726d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35120/0 (socket says 172.21.15.107:35120) 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0abe00640 1 -- 172.21.15.107:0/2591847906 learned_addr learned my addr 172.21.15.107:0/2591847906 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0b1c00640 1 --2- 172.21.15.107:0/2591847906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a4724eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0b1200640 1 --2- 172.21.15.107:0/2591847906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4726caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0abe00640 1 -- 172.21.15.107:0/2591847906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 msgr2=0x558a4726caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0abe00640 1 --2- 172.21.15.107:0/2591847906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4726caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0abe00640 1 -- 172.21.15.107:0/2591847906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 msgr2=0x558a4724eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0abe00640 1 --2- 172.21.15.107:0/2591847906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a4724eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0abe00640 1 -- 172.21.15.107:0/2591847906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a47246030 con 0x558a4724a590 2024-09-19T04:25:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.163+0000 7fc0abe00640 1 --2- 172.21.15.107:0/2591847906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4726d030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc098014040 tx=0x7fc0980048e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=542fd3294ee2a373 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.164+0000 7fc09d600640 1 -- 172.21.15.107:0/2591847906 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098005250 con 0x558a4724a590 2024-09-19T04:25:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.164+0000 7fc09d600640 1 -- 172.21.15.107:0/2591847906 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0980053f0 con 0x558a4724a590 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.164+0000 7fc09d600640 1 -- 172.21.15.107:0/2591847906 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098005020 con 0x558a4724a590 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.164+0000 7fc0b6681740 1 -- 172.21.15.107:0/2591847906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 msgr2=0x558a4726d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.164+0000 7fc0b6681740 1 --2- 172.21.15.107:0/2591847906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4726d030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc098014040 tx=0x7fc0980048e0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.164+0000 7fc0b6681740 1 -- 172.21.15.107:0/2591847906 shutdown_connections 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.164+0000 7fc0b6681740 1 --2- 172.21.15.107:0/2591847906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a4724eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.164+0000 7fc0b6681740 1 --2- 172.21.15.107:0/2591847906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4726d030 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.164+0000 7fc0b6681740 1 --2- 172.21.15.107:0/2591847906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4726caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.164+0000 7fc0b6681740 1 -- 172.21.15.107:0/2591847906 >> 172.21.15.107:0/2591847906 conn(0x558a47246200 msgr2=0x558a47245380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.165+0000 7fc0b6681740 1 -- 172.21.15.107:0/2591847906 shutdown_connections 2024-09-19T04:25:15.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.165+0000 7fc0b6681740 1 -- 172.21.15.107:0/2591847906 wait complete. 2024-09-19T04:25:15.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.165+0000 7fc0b6681740 1 -- start start 2024-09-19T04:25:15.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.165+0000 7fc0b6681740 1 Processor -- start 2024-09-19T04:25:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4725d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4724eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0b6681740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a4724f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0b6681740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c00fa70 con 0x558a47254e50 2024-09-19T04:25:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0b6681740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c00f3a0 con 0x558a4724a590 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0b6681740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c00c5e0 con 0x558a47249c70 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0b6681740 10 client.0 _ll_register_callbacks cb 0x7fc0a00f3790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0abe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4724eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0abe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4724eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35134/0 (socket says 172.21.15.107:35134) 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0abe00640 1 -- 172.21.15.107:0/1679318576 learned_addr learned my addr 172.21.15.107:0/1679318576 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0b1c00640 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a4724f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0b1200640 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4725d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0abe00640 1 -- 172.21.15.107:0/1679318576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 msgr2=0x558a4725d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0abe00640 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4725d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0abe00640 1 -- 172.21.15.107:0/1679318576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 msgr2=0x558a4724f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0abe00640 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a4724f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.166+0000 7fc0abe00640 1 -- 172.21.15.107:0/1679318576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a4724cae0 con 0x558a4724a590 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.167+0000 7fc0abe00640 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4724eb20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc09802e040 tx=0x7fc098005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46a3fd4e5607719a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.167+0000 7fc092a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.167+0000 7fc092a00640 1 -- 172.21.15.107:0/1679318576 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098039070 con 0x558a4724a590 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.167+0000 7fc0b6681740 1 -- 172.21.15.107:0/1679318576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0980053f0 con 0x558a4724a590 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.167+0000 7fc092a00640 1 -- 172.21.15.107:0/1679318576 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fc098004da0 con 0x558a4724a590 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.167+0000 7fc092a00640 1 -- 172.21.15.107:0/1679318576 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc098005250 con 0x558a4724a590 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc0b6681740 10 client.26099 fetch_fsmap learned FSMap version 33 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc0b6681740 1 -- 172.21.15.107:0/1679318576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558a47241770 con 0x558a4724a590 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc092a00640 1 -- 172.21.15.107:0/1679318576 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0980053f0 con 0x558a4724a590 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc092a00640 1 -- 172.21.15.107:0/1679318576 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc09802b020 con 0x558a4724a590 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc0b6681740 10 client.26099 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc0b6681740 10 client.26099 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:25:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc0b6681740 20 client.26099 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc0b6681740 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x558a472cfdc0 0x558a472d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc0b6681740 4 client.26099 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc0b6681740 1 -- 172.21.15.107:0/1679318576 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a471b7e60 con 0x558a472cfdc0 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.168+0000 7fc0b1200640 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x558a472cfdc0 0x558a472d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc0b1200640 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x558a472cfdc0 0x558a472d21e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21f23ef1463c2ac8 server_cookie=671c018c22e4c87a in_seq=0 out_seq=0 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc092a00640 10 client.26099 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc092a00640 1 -- 172.21.15.107:0/1679318576 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a471b7e60 con 0x558a472cfdc0 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc092a00640 10 client.26099 handle_command_reply: tid=0 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc0b6681740 1 client.26099 shutdown 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc0b6681740 1 -- 172.21.15.107:0/1679318576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 msgr2=0x558a4724eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc0b6681740 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4724eb20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc09802e040 tx=0x7fc098005d50 comp rx=0 tx=0).stop 2024-09-19T04:25:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc0b6681740 1 -- 172.21.15.107:0/1679318576 shutdown_connections 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc0b6681740 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x558a472cfdc0 0x558a472d21e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc0b6681740 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a47254e50 0x558a4724f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc0b6681740 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558a4724a590 0x558a4724eb20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc0b6681740 1 --2- 172.21.15.107:0/1679318576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558a47249c70 0x558a4725d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.169+0000 7fc0b6681740 1 -- 172.21.15.107:0/1679318576 >> 172.21.15.107:0/1679318576 conn(0x558a47246200 msgr2=0x558a4724b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.170+0000 7fc092a00640 0 client.26099 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.170+0000 7fc0b6681740 1 -- 172.21.15.107:0/1679318576 shutdown_connections 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.170+0000 7fc0b6681740 1 -- 172.21.15.107:0/1679318576 wait complete. 2024-09-19T04:25:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.170+0000 7fc0b6681740 20 client.26099 trim_cache size 0 max 16384 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 -- 172.21.15.107:0/3991685474 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc0940a3430 msgr2=0x7fc0940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc0940a3430 0x7fc0940a58f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009d60 tx=0x7fc0a4009500 comp rx=0 tx=0).stop 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 -- 172.21.15.107:0/3991685474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ac104010 msgr2=0x7fc0ac19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ac104010 0x7fc0ac19a6e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0004770 tx=0x7fc0a0003c10 comp rx=0 tx=0).stop 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 -- 172.21.15.107:0/3991685474 shutdown_connections 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc0940a3430 0x7fc0940a58f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac109140 0x7fc0ac19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ac1049c0 0x7fc0ac19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 --2- 172.21.15.107:0/3991685474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ac104010 0x7fc0ac19a6e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 -- 172.21.15.107:0/3991685474 >> 172.21.15.107:0/3991685474 conn(0x7fc0ac0ffa50 msgr2=0x7fc0ac100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 -- 172.21.15.107:0/3991685474 shutdown_connections 2024-09-19T04:25:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:15.171+0000 7fc0b3c00640 1 -- 172.21.15.107:0/3991685474 wait complete. 2024-09-19T04:25:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:14 smithi115 ceph-mon[35477]: pgmap v150: 97 pgs: 97 active+clean; 119 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-19T04:25:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:14 smithi145 ceph-mon[35096]: pgmap v150: 97 pgs: 97 active+clean; 119 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-19T04:25:15.663 INFO:tasks.workunit.client.0.smithi107.stdout: 1 187966 3.47 MB/sec execute 93 sec latency 16.469 ms 2024-09-19T04:25:16.070 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:25:16.460 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:16.663 INFO:tasks.workunit.client.0.smithi107.stdout: 1 188672 3.46 MB/sec execute 94 sec latency 299.039 ms 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.005+0000 7f364ca00640 1 -- 172.21.15.107:0/1675776606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3648103e30 msgr2=0x7f3648104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.005+0000 7f364ca00640 1 --2- 172.21.15.107:0/1675776606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3648103e30 0x7f3648104210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3638009900 tx=0x7f363802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.005+0000 7f364ca00640 1 -- 172.21.15.107:0/1675776606 shutdown_connections 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.005+0000 7f364ca00640 1 --2- 172.21.15.107:0/1675776606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3648108f60 0x7f3648110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.005+0000 7f364ca00640 1 --2- 172.21.15.107:0/1675776606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36481047e0 0x7f3648108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.005+0000 7f364ca00640 1 --2- 172.21.15.107:0/1675776606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3648103e30 0x7f3648104210 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.005+0000 7f364ca00640 1 -- 172.21.15.107:0/1675776606 >> 172.21.15.107:0/1675776606 conn(0x7f36480ff910 msgr2=0x7f3648101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.005+0000 7f364ca00640 1 -- 172.21.15.107:0/1675776606 shutdown_connections 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.006+0000 7f364ca00640 1 -- 172.21.15.107:0/1675776606 wait complete. 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.006+0000 7f364ca00640 1 Processor -- start 2024-09-19T04:25:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f364ca00640 1 -- start start 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f364ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3648103e30 0x7f364819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f364ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36481047e0 0x7f364819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f364ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3648108f60 0x7f364819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f364ca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3648114010 con 0x7f36481047e0 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f364ca00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3648113e90 con 0x7f3648108f60 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f364ca00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3648114190 con 0x7f3648103e30 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f363fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36481047e0 0x7f364819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f363fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36481047e0 0x7f364819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50786/0 (socket says 172.21.15.107:50786) 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f363fe00640 1 -- 172.21.15.107:0/3704526857 learned_addr learned my addr 172.21.15.107:0/3704526857 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f3647e00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3648108f60 0x7f364819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.007+0000 7f3647400640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3648103e30 0x7f364819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f363fe00640 1 -- 172.21.15.107:0/3704526857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3648103e30 msgr2=0x7f364819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f363fe00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3648103e30 0x7f364819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f363fe00640 1 -- 172.21.15.107:0/3704526857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3648108f60 msgr2=0x7f364819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f363fe00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3648108f60 0x7f364819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f363fe00640 1 -- 172.21.15.107:0/3704526857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364819f420 con 0x7f36481047e0 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f3647e00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3648108f60 0x7f364819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f363fe00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36481047e0 0x7f364819a9b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f363000ea60 tx=0x7f363000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f3644c00640 1 -- 172.21.15.107:0/3704526857 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363000cde0 con 0x7f36481047e0 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f364ca00640 1 -- 172.21.15.107:0/3704526857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364819f710 con 0x7f36481047e0 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f3644c00640 1 -- 172.21.15.107:0/3704526857 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36300035f0 con 0x7f36481047e0 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f3644c00640 1 -- 172.21.15.107:0/3704526857 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630010670 con 0x7f36481047e0 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.008+0000 7f364ca00640 1 -- 172.21.15.107:0/3704526857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36481a6ff0 con 0x7f36481047e0 2024-09-19T04:25:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.011+0000 7f3650bbf740 1 Processor -- start 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.011+0000 7f3644c00640 1 -- 172.21.15.107:0/3704526857 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f36300037a0 con 0x7f36481047e0 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.011+0000 7f3650bbf740 1 -- start start 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 0x564cb5594220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb559eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb559f020 0x564cb55a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f3650bbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564cb54fa3e0 con 0x564cb5594760 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f3650bbf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564cb54a1d80 con 0x564cb559f020 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f3650bbf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564cb54f63a0 con 0x564cb5593e40 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f3647400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb559eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f3647400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb559eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50790/0 (socket says 172.21.15.107:50790) 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f3647400640 1 -- 172.21.15.107:0/3662527206 learned_addr learned my addr 172.21.15.107:0/3662527206 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f3647e00640 1 --2- 172.21.15.107:0/3662527206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 0x564cb5594220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.012+0000 7f363fe00640 1 --2- 172.21.15.107:0/3662527206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb559f020 0x564cb55a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3647400640 1 -- 172.21.15.107:0/3662527206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 msgr2=0x564cb5594220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3647400640 1 --2- 172.21.15.107:0/3662527206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 0x564cb5594220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3647400640 1 -- 172.21.15.107:0/3662527206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb559f020 msgr2=0x564cb55a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3647400640 1 --2- 172.21.15.107:0/3662527206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb559f020 0x564cb55a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3647400640 1 -- 172.21.15.107:0/3662527206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb54c0ae0 con 0x564cb5594760 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3647400640 1 --2- 172.21.15.107:0/3662527206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb559eae0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3638009900 tx=0x7f3638005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9ca81d23512916 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3644c00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f362c0a33b0 0x7f362c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3644c00640 1 -- 172.21.15.107:0/3704526857 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f36300cffe0 con 0x7f36481047e0 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3647e00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f362c0a33b0 0x7f362c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3637e00640 1 -- 172.21.15.107:0/3662527206 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638035040 con 0x564cb5594760 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3637e00640 1 -- 172.21.15.107:0/3662527206 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3638004400 con 0x564cb5594760 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.013+0000 7f3637e00640 1 -- 172.21.15.107:0/3662527206 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36380035f0 con 0x564cb5594760 2024-09-19T04:25:17.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.014+0000 7f3650bbf740 1 -- 172.21.15.107:0/3662527206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 msgr2=0x564cb559eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.014+0000 7f3650bbf740 1 --2- 172.21.15.107:0/3662527206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb559eae0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3638009900 tx=0x7f3638005de0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.014+0000 7f3647e00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f362c0a33b0 0x7f362c0a5870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3640009fd0 tx=0x7f3640009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.014+0000 7f3650bbf740 1 -- 172.21.15.107:0/3662527206 shutdown_connections 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.014+0000 7f3650bbf740 1 --2- 172.21.15.107:0/3662527206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb559f020 0x564cb55a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.014+0000 7f3650bbf740 1 --2- 172.21.15.107:0/3662527206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb559eae0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.014+0000 7f3650bbf740 1 --2- 172.21.15.107:0/3662527206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 0x564cb5594220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.014+0000 7f3650bbf740 1 -- 172.21.15.107:0/3662527206 >> 172.21.15.107:0/3662527206 conn(0x564cb55903d0 msgr2=0x564cb55907b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.014+0000 7f3650bbf740 1 -- 172.21.15.107:0/3662527206 shutdown_connections 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.014+0000 7f3650bbf740 1 -- 172.21.15.107:0/3662527206 wait complete. 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.015+0000 7f3650bbf740 1 -- start start 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.015+0000 7f3650bbf740 1 Processor -- start 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.015+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5593e40 0x564cb559bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.015+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5594760 0x564cb559c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.015+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 0x564cb559c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.015+0000 7f3650bbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564cb55a1bf0 con 0x564cb5593e40 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.015+0000 7f3650bbf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564cb54b50f0 con 0x564cb5594760 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.015+0000 7f3650bbf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564cb55a1d70 con 0x564cb559f020 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.015+0000 7f3650bbf740 10 client.0 _ll_register_callbacks cb 0x564cb549cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f363fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5594760 0x564cb559c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f363fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5594760 0x564cb559c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35184/0 (socket says 172.21.15.107:35184) 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f363fe00640 1 -- 172.21.15.107:0/1106562998 learned_addr learned my addr 172.21.15.107:0/1106562998 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f3647400640 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5593e40 0x564cb559bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f3647e00640 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 0x564cb559c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f363fe00640 1 -- 172.21.15.107:0/1106562998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 msgr2=0x564cb559c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f363fe00640 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 0x564cb559c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f363fe00640 1 -- 172.21.15.107:0/1106562998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5593e40 msgr2=0x564cb559bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f363fe00640 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5593e40 0x564cb559bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f363fe00640 1 -- 172.21.15.107:0/1106562998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb526b3c0 con 0x564cb5594760 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f363fe00640 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5594760 0x564cb559c460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f363001f000 tx=0x7f36300d3600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c799b5015a28ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f3634c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.016+0000 7f3634c00640 1 -- 172.21.15.107:0/1106562998 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363009be30 con 0x564cb5594760 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3634c00640 1 -- 172.21.15.107:0/1106562998 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f36300d20d0 con 0x564cb5594760 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3650bbf740 1 -- 172.21.15.107:0/1106562998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3638004400 con 0x564cb5594760 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3634c00640 1 -- 172.21.15.107:0/1106562998 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f363009bc50 con 0x564cb5594760 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3650bbf740 10 client.26111 fetch_fsmap learned FSMap version 34 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3650bbf740 1 -- 172.21.15.107:0/1106562998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564cb55adca0 con 0x564cb5594760 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3634c00640 1 -- 172.21.15.107:0/1106562998 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3638004400 con 0x564cb5594760 2024-09-19T04:25:17.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3634c00640 1 -- 172.21.15.107:0/1106562998 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f36300cd9e0 con 0x564cb5594760 2024-09-19T04:25:17.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3650bbf740 10 client.26111 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:25:17.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3650bbf740 10 client.26111 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:25:17.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3650bbf740 20 client.26111 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:17.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3650bbf740 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x564cb55af160 0x564cb55b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3650bbf740 4 client.26111 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:17.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.017+0000 7f3650bbf740 1 -- 172.21.15.107:0/1106562998 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cb5507950 con 0x564cb55af160 2024-09-19T04:25:17.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.018+0000 7f3647400640 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x564cb55af160 0x564cb55b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.018+0000 7f3647400640 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x564cb55af160 0x564cb55b15b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cc835cc3d6747eed server_cookie=3d98605fc75dbe3b in_seq=0 out_seq=0 2024-09-19T04:25:17.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.018+0000 7f3634c00640 10 client.26111 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:25:17.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3634c00640 1 -- 172.21.15.107:0/1106562998 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564cb5507950 con 0x564cb55af160 2024-09-19T04:25:17.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3634c00640 10 client.26111 handle_command_reply: tid=0 2024-09-19T04:25:17.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 client.26111 shutdown 2024-09-19T04:25:17.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 -- 172.21.15.107:0/1106562998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5594760 msgr2=0x564cb559c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5594760 0x564cb559c460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f363001f000 tx=0x7f36300d3600 comp rx=0 tx=0).stop 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 -- 172.21.15.107:0/1106562998 shutdown_connections 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x564cb55af160 0x564cb55b15b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 0x564cb559c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3634c00640 0 client.26111 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5594760 0x564cb559c460 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 --2- 172.21.15.107:0/1106562998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5593e40 0x564cb559bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 -- 172.21.15.107:0/1106562998 >> 172.21.15.107:0/1106562998 conn(0x564cb55903d0 msgr2=0x564cb55960b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 -- 172.21.15.107:0/1106562998 shutdown_connections 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.019+0000 7f3650bbf740 1 -- 172.21.15.107:0/1106562998 wait complete. 2024-09-19T04:25:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.020+0000 7f3650bbf740 20 client.26111 trim_cache size 0 max 16384 2024-09-19T04:25:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.033+0000 7f3650bbf740 1 Processor -- start 2024-09-19T04:25:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.033+0000 7f3650bbf740 1 -- start start 2024-09-19T04:25:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.033+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5593e40 0x564cb559de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.033+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb55b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.033+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 0x564cb55b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.033+0000 7f3650bbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36481af770 con 0x564cb5594760 2024-09-19T04:25:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.033+0000 7f3650bbf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f36481b12e0 con 0x564cb5593e40 2024-09-19T04:25:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.033+0000 7f3650bbf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f36481afe40 con 0x564cb559f020 2024-09-19T04:25:17.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f363fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb55b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5593e40 0x564cb559de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5593e40 0x564cb559de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35194/0 (socket says 172.21.15.107:35194) 2024-09-19T04:25:17.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647400640 1 -- 172.21.15.107:0/2264344122 learned_addr learned my addr 172.21.15.107:0/2264344122 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:17.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647400640 1 -- 172.21.15.107:0/2264344122 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 msgr2=0x564cb55b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647e00640 1 --2- 172.21.15.107:0/2264344122 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 0x564cb55b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647400640 1 --2- 172.21.15.107:0/2264344122 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 0x564cb55b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647400640 1 -- 172.21.15.107:0/2264344122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 msgr2=0x564cb55b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647400640 1 --2- 172.21.15.107:0/2264344122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb55b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647400640 1 -- 172.21.15.107:0/2264344122 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb5590200 con 0x564cb5593e40 2024-09-19T04:25:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647e00640 1 --2- 172.21.15.107:0/2264344122 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 0x564cb55b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:25:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3647400640 1 --2- 172.21.15.107:0/2264344122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5593e40 0x564cb559de90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3638009900 tx=0x7f3638004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79215708932e4673 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3637400640 1 -- 172.21.15.107:0/2264344122 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638035040 con 0x564cb5593e40 2024-09-19T04:25:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3637400640 1 -- 172.21.15.107:0/2264344122 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3638004200 con 0x564cb5593e40 2024-09-19T04:25:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.034+0000 7f3637400640 1 -- 172.21.15.107:0/2264344122 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638003850 con 0x564cb5593e40 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.035+0000 7f3650bbf740 1 -- 172.21.15.107:0/2264344122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5593e40 msgr2=0x564cb559de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.035+0000 7f3650bbf740 1 --2- 172.21.15.107:0/2264344122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5593e40 0x564cb559de90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3638009900 tx=0x7f3638004b40 comp rx=0 tx=0).stop 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.035+0000 7f3650bbf740 1 -- 172.21.15.107:0/2264344122 shutdown_connections 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.035+0000 7f3650bbf740 1 --2- 172.21.15.107:0/2264344122 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb559f020 0x564cb55b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.035+0000 7f3650bbf740 1 --2- 172.21.15.107:0/2264344122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb55b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.035+0000 7f3650bbf740 1 --2- 172.21.15.107:0/2264344122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb5593e40 0x564cb559de90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.035+0000 7f3650bbf740 1 -- 172.21.15.107:0/2264344122 >> 172.21.15.107:0/2264344122 conn(0x564cb55903d0 msgr2=0x564cb559d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.035+0000 7f3650bbf740 1 -- 172.21.15.107:0/2264344122 shutdown_connections 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.035+0000 7f3650bbf740 1 -- 172.21.15.107:0/2264344122 wait complete. 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.036+0000 7f3650bbf740 1 -- start start 2024-09-19T04:25:17.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.036+0000 7f3650bbf740 1 Processor -- start 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.036+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 0x564cb55a7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.036+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb55a2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.036+0000 7f3650bbf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb559f020 0x564cb55a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.036+0000 7f3650bbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36481b0c10 con 0x564cb5594760 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.036+0000 7f3650bbf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f36481b0510 con 0x564cb559f020 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.036+0000 7f3650bbf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f36481ad750 con 0x564cb5593e40 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.036+0000 7f3650bbf740 10 client.0 _ll_register_callbacks cb 0x7f36300d37c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3647400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 0x564cb55a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3647400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 0x564cb55a7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:45746/0 (socket says 172.21.15.107:45746) 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3647400640 1 -- 172.21.15.107:0/3430441197 learned_addr learned my addr 172.21.15.107:0/3430441197 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3647e00640 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb559f020 0x564cb55a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f363fe00640 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb55a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3647400640 1 -- 172.21.15.107:0/3430441197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb559f020 msgr2=0x564cb55a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3647400640 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb559f020 0x564cb55a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3647400640 1 -- 172.21.15.107:0/3430441197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 msgr2=0x564cb55a2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:17.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3647400640 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb55a2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3647400640 1 -- 172.21.15.107:0/3430441197 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb5591060 con 0x564cb5593e40 2024-09-19T04:25:17.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3647400640 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 0x564cb55a7520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f363802a910 tx=0x7f3638009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=def32635deda22ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:17.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.037+0000 7f3634c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3634c00640 1 -- 172.21.15.107:0/3430441197 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638035070 con 0x564cb5593e40 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3634c00640 1 -- 172.21.15.107:0/3430441197 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f36380317b0 con 0x564cb5593e40 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3650bbf740 1 -- 172.21.15.107:0/3430441197 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3638004200 con 0x564cb5593e40 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3634c00640 1 -- 172.21.15.107:0/3430441197 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3638031e60 con 0x564cb5593e40 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3650bbf740 10 client.26077 fetch_fsmap learned FSMap version 34 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3650bbf740 1 -- 172.21.15.107:0/3430441197 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564cb5619a00 con 0x564cb5593e40 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3634c00640 1 -- 172.21.15.107:0/3430441197 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3638004200 con 0x564cb5593e40 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3634c00640 1 -- 172.21.15.107:0/3430441197 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f3638004c50 con 0x564cb5593e40 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3650bbf740 10 client.26077 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3650bbf740 10 client.26077 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:25:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3650bbf740 20 client.26077 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3650bbf740 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x564cb55af160 0x564cb561bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3650bbf740 4 client.26077 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.038+0000 7f3650bbf740 1 -- 172.21.15.107:0/3430441197 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564cb5501e60 con 0x564cb55af160 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f363fe00640 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x564cb55af160 0x564cb561bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f363fe00640 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x564cb55af160 0x564cb561bce0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b4be0875fd459ab2 server_cookie=cacc8982c980e7bb in_seq=0 out_seq=0 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3634c00640 10 client.26077 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3634c00640 1 -- 172.21.15.107:0/3430441197 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564cb5501e60 con 0x564cb55af160 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3634c00640 10 client.26077 handle_command_reply: tid=0 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3650bbf740 1 client.26077 shutdown 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3650bbf740 1 -- 172.21.15.107:0/3430441197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 msgr2=0x564cb55a7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3650bbf740 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 0x564cb55a7520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f363802a910 tx=0x7f3638009650 comp rx=0 tx=0).stop 2024-09-19T04:25:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3650bbf740 1 -- 172.21.15.107:0/3430441197 shutdown_connections 2024-09-19T04:25:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3650bbf740 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x564cb55af160 0x564cb561bce0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3650bbf740 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564cb559f020 0x564cb55a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3650bbf740 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564cb5594760 0x564cb55a2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3634c00640 0 client.26077 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:25:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3650bbf740 1 --2- 172.21.15.107:0/3430441197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564cb5593e40 0x564cb55a7520 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.039+0000 7f3650bbf740 1 -- 172.21.15.107:0/3430441197 >> 172.21.15.107:0/3430441197 conn(0x564cb55903d0 msgr2=0x564cb55960b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f3650bbf740 1 -- 172.21.15.107:0/3430441197 shutdown_connections 2024-09-19T04:25:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f3650bbf740 1 -- 172.21.15.107:0/3430441197 wait complete. 2024-09-19T04:25:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f3650bbf740 20 client.26077 trim_cache size 0 max 16384 2024-09-19T04:25:17.046 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:25:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f364ca00640 1 -- 172.21.15.107:0/3704526857 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f362c0a33b0 msgr2=0x7f362c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f364ca00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f362c0a33b0 0x7f362c0a5870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3640009fd0 tx=0x7f3640009500 comp rx=0 tx=0).stop 2024-09-19T04:25:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f364ca00640 1 -- 172.21.15.107:0/3704526857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36481047e0 msgr2=0x7f364819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f364ca00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36481047e0 0x7f364819a9b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f363000ea60 tx=0x7f363000ef20 comp rx=0 tx=0).stop 2024-09-19T04:25:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f364ca00640 1 -- 172.21.15.107:0/3704526857 shutdown_connections 2024-09-19T04:25:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f364ca00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f362c0a33b0 0x7f362c0a5870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f364ca00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3648108f60 0x7f364819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f364ca00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36481047e0 0x7f364819a9b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.040+0000 7f364ca00640 1 --2- 172.21.15.107:0/3704526857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3648103e30 0x7f364819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.041+0000 7f364ca00640 1 -- 172.21.15.107:0/3704526857 >> 172.21.15.107:0/3704526857 conn(0x7f36480ff910 msgr2=0x7f3648100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.041+0000 7f364ca00640 1 -- 172.21.15.107:0/3704526857 shutdown_connections 2024-09-19T04:25:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:17.041+0000 7f364ca00640 1 -- 172.21.15.107:0/3704526857 wait complete. 2024-09-19T04:25:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:16 smithi115 ceph-mon[35477]: pgmap v151: 97 pgs: 97 active+clean; 123 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-19T04:25:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:16 smithi145 ceph-mon[35096]: pgmap v151: 97 pgs: 97 active+clean; 123 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-19T04:25:17.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:16 smithi107 ceph-mon[30659]: pgmap v151: 97 pgs: 97 active+clean; 123 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-19T04:25:17.670 INFO:tasks.workunit.client.0.smithi107.stdout: 1 188672 3.42 MB/sec execute 95 sec latency 1302.688 ms 2024-09-19T04:25:18.053 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:25:18.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:17 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:18.200 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:17 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:17 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:18.425 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:18.666 INFO:tasks.workunit.client.0.smithi107.stdout: 1 188672 3.38 MB/sec execute 96 sec latency 2302.765 ms 2024-09-19T04:25:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.987+0000 7f82d3a00640 1 -- 172.21.15.107:0/57734836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc103670 msgr2=0x7f82cc10bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.987+0000 7f82d3a00640 1 --2- 172.21.15.107:0/57734836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc103670 0x7f82cc10bb60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009900 tx=0x7f82b402a480 comp rx=0 tx=0).stop 2024-09-19T04:25:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.987+0000 7f82d3a00640 1 -- 172.21.15.107:0/57734836 shutdown_connections 2024-09-19T04:25:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.987+0000 7f82d3a00640 1 --2- 172.21.15.107:0/57734836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82cc10c180 0x7f82cc10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.987+0000 7f82d3a00640 1 --2- 172.21.15.107:0/57734836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc103670 0x7f82cc10bb60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.987+0000 7f82d3a00640 1 --2- 172.21.15.107:0/57734836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc102d50 0x7f82cc103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.987+0000 7f82d3a00640 1 -- 172.21.15.107:0/57734836 >> 172.21.15.107:0/57734836 conn(0x7f82cc0fc7d0 msgr2=0x7f82cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:18.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.987+0000 7f82d3a00640 1 -- 172.21.15.107:0/57734836 shutdown_connections 2024-09-19T04:25:18.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.987+0000 7f82d3a00640 1 -- 172.21.15.107:0/57734836 wait complete. 2024-09-19T04:25:18.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.988+0000 7f82d3a00640 1 Processor -- start 2024-09-19T04:25:18.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.988+0000 7f82d3a00640 1 -- start start 2024-09-19T04:25:18.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.988+0000 7f82d3a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc102d50 0x7f82cc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.988+0000 7f82d3a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82cc103670 0x7f82cc19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.988+0000 7f82d3a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc10c180 0x7f82cc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.988+0000 7f82d3a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc111f80 con 0x7f82cc10c180 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.988+0000 7f82d3a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc111e00 con 0x7f82cc102d50 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.988+0000 7f82d3a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc112100 con 0x7f82cc103670 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc10c180 0x7f82cc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc102d50 0x7f82cc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc10c180 0x7f82cc19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56882/0 (socket says 172.21.15.107:56882) 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1a00640 1 -- 172.21.15.107:0/3868915524 learned_addr learned my addr 172.21.15.107:0/3868915524 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82c3e00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82cc103670 0x7f82cc19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1a00640 1 -- 172.21.15.107:0/3868915524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82cc103670 msgr2=0x7f82cc19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1a00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82cc103670 0x7f82cc19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1a00640 1 -- 172.21.15.107:0/3868915524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc102d50 msgr2=0x7f82cc19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1a00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc102d50 0x7f82cc19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1a00640 1 -- 172.21.15.107:0/3868915524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc19f6a0 con 0x7f82cc10c180 2024-09-19T04:25:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1000640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc102d50 0x7f82cc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.989+0000 7f82d1a00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc10c180 0x7f82cc19efc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f82c400eb10 tx=0x7f82c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.990+0000 7f82ca000640 1 -- 172.21.15.107:0/3868915524 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4004b20 con 0x7f82cc10c180 2024-09-19T04:25:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.990+0000 7f82ca000640 1 -- 172.21.15.107:0/3868915524 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c4004cc0 con 0x7f82cc10c180 2024-09-19T04:25:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.990+0000 7f82d3a00640 1 -- 172.21.15.107:0/3868915524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82cc19f990 con 0x7f82cc10c180 2024-09-19T04:25:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.990+0000 7f82ca000640 1 -- 172.21.15.107:0/3868915524 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4010760 con 0x7f82cc10c180 2024-09-19T04:25:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.990+0000 7f82d3a00640 1 -- 172.21.15.107:0/3868915524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82cc1a7270 con 0x7f82cc10c180 2024-09-19T04:25:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.993+0000 7f82d6649740 1 Processor -- start 2024-09-19T04:25:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.993+0000 7f82ca000640 1 -- 172.21.15.107:0/3868915524 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f82c4005020 con 0x7f82cc10c180 2024-09-19T04:25:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.993+0000 7f82d6649740 1 -- start start 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.993+0000 7f82d6649740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2cc70 0x5594ddd2d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.993+0000 7f82d6649740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd37910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.993+0000 7f82d6649740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd37e50 0x5594ddd3a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.993+0000 7f82d6649740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5594ddc933e0 con 0x5594ddd2d590 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.993+0000 7f82d6649740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5594ddc3ad80 con 0x5594ddd37e50 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.993+0000 7f82d6649740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5594ddc8f3a0 con 0x5594ddd2cc70 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82d1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd37910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82d1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd37910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56886/0 (socket says 172.21.15.107:56886) 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82d1000640 1 -- 172.21.15.107:0/2321915764 learned_addr learned my addr 172.21.15.107:0/2321915764 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82c3e00640 1 --2- 172.21.15.107:0/2321915764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2cc70 0x5594ddd2d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82c3e00640 1 --2- 172.21.15.107:0/2321915764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd37e50 0x5594ddd3a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:18.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82d1000640 1 -- 172.21.15.107:0/2321915764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2cc70 msgr2=0x5594ddd2d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:18.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82d1000640 1 --2- 172.21.15.107:0/2321915764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2cc70 0x5594ddd2d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82d1000640 1 -- 172.21.15.107:0/2321915764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd37e50 msgr2=0x5594ddd3a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:18.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82d1000640 1 --2- 172.21.15.107:0/2321915764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd37e50 0x5594ddd3a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82d1000640 1 -- 172.21.15.107:0/2321915764 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ddc59ae0 con 0x5594ddd2d590 2024-09-19T04:25:18.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.994+0000 7f82c3e00640 1 --2- 172.21.15.107:0/2321915764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd37e50 0x5594ddd3a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:18.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.995+0000 7f82d1000640 1 --2- 172.21.15.107:0/2321915764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd37910 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0115f0 tx=0x7f82bc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621d3a15d12d7799 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:18.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.995+0000 7f82ca000640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f82b00a33b0 0x7f82b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:18.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.995+0000 7f82ca000640 1 -- 172.21.15.107:0/3868915524 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f82c40d0220 con 0x7f82cc10c180 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.995+0000 7f82c2000640 1 -- 172.21.15.107:0/2321915764 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc01e070 con 0x5594ddd2d590 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.995+0000 7f82c2000640 1 -- 172.21.15.107:0/2321915764 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc00fb20 con 0x5594ddd2d590 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.995+0000 7f82c2000640 1 -- 172.21.15.107:0/2321915764 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc019c40 con 0x5594ddd2d590 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.995+0000 7f82c3e00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f82b00a33b0 0x7f82b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.996+0000 7f82d6649740 1 -- 172.21.15.107:0/2321915764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 msgr2=0x5594ddd37910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.996+0000 7f82c3e00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f82b00a33b0 0x7f82b00a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009230 tx=0x7f82b40058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.996+0000 7f82d6649740 1 --2- 172.21.15.107:0/2321915764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd37910 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0115f0 tx=0x7f82bc011ab0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.996+0000 7f82d6649740 1 -- 172.21.15.107:0/2321915764 shutdown_connections 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.996+0000 7f82d6649740 1 --2- 172.21.15.107:0/2321915764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd37e50 0x5594ddd3a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.996+0000 7f82d6649740 1 --2- 172.21.15.107:0/2321915764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd37910 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.996+0000 7f82d6649740 1 --2- 172.21.15.107:0/2321915764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2cc70 0x5594ddd2d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.996+0000 7f82d6649740 1 -- 172.21.15.107:0/2321915764 >> 172.21.15.107:0/2321915764 conn(0x5594ddd29200 msgr2=0x5594ddd295e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:18.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.996+0000 7f82d6649740 1 -- 172.21.15.107:0/2321915764 shutdown_connections 2024-09-19T04:25:19.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.996+0000 7f82d6649740 1 -- 172.21.15.107:0/2321915764 wait complete. 2024-09-19T04:25:19.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.997+0000 7f82d6649740 1 -- start start 2024-09-19T04:25:19.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.997+0000 7f82d6649740 1 Processor -- start 2024-09-19T04:25:19.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.997+0000 7f82d6649740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 0x5594ddd34d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.997+0000 7f82d6649740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2d590 0x5594ddd35290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.997+0000 7f82d6649740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd357d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.997+0000 7f82d6649740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5594ddd3aa20 con 0x5594ddd37e50 2024-09-19T04:25:19.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.997+0000 7f82d6649740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5594ddc4e0f0 con 0x5594ddd2cc70 2024-09-19T04:25:19.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.997+0000 7f82d6649740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5594ddd3aba0 con 0x5594ddd2d590 2024-09-19T04:25:19.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.997+0000 7f82d6649740 10 client.0 _ll_register_callbacks cb 0x5594ddc35bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:19.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd357d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56898/0 (socket says 172.21.15.107:56898) 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1a00640 1 -- 172.21.15.107:0/1407220672 learned_addr learned my addr 172.21.15.107:0/1407220672 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1000640 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 0x5594ddd34d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82c3e00640 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2d590 0x5594ddd35290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1a00640 1 -- 172.21.15.107:0/1407220672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2d590 msgr2=0x5594ddd35290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1a00640 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2d590 0x5594ddd35290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1a00640 1 -- 172.21.15.107:0/1407220672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 msgr2=0x5594ddd34d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1a00640 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 0x5594ddd34d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1a00640 1 -- 172.21.15.107:0/1407220672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594dda043c0 con 0x5594ddd37e50 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1000640 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 0x5594ddd34d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d1a00640 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd357d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f82c4098ec0 tx=0x7f82c409c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246402d9b25ec7ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82baa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82baa00640 1 -- 172.21.15.107:0/1407220672 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c40a4ed0 con 0x5594ddd37e50 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82baa00640 1 -- 172.21.15.107:0/1407220672 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f82c40b5210 con 0x5594ddd37e50 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.998+0000 7f82d6649740 1 -- 172.21.15.107:0/1407220672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82bc00fb20 con 0x5594ddd37e50 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82baa00640 1 -- 172.21.15.107:0/1407220672 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f82c407d020 con 0x5594ddd37e50 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82d6649740 10 client.16074 fetch_fsmap learned FSMap version 34 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82d6649740 1 -- 172.21.15.107:0/1407220672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5594ddd46a70 con 0x5594ddd37e50 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82baa00640 1 -- 172.21.15.107:0/1407220672 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc00fb20 con 0x5594ddd37e50 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82baa00640 1 -- 172.21.15.107:0/1407220672 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f82c4005310 con 0x5594ddd37e50 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82d6649740 10 client.16074 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82d6649740 10 client.16074 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82d6649740 20 client.16074 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:19.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82d6649740 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5594ddd47f30 0x5594ddd4a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82d6649740 4 client.16074 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:19.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:18.999+0000 7f82d6649740 1 -- 172.21.15.107:0/1407220672 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594ddca0950 con 0x5594ddd47f30 2024-09-19T04:25:19.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.000+0000 7f82d1000640 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5594ddd47f30 0x5594ddd4a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.000+0000 7f82d1000640 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5594ddd47f30 0x5594ddd4a3a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc5978807a8a0928 server_cookie=2a19beb1d806e3f0 in_seq=0 out_seq=0 2024-09-19T04:25:19.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.000+0000 7f82baa00640 10 client.16074 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:25:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82baa00640 1 -- 172.21.15.107:0/1407220672 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594ddca0950 con 0x5594ddd47f30 2024-09-19T04:25:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82baa00640 10 client.16074 handle_command_reply: tid=0 2024-09-19T04:25:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82d6649740 1 client.16074 shutdown 2024-09-19T04:25:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82d6649740 1 -- 172.21.15.107:0/1407220672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 msgr2=0x5594ddd357d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82d6649740 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd357d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f82c4098ec0 tx=0x7f82c409c3b0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82d6649740 1 -- 172.21.15.107:0/1407220672 shutdown_connections 2024-09-19T04:25:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82d6649740 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5594ddd47f30 0x5594ddd4a3a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82d6649740 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd357d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82d6649740 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2d590 0x5594ddd35290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82d6649740 1 --2- 172.21.15.107:0/1407220672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 0x5594ddd34d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82baa00640 0 client.16074 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:25:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.001+0000 7f82d6649740 1 -- 172.21.15.107:0/1407220672 >> 172.21.15.107:0/1407220672 conn(0x5594ddd29200 msgr2=0x5594ddd2eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.002+0000 7f82d6649740 1 -- 172.21.15.107:0/1407220672 shutdown_connections 2024-09-19T04:25:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.002+0000 7f82d6649740 1 -- 172.21.15.107:0/1407220672 wait complete. 2024-09-19T04:25:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.002+0000 7f82d6649740 20 client.16074 trim_cache size 0 max 16384 2024-09-19T04:25:19.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.015+0000 7f82d6649740 1 Processor -- start 2024-09-19T04:25:19.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.015+0000 7f82d6649740 1 -- start start 2024-09-19T04:25:19.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.015+0000 7f82d6649740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 0x5594ddd34610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.015+0000 7f82d6649740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd36cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.015+0000 7f82d6649740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd37e50 0x5594ddd4fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.015+0000 7f82d6649740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1af9c0 con 0x5594ddd2d590 2024-09-19T04:25:19.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.015+0000 7f82d6649740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b1530 con 0x5594ddd2cc70 2024-09-19T04:25:19.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.015+0000 7f82d6649740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b0090 con 0x5594ddd37e50 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd36cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd36cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56910/0 (socket says 172.21.15.107:56910) 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82c3e00640 1 -- 172.21.15.107:0/2145884225 learned_addr learned my addr 172.21.15.107:0/2145884225 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82d1000640 1 --2- 172.21.15.107:0/2145884225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 0x5594ddd34610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82d1a00640 1 --2- 172.21.15.107:0/2145884225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd37e50 0x5594ddd4fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82c3e00640 1 -- 172.21.15.107:0/2145884225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd37e50 msgr2=0x5594ddd4fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82c3e00640 1 --2- 172.21.15.107:0/2145884225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd37e50 0x5594ddd4fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82c3e00640 1 -- 172.21.15.107:0/2145884225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 msgr2=0x5594ddd34610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82c3e00640 1 --2- 172.21.15.107:0/2145884225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 0x5594ddd34610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82c3e00640 1 -- 172.21.15.107:0/2145884225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ddd33d00 con 0x5594ddd2d590 2024-09-19T04:25:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.016+0000 7f82d1000640 1 --2- 172.21.15.107:0/2145884225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 0x5594ddd34610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:19.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.017+0000 7f82c3e00640 1 --2- 172.21.15.107:0/2145884225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd36cc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009900 tx=0x7f82b402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24293f358f7c70d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:19.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.017+0000 7f82c1600640 1 -- 172.21.15.107:0/2145884225 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4035040 con 0x5594ddd2d590 2024-09-19T04:25:19.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.017+0000 7f82c1600640 1 -- 172.21.15.107:0/2145884225 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b40047a0 con 0x5594ddd2d590 2024-09-19T04:25:19.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.017+0000 7f82c1600640 1 -- 172.21.15.107:0/2145884225 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4003850 con 0x5594ddd2d590 2024-09-19T04:25:19.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 -- 172.21.15.107:0/2145884225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 msgr2=0x5594ddd36cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 --2- 172.21.15.107:0/2145884225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd36cc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009900 tx=0x7f82b402a480 comp rx=0 tx=0).stop 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 -- 172.21.15.107:0/2145884225 shutdown_connections 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 --2- 172.21.15.107:0/2145884225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd37e50 0x5594ddd4fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 --2- 172.21.15.107:0/2145884225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd2d590 0x5594ddd36cc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 --2- 172.21.15.107:0/2145884225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2cc70 0x5594ddd34610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 -- 172.21.15.107:0/2145884225 >> 172.21.15.107:0/2145884225 conn(0x5594ddd29200 msgr2=0x5594ddd365b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 -- 172.21.15.107:0/2145884225 shutdown_connections 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 -- 172.21.15.107:0/2145884225 wait complete. 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 -- start start 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.018+0000 7f82d6649740 1 Processor -- start 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82d6649740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2cc70 0x5594ddd40260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82d6649740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2d590 0x5594ddd3b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82d6649740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd3bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82d6649740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b0e60 con 0x5594ddd37e50 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82d6649740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b0760 con 0x5594ddd2d590 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82d6649740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1ad9a0 con 0x5594ddd2cc70 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82d6649740 10 client.0 _ll_register_callbacks cb 0x7f82c4098840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82d1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd3bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82c3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2d590 0x5594ddd3b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82c3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2d590 0x5594ddd3b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35470/0 (socket says 172.21.15.107:35470) 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82c3e00640 1 -- 172.21.15.107:0/1435186849 learned_addr learned my addr 172.21.15.107:0/1435186849 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.019+0000 7f82d1000640 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2cc70 0x5594ddd40260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82c3e00640 1 -- 172.21.15.107:0/1435186849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2cc70 msgr2=0x5594ddd40260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:19.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82c3e00640 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2cc70 0x5594ddd40260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82c3e00640 1 -- 172.21.15.107:0/1435186849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 msgr2=0x5594ddd3bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82c3e00640 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd3bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82c3e00640 1 -- 172.21.15.107:0/1435186849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ddd3c010 con 0x5594ddd2d590 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82d1a00640 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd3bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82c3e00640 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2d590 0x5594ddd3b590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006010 tx=0x7f82b4009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e209ff0676b9b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82baa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82baa00640 1 -- 172.21.15.107:0/1435186849 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b405a070 con 0x5594ddd2d590 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82d6649740 1 -- 172.21.15.107:0/1435186849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82b40047a0 con 0x5594ddd2d590 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82baa00640 1 -- 172.21.15.107:0/1435186849 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f82b4004b50 con 0x5594ddd2d590 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82baa00640 1 -- 172.21.15.107:0/1435186849 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f82b4035040 con 0x5594ddd2d590 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82d6649740 10 client.26135 fetch_fsmap learned FSMap version 34 2024-09-19T04:25:19.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.020+0000 7f82d6649740 1 -- 172.21.15.107:0/1435186849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5594dddb29a0 con 0x5594ddd2d590 2024-09-19T04:25:19.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.021+0000 7f82baa00640 1 -- 172.21.15.107:0/1435186849 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b40047a0 con 0x5594ddd2d590 2024-09-19T04:25:19.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.021+0000 7f82baa00640 1 -- 172.21.15.107:0/1435186849 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f82b4047040 con 0x5594ddd2d590 2024-09-19T04:25:19.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.021+0000 7f82d6649740 10 client.26135 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:25:19.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.021+0000 7f82d6649740 10 client.26135 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:25:19.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.021+0000 7f82d6649740 20 client.26135 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:19.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.021+0000 7f82d6649740 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5594dddb30b0 0x5594dddb54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:19.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.021+0000 7f82d6649740 4 client.26135 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:25:19.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.021+0000 7f82d6649740 1 -- 172.21.15.107:0/1435186849 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594ddc9ae60 con 0x5594dddb30b0 2024-09-19T04:25:19.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.021+0000 7f82d1000640 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5594dddb30b0 0x5594dddb54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:19.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82d1000640 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5594dddb30b0 0x5594dddb54d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2df24da145c485f7 server_cookie=4db7a2676f313ebc in_seq=0 out_seq=0 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82baa00640 10 client.26135 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82baa00640 1 -- 172.21.15.107:0/1435186849 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594ddc9ae60 con 0x5594dddb30b0 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82baa00640 10 client.26135 handle_command_reply: tid=0 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82d6649740 1 client.26135 shutdown 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82d6649740 1 -- 172.21.15.107:0/1435186849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2d590 msgr2=0x5594ddd3b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82d6649740 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2d590 0x5594ddd3b590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006010 tx=0x7f82b4009640 comp rx=0 tx=0).stop 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82d6649740 1 -- 172.21.15.107:0/1435186849 shutdown_connections 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82d6649740 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5594dddb30b0 0x5594dddb54d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82d6649740 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594ddd37e50 0x5594ddd3bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82d6649740 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ddd2d590 0x5594ddd3b590 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82d6649740 1 --2- 172.21.15.107:0/1435186849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5594ddd2cc70 0x5594ddd40260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82d6649740 1 -- 172.21.15.107:0/1435186849 >> 172.21.15.107:0/1435186849 conn(0x5594ddd29200 msgr2=0x5594ddd35530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:19.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.022+0000 7f82baa00640 0 client.26135 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:25:19.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d6649740 1 -- 172.21.15.107:0/1435186849 shutdown_connections 2024-09-19T04:25:19.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d6649740 1 -- 172.21.15.107:0/1435186849 wait complete. 2024-09-19T04:25:19.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d6649740 20 client.26135 trim_cache size 0 max 16384 2024-09-19T04:25:19.029 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:25:19.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d3a00640 1 -- 172.21.15.107:0/3868915524 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f82b00a33b0 msgr2=0x7f82b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:19.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d3a00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f82b00a33b0 0x7f82b00a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009230 tx=0x7f82b40058d0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d3a00640 1 -- 172.21.15.107:0/3868915524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc10c180 msgr2=0x7f82cc19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:19.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d3a00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc10c180 0x7f82cc19efc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f82c400eb10 tx=0x7f82c400c2d0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d3a00640 1 -- 172.21.15.107:0/3868915524 shutdown_connections 2024-09-19T04:25:19.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d3a00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f82b00a33b0 0x7f82b00a5870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d3a00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc10c180 0x7f82cc19efc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d3a00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82cc103670 0x7f82cc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.023+0000 7f82d3a00640 1 --2- 172.21.15.107:0/3868915524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc102d50 0x7f82cc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:19.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.024+0000 7f82d3a00640 1 -- 172.21.15.107:0/3868915524 >> 172.21.15.107:0/3868915524 conn(0x7f82cc0fc7d0 msgr2=0x7f82cc0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:19.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.024+0000 7f82d3a00640 1 -- 172.21.15.107:0/3868915524 shutdown_connections 2024-09-19T04:25:19.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:19.024+0000 7f82d3a00640 1 -- 172.21.15.107:0/3868915524 wait complete. 2024-09-19T04:25:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:18 smithi115 ceph-mon[35477]: pgmap v152: 97 pgs: 97 active+clean; 123 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-09-19T04:25:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:18 smithi145 ceph-mon[35096]: pgmap v152: 97 pgs: 97 active+clean; 123 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-09-19T04:25:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:18 smithi107 ceph-mon[30659]: pgmap v152: 97 pgs: 97 active+clean; 123 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-09-19T04:25:19.667 INFO:tasks.workunit.client.0.smithi107.stdout: 1 188796 3.35 MB/sec execute 97 sec latency 2856.202 ms 2024-09-19T04:25:19.789 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:25:20.161 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:20.667 INFO:tasks.workunit.client.0.smithi107.stdout: 1 188796 3.32 MB/sec execute 98 sec latency 1350.933 ms 2024-09-19T04:25:20.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.725+0000 7f17cce00640 1 -- 172.21.15.107:0/29690147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c81019c0 msgr2=0x7f17c8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.725+0000 7f17cce00640 1 --2- 172.21.15.107:0/29690147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c81019c0 0x7f17c8101e20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009930 tx=0x7f17b002a420 comp rx=0 tx=0).stop 2024-09-19T04:25:20.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.725+0000 7f17cce00640 1 -- 172.21.15.107:0/29690147 shutdown_connections 2024-09-19T04:25:20.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.725+0000 7f17cce00640 1 --2- 172.21.15.107:0/29690147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c8102360 0x7f17c810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.725+0000 7f17cce00640 1 --2- 172.21.15.107:0/29690147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c81019c0 0x7f17c8101e20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.725+0000 7f17cce00640 1 --2- 172.21.15.107:0/29690147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81079c0 0x7f17c8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.725+0000 7f17cce00640 1 -- 172.21.15.107:0/29690147 >> 172.21.15.107:0/29690147 conn(0x7f17c80fd740 msgr2=0x7f17c80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.725+0000 7f17cce00640 1 -- 172.21.15.107:0/29690147 shutdown_connections 2024-09-19T04:25:20.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.725+0000 7f17cce00640 1 -- 172.21.15.107:0/29690147 wait complete. 2024-09-19T04:25:20.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.726+0000 7f17cce00640 1 Processor -- start 2024-09-19T04:25:20.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.726+0000 7f17cce00640 1 -- start start 2024-09-19T04:25:20.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17cce00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c81019c0 0x7f17c819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17cce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c8102360 0x7f17c819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17cce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81079c0 0x7f17c819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17cce00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8111e10 con 0x7f17c8102360 2024-09-19T04:25:20.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17cce00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8111c90 con 0x7f17c81019c0 2024-09-19T04:25:20.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17cce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8111f90 con 0x7f17c81079c0 2024-09-19T04:25:20.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c8102360 0x7f17c819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c81019c0 0x7f17c819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c81019c0 0x7f17c819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35496/0 (socket says 172.21.15.107:35496) 2024-09-19T04:25:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17c7400640 1 -- 172.21.15.107:0/718005963 learned_addr learned my addr 172.21.15.107:0/718005963 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:20.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17c7e00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81079c0 0x7f17c819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17bfe00640 1 -- 172.21.15.107:0/718005963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81079c0 msgr2=0x7f17c819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17bfe00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81079c0 0x7f17c819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17bfe00640 1 -- 172.21.15.107:0/718005963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c81019c0 msgr2=0x7f17c819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17bfe00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c81019c0 0x7f17c819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17bfe00640 1 -- 172.21.15.107:0/718005963 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c819f4e0 con 0x7f17c8102360 2024-09-19T04:25:20.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.727+0000 7f17c7400640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c81019c0 0x7f17c819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:20.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.728+0000 7f17bfe00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c8102360 0x7f17c819aa70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f17b002a940 tx=0x7f17b0003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.728+0000 7f17c4c00640 1 -- 172.21.15.107:0/718005963 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b003f070 con 0x7f17c8102360 2024-09-19T04:25:20.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.728+0000 7f17c4c00640 1 -- 172.21.15.107:0/718005963 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b0031c90 con 0x7f17c8102360 2024-09-19T04:25:20.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.728+0000 7f17c4c00640 1 -- 172.21.15.107:0/718005963 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0035030 con 0x7f17c8102360 2024-09-19T04:25:20.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.728+0000 7f17cce00640 1 -- 172.21.15.107:0/718005963 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c819f770 con 0x7f17c8102360 2024-09-19T04:25:20.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.728+0000 7f17cce00640 1 -- 172.21.15.107:0/718005963 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c81a70b0 con 0x7f17c8102360 2024-09-19T04:25:20.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.732+0000 7f17d0fd4740 1 Processor -- start 2024-09-19T04:25:20.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.732+0000 7f17d0fd4740 1 -- start start 2024-09-19T04:25:20.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbfece40 0x556abbfed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17c4c00640 1 -- 172.21.15.107:0/718005963 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f17b0042070 con 0x7f17c8102360 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbff8020 0x556abbffa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556abbf533e0 con 0x556abbff8020 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556abbefad80 con 0x556abbfece40 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556abbf4f3a0 con 0x556abbfed760 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17c7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbfece40 0x556abbfed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbff8020 0x556abbffa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17c7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbfece40 0x556abbfed220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35500/0 (socket says 172.21.15.107:35500) 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17c7e00640 1 -- 172.21.15.107:0/796315520 learned_addr learned my addr 172.21.15.107:0/796315520 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17c7400640 1 --2- 172.21.15.107:0/796315520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17c7e00640 1 -- 172.21.15.107:0/796315520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 msgr2=0x556abbff7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17c7e00640 1 --2- 172.21.15.107:0/796315520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17c7e00640 1 -- 172.21.15.107:0/796315520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbff8020 msgr2=0x556abbffa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17c7e00640 1 --2- 172.21.15.107:0/796315520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbff8020 0x556abbffa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.733+0000 7f17c7e00640 1 -- 172.21.15.107:0/796315520 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556abbf19ae0 con 0x556abbfece40 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.734+0000 7f17c7e00640 1 --2- 172.21.15.107:0/796315520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbfece40 0x556abbfed220 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009fd0 tx=0x7f17c0010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5484ad0452e4ae18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.734+0000 7f17c4c00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17ac0a3430 0x7f17ac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.734+0000 7f17bcc00640 1 -- 172.21.15.107:0/796315520 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0011700 con 0x556abbfece40 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.734+0000 7f17bcc00640 1 -- 172.21.15.107:0/796315520 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c0011d80 con 0x556abbfece40 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.734+0000 7f17bcc00640 1 -- 172.21.15.107:0/796315520 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c001a380 con 0x556abbfece40 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.734+0000 7f17c4c00640 1 -- 172.21.15.107:0/718005963 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f17b00f05b0 con 0x7f17c8102360 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17c7400640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17ac0a3430 0x7f17ac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17d0fd4740 1 -- 172.21.15.107:0/796315520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbfece40 msgr2=0x556abbfed220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/796315520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbfece40 0x556abbfed220 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009fd0 tx=0x7f17c0010860 comp rx=0 tx=0).stop 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17c7400640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17ac0a3430 0x7f17ac0a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009640 tx=0x7f17b8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17d0fd4740 1 -- 172.21.15.107:0/796315520 shutdown_connections 2024-09-19T04:25:20.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/796315520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbff8020 0x556abbffa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/796315520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/796315520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbfece40 0x556abbfed220 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17d0fd4740 1 -- 172.21.15.107:0/796315520 >> 172.21.15.107:0/796315520 conn(0x556abbfe93d0 msgr2=0x556abbfe97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17d0fd4740 1 -- 172.21.15.107:0/796315520 shutdown_connections 2024-09-19T04:25:20.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.735+0000 7f17d0fd4740 1 -- 172.21.15.107:0/796315520 wait complete. 2024-09-19T04:25:20.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.736+0000 7f17d0fd4740 1 -- start start 2024-09-19T04:25:20.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.736+0000 7f17d0fd4740 1 Processor -- start 2024-09-19T04:25:20.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abbff4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556abbffabf0 con 0x556abbfece40 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556abbf0e0f0 con 0x556abbff8020 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556abbffad70 con 0x556abbfed760 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17d0fd4740 10 client.0 _ll_register_callbacks cb 0x556abbef5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17c7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abbff4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17c7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abbff4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56964/0 (socket says 172.21.15.107:56964) 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17c7400640 1 -- 172.21.15.107:0/506024976 learned_addr learned my addr 172.21.15.107:0/506024976 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17c7e00640 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17bfe00640 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17c7400640 1 -- 172.21.15.107:0/506024976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 msgr2=0x556abbff5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17c7400640 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17c7400640 1 -- 172.21.15.107:0/506024976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 msgr2=0x556abbff5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17c7400640 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.737+0000 7f17c7400640 1 -- 172.21.15.107:0/506024976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556abbcc43c0 con 0x556abbfece40 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.738+0000 7f17c7e00640 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.738+0000 7f17c7400640 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abbff4f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f17b800f8e0 tx=0x7f17b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e8d65a71010c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.738+0000 7f17b5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.738+0000 7f17b5600640 1 -- 172.21.15.107:0/506024976 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8011b50 con 0x556abbfece40 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.738+0000 7f17d0fd4740 1 -- 172.21.15.107:0/506024976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c0011d80 con 0x556abbfece40 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.738+0000 7f17b5600640 1 -- 172.21.15.107:0/506024976 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f17b8011cf0 con 0x556abbfece40 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.738+0000 7f17b5600640 1 -- 172.21.15.107:0/506024976 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f17b8005020 con 0x556abbfece40 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.738+0000 7f17d0fd4740 10 client.16092 fetch_fsmap learned FSMap version 34 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.738+0000 7f17d0fd4740 1 -- 172.21.15.107:0/506024976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556abc006d10 con 0x556abbfece40 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.739+0000 7f17b5600640 1 -- 172.21.15.107:0/506024976 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c0011d80 con 0x556abbfece40 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.739+0000 7f17b5600640 1 -- 172.21.15.107:0/506024976 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f17b8027030 con 0x556abbfece40 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.739+0000 7f17d0fd4740 10 client.16092 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:25:20.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.739+0000 7f17d0fd4740 10 client.16092 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.739+0000 7f17d0fd4740 20 client.16092 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.739+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556abc008120 0x556abc00a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.739+0000 7f17d0fd4740 4 client.16092 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.739+0000 7f17d0fd4740 1 -- 172.21.15.107:0/506024976 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556abbf60950 con 0x556abc008120 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.739+0000 7f17bfe00640 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556abc008120 0x556abc00a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.740+0000 7f17bfe00640 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556abc008120 0x556abc00a530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a703772d0cff3283 server_cookie=28dbcb0a486f7c19 in_seq=0 out_seq=0 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.740+0000 7f17b5600640 10 client.16092 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17b5600640 1 -- 172.21.15.107:0/506024976 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556abbf60950 con 0x556abc008120 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17b5600640 10 client.16092 handle_command_reply: tid=0 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 client.16092 shutdown 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 -- 172.21.15.107:0/506024976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 msgr2=0x556abbff4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abbff4f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f17b800f8e0 tx=0x7f17b8004730 comp rx=0 tx=0).stop 2024-09-19T04:25:20.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 -- 172.21.15.107:0/506024976 shutdown_connections 2024-09-19T04:25:20.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556abc008120 0x556abc00a530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/506024976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abbff4f00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 -- 172.21.15.107:0/506024976 >> 172.21.15.107:0/506024976 conn(0x556abbfe93d0 msgr2=0x556abbfef0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17b5600640 0 client.16092 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:25:20.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 -- 172.21.15.107:0/506024976 shutdown_connections 2024-09-19T04:25:20.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 1 -- 172.21.15.107:0/506024976 wait complete. 2024-09-19T04:25:20.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.741+0000 7f17d0fd4740 20 client.16092 trim_cache size 0 max 16384 2024-09-19T04:25:20.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17d0fd4740 1 Processor -- start 2024-09-19T04:25:20.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17d0fd4740 1 -- start start 2024-09-19T04:25:20.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfece40 0x556abbfef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfed760 0x556abbfe4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81af830 con 0x556abbfed760 2024-09-19T04:25:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81b13a0 con 0x556abbff8020 2024-09-19T04:25:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81aff00 con 0x556abbfece40 2024-09-19T04:25:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfed760 0x556abbfe4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfed760 0x556abbfe4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56968/0 (socket says 172.21.15.107:56968) 2024-09-19T04:25:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17bfe00640 1 -- 172.21.15.107:0/1016104950 learned_addr learned my addr 172.21.15.107:0/1016104950 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.755+0000 7f17c7400640 1 --2- 172.21.15.107:0/1016104950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfece40 0x556abbfef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17c7e00640 1 --2- 172.21.15.107:0/1016104950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17bfe00640 1 -- 172.21.15.107:0/1016104950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfece40 msgr2=0x556abbfef8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17bfe00640 1 --2- 172.21.15.107:0/1016104950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfece40 0x556abbfef8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17bfe00640 1 -- 172.21.15.107:0/1016104950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 msgr2=0x556abbff6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17bfe00640 1 --2- 172.21.15.107:0/1016104950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17bfe00640 1 -- 172.21.15.107:0/1016104950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556abbff3eb0 con 0x556abbfed760 2024-09-19T04:25:20.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17c7400640 1 --2- 172.21.15.107:0/1016104950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfece40 0x556abbfef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:20.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17bfe00640 1 --2- 172.21.15.107:0/1016104950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfed760 0x556abbfe4300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f17b0003fd0 tx=0x7f17b00f60e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad728eddc250293 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17b7e00640 1 -- 172.21.15.107:0/1016104950 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0049040 con 0x556abbfed760 2024-09-19T04:25:20.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17b7e00640 1 -- 172.21.15.107:0/1016104950 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b0031e40 con 0x556abbfed760 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.756+0000 7f17b7e00640 1 -- 172.21.15.107:0/1016104950 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b00c54f0 con 0x556abbfed760 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.757+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1016104950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfed760 msgr2=0x556abbfe4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.757+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/1016104950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfed760 0x556abbfe4300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f17b0003fd0 tx=0x7f17b00f60e0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.757+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1016104950 shutdown_connections 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.757+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/1016104950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.757+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/1016104950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfed760 0x556abbfe4300 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.757+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/1016104950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfece40 0x556abbfef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.757+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1016104950 >> 172.21.15.107:0/1016104950 conn(0x556abbfe93d0 msgr2=0x556abbff6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.757+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1016104950 shutdown_connections 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.757+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1016104950 wait complete. 2024-09-19T04:25:20.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.758+0000 7f17d0fd4740 1 -- start start 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.758+0000 7f17d0fd4740 1 Processor -- start 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.758+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abc0003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.758+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.758+0000 7f17d0fd4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.758+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81b0cd0 con 0x556abbfece40 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.758+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81b05d0 con 0x556abbff8020 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.758+0000 7f17d0fd4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81ad810 con 0x556abbfed760 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.758+0000 7f17d0fd4740 10 client.0 _ll_register_callbacks cb 0x7f17b8012030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abc0003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35540/0 (socket says 172.21.15.107:35540) 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7e00640 1 -- 172.21.15.107:0/1754622104 learned_addr learned my addr 172.21.15.107:0/1754622104 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:20.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17bfe00640 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7e00640 1 -- 172.21.15.107:0/1754622104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 msgr2=0x556abbff18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7e00640 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7e00640 1 -- 172.21.15.107:0/1754622104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 msgr2=0x556abc0003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7e00640 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abc0003f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7e00640 1 -- 172.21.15.107:0/1754622104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556abbff2350 con 0x556abbff8020 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7400640 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abc0003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17c7e00640 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff1e10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f17c0010d60 tx=0x7f17c001ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fcab416eac13c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.759+0000 7f17b5600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17b5600640 1 -- 172.21.15.107:0/1754622104 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c002c020 con 0x556abbff8020 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1754622104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556abbf5ae60 con 0x556abbff8020 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17b5600640 1 -- 172.21.15.107:0/1754622104 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f17c0003970 con 0x556abbff8020 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17b5600640 1 -- 172.21.15.107:0/1754622104 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f17c0009c50 con 0x556abbff8020 2024-09-19T04:25:20.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17d0fd4740 10 client.26153 fetch_fsmap learned FSMap version 34 2024-09-19T04:25:20.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1754622104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556abc072ae0 con 0x556abbff8020 2024-09-19T04:25:20.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17b5600640 1 -- 172.21.15.107:0/1754622104 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556abbf5ae60 con 0x556abbff8020 2024-09-19T04:25:20.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17b5600640 1 -- 172.21.15.107:0/1754622104 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f17c0004020 con 0x556abbff8020 2024-09-19T04:25:20.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17d0fd4740 10 client.26153 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:25:20.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17d0fd4740 10 client.26153 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:25:20.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17d0fd4740 20 client.26153 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:20.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556abc072f30 0x556abc075320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:20.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.760+0000 7f17d0fd4740 4 client.26153 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.761+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1754622104 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556abbed7010 con 0x556abc072f30 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.761+0000 7f17c7400640 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556abc072f30 0x556abc075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.761+0000 7f17c7400640 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556abc072f30 0x556abc075320 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cde31910b97c13f2 server_cookie=bca19b25b91e19aa in_seq=0 out_seq=0 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.761+0000 7f17b5600640 10 client.26153 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17b5600640 1 -- 172.21.15.107:0/1754622104 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556abbed7010 con 0x556abc072f30 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17b5600640 10 client.26153 handle_command_reply: tid=0 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 client.26153 shutdown 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1754622104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 msgr2=0x556abbff1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff1e10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f17c0010d60 tx=0x7f17c001ca10 comp rx=0 tx=0).stop 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1754622104 shutdown_connections 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x556abc072f30 0x556abc075320 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556abbff8020 0x556abbff1e10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556abbfed760 0x556abbff18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 --2- 172.21.15.107:0/1754622104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556abbfece40 0x556abc0003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17b5600640 0 client.26153 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:25:20.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1754622104 >> 172.21.15.107:0/1754622104 conn(0x556abbfe93d0 msgr2=0x556abbff5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1754622104 shutdown_connections 2024-09-19T04:25:20.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 1 -- 172.21.15.107:0/1754622104 wait complete. 2024-09-19T04:25:20.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.762+0000 7f17d0fd4740 20 client.26153 trim_cache size 0 max 16384 2024-09-19T04:25:20.768 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 -- 172.21.15.107:0/718005963 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17ac0a3430 msgr2=0x7f17ac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17ac0a3430 0x7f17ac0a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009640 tx=0x7f17b8005f90 comp rx=0 tx=0).stop 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 -- 172.21.15.107:0/718005963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c8102360 msgr2=0x7f17c819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c8102360 0x7f17c819aa70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f17b002a940 tx=0x7f17b0003b20 comp rx=0 tx=0).stop 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 -- 172.21.15.107:0/718005963 shutdown_connections 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17ac0a3430 0x7f17ac0a58f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81079c0 0x7f17c819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c8102360 0x7f17c819aa70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 --2- 172.21.15.107:0/718005963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c81019c0 0x7f17c819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 -- 172.21.15.107:0/718005963 >> 172.21.15.107:0/718005963 conn(0x7f17c80fd740 msgr2=0x7f17c810cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:20.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 -- 172.21.15.107:0/718005963 shutdown_connections 2024-09-19T04:25:20.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:20.763+0000 7f17cce00640 1 -- 172.21.15.107:0/718005963 wait complete. 2024-09-19T04:25:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:20 smithi115 ceph-mon[35477]: pgmap v153: 97 pgs: 97 active+clean; 123 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-09-19T04:25:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:20 smithi145 ceph-mon[35096]: pgmap v153: 97 pgs: 97 active+clean; 123 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-09-19T04:25:21.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:20 smithi107 ceph-mon[30659]: pgmap v153: 97 pgs: 97 active+clean; 123 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-09-19T04:25:21.516 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:25:21.667 INFO:tasks.workunit.client.0.smithi107.stdout: 1 188796 3.28 MB/sec execute 99 sec latency 2351.003 ms 2024-09-19T04:25:21.889 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:22.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.468+0000 7f8b7c800640 1 -- 172.21.15.107:0/1323065002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b74109120 msgr2=0x7f8b74110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:22.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.468+0000 7f8b7c800640 1 --2- 172.21.15.107:0/1323065002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b74109120 0x7f8b74110ca0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009ca0 tx=0x7f8b7002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.468+0000 7f8b7c800640 1 -- 172.21.15.107:0/1323065002 shutdown_connections 2024-09-19T04:25:22.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.468+0000 7f8b7c800640 1 --2- 172.21.15.107:0/1323065002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b74109120 0x7f8b74110ca0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.468+0000 7f8b7c800640 1 --2- 172.21.15.107:0/1323065002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b741049a0 0x7f8b741089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.468+0000 7f8b7c800640 1 --2- 172.21.15.107:0/1323065002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b74103ff0 0x7f8b741043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.468+0000 7f8b7c800640 1 -- 172.21.15.107:0/1323065002 >> 172.21.15.107:0/1323065002 conn(0x7f8b740ffa90 msgr2=0x7f8b74101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:22.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.468+0000 7f8b7c800640 1 -- 172.21.15.107:0/1323065002 shutdown_connections 2024-09-19T04:25:22.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.468+0000 7f8b7c800640 1 -- 172.21.15.107:0/1323065002 wait complete. 2024-09-19T04:25:22.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b7c800640 1 Processor -- start 2024-09-19T04:25:22.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b7c800640 1 -- start start 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b7c800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b74103ff0 0x7f8b7419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b7c800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b741049a0 0x7f8b7419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b7c800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b74109120 0x7f8b7419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b7c800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74114150 con 0x7f8b741049a0 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b7c800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74113fd0 con 0x7f8b74103ff0 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b7c800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741142d0 con 0x7f8b74109120 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b79400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b741049a0 0x7f8b7419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b79400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b741049a0 0x7f8b7419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56980/0 (socket says 172.21.15.107:56980) 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.469+0000 7f8b79400640 1 -- 172.21.15.107:0/688771852 learned_addr learned my addr 172.21.15.107:0/688771852 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.470+0000 7f8b79e00640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b74103ff0 0x7f8b7419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.470+0000 7f8b7a800640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b74109120 0x7f8b7419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.470+0000 7f8b79400640 1 -- 172.21.15.107:0/688771852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b74109120 msgr2=0x7f8b7419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.470+0000 7f8b79400640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b74109120 0x7f8b7419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.470+0000 7f8b79400640 1 -- 172.21.15.107:0/688771852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b74103ff0 msgr2=0x7f8b7419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.470+0000 7f8b79400640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b74103ff0 0x7f8b7419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.470+0000 7f8b79400640 1 -- 172.21.15.107:0/688771852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7419f5e0 con 0x7f8b741049a0 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.470+0000 7f8b7a800640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b74109120 0x7f8b7419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.470+0000 7f8b79e00640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b74103ff0 0x7f8b7419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.470+0000 7f8b79400640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b741049a0 0x7f8b7419ab70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400e9f0 tx=0x7f8b6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.471+0000 7f8b6ea00640 1 -- 172.21.15.107:0/688771852 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400cd80 con 0x7f8b741049a0 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.471+0000 7f8b6ea00640 1 -- 172.21.15.107:0/688771852 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b64010910 con 0x7f8b741049a0 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.471+0000 7f8b7c800640 1 -- 172.21.15.107:0/688771852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7419f8d0 con 0x7f8b741049a0 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.471+0000 7f8b6ea00640 1 -- 172.21.15.107:0/688771852 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64018990 con 0x7f8b741049a0 2024-09-19T04:25:22.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.471+0000 7f8b7c800640 1 -- 172.21.15.107:0/688771852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b741a71b0 con 0x7f8b741049a0 2024-09-19T04:25:22.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.474+0000 7f8b6ea00640 1 -- 172.21.15.107:0/688771852 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f8b64010ac0 con 0x7f8b741049a0 2024-09-19T04:25:22.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.474+0000 7f8b7f289740 1 Processor -- start 2024-09-19T04:25:22.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.474+0000 7f8b7f289740 1 -- start start 2024-09-19T04:25:22.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.474+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad15c70 0x55683ad16050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.474+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad16590 0x55683ad20910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.474+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad20e50 0x55683ad23240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.474+0000 7f8b7f289740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55683ac7c3e0 con 0x55683ad16590 2024-09-19T04:25:22.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.474+0000 7f8b7f289740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55683ac23d80 con 0x55683ad20e50 2024-09-19T04:25:22.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.474+0000 7f8b7f289740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55683ac783a0 con 0x55683ad15c70 2024-09-19T04:25:22.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad16590 0x55683ad20910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b7a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad15c70 0x55683ad16050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b7a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad15c70 0x55683ad16050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:40666/0 (socket says 172.21.15.107:40666) 2024-09-19T04:25:22.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b7a800640 1 -- 172.21.15.107:0/2092967859 learned_addr learned my addr 172.21.15.107:0/2092967859 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:22.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b79400640 1 --2- 172.21.15.107:0/2092967859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad20e50 0x55683ad23240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b7a800640 1 -- 172.21.15.107:0/2092967859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad20e50 msgr2=0x55683ad23240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:22.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b7a800640 1 --2- 172.21.15.107:0/2092967859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad20e50 0x55683ad23240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b7a800640 1 -- 172.21.15.107:0/2092967859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad16590 msgr2=0x55683ad20910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b7a800640 1 --2- 172.21.15.107:0/2092967859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad16590 0x55683ad20910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b7a800640 1 -- 172.21.15.107:0/2092967859 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683ac42ae0 con 0x55683ad15c70 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b6ea00640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8b5c0a33b0 0x7f8b5c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.475+0000 7f8b7a800640 1 --2- 172.21.15.107:0/2092967859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad15c70 0x55683ad16050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8b70005f60 tx=0x7f8b70005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ce091cc6387828a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.476+0000 7f8b6ea00640 1 -- 172.21.15.107:0/688771852 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f8b640d0070 con 0x7f8b741049a0 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.476+0000 7f8b79e00640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8b5c0a33b0 0x7f8b5c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.476+0000 7f8b63000640 1 -- 172.21.15.107:0/2092967859 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70031db0 con 0x55683ad15c70 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.476+0000 7f8b63000640 1 -- 172.21.15.107:0/2092967859 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b70036070 con 0x55683ad15c70 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.476+0000 7f8b63000640 1 -- 172.21.15.107:0/2092967859 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b700042b0 con 0x55683ad15c70 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 -- 172.21.15.107:0/2092967859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad15c70 msgr2=0x55683ad16050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 --2- 172.21.15.107:0/2092967859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad15c70 0x55683ad16050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8b70005f60 tx=0x7f8b70005f90 comp rx=0 tx=0).stop 2024-09-19T04:25:22.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b79e00640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8b5c0a33b0 0x7f8b5c0a5870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55683ad13060 tx=0x7f8b68005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:22.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 -- 172.21.15.107:0/2092967859 shutdown_connections 2024-09-19T04:25:22.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 --2- 172.21.15.107:0/2092967859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad20e50 0x55683ad23240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 --2- 172.21.15.107:0/2092967859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad16590 0x55683ad20910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 --2- 172.21.15.107:0/2092967859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad15c70 0x55683ad16050 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 -- 172.21.15.107:0/2092967859 >> 172.21.15.107:0/2092967859 conn(0x55683ad12200 msgr2=0x55683ad125e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:22.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 -- 172.21.15.107:0/2092967859 shutdown_connections 2024-09-19T04:25:22.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 -- 172.21.15.107:0/2092967859 wait complete. 2024-09-19T04:25:22.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 -- start start 2024-09-19T04:25:22.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.477+0000 7f8b7f289740 1 Processor -- start 2024-09-19T04:25:22.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad1dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b7f289740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55683ad238a0 con 0x55683ad15c70 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b7f289740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55683ac370f0 con 0x55683ad16590 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b7f289740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55683ad23a20 con 0x55683ad20e50 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b7f289740 10 client.0 _ll_register_callbacks cb 0x55683ac1ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad1dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad1dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56996/0 (socket says 172.21.15.107:56996) 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b79e00640 1 -- 172.21.15.107:0/1360625893 learned_addr learned my addr 172.21.15.107:0/1360625893 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b79400640 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.478+0000 7f8b7a800640 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b7a800640 1 -- 172.21.15.107:0/1360625893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 msgr2=0x55683ad1e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:22.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b7a800640 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b7a800640 1 -- 172.21.15.107:0/1360625893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 msgr2=0x55683ad1dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b7a800640 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad1dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b7a800640 1 -- 172.21.15.107:0/1360625893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683a9ed3c0 con 0x55683ad20e50 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b79400640 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b79e00640 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad1dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b7a800640 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1e790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009bf0 tx=0x7f8b70031e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f24d243583d7a2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b57400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b57400640 1 -- 172.21.15.107:0/1360625893 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7004c060 con 0x55683ad20e50 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.479+0000 7f8b7f289740 1 -- 172.21.15.107:0/1360625893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b70036070 con 0x55683ad20e50 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.480+0000 7f8b57400640 1 -- 172.21.15.107:0/1360625893 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f8b70003d80 con 0x55683ad20e50 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.480+0000 7f8b57400640 1 -- 172.21.15.107:0/1360625893 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8b70004050 con 0x55683ad20e50 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.480+0000 7f8b7f289740 10 client.26125 fetch_fsmap learned FSMap version 35 2024-09-19T04:25:22.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.480+0000 7f8b7f289740 1 -- 172.21.15.107:0/1360625893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55683ad238a0 con 0x55683ad20e50 2024-09-19T04:25:22.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.480+0000 7f8b57400640 1 -- 172.21.15.107:0/1360625893 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b70036070 con 0x55683ad20e50 2024-09-19T04:25:22.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.480+0000 7f8b57400640 1 -- 172.21.15.107:0/1360625893 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f8b70042020 con 0x55683ad20e50 2024-09-19T04:25:22.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.480+0000 7f8b7f289740 10 client.26125 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T04:25:22.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.480+0000 7f8b7f289740 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:22.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.480+0000 7f8b7f289740 20 client.26125 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:22.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.480+0000 7f8b7f289740 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55683ad30a30 0x55683ad32e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.481+0000 7f8b7f289740 4 client.26125 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:22.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.481+0000 7f8b7f289740 1 -- 172.21.15.107:0/1360625893 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55683ac89950 con 0x55683ad30a30 2024-09-19T04:25:22.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.481+0000 7f8b79e00640 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55683ad30a30 0x55683ad32e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.481+0000 7f8b79e00640 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55683ad30a30 0x55683ad32e20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241c2aa159f55de0 server_cookie=9da8d0337558f6a6 in_seq=0 out_seq=0 2024-09-19T04:25:22.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.481+0000 7f8b57400640 10 client.26125 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:22.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.482+0000 7f8b57400640 1 -- 172.21.15.107:0/1360625893 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55683ac89950 con 0x55683ad30a30 2024-09-19T04:25:22.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.482+0000 7f8b57400640 10 client.26125 handle_command_reply: tid=0 2024-09-19T04:25:22.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.482+0000 7f8b7f289740 1 client.26125 shutdown 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.482+0000 7f8b7f289740 1 -- 172.21.15.107:0/1360625893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 msgr2=0x55683ad1e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.482+0000 7f8b7f289740 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1e790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009bf0 tx=0x7f8b70031e60 comp rx=0 tx=0).stop 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.482+0000 7f8b7f289740 1 -- 172.21.15.107:0/1360625893 shutdown_connections 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.482+0000 7f8b7f289740 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55683ad30a30 0x55683ad32e20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.483+0000 7f8b57400640 0 client.26125 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.482+0000 7f8b7f289740 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1e790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.483+0000 7f8b7f289740 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.483+0000 7f8b7f289740 1 --2- 172.21.15.107:0/1360625893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad1dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.483+0000 7f8b7f289740 1 -- 172.21.15.107:0/1360625893 >> 172.21.15.107:0/1360625893 conn(0x55683ad12200 msgr2=0x55683ad11380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.483+0000 7f8b7f289740 1 -- 172.21.15.107:0/1360625893 shutdown_connections 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.483+0000 7f8b7f289740 1 -- 172.21.15.107:0/1360625893 wait complete. 2024-09-19T04:25:22.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.483+0000 7f8b7f289740 20 client.26125 trim_cache size 0 max 16384 2024-09-19T04:25:22.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.496+0000 7f8b7f289740 1 Processor -- start 2024-09-19T04:25:22.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.496+0000 7f8b7f289740 1 -- start start 2024-09-19T04:25:22.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad38680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad38bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b7f289740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741af910 con 0x55683ad15c70 2024-09-19T04:25:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b7f289740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741b14e0 con 0x55683ad16590 2024-09-19T04:25:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b7f289740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741affe0 con 0x55683ad20e50 2024-09-19T04:25:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad38680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b79400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad38bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b79400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad38bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35586/0 (socket says 172.21.15.107:35586) 2024-09-19T04:25:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b79400640 1 -- 172.21.15.107:0/4247577821 learned_addr learned my addr 172.21.15.107:0/4247577821 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad38680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57008/0 (socket says 172.21.15.107:57008) 2024-09-19T04:25:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b7a800640 1 --2- 172.21.15.107:0/4247577821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b79400640 1 -- 172.21.15.107:0/4247577821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 msgr2=0x55683ad1dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b79400640 1 --2- 172.21.15.107:0/4247577821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b79400640 1 -- 172.21.15.107:0/4247577821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 msgr2=0x55683ad38680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b79400640 1 --2- 172.21.15.107:0/4247577821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad38680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.497+0000 7f8b79400640 1 -- 172.21.15.107:0/4247577821 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683ad163c0 con 0x55683ad16590 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b79400640 1 --2- 172.21.15.107:0/4247577821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad38bc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8b640987a0 tx=0x7f8b640d3b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ea61309d7e4abc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b62600640 1 -- 172.21.15.107:0/4247577821 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6409bca0 con 0x55683ad16590 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b62600640 1 -- 172.21.15.107:0/4247577821 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6409ca40 con 0x55683ad16590 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b62600640 1 -- 172.21.15.107:0/4247577821 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b640d20f0 con 0x55683ad16590 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b7f289740 1 -- 172.21.15.107:0/4247577821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 msgr2=0x55683ad38bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b7f289740 1 --2- 172.21.15.107:0/4247577821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad38bc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8b640987a0 tx=0x7f8b640d3b20 comp rx=0 tx=0).stop 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b7f289740 1 -- 172.21.15.107:0/4247577821 shutdown_connections 2024-09-19T04:25:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b7f289740 1 --2- 172.21.15.107:0/4247577821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b7f289740 1 --2- 172.21.15.107:0/4247577821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad38bc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b7f289740 1 --2- 172.21.15.107:0/4247577821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad38680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b7f289740 1 -- 172.21.15.107:0/4247577821 >> 172.21.15.107:0/4247577821 conn(0x55683ad12200 msgr2=0x55683ad1fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.498+0000 7f8b7f289740 1 -- 172.21.15.107:0/4247577821 shutdown_connections 2024-09-19T04:25:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.499+0000 7f8b7f289740 1 -- 172.21.15.107:0/4247577821 wait complete. 2024-09-19T04:25:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.499+0000 7f8b7f289740 1 -- start start 2024-09-19T04:25:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.499+0000 7f8b7f289740 1 Processor -- start 2024-09-19T04:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.499+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad29070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.499+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.499+0000 7f8b7f289740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.499+0000 7f8b7f289740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741b0de0 con 0x55683ad15c70 2024-09-19T04:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.499+0000 7f8b7f289740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741b06b0 con 0x55683ad16590 2024-09-19T04:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.499+0000 7f8b7f289740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741ad8f0 con 0x55683ad20e50 2024-09-19T04:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b7f289740 10 client.0 _ll_register_callbacks cb 0x7f8b640d39f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad29070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35602/0 (socket says 172.21.15.107:35602) 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79400640 1 -- 172.21.15.107:0/4070548851 learned_addr learned my addr 172.21.15.107:0/4070548851 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b7a800640 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79400640 1 -- 172.21.15.107:0/4070548851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 msgr2=0x55683ad1ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79400640 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79400640 1 -- 172.21.15.107:0/4070548851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 msgr2=0x55683ad29070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79400640 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad29070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79400640 1 -- 172.21.15.107:0/4070548851 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683ad1b150 con 0x55683ad16590 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b7a800640 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79e00640 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad29070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b79400640 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1a6d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c6a0 tx=0x7f8b640d2380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad0d4e41d4e723c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.500+0000 7f8b57400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:22.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b57400640 1 -- 172.21.15.107:0/4070548851 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6409bca0 con 0x55683ad16590 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b57400640 1 -- 172.21.15.107:0/4070548851 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f8b64018b30 con 0x55683ad16590 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b7f289740 1 -- 172.21.15.107:0/4070548851 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b6409ca40 con 0x55683ad16590 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b57400640 1 -- 172.21.15.107:0/4070548851 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8b640cdc40 con 0x55683ad16590 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b7f289740 10 client.26171 fetch_fsmap learned FSMap version 35 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b7f289740 1 -- 172.21.15.107:0/4070548851 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55683ad9b2d0 con 0x55683ad16590 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b57400640 1 -- 172.21.15.107:0/4070548851 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6409ca40 con 0x55683ad16590 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b57400640 1 -- 172.21.15.107:0/4070548851 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f8b6409b790 con 0x55683ad16590 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b7f289740 10 client.26171 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b7f289740 10 client.26171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b7f289740 20 client.26171 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b7f289740 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55683ad9b9f0 0x55683ad9de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b7f289740 4 client.26171 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:25:22.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.501+0000 7f8b7f289740 1 -- 172.21.15.107:0/4070548851 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55683ac83e60 con 0x55683ad9b9f0 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.502+0000 7f8b79e00640 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55683ad9b9f0 0x55683ad9de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.502+0000 7f8b79e00640 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55683ad9b9f0 0x55683ad9de10 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69424995d8d279ed server_cookie=ab68bfc0266b77c in_seq=0 out_seq=0 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.502+0000 7f8b57400640 10 client.26171 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.502+0000 7f8b57400640 1 -- 172.21.15.107:0/4070548851 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55683ac83e60 con 0x55683ad9b9f0 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.502+0000 7f8b57400640 10 client.26171 handle_command_reply: tid=0 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.502+0000 7f8b7f289740 1 client.26171 shutdown 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.502+0000 7f8b7f289740 1 -- 172.21.15.107:0/4070548851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 msgr2=0x55683ad1a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.502+0000 7f8b7f289740 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1a6d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c6a0 tx=0x7f8b640d2380 comp rx=0 tx=0).stop 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7f289740 1 -- 172.21.15.107:0/4070548851 shutdown_connections 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7f289740 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55683ad9b9f0 0x55683ad9de10 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7f289740 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55683ad20e50 0x55683ad1ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7f289740 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55683ad16590 0x55683ad1a6d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b57400640 0 client.26171 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:22.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7f289740 1 --2- 172.21.15.107:0/4070548851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55683ad15c70 0x55683ad29070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7f289740 1 -- 172.21.15.107:0/4070548851 >> 172.21.15.107:0/4070548851 conn(0x55683ad12200 msgr2=0x55683ad1c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:22.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7f289740 1 -- 172.21.15.107:0/4070548851 shutdown_connections 2024-09-19T04:25:22.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7f289740 1 -- 172.21.15.107:0/4070548851 wait complete. 2024-09-19T04:25:22.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7f289740 20 client.26171 trim_cache size 0 max 16384 2024-09-19T04:25:22.509 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:25:22.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7c800640 1 -- 172.21.15.107:0/688771852 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8b5c0a33b0 msgr2=0x7f8b5c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:22.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7c800640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8b5c0a33b0 0x7f8b5c0a5870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55683ad13060 tx=0x7f8b68005f90 comp rx=0 tx=0).stop 2024-09-19T04:25:22.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7c800640 1 -- 172.21.15.107:0/688771852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b741049a0 msgr2=0x7f8b7419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:22.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.503+0000 7f8b7c800640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b741049a0 0x7f8b7419ab70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400e9f0 tx=0x7f8b6400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.504+0000 7f8b7c800640 1 -- 172.21.15.107:0/688771852 shutdown_connections 2024-09-19T04:25:22.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.504+0000 7f8b7c800640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8b5c0a33b0 0x7f8b5c0a5870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.504+0000 7f8b7c800640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b74109120 0x7f8b7419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.504+0000 7f8b7c800640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b741049a0 0x7f8b7419ab70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.504+0000 7f8b7c800640 1 --2- 172.21.15.107:0/688771852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b74103ff0 0x7f8b7419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:22.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.504+0000 7f8b7c800640 1 -- 172.21.15.107:0/688771852 >> 172.21.15.107:0/688771852 conn(0x7f8b740ffa90 msgr2=0x7f8b741005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:22.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.504+0000 7f8b7c800640 1 -- 172.21.15.107:0/688771852 shutdown_connections 2024-09-19T04:25:22.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:22.504+0000 7f8b7c800640 1 -- 172.21.15.107:0/688771852 wait complete. 2024-09-19T04:25:22.667 INFO:tasks.workunit.client.0.smithi107.stdout: 1 188796 3.25 MB/sec execute 100 sec latency 3351.093 ms 2024-09-19T04:25:23.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:22 smithi107 ceph-mon[30659]: pgmap v154: 97 pgs: 97 active+clean; 125 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-19T04:25:23.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:22 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:22 smithi115 ceph-mon[35477]: pgmap v154: 97 pgs: 97 active+clean; 125 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-19T04:25:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:22 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:22 smithi145 ceph-mon[35096]: pgmap v154: 97 pgs: 97 active+clean; 125 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-19T04:25:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:22 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:23.667 INFO:tasks.workunit.client.0.smithi107.stdout: 1 188796 3.22 MB/sec execute 101 sec latency 4351.229 ms 2024-09-19T04:25:24.420 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:25:24.420 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:25:24.667 INFO:tasks.workunit.client.0.smithi107.stdout: 1 189655 3.24 MB/sec execute 102 sec latency 4898.586 ms 2024-09-19T04:25:24.824 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:24 smithi145 ceph-mon[35096]: pgmap v155: 97 pgs: 97 active+clean; 125 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-09-19T04:25:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:24 smithi107 ceph-mon[30659]: pgmap v155: 97 pgs: 97 active+clean; 125 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-09-19T04:25:25.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.434+0000 7fedfac00640 1 -- 172.21.15.107:0/152705058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf410e4e0 msgr2=0x7fedf41108d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.434+0000 7fedfac00640 1 --2- 172.21.15.107:0/152705058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf410e4e0 0x7fedf41108d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fedec009ca0 tx=0x7fedec02a8b0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.434+0000 7fedfac00640 1 -- 172.21.15.107:0/152705058 shutdown_connections 2024-09-19T04:25:25.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.434+0000 7fedfac00640 1 --2- 172.21.15.107:0/152705058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf410e4e0 0x7fedf41108d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.434+0000 7fedfac00640 1 --2- 172.21.15.107:0/152705058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf41012e0 0x7fedf410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.434+0000 7fedfac00640 1 --2- 172.21.15.107:0/152705058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedf4100930 0x7fedf4100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.434+0000 7fedfac00640 1 -- 172.21.15.107:0/152705058 >> 172.21.15.107:0/152705058 conn(0x7fedf40fc7a0 msgr2=0x7fedf40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.434+0000 7fedfac00640 1 -- 172.21.15.107:0/152705058 shutdown_connections 2024-09-19T04:25:25.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.434+0000 7fedfac00640 1 -- 172.21.15.107:0/152705058 wait complete. 2024-09-19T04:25:25.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.435+0000 7fedfac00640 1 Processor -- start 2024-09-19T04:25:25.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.435+0000 7fedfac00640 1 -- start start 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.435+0000 7fedfac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf4100930 0x7fedf419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedfac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedf41012e0 0x7fedf419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedfac00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf410e4e0 0x7fedf419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedfac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fedf4114160 con 0x7fedf4100930 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedfac00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fedf4113fe0 con 0x7fedf410e4e0 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedfac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41142e0 con 0x7fedf41012e0 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf4100930 0x7fedf419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf8c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf410e4e0 0x7fedf419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedf41012e0 0x7fedf419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf4100930 0x7fedf419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57042/0 (socket says 172.21.15.107:57042) 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf8c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf410e4e0 0x7fedf419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35616/0 (socket says 172.21.15.107:35616) 2024-09-19T04:25:25.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf3e00640 1 -- 172.21.15.107:0/85234911 learned_addr learned my addr 172.21.15.107:0/85234911 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:25.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf8c00640 1 -- 172.21.15.107:0/85234911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedf41012e0 msgr2=0x7fedf419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf8c00640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedf41012e0 0x7fedf419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf8c00640 1 -- 172.21.15.107:0/85234911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf4100930 msgr2=0x7fedf419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf8c00640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf4100930 0x7fedf419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf8c00640 1 -- 172.21.15.107:0/85234911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf419f670 con 0x7fedf410e4e0 2024-09-19T04:25:25.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.436+0000 7fedf3400640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedf41012e0 0x7fedf419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:25.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.437+0000 7fedf8c00640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf410e4e0 0x7fedf419ef90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fedf4102b10 tx=0x7fedec030ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.437+0000 7fedf0c00640 1 -- 172.21.15.107:0/85234911 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec003d80 con 0x7fedf410e4e0 2024-09-19T04:25:25.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.437+0000 7fedf0c00640 1 -- 172.21.15.107:0/85234911 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedec040070 con 0x7fedf410e4e0 2024-09-19T04:25:25.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.437+0000 7fedfac00640 1 -- 172.21.15.107:0/85234911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf419f900 con 0x7fedf410e4e0 2024-09-19T04:25:25.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.437+0000 7fedf0c00640 1 -- 172.21.15.107:0/85234911 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec045a60 con 0x7fedf410e4e0 2024-09-19T04:25:25.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.437+0000 7fedfac00640 1 -- 172.21.15.107:0/85234911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf41a7240 con 0x7fedf410e4e0 2024-09-19T04:25:25.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.440+0000 7fedfd689740 1 Processor -- start 2024-09-19T04:25:25.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.440+0000 7fedf0c00640 1 -- 172.21.15.107:0/85234911 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fedec004690 con 0x7fedf410e4e0 2024-09-19T04:25:25.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedfd689740 1 -- start start 2024-09-19T04:25:25.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedfd689740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c68ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedfd689740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 0x561537c73760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedfd689740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 0x561537c76090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedfd689740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561537bcf290 con 0x561537c73ca0 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedfd689740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561537b76b40 con 0x561537c68ac0 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedfd689740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561537bcb250 con 0x561537c693e0 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 0x561537c73760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedf3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c68ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 0x561537c73760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:40744/0 (socket says 172.21.15.107:40744) 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedf3e00640 1 -- 172.21.15.107:0/2518578973 learned_addr learned my addr 172.21.15.107:0/2518578973 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.441+0000 7fedf3400640 1 --2- 172.21.15.107:0/2518578973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 0x561537c76090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.442+0000 7fedf3e00640 1 -- 172.21.15.107:0/2518578973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 msgr2=0x561537c68ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.442+0000 7fedf3e00640 1 --2- 172.21.15.107:0/2518578973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c68ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.442+0000 7fedf3e00640 1 -- 172.21.15.107:0/2518578973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 msgr2=0x561537c76090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.442+0000 7fedf3e00640 1 --2- 172.21.15.107:0/2518578973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 0x561537c76090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.442+0000 7fedf3e00640 1 -- 172.21.15.107:0/2518578973 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615379406a0 con 0x561537c693e0 2024-09-19T04:25:25.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.442+0000 7fedf3400640 1 --2- 172.21.15.107:0/2518578973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 0x561537c76090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:25.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.442+0000 7fedf3e00640 1 --2- 172.21.15.107:0/2518578973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 0x561537c73760 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fede4009e10 tx=0x7fede4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15db6cdf9a242f2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.442+0000 7fede9200640 1 -- 172.21.15.107:0/2518578973 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede401e070 con 0x561537c693e0 2024-09-19T04:25:25.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.442+0000 7fede9200640 1 -- 172.21.15.107:0/2518578973 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede4019b10 con 0x561537c693e0 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.442+0000 7fede9200640 1 -- 172.21.15.107:0/2518578973 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede400fbf0 con 0x561537c693e0 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedf0c00640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fedd80a3430 0x7fedd80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedf0c00640 1 -- 172.21.15.107:0/85234911 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fedec0f1080 con 0x7fedf410e4e0 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedf3400640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fedd80a3430 0x7fedd80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedfd689740 1 -- 172.21.15.107:0/2518578973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 msgr2=0x561537c73760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedfd689740 1 --2- 172.21.15.107:0/2518578973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 0x561537c73760 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fede4009e10 tx=0x7fede4011810 comp rx=0 tx=0).stop 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedfd689740 1 -- 172.21.15.107:0/2518578973 shutdown_connections 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedfd689740 1 --2- 172.21.15.107:0/2518578973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 0x561537c76090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedfd689740 1 --2- 172.21.15.107:0/2518578973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 0x561537c73760 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedfd689740 1 --2- 172.21.15.107:0/2518578973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c68ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedfd689740 1 -- 172.21.15.107:0/2518578973 >> 172.21.15.107:0/2518578973 conn(0x561537c65050 msgr2=0x561537c65430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.443+0000 7fedfd689740 1 -- 172.21.15.107:0/2518578973 shutdown_connections 2024-09-19T04:25:25.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.444+0000 7fedfd689740 1 -- 172.21.15.107:0/2518578973 wait complete. 2024-09-19T04:25:25.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.444+0000 7fedf3400640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fedd80a3430 0x7fedd80a58f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x561537c663d0 tx=0x7fede0013040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.445+0000 7fedfd689740 1 -- start start 2024-09-19T04:25:25.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.445+0000 7fedfd689740 1 Processor -- start 2024-09-19T04:25:25.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:24 smithi115 ceph-mon[35477]: pgmap v155: 97 pgs: 97 active+clean; 125 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-09-19T04:25:25.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.445+0000 7fedfd689740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.445+0000 7fedfd689740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 0x561537c70860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.445+0000 7fedfd689740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 0x561537c70da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.445+0000 7fedfd689740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561537c76910 con 0x561537c73ca0 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.445+0000 7fedfd689740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561537b89fa0 con 0x561537c68ac0 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.445+0000 7fedfd689740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561537c76a90 con 0x561537c693e0 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.445+0000 7fedfd689740 10 client.0 _ll_register_callbacks cb 0x5615378ddba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 0x561537c70860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c735b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35642/0 (socket says 172.21.15.107:35642) 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf8c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 0x561537c70da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf3e00640 1 -- 172.21.15.107:0/3841475406 learned_addr learned my addr 172.21.15.107:0/3841475406 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf3e00640 1 -- 172.21.15.107:0/3841475406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 msgr2=0x561537c70860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf3e00640 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 0x561537c70860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf3e00640 1 -- 172.21.15.107:0/3841475406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 msgr2=0x561537c70da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf3e00640 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 0x561537c70da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf3e00640 1 -- 172.21.15.107:0/3841475406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561537c715a0 con 0x561537c68ac0 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7fedf3e00640 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c735b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fede401c040 tx=0x7fede40198a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=402cfa73912ac850 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.446+0000 7feddd600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.447+0000 7feddd600640 1 -- 172.21.15.107:0/3841475406 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4026020 con 0x561537c68ac0 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.447+0000 7feddd600640 1 -- 172.21.15.107:0/3841475406 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fede400f930 con 0x561537c68ac0 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.447+0000 7fedfd689740 1 -- 172.21.15.107:0/3841475406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede4019b10 con 0x561537c68ac0 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.447+0000 7feddd600640 1 -- 172.21.15.107:0/3841475406 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fede4031050 con 0x561537c68ac0 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.447+0000 7fedfd689740 10 client.26183 fetch_fsmap learned FSMap version 35 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.447+0000 7fedfd689740 1 -- 172.21.15.107:0/3841475406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561537c82820 con 0x561537c68ac0 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.447+0000 7feddd600640 1 -- 172.21.15.107:0/3841475406 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede4019b10 con 0x561537c68ac0 2024-09-19T04:25:25.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.447+0000 7feddd600640 1 -- 172.21.15.107:0/3841475406 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fede40046c0 con 0x561537c68ac0 2024-09-19T04:25:25.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.448+0000 7fedfd689740 10 client.26183 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T04:25:25.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.448+0000 7fedfd689740 10 client.26183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:25.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.448+0000 7fedfd689740 20 client.26183 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:25.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.448+0000 7fedfd689740 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561537c83ce0 0x561537c860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.448+0000 7fedfd689740 4 client.26183 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:25.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.448+0000 7fedfd689740 1 -- 172.21.15.107:0/3841475406 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561537bdc800 con 0x561537c83ce0 2024-09-19T04:25:25.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.448+0000 7fedf3400640 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561537c83ce0 0x561537c860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.449+0000 7fedf3400640 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561537c83ce0 0x561537c860d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578421bce976d800 server_cookie=4621778f65dfa20 in_seq=0 out_seq=0 2024-09-19T04:25:25.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.449+0000 7feddd600640 10 client.26183 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:25.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.449+0000 7feddd600640 1 -- 172.21.15.107:0/3841475406 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561537bdc800 con 0x561537c83ce0 2024-09-19T04:25:25.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.449+0000 7feddd600640 10 client.26183 handle_command_reply: tid=0 2024-09-19T04:25:25.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.449+0000 7fedfd689740 1 client.26183 shutdown 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.449+0000 7fedfd689740 1 -- 172.21.15.107:0/3841475406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 msgr2=0x561537c735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.449+0000 7fedfd689740 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c735b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fede401c040 tx=0x7fede40198a0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.450+0000 7fedfd689740 1 -- 172.21.15.107:0/3841475406 shutdown_connections 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.450+0000 7fedfd689740 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561537c83ce0 0x561537c860d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.450+0000 7fedfd689740 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c73ca0 0x561537c70da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.450+0000 7fedfd689740 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c693e0 0x561537c70860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.450+0000 7feddd600640 0 client.26183 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.450+0000 7fedfd689740 1 --2- 172.21.15.107:0/3841475406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c735b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.450+0000 7fedfd689740 1 -- 172.21.15.107:0/3841475406 >> 172.21.15.107:0/3841475406 conn(0x561537c65050 msgr2=0x561537c6ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.450+0000 7fedfd689740 1 -- 172.21.15.107:0/3841475406 shutdown_connections 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.450+0000 7fedfd689740 1 -- 172.21.15.107:0/3841475406 wait complete. 2024-09-19T04:25:25.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.450+0000 7fedfd689740 20 client.26183 trim_cache size 0 max 16384 2024-09-19T04:25:25.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.471+0000 7fedfd689740 1 Processor -- start 2024-09-19T04:25:25.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.471+0000 7fedfd689740 1 -- start start 2024-09-19T04:25:25.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.471+0000 7fedfd689740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.471+0000 7fedfd689740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c693e0 0x561537c7bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.471+0000 7fedfd689740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537c7c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.471+0000 7fedfd689740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41af9d0 con 0x561537c693e0 2024-09-19T04:25:25.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.471+0000 7fedfd689740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41b15a0 con 0x561537c68ac0 2024-09-19T04:25:25.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.471+0000 7fedfd689740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41b00a0 con 0x561537c73ca0 2024-09-19T04:25:25.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537c7c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c640b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35644/0 (socket says 172.21.15.107:35644) 2024-09-19T04:25:25.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf3e00640 1 -- 172.21.15.107:0/1811069524 learned_addr learned my addr 172.21.15.107:0/1811069524 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:25.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf3400640 1 --2- 172.21.15.107:0/1811069524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c693e0 0x561537c7bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf8c00640 1 -- 172.21.15.107:0/1811069524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 msgr2=0x561537c640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf8c00640 1 --2- 172.21.15.107:0/1811069524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf8c00640 1 -- 172.21.15.107:0/1811069524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c693e0 msgr2=0x561537c7bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf8c00640 1 --2- 172.21.15.107:0/1811069524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c693e0 0x561537c7bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf8c00640 1 -- 172.21.15.107:0/1811069524 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561537c69210 con 0x561537c73ca0 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7fedf8c00640 1 --2- 172.21.15.107:0/1811069524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537c7c250 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fedec0ee8d0 tx=0x7fedec045500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf53ad55bffa82a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.472+0000 7feddfe00640 1 -- 172.21.15.107:0/1811069524 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec0bd460 con 0x561537c73ca0 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.473+0000 7feddfe00640 1 -- 172.21.15.107:0/1811069524 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedec036030 con 0x561537c73ca0 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.473+0000 7feddfe00640 1 -- 172.21.15.107:0/1811069524 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec0bd6c0 con 0x561537c73ca0 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.473+0000 7fedfd689740 1 -- 172.21.15.107:0/1811069524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 msgr2=0x561537c7c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.473+0000 7fedfd689740 1 --2- 172.21.15.107:0/1811069524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537c7c250 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fedec0ee8d0 tx=0x7fedec045500 comp rx=0 tx=0).stop 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.473+0000 7fedfd689740 1 -- 172.21.15.107:0/1811069524 shutdown_connections 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.473+0000 7fedfd689740 1 --2- 172.21.15.107:0/1811069524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537c7c250 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.473+0000 7fedfd689740 1 --2- 172.21.15.107:0/1811069524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c693e0 0x561537c7bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.473+0000 7fedfd689740 1 --2- 172.21.15.107:0/1811069524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.473+0000 7fedfd689740 1 -- 172.21.15.107:0/1811069524 >> 172.21.15.107:0/1811069524 conn(0x561537c65050 msgr2=0x561537c639a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.473+0000 7fedfd689740 1 -- 172.21.15.107:0/1811069524 shutdown_connections 2024-09-19T04:25:25.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.474+0000 7fedfd689740 1 -- 172.21.15.107:0/1811069524 wait complete. 2024-09-19T04:25:25.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.474+0000 7fedfd689740 1 -- start start 2024-09-19T04:25:25.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.474+0000 7fedfd689740 1 Processor -- start 2024-09-19T04:25:25.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedfd689740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c7a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedfd689740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c693e0 0x561537c7a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedfd689740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537ced8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedfd689740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41b0ea0 con 0x561537c693e0 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedfd689740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41b0770 con 0x561537c68ac0 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedfd689740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fedf41ad9b0 con 0x561537c73ca0 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedfd689740 10 client.0 _ll_register_callbacks cb 0x7fede401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedf8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537ced8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedf8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537ced8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:40762/0 (socket says 172.21.15.107:40762) 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedf8c00640 1 -- 172.21.15.107:0/2441427379 learned_addr learned my addr 172.21.15.107:0/2441427379 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedf3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c7a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedf3400640 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c693e0 0x561537c7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedf8c00640 1 -- 172.21.15.107:0/2441427379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 msgr2=0x561537c7a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedf8c00640 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c7a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedf8c00640 1 -- 172.21.15.107:0/2441427379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c693e0 msgr2=0x561537c7a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedf8c00640 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c693e0 0x561537c7a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.475+0000 7fedf8c00640 1 -- 172.21.15.107:0/2441427379 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561537cede00 con 0x561537c73ca0 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7fedf8c00640 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537ced8c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fedec0bd3f0 tx=0x7fedec045470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1428ed02c557152 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7feddd600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7feddd600640 1 -- 172.21.15.107:0/2441427379 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec0c60d0 con 0x561537c73ca0 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7fedfd689740 1 -- 172.21.15.107:0/2441427379 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedec036030 con 0x561537c73ca0 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7feddd600640 1 -- 172.21.15.107:0/2441427379 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fedec0bd6c0 con 0x561537c73ca0 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7feddd600640 1 -- 172.21.15.107:0/2441427379 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fedec0a0020 con 0x561537c73ca0 2024-09-19T04:25:25.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7fedfd689740 10 client.26149 fetch_fsmap learned FSMap version 35 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7fedfd689740 1 -- 172.21.15.107:0/2441427379 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561537cef0a0 con 0x561537c73ca0 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7feddd600640 1 -- 172.21.15.107:0/2441427379 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedec036030 con 0x561537c73ca0 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7feddd600640 1 -- 172.21.15.107:0/2441427379 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fedec04d070 con 0x561537c73ca0 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7fedfd689740 10 client.26149 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7fedfd689740 10 client.26149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7fedfd689740 20 client.26149 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7fedfd689740 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561537cef760 0x561537cf1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7fedfd689740 4 client.26149 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.476+0000 7fedfd689740 1 -- 172.21.15.107:0/2441427379 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561537bd6d10 con 0x561537cef760 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.477+0000 7fedf3e00640 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561537cef760 0x561537cf1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.477+0000 7fedf3e00640 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561537cef760 0x561537cf1b50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe1c495dedf5d8e server_cookie=7a15e3bdfddfd465 in_seq=0 out_seq=0 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.477+0000 7feddd600640 10 client.26149 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:25.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7feddd600640 1 -- 172.21.15.107:0/2441427379 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561537bd6d10 con 0x561537cef760 2024-09-19T04:25:25.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7feddd600640 10 client.26149 handle_command_reply: tid=0 2024-09-19T04:25:25.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7fedfd689740 1 client.26149 shutdown 2024-09-19T04:25:25.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7fedfd689740 1 -- 172.21.15.107:0/2441427379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 msgr2=0x561537ced8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7fedfd689740 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537ced8c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fedec0bd3f0 tx=0x7fedec045470 comp rx=0 tx=0).stop 2024-09-19T04:25:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7fedfd689740 1 -- 172.21.15.107:0/2441427379 shutdown_connections 2024-09-19T04:25:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7fedfd689740 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561537cef760 0x561537cf1b50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7fedfd689740 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561537c73ca0 0x561537ced8c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7fedfd689740 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561537c693e0 0x561537c7a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7feddd600640 0 client.26149 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7fedfd689740 1 --2- 172.21.15.107:0/2441427379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561537c68ac0 0x561537c7a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.478+0000 7fedfd689740 1 -- 172.21.15.107:0/2441427379 >> 172.21.15.107:0/2441427379 conn(0x561537c65050 msgr2=0x561537c641c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfd689740 1 -- 172.21.15.107:0/2441427379 shutdown_connections 2024-09-19T04:25:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfd689740 1 -- 172.21.15.107:0/2441427379 wait complete. 2024-09-19T04:25:25.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfd689740 20 client.26149 trim_cache size 0 max 16384 2024-09-19T04:25:25.485 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:25:25.485 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:25:25.485 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "c34abd9f-6032-4938-a54f-16595eab6679", 2024-09-19T04:25:25.485 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:25:25.485 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:25:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfac00640 1 -- 172.21.15.107:0/85234911 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fedd80a3430 msgr2=0x7fedd80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfac00640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fedd80a3430 0x7fedd80a58f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x561537c663d0 tx=0x7fede0013040 comp rx=0 tx=0).stop 2024-09-19T04:25:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfac00640 1 -- 172.21.15.107:0/85234911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf410e4e0 msgr2=0x7fedf419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfac00640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf410e4e0 0x7fedf419ef90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fedf4102b10 tx=0x7fedec030ed0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfac00640 1 -- 172.21.15.107:0/85234911 shutdown_connections 2024-09-19T04:25:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfac00640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fedd80a3430 0x7fedd80a58f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfac00640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf410e4e0 0x7fedf419ef90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfac00640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedf41012e0 0x7fedf419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfac00640 1 --2- 172.21.15.107:0/85234911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf4100930 0x7fedf419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:25.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.479+0000 7fedfac00640 1 -- 172.21.15.107:0/85234911 >> 172.21.15.107:0/85234911 conn(0x7fedf40fc7a0 msgr2=0x7fedf40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:25.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.480+0000 7fedfac00640 1 -- 172.21.15.107:0/85234911 shutdown_connections 2024-09-19T04:25:25.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:25.480+0000 7fedfac00640 1 -- 172.21.15.107:0/85234911 wait complete. 2024-09-19T04:25:25.667 INFO:tasks.workunit.client.0.smithi107.stdout: 1 191688 3.33 MB/sec execute 103 sec latency 98.674 ms 2024-09-19T04:25:26.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:25 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:25:26.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:25 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:25 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:25:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:25 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:25:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:25 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:25 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:25:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:25 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:25 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:25:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:25 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:25 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:25 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:25:26.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:25 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:25 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:25:26.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:25 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:25:26.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:25 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:25 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:25:26.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:25 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:25 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:25:26.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:25 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:25 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.389 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:25:26.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:25 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:25:26.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:25 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:25 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:25:26.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:25 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:25:26.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:25 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:25 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:25:26.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:25 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:25 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:25:26.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:25 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:25 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:26.667 INFO:tasks.workunit.client.0.smithi107.stdout: 1 191688 3.29 MB/sec execute 104 sec latency 1098.715 ms 2024-09-19T04:25:26.759 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:27.182 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:26 smithi107 ceph-mon[30659]: pgmap v156: 97 pgs: 97 active+clean; 125 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T04:25:27.182 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:26 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:25:27.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:26 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:25:27.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:26 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:26 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:26 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:25:27.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:26 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:26 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:25:27.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:26 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:26 smithi145 ceph-mon[35096]: pgmap v156: 97 pgs: 97 active+clean; 125 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T04:25:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:26 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:25:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:26 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:25:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:26 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:26 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:26 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:25:27.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:26 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:26 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:25:27.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:26 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.292+0000 7f59acc00640 1 -- 172.21.15.107:0/3183588536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59a8106f10 msgr2=0x7f59a810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:27.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.292+0000 7f59acc00640 1 --2- 172.21.15.107:0/3183588536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59a8106f10 0x7f59a810ea90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009ca0 tx=0x7f59a002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.292+0000 7f59acc00640 1 -- 172.21.15.107:0/3183588536 shutdown_connections 2024-09-19T04:25:27.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.292+0000 7f59acc00640 1 --2- 172.21.15.107:0/3183588536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59a8106f10 0x7f59a810ea90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.292+0000 7f59acc00640 1 --2- 172.21.15.107:0/3183588536 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59a8102790 0x7f59a81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.292+0000 7f59acc00640 1 --2- 172.21.15.107:0/3183588536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a8101de0 0x7f59a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.292+0000 7f59acc00640 1 -- 172.21.15.107:0/3183588536 >> 172.21.15.107:0/3183588536 conn(0x7f59a80fd8c0 msgr2=0x7f59a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.292+0000 7f59acc00640 1 -- 172.21.15.107:0/3183588536 shutdown_connections 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.292+0000 7f59acc00640 1 -- 172.21.15.107:0/3183588536 wait complete. 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.293+0000 7f59acc00640 1 Processor -- start 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.293+0000 7f59acc00640 1 -- start start 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59acc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a8101de0 0x7f59a819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59acc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59a8102790 0x7f59a819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59acc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59a8106f10 0x7f59a819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59acc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59a8111f70 con 0x7f59a8106f10 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59acc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f59a8111df0 con 0x7f59a8101de0 2024-09-19T04:25:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59acc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81120f0 con 0x7f59a8102790 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59a8106f10 0x7f59a819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a5e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a8101de0 0x7f59a819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a5e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a8101de0 0x7f59a819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35678/0 (socket says 172.21.15.107:35678) 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59a8106f10 0x7f59a819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57092/0 (socket says 172.21.15.107:57092) 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a5e00640 1 -- 172.21.15.107:0/2510484823 learned_addr learned my addr 172.21.15.107:0/2510484823 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a5400640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59a8102790 0x7f59a819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a5e00640 1 -- 172.21.15.107:0/2510484823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59a8102790 msgr2=0x7f59a819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a5e00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59a8102790 0x7f59a819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a5e00640 1 -- 172.21.15.107:0/2510484823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59a8106f10 msgr2=0x7f59a819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a5e00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59a8106f10 0x7f59a819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a5e00640 1 -- 172.21.15.107:0/2510484823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a819f600 con 0x7f59a8101de0 2024-09-19T04:25:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.294+0000 7f59a6800640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59a8106f10 0x7f59a819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.295+0000 7f59a5e00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a8101de0 0x7f59a819a650 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f599800d950 tx=0x7f599800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.295+0000 7f599ea00640 1 -- 172.21.15.107:0/2510484823 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998017070 con 0x7f59a8101de0 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.295+0000 7f59acc00640 1 -- 172.21.15.107:0/2510484823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a819f8f0 con 0x7f59a8101de0 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.295+0000 7f599ea00640 1 -- 172.21.15.107:0/2510484823 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5998009c30 con 0x7f59a8101de0 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.295+0000 7f599ea00640 1 -- 172.21.15.107:0/2510484823 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599800f6b0 con 0x7f59a8101de0 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.295+0000 7f59acc00640 1 -- 172.21.15.107:0/2510484823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a81a71d0 con 0x7f59a8101de0 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.298+0000 7f59af7f7740 1 Processor -- start 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.298+0000 7f59af7f7740 1 -- start start 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.298+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.298+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 0x5606f72c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72c6d70 0x5606f72c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59af7f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5606f72223e0 con 0x5606f72bc4b0 2024-09-19T04:25:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59af7f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5606f71c9d80 con 0x5606f72bbb90 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59af7f7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5606f721e3a0 con 0x5606f72c6d70 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f599ea00640 1 -- 172.21.15.107:0/2510484823 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f59980109d0 con 0x7f59a8101de0 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59a6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59a6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72bbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35694/0 (socket says 172.21.15.107:35694) 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59a6800640 1 -- 172.21.15.107:0/3069520185 learned_addr learned my addr 172.21.15.107:0/3069520185 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59a5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 0x5606f72c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59a5400640 1 --2- 172.21.15.107:0/3069520185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72c6d70 0x5606f72c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59a6800640 1 -- 172.21.15.107:0/3069520185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72c6d70 msgr2=0x5606f72c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59a6800640 1 --2- 172.21.15.107:0/3069520185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72c6d70 0x5606f72c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59a6800640 1 -- 172.21.15.107:0/3069520185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 msgr2=0x5606f72c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59a6800640 1 --2- 172.21.15.107:0/3069520185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 0x5606f72c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.299+0000 7f59a6800640 1 -- 172.21.15.107:0/3069520185 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f71e8ae0 con 0x5606f72bbb90 2024-09-19T04:25:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.300+0000 7f59a6800640 1 --2- 172.21.15.107:0/3069520185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72bbf70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009ca0 tx=0x7f59a00098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99123eea157a5632 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.300+0000 7f5993000640 1 -- 172.21.15.107:0/3069520185 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0003bf0 con 0x5606f72bbb90 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.300+0000 7f5993000640 1 -- 172.21.15.107:0/3069520185 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a0036040 con 0x5606f72bbb90 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.300+0000 7f5993000640 1 -- 172.21.15.107:0/3069520185 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a00047b0 con 0x5606f72bbb90 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.300+0000 7f599ea00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f598c0a33b0 0x7f598c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f599ea00640 1 -- 172.21.15.107:0/2510484823 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f59980cf890 con 0x7f59a8101de0 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59af7f7740 1 -- 172.21.15.107:0/3069520185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 msgr2=0x5606f72bbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59af7f7740 1 --2- 172.21.15.107:0/3069520185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72bbf70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009ca0 tx=0x7f59a00098b0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59a5e00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f598c0a33b0 0x7f598c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59af7f7740 1 -- 172.21.15.107:0/3069520185 shutdown_connections 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59af7f7740 1 --2- 172.21.15.107:0/3069520185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72c6d70 0x5606f72c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59af7f7740 1 --2- 172.21.15.107:0/3069520185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 0x5606f72c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59af7f7740 1 --2- 172.21.15.107:0/3069520185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72bbf70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59af7f7740 1 -- 172.21.15.107:0/3069520185 >> 172.21.15.107:0/3069520185 conn(0x5606f72b8120 msgr2=0x5606f72b8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59af7f7740 1 -- 172.21.15.107:0/3069520185 shutdown_connections 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59af7f7740 1 -- 172.21.15.107:0/3069520185 wait complete. 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59a5e00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f598c0a33b0 0x7f598c0a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59980cd0e0 tx=0x7f59980d5550 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.301+0000 7f59af7f7740 1 -- start start 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.302+0000 7f59af7f7740 1 Processor -- start 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.302+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.302+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 0x5606f72c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.302+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72c6d70 0x5606f72c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.302+0000 7f59af7f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5606f72c9860 con 0x5606f72bc4b0 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.302+0000 7f59af7f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5606f71dd0f0 con 0x5606f72bbb90 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.302+0000 7f59af7f7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5606f72c99e0 con 0x5606f72c6d70 2024-09-19T04:25:27.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.302+0000 7f59af7f7740 10 client.0 _ll_register_callbacks cb 0x5606f71ce9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.302+0000 7f59a5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 0x5606f72c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f59a6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.302+0000 7f59a5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 0x5606f72c4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57114/0 (socket says 172.21.15.107:57114) 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f59a5400640 1 -- 172.21.15.107:0/700595980 learned_addr learned my addr 172.21.15.107:0/700595980 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f59a6800640 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72c6d70 0x5606f72c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f59a6800640 1 -- 172.21.15.107:0/700595980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72c6d70 msgr2=0x5606f72c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f59a6800640 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72c6d70 0x5606f72c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f59a6800640 1 -- 172.21.15.107:0/700595980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 msgr2=0x5606f72c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f59a6800640 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 0x5606f72c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f59a6800640 1 -- 172.21.15.107:0/700595980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f6f933c0 con 0x5606f72bbb90 2024-09-19T04:25:27.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f59a6800640 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72c3c30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009bf0 tx=0x7f59a0004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50cf62b6fb778763 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f5987400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f5987400640 1 -- 172.21.15.107:0/700595980 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0058060 con 0x5606f72bbb90 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f5987400640 1 -- 172.21.15.107:0/700595980 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f59a0003c70 con 0x5606f72bbb90 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.303+0000 7f59af7f7740 1 -- 172.21.15.107:0/700595980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a0036040 con 0x5606f72bbb90 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.304+0000 7f5987400640 1 -- 172.21.15.107:0/700595980 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f59a0031450 con 0x5606f72bbb90 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.304+0000 7f59af7f7740 10 client.26201 fetch_fsmap learned FSMap version 36 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.304+0000 7f59af7f7740 1 -- 172.21.15.107:0/700595980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5606f72c9860 con 0x5606f72bbb90 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.304+0000 7f5987400640 1 -- 172.21.15.107:0/700595980 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a0036040 con 0x5606f72bbb90 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.304+0000 7f5987400640 1 -- 172.21.15.107:0/700595980 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) ==== 8382+0+0 (secure 0 0 0) 0x7f59a004d030 con 0x5606f72bbb90 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.304+0000 7f59af7f7740 10 client.26201 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.304+0000 7f59af7f7740 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.304+0000 7f59af7f7740 20 client.26201 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.305+0000 7f59af7f7740 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5606f72d6700 0x5606f72d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.305+0000 7f59af7f7740 4 client.26201 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:27.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.305+0000 7f59af7f7740 1 -- 172.21.15.107:0/700595980 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606f722f950 con 0x5606f72d6700 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.305+0000 7f59a5400640 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5606f72d6700 0x5606f72d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.305+0000 7f59a5400640 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5606f72d6700 0x5606f72d8af0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6457ae57dfc631e server_cookie=8c6e7f212300fe21 in_seq=0 out_seq=0 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.305+0000 7f5987400640 10 client.26201 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.306+0000 7f5987400640 1 -- 172.21.15.107:0/700595980 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606f722f950 con 0x5606f72d6700 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.306+0000 7f5987400640 10 client.26201 handle_command_reply: tid=0 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.306+0000 7f59af7f7740 1 client.26201 shutdown 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.306+0000 7f59af7f7740 1 -- 172.21.15.107:0/700595980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 msgr2=0x5606f72c3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.306+0000 7f59af7f7740 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72c3c30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009bf0 tx=0x7f59a0004390 comp rx=0 tx=0).stop 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.306+0000 7f59af7f7740 1 -- 172.21.15.107:0/700595980 shutdown_connections 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.306+0000 7f59af7f7740 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5606f72d6700 0x5606f72d8af0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.307+0000 7f59af7f7740 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72c6d70 0x5606f72c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.307+0000 7f59af7f7740 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72bc4b0 0x5606f72c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.307+0000 7f59af7f7740 1 --2- 172.21.15.107:0/700595980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72c3c30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.307+0000 7f59af7f7740 1 -- 172.21.15.107:0/700595980 >> 172.21.15.107:0/700595980 conn(0x5606f72b8120 msgr2=0x5606f72bde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.307+0000 7f5987400640 0 client.26201 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.307+0000 7f59af7f7740 1 -- 172.21.15.107:0/700595980 shutdown_connections 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.307+0000 7f59af7f7740 1 -- 172.21.15.107:0/700595980 wait complete. 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.307+0000 7f59af7f7740 20 client.26201 trim_cache size 0 max 16384 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.324+0000 7f59af7f7740 1 Processor -- start 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59af7f7740 1 -- start start 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72bbb90 0x5606f72c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bc4b0 0x5606f72de3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 0x5606f72de8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59af7f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81af8d0 con 0x5606f72c6d70 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59af7f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81b1500 con 0x5606f72bc4b0 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59af7f7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81affa0 con 0x5606f72bbb90 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59a6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 0x5606f72de8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59a6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 0x5606f72de8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57128/0 (socket says 172.21.15.107:57128) 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59a6800640 1 -- 172.21.15.107:0/2949731721 learned_addr learned my addr 172.21.15.107:0/2949731721 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59a5400640 1 --2- 172.21.15.107:0/2949731721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bc4b0 0x5606f72de3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59a6800640 1 --2- 172.21.15.107:0/2949731721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72bbb90 0x5606f72c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59a6800640 1 -- 172.21.15.107:0/2949731721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72bbb90 msgr2=0x5606f72c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59a6800640 1 --2- 172.21.15.107:0/2949731721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72bbb90 0x5606f72c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59a6800640 1 -- 172.21.15.107:0/2949731721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bc4b0 msgr2=0x5606f72de3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.325+0000 7f59a6800640 1 --2- 172.21.15.107:0/2949731721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bc4b0 0x5606f72de3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.326+0000 7f59a6800640 1 -- 172.21.15.107:0/2949731721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f72d90d0 con 0x5606f72c6d70 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.326+0000 7f59a6800640 1 --2- 172.21.15.107:0/2949731721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 0x5606f72de8e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009ca0 tx=0x7f59a0033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f7efda6f687bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.326+0000 7f5992600640 1 -- 172.21.15.107:0/2949731721 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0058070 con 0x5606f72c6d70 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.326+0000 7f5992600640 1 -- 172.21.15.107:0/2949731721 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a0036070 con 0x5606f72c6d70 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.326+0000 7f5992600640 1 -- 172.21.15.107:0/2949731721 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0003dd0 con 0x5606f72c6d70 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.326+0000 7f59af7f7740 1 -- 172.21.15.107:0/2949731721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 msgr2=0x5606f72de8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.327+0000 7f59af7f7740 1 --2- 172.21.15.107:0/2949731721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 0x5606f72de8e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009ca0 tx=0x7f59a0033040 comp rx=0 tx=0).stop 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.327+0000 7f59af7f7740 1 -- 172.21.15.107:0/2949731721 shutdown_connections 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.327+0000 7f59af7f7740 1 --2- 172.21.15.107:0/2949731721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 0x5606f72de8e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.327+0000 7f59af7f7740 1 --2- 172.21.15.107:0/2949731721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bc4b0 0x5606f72de3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.327+0000 7f59af7f7740 1 --2- 172.21.15.107:0/2949731721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72bbb90 0x5606f72c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.327+0000 7f59af7f7740 1 -- 172.21.15.107:0/2949731721 >> 172.21.15.107:0/2949731721 conn(0x5606f72b8120 msgr2=0x5606f72c2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:27.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.327+0000 7f59af7f7740 1 -- 172.21.15.107:0/2949731721 shutdown_connections 2024-09-19T04:25:27.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.327+0000 7f59af7f7740 1 -- 172.21.15.107:0/2949731721 wait complete. 2024-09-19T04:25:27.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.327+0000 7f59af7f7740 1 -- start start 2024-09-19T04:25:27.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.327+0000 7f59af7f7740 1 Processor -- start 2024-09-19T04:25:27.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72bc4b0 0x5606f72cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59af7f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 0x5606f72c0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59af7f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81b0da0 con 0x5606f72c6d70 2024-09-19T04:25:27.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59af7f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81b0670 con 0x5606f72bbb90 2024-09-19T04:25:27.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59af7f7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81ad8b0 con 0x5606f72bc4b0 2024-09-19T04:25:27.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59af7f7740 10 client.0 _ll_register_callbacks cb 0x7f59a0009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:27.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59a6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59a6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 0x5606f72c0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59a6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:35732/0 (socket says 172.21.15.107:35732) 2024-09-19T04:25:27.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59a6800640 1 -- 172.21.15.107:0/4035254149 learned_addr learned my addr 172.21.15.107:0/4035254149 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:27.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59a5400640 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72bc4b0 0x5606f72cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59a6800640 1 -- 172.21.15.107:0/4035254149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72bc4b0 msgr2=0x5606f72cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:27.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59a6800640 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72bc4b0 0x5606f72cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59a6800640 1 -- 172.21.15.107:0/4035254149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 msgr2=0x5606f72c0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59a6800640 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 0x5606f72c0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.328+0000 7f59a6800640 1 -- 172.21.15.107:0/4035254149 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f72c0840 con 0x5606f72bbb90 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.329+0000 7f59a6800640 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72cec00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f59a0030040 tx=0x7f59a002b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8702393529ecbcc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.329+0000 7f5987400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.329+0000 7f5987400640 1 -- 172.21.15.107:0/4035254149 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0040060 con 0x5606f72bbb90 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.329+0000 7f59af7f7740 1 -- 172.21.15.107:0/4035254149 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a0036070 con 0x5606f72bbb90 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.329+0000 7f5987400640 1 -- 172.21.15.107:0/4035254149 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f59a002f680 con 0x5606f72bbb90 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.329+0000 7f5987400640 1 -- 172.21.15.107:0/4035254149 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f59a0058070 con 0x5606f72bbb90 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.329+0000 7f59af7f7740 10 client.26207 fetch_fsmap learned FSMap version 36 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.329+0000 7f59af7f7740 1 -- 172.21.15.107:0/4035254149 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5606f7340be0 con 0x5606f72bbb90 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.329+0000 7f5987400640 1 -- 172.21.15.107:0/4035254149 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a0036070 con 0x5606f72bbb90 2024-09-19T04:25:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.330+0000 7f5987400640 1 -- 172.21.15.107:0/4035254149 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) ==== 8382+0+0 (secure 0 0 0) 0x7f59a0065070 con 0x5606f72bbb90 2024-09-19T04:25:27.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.330+0000 7f59af7f7740 10 client.26207 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:25:27.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.330+0000 7f59af7f7740 10 client.26207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:27.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.330+0000 7f59af7f7740 20 client.26207 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:27.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.330+0000 7f59af7f7740 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5606f7341170 0x5606f7343590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:27.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.330+0000 7f59af7f7740 4 client.26207 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:25:27.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.330+0000 7f59af7f7740 1 -- 172.21.15.107:0/4035254149 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606f7229e60 con 0x5606f7341170 2024-09-19T04:25:27.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.330+0000 7f59a5400640 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5606f7341170 0x5606f7343590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:27.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f59a5400640 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5606f7341170 0x5606f7343590 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0fa3a8bb46946e server_cookie=844a2ee4b372ceca in_seq=0 out_seq=0 2024-09-19T04:25:27.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f5987400640 10 client.26207 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:27.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f5987400640 1 -- 172.21.15.107:0/4035254149 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5606f7229e60 con 0x5606f7341170 2024-09-19T04:25:27.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f5987400640 10 client.26207 handle_command_reply: tid=0 2024-09-19T04:25:27.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f59af7f7740 1 client.26207 shutdown 2024-09-19T04:25:27.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f59af7f7740 1 -- 172.21.15.107:0/4035254149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 msgr2=0x5606f72cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:27.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f59af7f7740 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72cec00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f59a0030040 tx=0x7f59a002b040 comp rx=0 tx=0).stop 2024-09-19T04:25:27.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f59af7f7740 1 -- 172.21.15.107:0/4035254149 shutdown_connections 2024-09-19T04:25:27.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f59af7f7740 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5606f7341170 0x5606f7343590 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f59af7f7740 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606f72c6d70 0x5606f72c0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f59af7f7740 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606f72bc4b0 0x5606f72cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f59af7f7740 1 --2- 172.21.15.107:0/4035254149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606f72bbb90 0x5606f72cec00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f5987400640 0 client.26207 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:27.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.331+0000 7f59af7f7740 1 -- 172.21.15.107:0/4035254149 >> 172.21.15.107:0/4035254149 conn(0x5606f72b8120 msgr2=0x5606f72b69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:27.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.332+0000 7f59af7f7740 1 -- 172.21.15.107:0/4035254149 shutdown_connections 2024-09-19T04:25:27.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.332+0000 7f59af7f7740 1 -- 172.21.15.107:0/4035254149 wait complete. 2024-09-19T04:25:27.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.332+0000 7f59af7f7740 20 client.26207 trim_cache size 0 max 16384 2024-09-19T04:25:27.353 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:25:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:25:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:25:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "c34abd9f-6032-4938-a54f-16595eab6679": { 2024-09-19T04:25:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:25:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "c34abd9f-6032-4938-a54f-16595eab6679", 2024-09-19T04:25:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:25:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:25:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:25:27.354 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 -- 172.21.15.107:0/2510484823 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f598c0a33b0 msgr2=0x7f598c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f598c0a33b0 0x7f598c0a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59980cd0e0 tx=0x7f59980d5550 comp rx=0 tx=0).stop 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 -- 172.21.15.107:0/2510484823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a8101de0 msgr2=0x7f59a819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a8101de0 0x7f59a819a650 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f599800d950 tx=0x7f599800de10 comp rx=0 tx=0).stop 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 -- 172.21.15.107:0/2510484823 shutdown_connections 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f598c0a33b0 0x7f598c0a5870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59a8106f10 0x7f59a819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59a8102790 0x7f59a819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 --2- 172.21.15.107:0/2510484823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a8101de0 0x7f59a819a650 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 -- 172.21.15.107:0/2510484823 >> 172.21.15.107:0/2510484823 conn(0x7f59a80fd8c0 msgr2=0x7f59a80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 -- 172.21.15.107:0/2510484823 shutdown_connections 2024-09-19T04:25:27.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:27.333+0000 7f59acc00640 1 -- 172.21.15.107:0/2510484823 wait complete. 2024-09-19T04:25:27.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:26 smithi115 ceph-mon[35477]: pgmap v156: 97 pgs: 97 active+clean; 125 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-19T04:25:27.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:26 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:25:27.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:26 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:25:27.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:26 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:26 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:26 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:25:27.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:26 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:26 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:25:27.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:26 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:25:27.668 INFO:tasks.workunit.client.0.smithi107.stdout: 1 191707 3.26 MB/sec execute 105 sec latency 1401.919 ms 2024-09-19T04:25:28.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:27 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:28.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:27 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:28.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:27 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:28.668 INFO:tasks.workunit.client.0.smithi107.stdout: 1 191707 3.23 MB/sec execute 106 sec latency 1681.000 ms 2024-09-19T04:25:28.811 INFO:tasks.cephfs.filesystem:scrub status for tag:c34abd9f-6032-4938-a54f-16595eab6679 - {'path': '/', 'tag': 'c34abd9f-6032-4938-a54f-16595eab6679', 'options': 'recursive,force'} 2024-09-19T04:25:29.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:28 smithi107 ceph-mon[30659]: pgmap v157: 97 pgs: 97 active+clean; 125 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 578 KiB/s wr, 69 op/s 2024-09-19T04:25:29.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:28 smithi115 ceph-mon[35477]: pgmap v157: 97 pgs: 97 active+clean; 125 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 578 KiB/s wr, 69 op/s 2024-09-19T04:25:29.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:28 smithi145 ceph-mon[35096]: pgmap v157: 97 pgs: 97 active+clean; 125 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 578 KiB/s wr, 69 op/s 2024-09-19T04:25:29.668 INFO:tasks.workunit.client.0.smithi107.stdout: 1 192213 3.21 MB/sec execute 107 sec latency 2066.416 ms 2024-09-19T04:25:30.668 INFO:tasks.workunit.client.0.smithi107.stdout: 1 192213 3.18 MB/sec execute 108 sec latency 1055.549 ms 2024-09-19T04:25:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:30 smithi107 ceph-mon[30659]: pgmap v158: 97 pgs: 97 active+clean; 125 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 578 KiB/s wr, 69 op/s 2024-09-19T04:25:31.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:30 smithi115 ceph-mon[35477]: pgmap v158: 97 pgs: 97 active+clean; 125 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 578 KiB/s wr, 69 op/s 2024-09-19T04:25:31.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:30 smithi145 ceph-mon[35096]: pgmap v158: 97 pgs: 97 active+clean; 125 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 578 KiB/s wr, 69 op/s 2024-09-19T04:25:31.668 INFO:tasks.workunit.client.0.smithi107.stdout: 1 192213 3.16 MB/sec execute 109 sec latency 2055.700 ms 2024-09-19T04:25:32.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:31 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:25:32.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:31 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:25:32.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:31 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:25:32.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:31 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:25:32.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:31 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:25:32.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:31 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:25:32.668 INFO:tasks.workunit.client.0.smithi107.stdout: 1 192337 3.13 MB/sec execute 110 sec latency 2114.330 ms 2024-09-19T04:25:33.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:32 smithi115 ceph-mon[35477]: pgmap v159: 97 pgs: 97 active+clean; 127 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-19T04:25:33.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:32 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:25:33.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:32 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:25:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:32 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:25:33.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:32 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:33.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:32 smithi107 ceph-mon[30659]: pgmap v159: 97 pgs: 97 active+clean; 127 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-19T04:25:33.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:32 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:25:33.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:32 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:25:33.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:32 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:25:33.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:32 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:33.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:32 smithi145 ceph-mon[35096]: pgmap v159: 97 pgs: 97 active+clean; 127 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-19T04:25:33.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:32 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:25:33.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:32 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:25:33.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:32 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:25:33.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:32 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:25:33.668 INFO:tasks.workunit.client.0.smithi107.stdout: 1 192337 3.10 MB/sec execute 111 sec latency 1836.082 ms 2024-09-19T04:25:34.668 INFO:tasks.workunit.client.0.smithi107.stdout: 1 193120 3.12 MB/sec execute 112 sec latency 2380.865 ms 2024-09-19T04:25:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:34 smithi115 ceph-mon[35477]: pgmap v160: 97 pgs: 97 active+clean; 127 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 738 KiB/s wr, 69 op/s 2024-09-19T04:25:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:34 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:25:35.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:34 smithi145 ceph-mon[35096]: pgmap v160: 97 pgs: 97 active+clean; 127 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 738 KiB/s wr, 69 op/s 2024-09-19T04:25:35.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:34 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:25:35.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:34 smithi107 ceph-mon[30659]: pgmap v160: 97 pgs: 97 active+clean; 127 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 738 KiB/s wr, 69 op/s 2024-09-19T04:25:35.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:34 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:25:35.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 193239 3.09 MB/sec execute 113 sec latency 917.031 ms 2024-09-19T04:25:36.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:35 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:25:36.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:35 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:25:36.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:35 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:25:36.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:35 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:25:36.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:35 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:25:36.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:35 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:25:36.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:35 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:25:36.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:35 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:25:36.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:35 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:25:36.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 193239 3.06 MB/sec execute 114 sec latency 1917.168 ms 2024-09-19T04:25:37.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:36 smithi115 ceph-mon[35477]: pgmap v161: 97 pgs: 97 active+clean; 127 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 982 KiB/s wr, 105 op/s 2024-09-19T04:25:37.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:36 smithi145 ceph-mon[35096]: pgmap v161: 97 pgs: 97 active+clean; 127 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 982 KiB/s wr, 105 op/s 2024-09-19T04:25:37.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:36 smithi107 ceph-mon[30659]: pgmap v161: 97 pgs: 97 active+clean; 127 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 982 KiB/s wr, 105 op/s 2024-09-19T04:25:37.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 193239 3.04 MB/sec execute 115 sec latency 2917.296 ms 2024-09-19T04:25:38.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 193239 3.01 MB/sec execute 116 sec latency 3917.369 ms 2024-09-19T04:25:39.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:38 smithi115 ceph-mon[35477]: pgmap v162: 97 pgs: 97 active+clean; 127 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 815 KiB/s wr, 77 op/s 2024-09-19T04:25:39.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:38 smithi145 ceph-mon[35096]: pgmap v162: 97 pgs: 97 active+clean; 127 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 815 KiB/s wr, 77 op/s 2024-09-19T04:25:39.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:38 smithi107 ceph-mon[30659]: pgmap v162: 97 pgs: 97 active+clean; 127 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 815 KiB/s wr, 77 op/s 2024-09-19T04:25:39.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 194911 3.06 MB/sec execute 117 sec latency 4200.783 ms 2024-09-19T04:25:40.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 195819 3.08 MB/sec execute 118 sec latency 161.246 ms 2024-09-19T04:25:41.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:41 smithi115 ceph-mon[35477]: pgmap v163: 97 pgs: 97 active+clean; 127 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 815 KiB/s wr, 77 op/s 2024-09-19T04:25:41.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:41 smithi145 ceph-mon[35096]: pgmap v163: 97 pgs: 97 active+clean; 127 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 815 KiB/s wr, 77 op/s 2024-09-19T04:25:41.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:41 smithi107 ceph-mon[30659]: pgmap v163: 97 pgs: 97 active+clean; 127 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 815 KiB/s wr, 77 op/s 2024-09-19T04:25:41.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 195819 3.05 MB/sec execute 119 sec latency 1161.406 ms 2024-09-19T04:25:42.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 195819 3.03 MB/sec execute 120 sec latency 2161.559 ms 2024-09-19T04:25:43.279 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:43 smithi107 ceph-mon[30659]: pgmap v164: 97 pgs: 97 active+clean; 128 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-19T04:25:43.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:43 smithi115 ceph-mon[35477]: pgmap v164: 97 pgs: 97 active+clean; 128 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-19T04:25:43.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:43 smithi145 ceph-mon[35096]: pgmap v164: 97 pgs: 97 active+clean; 128 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-19T04:25:43.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 195819 3.00 MB/sec execute 121 sec latency 3161.629 ms 2024-09-19T04:25:44.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 196605 3.02 MB/sec execute 122 sec latency 3708.832 ms 2024-09-19T04:25:45.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:45 smithi115 ceph-mon[35477]: pgmap v165: 97 pgs: 97 active+clean; 128 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 651 KiB/s wr, 77 op/s 2024-09-19T04:25:45.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:45 smithi145 ceph-mon[35096]: pgmap v165: 97 pgs: 97 active+clean; 128 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 651 KiB/s wr, 77 op/s 2024-09-19T04:25:45.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:45 smithi107 ceph-mon[30659]: pgmap v165: 97 pgs: 97 active+clean; 128 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 651 KiB/s wr, 77 op/s 2024-09-19T04:25:45.669 INFO:tasks.workunit.client.0.smithi107.stdout: 1 196788 2.99 MB/sec execute 123 sec latency 870.649 ms 2024-09-19T04:25:46.670 INFO:tasks.workunit.client.0.smithi107.stdout: 1 196788 2.97 MB/sec execute 124 sec latency 1870.767 ms 2024-09-19T04:25:47.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:47 smithi145 ceph-mon[35096]: pgmap v166: 97 pgs: 97 active+clean; 129 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-19T04:25:47.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:47 smithi115 ceph-mon[35477]: pgmap v166: 97 pgs: 97 active+clean; 129 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-19T04:25:47.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:47 smithi107 ceph-mon[30659]: pgmap v166: 97 pgs: 97 active+clean; 129 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-19T04:25:47.670 INFO:tasks.workunit.client.0.smithi107.stdout: 1 196788 2.95 MB/sec execute 125 sec latency 2870.926 ms 2024-09-19T04:25:48.670 INFO:tasks.workunit.client.0.smithi107.stdout: 1 196788 2.92 MB/sec execute 126 sec latency 3870.970 ms 2024-09-19T04:25:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:49 smithi107 ceph-mon[30659]: pgmap v167: 97 pgs: 97 active+clean; 129 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 875 KiB/s wr, 81 op/s 2024-09-19T04:25:49.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:49 smithi115 ceph-mon[35477]: pgmap v167: 97 pgs: 97 active+clean; 129 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 875 KiB/s wr, 81 op/s 2024-09-19T04:25:49.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:49 smithi145 ceph-mon[35096]: pgmap v167: 97 pgs: 97 active+clean; 129 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 875 KiB/s wr, 81 op/s 2024-09-19T04:25:49.670 INFO:tasks.workunit.client.0.smithi107.stdout: 1 197861 2.94 MB/sec execute 127 sec latency 4417.906 ms 2024-09-19T04:25:50.670 INFO:tasks.workunit.client.0.smithi107.stdout: 1 199240 2.99 MB/sec execute 128 sec latency 13.500 ms 2024-09-19T04:25:51.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:51 smithi115 ceph-mon[35477]: pgmap v168: 97 pgs: 97 active+clean; 129 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 875 KiB/s wr, 81 op/s 2024-09-19T04:25:51.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:51 smithi145 ceph-mon[35096]: pgmap v168: 97 pgs: 97 active+clean; 129 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 875 KiB/s wr, 81 op/s 2024-09-19T04:25:51.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:51 smithi107 ceph-mon[30659]: pgmap v168: 97 pgs: 97 active+clean; 129 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 875 KiB/s wr, 81 op/s 2024-09-19T04:25:51.670 INFO:tasks.workunit.client.0.smithi107.stdout: 1 199429 2.97 MB/sec execute 129 sec latency 873.294 ms 2024-09-19T04:25:52.670 INFO:tasks.workunit.client.0.smithi107.stdout: 1 199429 2.94 MB/sec execute 130 sec latency 1873.412 ms 2024-09-19T04:25:53.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:53 smithi115 ceph-mon[35477]: pgmap v169: 97 pgs: 97 active+clean; 130 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-09-19T04:25:53.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:53 smithi145 ceph-mon[35096]: pgmap v169: 97 pgs: 97 active+clean; 130 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-09-19T04:25:53.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:53 smithi107 ceph-mon[30659]: pgmap v169: 97 pgs: 97 active+clean; 130 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-09-19T04:25:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: 1 199429 2.92 MB/sec execute 131 sec latency 2873.578 ms 2024-09-19T04:25:54.670 INFO:tasks.workunit.client.0.smithi107.stdout: 1 200108 2.93 MB/sec execute 132 sec latency 3421.973 ms 2024-09-19T04:25:55.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:55 smithi115 ceph-mon[35477]: pgmap v170: 97 pgs: 97 active+clean; 130 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 856 KiB/s wr, 78 op/s 2024-09-19T04:25:55.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:55 smithi145 ceph-mon[35096]: pgmap v170: 97 pgs: 97 active+clean; 130 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 856 KiB/s wr, 78 op/s 2024-09-19T04:25:55.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:55 smithi107 ceph-mon[30659]: pgmap v170: 97 pgs: 97 active+clean; 130 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 856 KiB/s wr, 78 op/s 2024-09-19T04:25:55.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 200331 2.91 MB/sec execute 133 sec latency 693.875 ms 2024-09-19T04:25:56.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 200331 2.89 MB/sec execute 134 sec latency 1693.978 ms 2024-09-19T04:25:57.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:57 smithi107 ceph-mon[30659]: pgmap v171: 97 pgs: 97 active+clean; 131 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1020 KiB/s wr, 111 op/s 2024-09-19T04:25:57.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:57 smithi115 ceph-mon[35477]: pgmap v171: 97 pgs: 97 active+clean; 131 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1020 KiB/s wr, 111 op/s 2024-09-19T04:25:57.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:57 smithi145 ceph-mon[35096]: pgmap v171: 97 pgs: 97 active+clean; 131 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1020 KiB/s wr, 111 op/s 2024-09-19T04:25:57.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 200331 2.87 MB/sec execute 135 sec latency 2694.099 ms 2024-09-19T04:25:58.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 200331 2.85 MB/sec execute 136 sec latency 3694.165 ms 2024-09-19T04:25:58.812 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:25:59.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:25:59 smithi107 ceph-mon[30659]: pgmap v172: 97 pgs: 97 active+clean; 131 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 552 KiB/s wr, 71 op/s 2024-09-19T04:25:59.265 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:25:59.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:25:59 smithi115 ceph-mon[35477]: pgmap v172: 97 pgs: 97 active+clean; 131 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 552 KiB/s wr, 71 op/s 2024-09-19T04:25:59.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:25:59 smithi145 ceph-mon[35096]: pgmap v172: 97 pgs: 97 active+clean; 131 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 552 KiB/s wr, 71 op/s 2024-09-19T04:25:59.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 201057 2.84 MB/sec execute 137 sec latency 3994.238 ms 2024-09-19T04:25:59.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.903+0000 7f09a3e00640 1 -- 172.21.15.107:0/1226969480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09a4108f60 msgr2=0x7f09a4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.903+0000 7f09a3e00640 1 --2- 172.21.15.107:0/1226969480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09a4108f60 0x7f09a4110ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f099c009ca0 tx=0x7f099c02a8b0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.904+0000 7f09a3e00640 1 -- 172.21.15.107:0/1226969480 shutdown_connections 2024-09-19T04:25:59.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.904+0000 7f09a3e00640 1 --2- 172.21.15.107:0/1226969480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09a4108f60 0x7f09a4110ae0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.904+0000 7f09a3e00640 1 --2- 172.21.15.107:0/1226969480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a41047e0 0x7f09a4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.904+0000 7f09a3e00640 1 --2- 172.21.15.107:0/1226969480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a4103e30 0x7f09a4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.904+0000 7f09a3e00640 1 -- 172.21.15.107:0/1226969480 >> 172.21.15.107:0/1226969480 conn(0x7f09a40ff910 msgr2=0x7f09a4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:59.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.904+0000 7f09a3e00640 1 -- 172.21.15.107:0/1226969480 shutdown_connections 2024-09-19T04:25:59.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.904+0000 7f09a3e00640 1 -- 172.21.15.107:0/1226969480 wait complete. 2024-09-19T04:25:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.905+0000 7f09a3e00640 1 Processor -- start 2024-09-19T04:25:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.905+0000 7f09a3e00640 1 -- start start 2024-09-19T04:25:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a4103e30 0x7f09a419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a41047e0 0x7f09a419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09a4108f60 0x7f09a419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4114010 con 0x7f09a4108f60 2024-09-19T04:25:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a3e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4113e90 con 0x7f09a41047e0 2024-09-19T04:25:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4114190 con 0x7f09a4103e30 2024-09-19T04:25:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a4103e30 0x7f09a419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a4103e30 0x7f09a419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42552/0 (socket says 172.21.15.107:42552) 2024-09-19T04:25:59.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a2a00640 1 -- 172.21.15.107:0/1509292645 learned_addr learned my addr 172.21.15.107:0/1509292645 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:59.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a2000640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a41047e0 0x7f09a419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a3400640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09a4108f60 0x7f09a419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a2a00640 1 -- 172.21.15.107:0/1509292645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a41047e0 msgr2=0x7f09a419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a2a00640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a41047e0 0x7f09a419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a2a00640 1 -- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09a4108f60 msgr2=0x7f09a419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a2a00640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09a4108f60 0x7f09a419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a2a00640 1 -- 172.21.15.107:0/1509292645 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a419f460 con 0x7f09a4103e30 2024-09-19T04:25:59.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a3400640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09a4108f60 0x7f09a419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:59.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.906+0000 7f09a2000640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a41047e0 0x7f09a419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:59.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.907+0000 7f09a2a00640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a4103e30 0x7f09a419a4b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f099400d950 tx=0x7f099400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:59.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.907+0000 7f09a8a00640 1 -- 172.21.15.107:0/1509292645 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994017030 con 0x7f09a4103e30 2024-09-19T04:25:59.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.907+0000 7f09a8a00640 1 -- 172.21.15.107:0/1509292645 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0994004020 con 0x7f09a4103e30 2024-09-19T04:25:59.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.907+0000 7f09a3e00640 1 -- 172.21.15.107:0/1509292645 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a419f750 con 0x7f09a4103e30 2024-09-19T04:25:59.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.907+0000 7f09a8a00640 1 -- 172.21.15.107:0/1509292645 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09940047a0 con 0x7f09a4103e30 2024-09-19T04:25:59.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.907+0000 7f09a3e00640 1 -- 172.21.15.107:0/1509292645 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a41a7030 con 0x7f09a4103e30 2024-09-19T04:25:59.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09aca68740 1 Processor -- start 2024-09-19T04:25:59.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09aca68740 1 -- start start 2024-09-19T04:25:59.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09aca68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442285e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09aca68740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56442285e590 0x564422871200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09aca68740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564422871740 0x564422873b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09aca68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5644227c43e0 con 0x56442285dc70 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09aca68740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56442276bd80 con 0x56442285e590 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09aca68740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5644227c03a0 con 0x564422871740 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09a2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564422871740 0x564422873b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09a2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564422871740 0x564422873b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42564/0 (socket says 172.21.15.107:42564) 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09a2000640 1 -- 172.21.15.107:0/386774379 learned_addr learned my addr 172.21.15.107:0/386774379 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.911+0000 7f09a3400640 1 --2- 172.21.15.107:0/386774379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442285e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.912+0000 7f09a8a00640 1 -- 172.21.15.107:0/1509292645 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f09940041d0 con 0x7f09a4103e30 2024-09-19T04:25:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.912+0000 7f09a2000640 1 -- 172.21.15.107:0/386774379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56442285e590 msgr2=0x564422871200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:25:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.912+0000 7f09a2000640 1 --2- 172.21.15.107:0/386774379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56442285e590 0x564422871200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.912+0000 7f09a2000640 1 -- 172.21.15.107:0/386774379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 msgr2=0x56442285e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.912+0000 7f09a2000640 1 --2- 172.21.15.107:0/386774379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442285e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.912+0000 7f09a2000640 1 -- 172.21.15.107:0/386774379 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442278aae0 con 0x564422871740 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.912+0000 7f09a3400640 1 --2- 172.21.15.107:0/386774379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442285e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.913+0000 7f09a2000640 1 --2- 172.21.15.107:0/386774379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564422871740 0x564422873b30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0990011470 tx=0x7f0990011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c050aa7dde92d688 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.913+0000 7f098fe00640 1 -- 172.21.15.107:0/386774379 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099001e070 con 0x564422871740 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.913+0000 7f098fe00640 1 -- 172.21.15.107:0/386774379 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09900035f0 con 0x564422871740 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.913+0000 7f098fe00640 1 -- 172.21.15.107:0/386774379 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099000f860 con 0x564422871740 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09aca68740 1 -- 172.21.15.107:0/386774379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564422871740 msgr2=0x564422873b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09aca68740 1 --2- 172.21.15.107:0/386774379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564422871740 0x564422873b30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0990011470 tx=0x7f0990011930 comp rx=0 tx=0).stop 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09aca68740 1 -- 172.21.15.107:0/386774379 shutdown_connections 2024-09-19T04:25:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09aca68740 1 --2- 172.21.15.107:0/386774379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564422871740 0x564422873b30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09aca68740 1 --2- 172.21.15.107:0/386774379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56442285e590 0x564422871200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09aca68740 1 --2- 172.21.15.107:0/386774379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442285e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09aca68740 1 -- 172.21.15.107:0/386774379 >> 172.21.15.107:0/386774379 conn(0x56442285a2e0 msgr2=0x56442285a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:59.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09a8a00640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09880a34b0 0x7f09880a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09aca68740 1 -- 172.21.15.107:0/386774379 shutdown_connections 2024-09-19T04:25:59.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09aca68740 1 -- 172.21.15.107:0/386774379 wait complete. 2024-09-19T04:25:59.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.914+0000 7f09a8a00640 1 -- 172.21.15.107:0/1509292645 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f09940cf6d0 con 0x7f09a4103e30 2024-09-19T04:25:59.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.915+0000 7f09a2000640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09880a34b0 0x7f09880a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.915+0000 7f09aca68740 1 -- start start 2024-09-19T04:25:59.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.915+0000 7f09aca68740 1 Processor -- start 2024-09-19T04:25:59.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.915+0000 7f09a2000640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09880a34b0 0x7f09880a5970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0990003f30 tx=0x7f0990000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.916+0000 7f09aca68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442286b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.916+0000 7f09aca68740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422871030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.916+0000 7f09aca68740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 0x56442286b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.916+0000 7f09aca68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564422860c70 con 0x56442285dc70 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.916+0000 7f09aca68740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56442277f0f0 con 0x564422871740 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.916+0000 7f09aca68740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564422860df0 con 0x56442285e590 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.916+0000 7f09aca68740 10 client.0 _ll_register_callbacks cb 0x5644227709e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.916+0000 7f09a2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422871030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.916+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442286b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.917+0000 7f09a2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422871030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42568/0 (socket says 172.21.15.107:42568) 2024-09-19T04:25:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.917+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442286b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35730/0 (socket says 172.21.15.107:35730) 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.917+0000 7f09a2000640 1 -- 172.21.15.107:0/1514031102 learned_addr learned my addr 172.21.15.107:0/1514031102 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.917+0000 7f09a3400640 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 0x56442286b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.917+0000 7f09a2a00640 1 -- 172.21.15.107:0/1514031102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 msgr2=0x564422871030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.917+0000 7f09a2a00640 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422871030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.917+0000 7f09a2a00640 1 -- 172.21.15.107:0/1514031102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 msgr2=0x56442286b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.917+0000 7f09a2a00640 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 0x56442286b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.917+0000 7f09a2a00640 1 -- 172.21.15.107:0/1514031102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644225353c0 con 0x56442285dc70 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.917+0000 7f09a2000640 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422871030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.918+0000 7f09a2a00640 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442286b2b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09940cce10 tx=0x7f099409b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6662a564139426 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.918+0000 7f098cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.918+0000 7f098cc00640 1 -- 172.21.15.107:0/1514031102 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099409fd50 con 0x56442285dc70 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.918+0000 7f098cc00640 1 -- 172.21.15.107:0/1514031102 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f09940cd090 con 0x56442285dc70 2024-09-19T04:25:59.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.918+0000 7f09aca68740 1 -- 172.21.15.107:0/1514031102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09900035f0 con 0x56442285dc70 2024-09-19T04:25:59.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.918+0000 7f098cc00640 1 -- 172.21.15.107:0/1514031102 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f09940b9050 con 0x56442285dc70 2024-09-19T04:25:59.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.918+0000 7f09aca68740 10 client.16164 fetch_fsmap learned FSMap version 37 2024-09-19T04:25:59.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.918+0000 7f09aca68740 1 -- 172.21.15.107:0/1514031102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564422888980 con 0x56442285dc70 2024-09-19T04:25:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.919+0000 7f098cc00640 1 -- 172.21.15.107:0/1514031102 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09900035f0 con 0x56442285dc70 2024-09-19T04:25:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.919+0000 7f098cc00640 1 -- 172.21.15.107:0/1514031102 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f09940b4c30 con 0x56442285dc70 2024-09-19T04:25:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.919+0000 7f09aca68740 10 client.16164 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.919+0000 7f09aca68740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.919+0000 7f09aca68740 20 client.16164 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.919+0000 7f09aca68740 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564422889e30 0x56442288c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.919+0000 7f09aca68740 4 client.16164 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.919+0000 7f09aca68740 1 -- 172.21.15.107:0/1514031102 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644227d1950 con 0x564422889e30 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.919+0000 7f09a2000640 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564422889e30 0x56442288c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.920+0000 7f09a2000640 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564422889e30 0x56442288c240 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f6348bde0eca2c5 server_cookie=a6331a84b15a2f78 in_seq=0 out_seq=0 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.920+0000 7f098cc00640 10 client.16164 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f098cc00640 1 -- 172.21.15.107:0/1514031102 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644227d1950 con 0x564422889e30 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f098cc00640 10 client.16164 handle_command_reply: tid=0 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f09aca68740 1 client.16164 shutdown 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f09aca68740 1 -- 172.21.15.107:0/1514031102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 msgr2=0x56442286b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f09aca68740 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442286b2b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09940cce10 tx=0x7f099409b610 comp rx=0 tx=0).stop 2024-09-19T04:25:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f09aca68740 1 -- 172.21.15.107:0/1514031102 shutdown_connections 2024-09-19T04:25:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f09aca68740 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564422889e30 0x56442288c240 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f09aca68740 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 0x56442286b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f09aca68740 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422871030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f09aca68740 1 --2- 172.21.15.107:0/1514031102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56442285dc70 0x56442286b2b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f09aca68740 1 -- 172.21.15.107:0/1514031102 >> 172.21.15.107:0/1514031102 conn(0x56442285a2e0 msgr2=0x56442285ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.921+0000 7f098cc00640 0 client.16164 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.922+0000 7f09aca68740 1 -- 172.21.15.107:0/1514031102 shutdown_connections 2024-09-19T04:25:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.922+0000 7f09aca68740 1 -- 172.21.15.107:0/1514031102 wait complete. 2024-09-19T04:25:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.922+0000 7f09aca68740 20 client.16164 trim_cache size 0 max 16384 2024-09-19T04:25:59.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09aca68740 1 Processor -- start 2024-09-19T04:25:59.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09aca68740 1 -- start start 2024-09-19T04:25:59.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09aca68740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422870f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09aca68740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 0x56442285fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09aca68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422891b50 0x564422892000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09aca68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41ab380 con 0x564422891b50 2024-09-19T04:25:59.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09aca68740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41acfe0 con 0x564422871740 2024-09-19T04:25:59.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09aca68740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41aba50 con 0x56442285e590 2024-09-19T04:25:59.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422870f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09a2000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 0x56442285fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422870f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42580/0 (socket says 172.21.15.107:42580) 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09a2000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 0x56442285fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47628/0 (socket says 172.21.15.107:47628) 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.943+0000 7f09a2a00640 1 -- 172.21.15.107:0/276597130 learned_addr learned my addr 172.21.15.107:0/276597130 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.944+0000 7f09a3400640 1 --2- 172.21.15.107:0/276597130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422891b50 0x564422892000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.944+0000 7f09a2a00640 1 -- 172.21.15.107:0/276597130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 msgr2=0x56442285fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.944+0000 7f09a2a00640 1 --2- 172.21.15.107:0/276597130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 0x56442285fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.944+0000 7f09a2a00640 1 -- 172.21.15.107:0/276597130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422891b50 msgr2=0x564422892000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.944+0000 7f09a2a00640 1 --2- 172.21.15.107:0/276597130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422891b50 0x564422892000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.944+0000 7f09a2a00640 1 -- 172.21.15.107:0/276597130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442288c8c0 con 0x56442285e590 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.944+0000 7f09a2a00640 1 --2- 172.21.15.107:0/276597130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422870f20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0994099c50 tx=0x7f09940045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db811c6d04e17027 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.944+0000 7f098f400640 1 -- 172.21.15.107:0/276597130 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099407f070 con 0x56442285e590 2024-09-19T04:25:59.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.944+0000 7f098f400640 1 -- 172.21.15.107:0/276597130 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099409b110 con 0x56442285e590 2024-09-19T04:25:59.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.944+0000 7f098f400640 1 -- 172.21.15.107:0/276597130 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09940a4610 con 0x56442285e590 2024-09-19T04:25:59.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.945+0000 7f09aca68740 1 -- 172.21.15.107:0/276597130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 msgr2=0x564422870f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.945+0000 7f09aca68740 1 --2- 172.21.15.107:0/276597130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422870f20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0994099c50 tx=0x7f09940045b0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.945+0000 7f09aca68740 1 -- 172.21.15.107:0/276597130 shutdown_connections 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.945+0000 7f09aca68740 1 --2- 172.21.15.107:0/276597130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422891b50 0x564422892000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.945+0000 7f09aca68740 1 --2- 172.21.15.107:0/276597130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422871740 0x56442285fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.945+0000 7f09aca68740 1 --2- 172.21.15.107:0/276597130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x564422870f20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.945+0000 7f09aca68740 1 -- 172.21.15.107:0/276597130 >> 172.21.15.107:0/276597130 conn(0x56442285a2e0 msgr2=0x56442285f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.945+0000 7f09aca68740 1 -- 172.21.15.107:0/276597130 shutdown_connections 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.945+0000 7f09aca68740 1 -- 172.21.15.107:0/276597130 wait complete. 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.946+0000 7f09aca68740 1 -- start start 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.946+0000 7f09aca68740 1 Processor -- start 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.946+0000 7f09aca68740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x56442289dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.946+0000 7f09aca68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422871740 0x56442289a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.946+0000 7f09aca68740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422891b50 0x56442289ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.946+0000 7f09aca68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41ac850 con 0x564422871740 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.946+0000 7f09aca68740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41ac120 con 0x564422891b50 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.946+0000 7f09aca68740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41a9380 con 0x56442285e590 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09aca68740 10 client.0 _ll_register_callbacks cb 0x5644227709e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422891b50 0x56442289ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422891b50 0x56442289ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47634/0 (socket says 172.21.15.107:47634) 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a3400640 1 -- 172.21.15.107:0/3718203856 learned_addr learned my addr 172.21.15.107:0/3718203856 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a2a00640 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x56442289dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a2000640 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422871740 0x56442289a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a3400640 1 -- 172.21.15.107:0/3718203856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 msgr2=0x56442289dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a3400640 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x56442289dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a3400640 1 -- 172.21.15.107:0/3718203856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422871740 msgr2=0x56442289a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:25:59.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a3400640 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422871740 0x56442289a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a3400640 1 -- 172.21.15.107:0/3718203856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56442286bb60 con 0x564422891b50 2024-09-19T04:25:59.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.947+0000 7f09a2000640 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422871740 0x56442289a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:25:59.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.948+0000 7f09a3400640 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422891b50 0x56442289ad90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f099c009ca0 tx=0x7f099c003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cae9562e0ea9fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:25:59.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.948+0000 7f098cc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:25:59.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.948+0000 7f098cc00640 1 -- 172.21.15.107:0/3718203856 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c030b30 con 0x564422891b50 2024-09-19T04:25:59.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.948+0000 7f09aca68740 1 -- 172.21.15.107:0/3718203856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099409b110 con 0x564422891b50 2024-09-19T04:25:59.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.948+0000 7f098cc00640 1 -- 172.21.15.107:0/3718203856 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f099c030cd0 con 0x564422891b50 2024-09-19T04:25:59.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.948+0000 7f098cc00640 1 -- 172.21.15.107:0/3718203856 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f099c030ed0 con 0x564422891b50 2024-09-19T04:25:59.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.948+0000 7f09aca68740 10 client.26222 fetch_fsmap learned FSMap version 37 2024-09-19T04:25:59.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.948+0000 7f09aca68740 1 -- 172.21.15.107:0/3718203856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56442285ed70 con 0x564422891b50 2024-09-19T04:25:59.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.949+0000 7f098cc00640 1 -- 172.21.15.107:0/3718203856 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099409b110 con 0x564422891b50 2024-09-19T04:25:59.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.949+0000 7f098cc00640 1 -- 172.21.15.107:0/3718203856 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f099c04f030 con 0x564422891b50 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.949+0000 7f09aca68740 10 client.26222 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.949+0000 7f09aca68740 10 client.26222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.949+0000 7f09aca68740 20 client.26222 populate_metadata read hostname 'smithi107' 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.949+0000 7f09aca68740 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5644228f5c90 0x5644228f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.949+0000 7f09aca68740 4 client.26222 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.949+0000 7f09aca68740 1 -- 172.21.15.107:0/3718203856 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644227cbe60 con 0x5644228f5c90 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.950+0000 7f09a2a00640 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5644228f5c90 0x5644228f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.950+0000 7f09a2a00640 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5644228f5c90 0x5644228f80b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d34c7f273c7104 server_cookie=5c4d85175984e492 in_seq=0 out_seq=0 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.950+0000 7f098cc00640 10 client.26222 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.950+0000 7f098cc00640 1 -- 172.21.15.107:0/3718203856 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5644227cbe60 con 0x5644228f5c90 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.950+0000 7f098cc00640 10 client.26222 handle_command_reply: tid=0 2024-09-19T04:25:59.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.950+0000 7f09aca68740 1 client.26222 shutdown 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 1 -- 172.21.15.107:0/3718203856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422891b50 msgr2=0x56442289ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422891b50 0x56442289ad90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f099c009ca0 tx=0x7f099c003b80 comp rx=0 tx=0).stop 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 1 -- 172.21.15.107:0/3718203856 shutdown_connections 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5644228f5c90 0x5644228f80b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564422891b50 0x56442289ad90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564422871740 0x56442289a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 1 --2- 172.21.15.107:0/3718203856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56442285e590 0x56442289dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 1 -- 172.21.15.107:0/3718203856 >> 172.21.15.107:0/3718203856 conn(0x56442285a2e0 msgr2=0x56442285f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f098cc00640 0 client.26222 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 1 -- 172.21.15.107:0/3718203856 shutdown_connections 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 1 -- 172.21.15.107:0/3718203856 wait complete. 2024-09-19T04:25:59.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.951+0000 7f09aca68740 20 client.26222 trim_cache size 0 max 16384 2024-09-19T04:25:59.957 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:25:59.957 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:25:59.957 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:25:59.957 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:25:59.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.952+0000 7f09a3e00640 1 -- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09880a34b0 msgr2=0x7f09880a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.952+0000 7f09a3e00640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09880a34b0 0x7f09880a5970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0990003f30 tx=0x7f0990000bc0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.952+0000 7f09a3e00640 1 -- 172.21.15.107:0/1509292645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a4103e30 msgr2=0x7f09a419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.952+0000 7f09a3e00640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a4103e30 0x7f09a419a4b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f099400d950 tx=0x7f099400de10 comp rx=0 tx=0).stop 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.953+0000 7f09a3e00640 1 -- 172.21.15.107:0/1509292645 shutdown_connections 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.953+0000 7f09a3e00640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09880a34b0 0x7f09880a5970 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.953+0000 7f09a3e00640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09a4108f60 0x7f09a419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.953+0000 7f09a3e00640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a41047e0 0x7f09a419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.953+0000 7f09a3e00640 1 --2- 172.21.15.107:0/1509292645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a4103e30 0x7f09a419a4b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.953+0000 7f09a3e00640 1 -- 172.21.15.107:0/1509292645 >> 172.21.15.107:0/1509292645 conn(0x7f09a40ff910 msgr2=0x7f09a4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.953+0000 7f09a3e00640 1 -- 172.21.15.107:0/1509292645 shutdown_connections 2024-09-19T04:25:59.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:25:59.953+0000 7f09a3e00640 1 -- 172.21.15.107:0/1509292645 wait complete. 2024-09-19T04:26:00.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 202412 2.91 MB/sec execute 138 sec latency 23.898 ms 2024-09-19T04:26:01.016 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:26:01.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:26:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:01 smithi107 ceph-mon[30659]: pgmap v173: 97 pgs: 97 active+clean; 131 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 552 KiB/s wr, 71 op/s 2024-09-19T04:26:01.400 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:01 smithi115 ceph-mon[35477]: pgmap v173: 97 pgs: 97 active+clean; 131 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 552 KiB/s wr, 71 op/s 2024-09-19T04:26:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:01 smithi145 ceph-mon[35096]: pgmap v173: 97 pgs: 97 active+clean; 131 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 552 KiB/s wr, 71 op/s 2024-09-19T04:26:01.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 202908 2.89 MB/sec execute 139 sec latency 418.394 ms 2024-09-19T04:26:02.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.046+0000 7f4263200640 1 -- 172.21.15.107:0/2456344804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f425c101de0 msgr2=0x7f425c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:02.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.046+0000 7f4263200640 1 --2- 172.21.15.107:0/2456344804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f425c101de0 0x7f425c1021c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f424c009900 tx=0x7f424c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.046+0000 7f4263200640 1 -- 172.21.15.107:0/2456344804 shutdown_connections 2024-09-19T04:26:02.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.046+0000 7f4263200640 1 --2- 172.21.15.107:0/2456344804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c106f10 0x7f425c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.046+0000 7f4263200640 1 --2- 172.21.15.107:0/2456344804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f425c102790 0x7f425c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.046+0000 7f4263200640 1 --2- 172.21.15.107:0/2456344804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f425c101de0 0x7f425c1021c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.047+0000 7f4263200640 1 -- 172.21.15.107:0/2456344804 >> 172.21.15.107:0/2456344804 conn(0x7f425c0fd8c0 msgr2=0x7f425c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.047+0000 7f4263200640 1 -- 172.21.15.107:0/2456344804 shutdown_connections 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.047+0000 7f4263200640 1 -- 172.21.15.107:0/2456344804 wait complete. 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.048+0000 7f4263200640 1 Processor -- start 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.048+0000 7f4263200640 1 -- start start 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4263200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c101de0 0x7f425c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4263200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f425c102790 0x7f425c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4263200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f425c106f10 0x7f425c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4263200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f425c111f70 con 0x7f425c106f10 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4263200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f425c111df0 con 0x7f425c101de0 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4263200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1120f0 con 0x7f425c102790 2024-09-19T04:26:02.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4261200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f425c106f10 0x7f425c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4261200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f425c106f10 0x7f425c19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35764/0 (socket says 172.21.15.107:35764) 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4261200640 1 -- 172.21.15.107:0/1630538149 learned_addr learned my addr 172.21.15.107:0/1630538149 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f425be00640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c101de0 0x7f425c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4253e00640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f425c102790 0x7f425c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4261200640 1 -- 172.21.15.107:0/1630538149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f425c102790 msgr2=0x7f425c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4261200640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f425c102790 0x7f425c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4261200640 1 -- 172.21.15.107:0/1630538149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c101de0 msgr2=0x7f425c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4261200640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c101de0 0x7f425c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.049+0000 7f4261200640 1 -- 172.21.15.107:0/1630538149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c19f5d0 con 0x7f425c106f10 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.050+0000 7f4261200640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f425c106f10 0x7f425c19eef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f425400eb10 tx=0x7f425400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:02.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.050+0000 7f4259600640 1 -- 172.21.15.107:0/1630538149 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254004b20 con 0x7f425c106f10 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.050+0000 7f4259600640 1 -- 172.21.15.107:0/1630538149 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4254004cc0 con 0x7f425c106f10 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.050+0000 7f4263200640 1 -- 172.21.15.107:0/1630538149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425c19f8c0 con 0x7f425c106f10 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.050+0000 7f4259600640 1 -- 172.21.15.107:0/1630538149 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254010760 con 0x7f425c106f10 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.050+0000 7f4263200640 1 -- 172.21.15.107:0/1630538149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425c1a71a0 con 0x7f425c106f10 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.051+0000 7f4263200640 1 -- 172.21.15.107:0/1630538149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f425c1021c0 con 0x7f425c106f10 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.053+0000 7f4259600640 1 -- 172.21.15.107:0/1630538149 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f4254005020 con 0x7f425c106f10 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.054+0000 7f4259600640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f42400a33b0 0x7f42400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.055+0000 7f4259600640 1 -- 172.21.15.107:0/1630538149 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f42540d0220 con 0x7f425c106f10 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.055+0000 7f425be00640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f42400a33b0 0x7f42400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.055+0000 7f425be00640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f42400a33b0 0x7f42400a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f424c009230 tx=0x7f424c0058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:02.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.060+0000 7f4259600640 1 -- 172.21.15.107:0/1630538149 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f425409cbf0 con 0x7f425c106f10 2024-09-19T04:26:02.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.271+0000 7f4263200640 1 -- 172.21.15.107:0/1630538149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f425c1a7480 con 0x7f425c106f10 2024-09-19T04:26:02.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.272+0000 7f4259600640 1 -- 172.21.15.107:0/1630538149 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12002 (secure 0 0 0) 0x7f425409c4f0 con 0x7f425c106f10 2024-09-19T04:26:02.274 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:26:02.274 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":37,"btime":"2024-09-19T04:25:31:977893+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:25:31.704942+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:26:02.275 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 37 2024-09-19T04:26:02.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 -- 172.21.15.107:0/1630538149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f42400a33b0 msgr2=0x7f42400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f42400a33b0 0x7f42400a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f424c009230 tx=0x7f424c0058d0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 -- 172.21.15.107:0/1630538149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f425c106f10 msgr2=0x7f425c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f425c106f10 0x7f425c19eef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f425400eb10 tx=0x7f425400c2d0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 -- 172.21.15.107:0/1630538149 shutdown_connections 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f42400a33b0 0x7f42400a5870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f425c106f10 0x7f425c19eef0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f425c102790 0x7f425c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 --2- 172.21.15.107:0/1630538149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c101de0 0x7f425c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 -- 172.21.15.107:0/1630538149 >> 172.21.15.107:0/1630538149 conn(0x7f425c0fd8c0 msgr2=0x7f425c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 -- 172.21.15.107:0/1630538149 shutdown_connections 2024-09-19T04:26:02.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:02.275+0000 7f4263200640 1 -- 172.21.15.107:0/1630538149 wait complete. 2024-09-19T04:26:02.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 202908 2.87 MB/sec execute 140 sec latency 1418.469 ms 2024-09-19T04:26:02.988 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:26:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:03 smithi107 ceph-mon[30659]: pgmap v174: 97 pgs: 97 active+clean; 132 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 892 KiB/s wr, 104 op/s 2024-09-19T04:26:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:03 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1630538149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:03.361 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:03.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:03 smithi115 ceph-mon[35477]: pgmap v174: 97 pgs: 97 active+clean; 132 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 892 KiB/s wr, 104 op/s 2024-09-19T04:26:03.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:03 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1630538149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:03.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:03 smithi145 ceph-mon[35096]: pgmap v174: 97 pgs: 97 active+clean; 132 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 892 KiB/s wr, 104 op/s 2024-09-19T04:26:03.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:03 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1630538149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:03.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 202908 2.85 MB/sec execute 141 sec latency 2418.510 ms 2024-09-19T04:26:03.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.940+0000 7f2077200640 1 -- 172.21.15.107:0/1169447856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20701049a0 msgr2=0x7f20701089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.940+0000 7f2077200640 1 --2- 172.21.15.107:0/1169447856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20701049a0 0x7f20701089f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f205c009900 tx=0x7f205c02a410 comp rx=0 tx=0).stop 2024-09-19T04:26:03.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.940+0000 7f2077200640 1 -- 172.21.15.107:0/1169447856 shutdown_connections 2024-09-19T04:26:03.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.940+0000 7f2077200640 1 --2- 172.21.15.107:0/1169447856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2070109120 0x7f2070110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.940+0000 7f2077200640 1 --2- 172.21.15.107:0/1169447856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20701049a0 0x7f20701089f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.940+0000 7f2077200640 1 --2- 172.21.15.107:0/1169447856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2070103ff0 0x7f20701043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.940+0000 7f2077200640 1 -- 172.21.15.107:0/1169447856 >> 172.21.15.107:0/1169447856 conn(0x7f20700ffa90 msgr2=0x7f2070101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.940+0000 7f2077200640 1 -- 172.21.15.107:0/1169447856 shutdown_connections 2024-09-19T04:26:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.940+0000 7f2077200640 1 -- 172.21.15.107:0/1169447856 wait complete. 2024-09-19T04:26:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.941+0000 7f2077200640 1 Processor -- start 2024-09-19T04:26:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.941+0000 7f2077200640 1 -- start start 2024-09-19T04:26:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f2077200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2070103ff0 0x7f207019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f2077200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20701049a0 0x7f207019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f2077200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2070109120 0x7f207019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f2077200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2070114170 con 0x7f20701049a0 2024-09-19T04:26:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f2077200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2070113ff0 con 0x7f2070109120 2024-09-19T04:26:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f2077200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f20701142f0 con 0x7f2070103ff0 2024-09-19T04:26:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f206fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2070103ff0 0x7f207019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f206fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2070103ff0 0x7f207019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42642/0 (socket says 172.21.15.107:42642) 2024-09-19T04:26:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f206fe00640 1 -- 172.21.15.107:0/2882139658 learned_addr learned my addr 172.21.15.107:0/2882139658 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f206f400640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20701049a0 0x7f207019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.942+0000 7f2075200640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2070109120 0x7f207019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f206fe00640 1 -- 172.21.15.107:0/2882139658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2070109120 msgr2=0x7f207019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f206fe00640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2070109120 0x7f207019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f206fe00640 1 -- 172.21.15.107:0/2882139658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20701049a0 msgr2=0x7f207019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f206fe00640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20701049a0 0x7f207019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f206fe00640 1 -- 172.21.15.107:0/2882139658 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207019f600 con 0x7f2070103ff0 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f206fe00640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2070103ff0 0x7f207019a650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f206000ea60 tx=0x7f206000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f206cc00640 1 -- 172.21.15.107:0/2882139658 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206000cde0 con 0x7f2070103ff0 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f2077200640 1 -- 172.21.15.107:0/2882139658 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207019f8f0 con 0x7f2070103ff0 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f206cc00640 1 -- 172.21.15.107:0/2882139658 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20600035f0 con 0x7f2070103ff0 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f206cc00640 1 -- 172.21.15.107:0/2882139658 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060010670 con 0x7f2070103ff0 2024-09-19T04:26:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.943+0000 7f2077200640 1 -- 172.21.15.107:0/2882139658 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20701a71d0 con 0x7f2070103ff0 2024-09-19T04:26:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.946+0000 7f2079bf7740 1 Processor -- start 2024-09-19T04:26:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.946+0000 7f2079bf7740 1 -- start start 2024-09-19T04:26:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 0x555dc0abf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f206cc00640 1 -- 172.21.15.107:0/2882139658 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f20600037a0 con 0x7f2070103ff0 2024-09-19T04:26:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0abf590 0x555dc0ac9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0ac9e50 0x555dc0acc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f2079bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555dc0a253e0 con 0x555dc0ac9e50 2024-09-19T04:26:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f2079bf7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555dc09ccd80 con 0x555dc0abec70 2024-09-19T04:26:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f2079bf7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555dc0a213a0 con 0x555dc0abf590 2024-09-19T04:26:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f2075200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 0x555dc0abf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f206fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0abf590 0x555dc0ac9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f2075200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 0x555dc0abf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47674/0 (socket says 172.21.15.107:47674) 2024-09-19T04:26:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f2075200640 1 -- 172.21.15.107:0/3321164780 learned_addr learned my addr 172.21.15.107:0/3321164780 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.947+0000 7f206f400640 1 --2- 172.21.15.107:0/3321164780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0ac9e50 0x555dc0acc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.948+0000 7f2075200640 1 -- 172.21.15.107:0/3321164780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0abf590 msgr2=0x555dc0ac9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.948+0000 7f2075200640 1 --2- 172.21.15.107:0/3321164780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0abf590 0x555dc0ac9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.948+0000 7f2075200640 1 -- 172.21.15.107:0/3321164780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0ac9e50 msgr2=0x555dc0acc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.948+0000 7f2075200640 1 --2- 172.21.15.107:0/3321164780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0ac9e50 0x555dc0acc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.948+0000 7f2075200640 1 -- 172.21.15.107:0/3321164780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc09ebae0 con 0x555dc0abec70 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.948+0000 7f2075200640 1 --2- 172.21.15.107:0/3321164780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 0x555dc0abf050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2068009f50 tx=0x7f20680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94b59660e8737f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.948+0000 7f2065600640 1 -- 172.21.15.107:0/3321164780 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068004d20 con 0x555dc0abec70 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.948+0000 7f2065600640 1 -- 172.21.15.107:0/3321164780 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2068004050 con 0x555dc0abec70 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.948+0000 7f2065600640 1 -- 172.21.15.107:0/3321164780 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068005280 con 0x555dc0abec70 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f206cc00640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20540a33b0 0x7f20540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f206cc00640 1 -- 172.21.15.107:0/2882139658 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f20600cffe0 con 0x7f2070103ff0 2024-09-19T04:26:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f2079bf7740 1 -- 172.21.15.107:0/3321164780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 msgr2=0x555dc0abf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f206fe00640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20540a33b0 0x7f20540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f2079bf7740 1 --2- 172.21.15.107:0/3321164780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 0x555dc0abf050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2068009f50 tx=0x7f20680118d0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f2079bf7740 1 -- 172.21.15.107:0/3321164780 shutdown_connections 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f2079bf7740 1 --2- 172.21.15.107:0/3321164780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0ac9e50 0x555dc0acc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f2079bf7740 1 --2- 172.21.15.107:0/3321164780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0abf590 0x555dc0ac9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f2079bf7740 1 --2- 172.21.15.107:0/3321164780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 0x555dc0abf050 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f2079bf7740 1 -- 172.21.15.107:0/3321164780 >> 172.21.15.107:0/3321164780 conn(0x555dc0abb200 msgr2=0x555dc0abb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f2079bf7740 1 -- 172.21.15.107:0/3321164780 shutdown_connections 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.949+0000 7f2079bf7740 1 -- 172.21.15.107:0/3321164780 wait complete. 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.950+0000 7f206fe00640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20540a33b0 0x7f20540a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x555dc0abc060 tx=0x7f20600d3a90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.950+0000 7f2079bf7740 1 -- start start 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.950+0000 7f2079bf7740 1 Processor -- start 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 0x555dc0ac6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abf590 0x555dc0ac72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ac77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f2079bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555dc0acca20 con 0x555dc0abf590 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f2079bf7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555dc09e00f0 con 0x555dc0abec70 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f2079bf7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555dc0accba0 con 0x555dc0ac9e50 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f2079bf7740 10 client.0 _ll_register_callbacks cb 0x555dc09c7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f206f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abf590 0x555dc0ac72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f2075200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ac77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f206f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abf590 0x555dc0ac72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35804/0 (socket says 172.21.15.107:35804) 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f206f400640 1 -- 172.21.15.107:0/3408985949 learned_addr learned my addr 172.21.15.107:0/3408985949 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.951+0000 7f2075200640 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 0x555dc0ac6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f2075200640 1 -- 172.21.15.107:0/3408985949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 msgr2=0x555dc0ac6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f2075200640 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 0x555dc0ac6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f2075200640 1 -- 172.21.15.107:0/3408985949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abf590 msgr2=0x555dc0ac72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f2075200640 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abf590 0x555dc0ac72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f2075200640 1 -- 172.21.15.107:0/3408985949 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc07963c0 con 0x555dc0ac9e50 2024-09-19T04:26:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f206f400640 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abf590 0x555dc0ac72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f2075200640 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ac77e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2068011dd0 tx=0x7f2068005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff47bbb57fcdfaf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f205a000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:26:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f205a000640 1 -- 172.21.15.107:0/3408985949 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206802b030 con 0x555dc0ac9e50 2024-09-19T04:26:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f205a000640 1 -- 172.21.15.107:0/3408985949 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f2068027440 con 0x555dc0ac9e50 2024-09-19T04:26:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f2079bf7740 1 -- 172.21.15.107:0/3408985949 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2068004050 con 0x555dc0ac9e50 2024-09-19T04:26:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f205a000640 1 -- 172.21.15.107:0/3408985949 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2068035050 con 0x555dc0ac9e50 2024-09-19T04:26:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f2079bf7740 10 client.26203 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.952+0000 7f2079bf7740 1 -- 172.21.15.107:0/3408985949 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555dc0ad8a90 con 0x555dc0ac9e50 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.953+0000 7f205a000640 1 -- 172.21.15.107:0/3408985949 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2068004050 con 0x555dc0ac9e50 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.953+0000 7f205a000640 1 -- 172.21.15.107:0/3408985949 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f206800f510 con 0x555dc0ac9e50 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.953+0000 7f2079bf7740 10 client.26203 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.953+0000 7f2079bf7740 10 client.26203 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.953+0000 7f2079bf7740 20 client.26203 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.953+0000 7f2079bf7740 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555dc0ad9ef0 0x555dc0adc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.953+0000 7f2079bf7740 4 client.26203 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.953+0000 7f2079bf7740 1 -- 172.21.15.107:0/3408985949 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dc0a32950 con 0x555dc0ad9ef0 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.953+0000 7f206f400640 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555dc0ad9ef0 0x555dc0adc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.954+0000 7f206f400640 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555dc0ad9ef0 0x555dc0adc2e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=946677ded4f8273a server_cookie=d28f8635cf2dd615 in_seq=0 out_seq=0 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.954+0000 7f205a000640 10 client.26203 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:26:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f205a000640 1 -- 172.21.15.107:0/3408985949 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555dc0a32950 con 0x555dc0ad9ef0 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f205a000640 10 client.26203 handle_command_reply: tid=0 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 client.26203 shutdown 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 -- 172.21.15.107:0/3408985949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 msgr2=0x555dc0ac77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ac77e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2068011dd0 tx=0x7f2068005bd0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 -- 172.21.15.107:0/3408985949 shutdown_connections 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555dc0ad9ef0 0x555dc0adc2e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ac77e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abf590 0x555dc0ac72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 --2- 172.21.15.107:0/3408985949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abec70 0x555dc0ac6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 -- 172.21.15.107:0/3408985949 >> 172.21.15.107:0/3408985949 conn(0x555dc0abb200 msgr2=0x555dc0ac0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f205a000640 0 client.26203 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 -- 172.21.15.107:0/3408985949 shutdown_connections 2024-09-19T04:26:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 1 -- 172.21.15.107:0/3408985949 wait complete. 2024-09-19T04:26:03.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.955+0000 7f2079bf7740 20 client.26203 trim_cache size 0 max 16384 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.968+0000 7f2079bf7740 1 Processor -- start 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2079bf7740 1 -- start start 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ac9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abf590 0x555dc0ac7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ae1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2079bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20701afa20 con 0x555dc0abec70 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2079bf7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20701b1680 con 0x555dc0abf590 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2079bf7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f20701b00f0 con 0x555dc0ac9e50 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2075200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ac9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f206f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abf590 0x555dc0ac7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2075200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ac9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35808/0 (socket says 172.21.15.107:35808) 2024-09-19T04:26:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2075200640 1 -- 172.21.15.107:0/2711844413 learned_addr learned my addr 172.21.15.107:0/2711844413 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.969+0000 7f2075200640 1 --2- 172.21.15.107:0/2711844413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ae1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.970+0000 7f2075200640 1 -- 172.21.15.107:0/2711844413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 msgr2=0x555dc0ae1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.970+0000 7f2075200640 1 --2- 172.21.15.107:0/2711844413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ae1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.970+0000 7f2075200640 1 -- 172.21.15.107:0/2711844413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abf590 msgr2=0x555dc0ac7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.970+0000 7f2075200640 1 --2- 172.21.15.107:0/2711844413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abf590 0x555dc0ac7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.970+0000 7f2075200640 1 -- 172.21.15.107:0/2711844413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc0ab6770 con 0x555dc0abec70 2024-09-19T04:26:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.970+0000 7f2075200640 1 --2- 172.21.15.107:0/2711844413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ac9090 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2068009600 tx=0x7f2068009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182f44da2de6111f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.970+0000 7f2064c00640 1 -- 172.21.15.107:0/2711844413 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206802b020 con 0x555dc0abec70 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.970+0000 7f2064c00640 1 -- 172.21.15.107:0/2711844413 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2068003e70 con 0x555dc0abec70 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.970+0000 7f2064c00640 1 -- 172.21.15.107:0/2711844413 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068027920 con 0x555dc0abec70 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.971+0000 7f2079bf7740 1 -- 172.21.15.107:0/2711844413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 msgr2=0x555dc0ac9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.971+0000 7f2079bf7740 1 --2- 172.21.15.107:0/2711844413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ac9090 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2068009600 tx=0x7f2068009590 comp rx=0 tx=0).stop 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.971+0000 7f2079bf7740 1 -- 172.21.15.107:0/2711844413 shutdown_connections 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.971+0000 7f2079bf7740 1 --2- 172.21.15.107:0/2711844413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ae1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.971+0000 7f2079bf7740 1 --2- 172.21.15.107:0/2711844413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abf590 0x555dc0ac7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.971+0000 7f2079bf7740 1 --2- 172.21.15.107:0/2711844413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ac9090 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.971+0000 7f2079bf7740 1 -- 172.21.15.107:0/2711844413 >> 172.21.15.107:0/2711844413 conn(0x555dc0abb200 msgr2=0x555dc0ac5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.971+0000 7f2079bf7740 1 -- 172.21.15.107:0/2711844413 shutdown_connections 2024-09-19T04:26:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.971+0000 7f2079bf7740 1 -- 172.21.15.107:0/2711844413 wait complete. 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.971+0000 7f2079bf7740 1 -- start start 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2079bf7740 1 Processor -- start 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ad2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abf590 0x555dc0ad26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2079bf7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ac3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2079bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20701b0ef0 con 0x555dc0abec70 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2079bf7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20701b07c0 con 0x555dc0abf590 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2079bf7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f20701ada00 con 0x555dc0ac9e50 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2079bf7740 10 client.0 _ll_register_callbacks cb 0x7f20600d3960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2075200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ad2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2075200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ad2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35824/0 (socket says 172.21.15.107:35824) 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f2075200640 1 -- 172.21.15.107:0/200974237 learned_addr learned my addr 172.21.15.107:0/200974237 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.972+0000 7f206f400640 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abf590 0x555dc0ad26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f2075200640 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ac3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f2075200640 1 -- 172.21.15.107:0/200974237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 msgr2=0x555dc0ac3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f2075200640 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ac3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f2075200640 1 -- 172.21.15.107:0/200974237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abf590 msgr2=0x555dc0ad26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:03.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f2075200640 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abf590 0x555dc0ad26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f2075200640 1 -- 172.21.15.107:0/200974237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc0ac4140 con 0x555dc0abec70 2024-09-19T04:26:03.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f2075200640 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ad2160 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2068009540 tx=0x7f2068000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a40cfc7613aa82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:03.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f205a000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:26:03.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f205a000640 1 -- 172.21.15.107:0/200974237 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206802b050 con 0x555dc0abec70 2024-09-19T04:26:03.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f205a000640 1 -- 172.21.15.107:0/200974237 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f20680044c0 con 0x555dc0abec70 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.973+0000 7f2079bf7740 1 -- 172.21.15.107:0/200974237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2068003e70 con 0x555dc0abec70 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f205a000640 1 -- 172.21.15.107:0/200974237 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f206804d050 con 0x555dc0abec70 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f2079bf7740 10 client.16194 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f2079bf7740 1 -- 172.21.15.107:0/200974237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555dc0b446e0 con 0x555dc0abec70 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f205a000640 1 -- 172.21.15.107:0/200974237 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2068003e70 con 0x555dc0abec70 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f205a000640 1 -- 172.21.15.107:0/200974237 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f2068046060 con 0x555dc0abec70 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f2079bf7740 10 client.16194 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f2079bf7740 10 client.16194 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f2079bf7740 20 client.16194 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f2079bf7740 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555dc0b44ce0 0x555dc0b47100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f2079bf7740 4 client.16194 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.974+0000 7f2079bf7740 1 -- 172.21.15.107:0/200974237 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555dc0a2ce60 con 0x555dc0b44ce0 2024-09-19T04:26:03.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.975+0000 7f206f400640 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555dc0b44ce0 0x555dc0b47100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.975+0000 7f206f400640 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555dc0b44ce0 0x555dc0b47100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ea23fa401790b22 server_cookie=d34f5f10329a13dd in_seq=0 out_seq=0 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.975+0000 7f205a000640 10 client.16194 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.975+0000 7f205a000640 1 -- 172.21.15.107:0/200974237 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555dc0a2ce60 con 0x555dc0b44ce0 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.975+0000 7f205a000640 10 client.16194 handle_command_reply: tid=0 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.975+0000 7f2079bf7740 1 client.16194 shutdown 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.975+0000 7f2079bf7740 1 -- 172.21.15.107:0/200974237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 msgr2=0x555dc0ad2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.975+0000 7f2079bf7740 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ad2160 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2068009540 tx=0x7f2068000f40 comp rx=0 tx=0).stop 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2079bf7740 1 -- 172.21.15.107:0/200974237 shutdown_connections 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2079bf7740 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x555dc0b44ce0 0x555dc0b47100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2079bf7740 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dc0ac9e50 0x555dc0ac3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2079bf7740 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555dc0abf590 0x555dc0ad26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2079bf7740 1 --2- 172.21.15.107:0/200974237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555dc0abec70 0x555dc0ad2160 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2079bf7740 1 -- 172.21.15.107:0/200974237 >> 172.21.15.107:0/200974237 conn(0x555dc0abb200 msgr2=0x555dc0ac5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f205a000640 0 client.16194 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:26:03.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2079bf7740 1 -- 172.21.15.107:0/200974237 shutdown_connections 2024-09-19T04:26:03.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2079bf7740 1 -- 172.21.15.107:0/200974237 wait complete. 2024-09-19T04:26:03.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2079bf7740 20 client.16194 trim_cache size 0 max 16384 2024-09-19T04:26:03.982 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:26:03.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2077200640 1 -- 172.21.15.107:0/2882139658 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20540a33b0 msgr2=0x7f20540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2077200640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20540a33b0 0x7f20540a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x555dc0abc060 tx=0x7f20600d3a90 comp rx=0 tx=0).stop 2024-09-19T04:26:03.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2077200640 1 -- 172.21.15.107:0/2882139658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2070103ff0 msgr2=0x7f207019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:03.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.976+0000 7f2077200640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2070103ff0 0x7f207019a650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f206000ea60 tx=0x7f206000ef20 comp rx=0 tx=0).stop 2024-09-19T04:26:03.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.977+0000 7f2077200640 1 -- 172.21.15.107:0/2882139658 shutdown_connections 2024-09-19T04:26:03.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.977+0000 7f2077200640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20540a33b0 0x7f20540a5870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.977+0000 7f2077200640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2070109120 0x7f207019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.977+0000 7f2077200640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20701049a0 0x7f207019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.977+0000 7f2077200640 1 --2- 172.21.15.107:0/2882139658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2070103ff0 0x7f207019a650 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:03.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.977+0000 7f2077200640 1 -- 172.21.15.107:0/2882139658 >> 172.21.15.107:0/2882139658 conn(0x7f20700ffa90 msgr2=0x7f2070100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:03.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.977+0000 7f2077200640 1 -- 172.21.15.107:0/2882139658 shutdown_connections 2024-09-19T04:26:03.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:03.977+0000 7f2077200640 1 -- 172.21.15.107:0/2882139658 wait complete. 2024-09-19T04:26:04.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 203679 2.86 MB/sec execute 142 sec latency 2964.120 ms 2024-09-19T04:26:05.033 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:26:05.404 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:05.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:05 smithi115 ceph-mon[35477]: pgmap v175: 97 pgs: 97 active+clean; 132 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 504 KiB/s wr, 66 op/s 2024-09-19T04:26:05.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:05 smithi145 ceph-mon[35096]: pgmap v175: 97 pgs: 97 active+clean; 132 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 504 KiB/s wr, 66 op/s 2024-09-19T04:26:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:05 smithi107 ceph-mon[30659]: pgmap v175: 97 pgs: 97 active+clean; 132 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 504 KiB/s wr, 66 op/s 2024-09-19T04:26:05.671 INFO:tasks.workunit.client.0.smithi107.stdout: 1 203878 2.84 MB/sec execute 143 sec latency 808.061 ms 2024-09-19T04:26:05.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.987+0000 7f2853000640 1 -- 172.21.15.107:0/4130931037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f284c104010 msgr2=0x7f284c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.987+0000 7f2853000640 1 --2- 172.21.15.107:0/4130931037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f284c104010 0x7f284c1043f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f283c009900 tx=0x7f283c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.987+0000 7f2853000640 1 -- 172.21.15.107:0/4130931037 shutdown_connections 2024-09-19T04:26:05.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.987+0000 7f2853000640 1 --2- 172.21.15.107:0/4130931037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f284c109140 0x7f284c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.987+0000 7f2853000640 1 --2- 172.21.15.107:0/4130931037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c1049c0 0x7f284c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.987+0000 7f2853000640 1 --2- 172.21.15.107:0/4130931037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f284c104010 0x7f284c1043f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.987+0000 7f2853000640 1 -- 172.21.15.107:0/4130931037 >> 172.21.15.107:0/4130931037 conn(0x7f284c0ffa50 msgr2=0x7f284c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:05.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.987+0000 7f2853000640 1 -- 172.21.15.107:0/4130931037 shutdown_connections 2024-09-19T04:26:05.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.987+0000 7f2853000640 1 -- 172.21.15.107:0/4130931037 wait complete. 2024-09-19T04:26:05.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.988+0000 7f2853000640 1 Processor -- start 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.988+0000 7f2853000640 1 -- start start 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.988+0000 7f2853000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f284c104010 0x7f284c110b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.988+0000 7f2853000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c1049c0 0x7f284c10dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.988+0000 7f2853000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f284c109140 0x7f284c10e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.988+0000 7f2853000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f284c114160 con 0x7f284c109140 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.988+0000 7f2853000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f284c113fe0 con 0x7f284c104010 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.988+0000 7f2853000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1142e0 con 0x7f284c1049c0 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.989+0000 7f284be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f284c104010 0x7f284c110b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.989+0000 7f284be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f284c104010 0x7f284c110b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47716/0 (socket says 172.21.15.107:47716) 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.989+0000 7f2851000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f284c109140 0x7f284c10e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.989+0000 7f284be00640 1 -- 172.21.15.107:0/4007610764 learned_addr learned my addr 172.21.15.107:0/4007610764 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:05.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.989+0000 7f2843400640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c1049c0 0x7f284c10dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.989+0000 7f284be00640 1 -- 172.21.15.107:0/4007610764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c1049c0 msgr2=0x7f284c10dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.989+0000 7f284be00640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c1049c0 0x7f284c10dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.989+0000 7f284be00640 1 -- 172.21.15.107:0/4007610764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f284c109140 msgr2=0x7f284c10e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:05.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.989+0000 7f284be00640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f284c109140 0x7f284c10e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.989+0000 7f284be00640 1 -- 172.21.15.107:0/4007610764 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c10ea10 con 0x7f284c104010 2024-09-19T04:26:05.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.990+0000 7f284be00640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f284c104010 0x7f284c110b60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f283c006010 tx=0x7f283c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:05.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.990+0000 7f2849600640 1 -- 172.21.15.107:0/4007610764 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c03f030 con 0x7f284c104010 2024-09-19T04:26:05.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.990+0000 7f2849600640 1 -- 172.21.15.107:0/4007610764 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283c004440 con 0x7f284c104010 2024-09-19T04:26:05.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.990+0000 7f2853000640 1 -- 172.21.15.107:0/4007610764 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c1aaf60 con 0x7f284c104010 2024-09-19T04:26:05.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.990+0000 7f2849600640 1 -- 172.21.15.107:0/4007610764 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c035070 con 0x7f284c104010 2024-09-19T04:26:05.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.990+0000 7f2853000640 1 -- 172.21.15.107:0/4007610764 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c1ab410 con 0x7f284c104010 2024-09-19T04:26:05.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.993+0000 7f2855bee740 1 Processor -- start 2024-09-19T04:26:05.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.993+0000 7f2855bee740 1 -- start start 2024-09-19T04:26:05.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.993+0000 7f2849600640 1 -- 172.21.15.107:0/4007610764 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f283c042030 con 0x7f284c104010 2024-09-19T04:26:05.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.993+0000 7f2855bee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100ac70 0x561bd100b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.993+0000 7f2855bee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100b590 0x561bd1015910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.993+0000 7f2855bee740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 0x561bd1018240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.993+0000 7f2855bee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561bd0f713e0 con 0x561bd100ac70 2024-09-19T04:26:05.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.993+0000 7f2855bee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561bd0f18d80 con 0x561bd100b590 2024-09-19T04:26:05.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.993+0000 7f2855bee740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561bd0f6d3a0 con 0x561bd1015e50 2024-09-19T04:26:05.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f2851000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100ac70 0x561bd100b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f2851000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100ac70 0x561bd100b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35858/0 (socket says 172.21.15.107:35858) 2024-09-19T04:26:05.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f2851000640 1 -- 172.21.15.107:0/615934478 learned_addr learned my addr 172.21.15.107:0/615934478 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:05.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f284be00640 1 --2- 172.21.15.107:0/615934478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100b590 0x561bd1015910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f2843400640 1 --2- 172.21.15.107:0/615934478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 0x561bd1018240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:05.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f284be00640 1 -- 172.21.15.107:0/615934478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 msgr2=0x561bd1018240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:05.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f284be00640 1 --2- 172.21.15.107:0/615934478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 0x561bd1018240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f284be00640 1 -- 172.21.15.107:0/615934478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100ac70 msgr2=0x561bd100b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:05.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f284be00640 1 --2- 172.21.15.107:0/615934478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100ac70 0x561bd100b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:05.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f284be00640 1 -- 172.21.15.107:0/615934478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd0f37ae0 con 0x561bd100b590 2024-09-19T04:26:05.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.994+0000 7f2851000640 1 --2- 172.21.15.107:0/615934478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100ac70 0x561bd100b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:05.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.995+0000 7f284be00640 1 --2- 172.21.15.107:0/615934478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100b590 0x561bd1015910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f283c0ed790 tx=0x7f283c0f8320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8143bc1d06d98785 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:05.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.995+0000 7f2849600640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28300a33b0 0x7f28300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:05.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.995+0000 7f2841600640 1 -- 172.21.15.107:0/615934478 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c0b8670 con 0x561bd100b590 2024-09-19T04:26:05.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.995+0000 7f2841600640 1 -- 172.21.15.107:0/615934478 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283c0fd030 con 0x561bd100b590 2024-09-19T04:26:05.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.995+0000 7f2841600640 1 -- 172.21.15.107:0/615934478 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c0bb820 con 0x561bd100b590 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.995+0000 7f2849600640 1 -- 172.21.15.107:0/4007610764 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f283c0efff0 con 0x7f284c104010 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.995+0000 7f2851000640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28300a33b0 0x7f28300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.996+0000 7f2855bee740 1 -- 172.21.15.107:0/615934478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100b590 msgr2=0x561bd1015910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.996+0000 7f2855bee740 1 --2- 172.21.15.107:0/615934478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100b590 0x561bd1015910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f283c0ed790 tx=0x7f283c0f8320 comp rx=0 tx=0).stop 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.996+0000 7f2851000640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28300a33b0 0x7f28300a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2844009fd0 tx=0x7f2844009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.996+0000 7f2855bee740 1 -- 172.21.15.107:0/615934478 shutdown_connections 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.996+0000 7f2855bee740 1 --2- 172.21.15.107:0/615934478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 0x561bd1018240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.996+0000 7f2855bee740 1 --2- 172.21.15.107:0/615934478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100b590 0x561bd1015910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.996+0000 7f2855bee740 1 --2- 172.21.15.107:0/615934478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100ac70 0x561bd100b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.996+0000 7f2855bee740 1 -- 172.21.15.107:0/615934478 >> 172.21.15.107:0/615934478 conn(0x561bd1007200 msgr2=0x561bd10075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.996+0000 7f2855bee740 1 -- 172.21.15.107:0/615934478 shutdown_connections 2024-09-19T04:26:06.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.996+0000 7f2855bee740 1 -- 172.21.15.107:0/615934478 wait complete. 2024-09-19T04:26:06.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.997+0000 7f2855bee740 1 -- start start 2024-09-19T04:26:06.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.997+0000 7f2855bee740 1 Processor -- start 2024-09-19T04:26:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.997+0000 7f2855bee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 0x561bd1012c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f2855bee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 0x561bd10131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f2855bee740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 0x561bd1013710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f2855bee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561bd10187e0 con 0x561bd100b590 2024-09-19T04:26:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f2855bee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561bd0f2c0f0 con 0x561bd100ac70 2024-09-19T04:26:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f2855bee740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561bd1018960 con 0x561bd1015e50 2024-09-19T04:26:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f2855bee740 10 client.0 _ll_register_callbacks cb 0x561bd0f13bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:06.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f284be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 0x561bd1012c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f284be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 0x561bd1012c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47724/0 (socket says 172.21.15.107:47724) 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f284be00640 1 -- 172.21.15.107:0/3030061532 learned_addr learned my addr 172.21.15.107:0/3030061532 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f2843400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 0x561bd10131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f2851000640 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 0x561bd1013710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f284be00640 1 -- 172.21.15.107:0/3030061532 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 msgr2=0x561bd1013710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f284be00640 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 0x561bd1013710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f284be00640 1 -- 172.21.15.107:0/3030061532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 msgr2=0x561bd10131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f284be00640 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 0x561bd10131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f284be00640 1 -- 172.21.15.107:0/3030061532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd0ce23c0 con 0x561bd100ac70 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.998+0000 7f2851000640 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 0x561bd1013710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f284be00640 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 0x561bd1012c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f283c0ed840 tx=0x7f283c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea51c67f94527cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:06.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f283a000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:26:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f283a000640 1 -- 172.21.15.107:0/3030061532 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c049070 con 0x561bd100ac70 2024-09-19T04:26:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f283a000640 1 -- 172.21.15.107:0/3030061532 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f283c0f3cf0 con 0x561bd100ac70 2024-09-19T04:26:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f2855bee740 1 -- 172.21.15.107:0/3030061532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283c0fd030 con 0x561bd100ac70 2024-09-19T04:26:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f283a000640 1 -- 172.21.15.107:0/3030061532 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f283c0bbb50 con 0x561bd100ac70 2024-09-19T04:26:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f2855bee740 10 client.26258 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f2855bee740 1 -- 172.21.15.107:0/3030061532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561bd10187e0 con 0x561bd100ac70 2024-09-19T04:26:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f283a000640 1 -- 172.21.15.107:0/3030061532 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283c0fd030 con 0x561bd100ac70 2024-09-19T04:26:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f283a000640 1 -- 172.21.15.107:0/3030061532 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f283c0eda10 con 0x561bd100ac70 2024-09-19T04:26:06.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:05.999+0000 7f2855bee740 10 client.26258 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.000+0000 7f2855bee740 10 client.26258 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.000+0000 7f2855bee740 20 client.26258 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.000+0000 7f2855bee740 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x561bd1025a10 0x561bd1027e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.000+0000 7f2855bee740 4 client.26258 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.000+0000 7f2855bee740 1 -- 172.21.15.107:0/3030061532 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd0f7e950 con 0x561bd1025a10 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.000+0000 7f2843400640 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x561bd1025a10 0x561bd1027e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.001+0000 7f2843400640 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x561bd1025a10 0x561bd1027e70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7707156207ff16aa server_cookie=ce087491e21c4f6 in_seq=0 out_seq=0 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.001+0000 7f283a000640 10 client.26258 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.001+0000 7f283a000640 1 -- 172.21.15.107:0/3030061532 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561bd0f7e950 con 0x561bd1025a10 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.001+0000 7f283a000640 10 client.26258 handle_command_reply: tid=0 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.001+0000 7f2855bee740 1 client.26258 shutdown 2024-09-19T04:26:06.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 1 -- 172.21.15.107:0/3030061532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 msgr2=0x561bd1012c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 0x561bd1012c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f283c0ed840 tx=0x7f283c005c90 comp rx=0 tx=0).stop 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 1 -- 172.21.15.107:0/3030061532 shutdown_connections 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x561bd1025a10 0x561bd1027e70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd1015e50 0x561bd1013710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 0x561bd10131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 1 --2- 172.21.15.107:0/3030061532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 0x561bd1012c90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f283a000640 0 client.26258 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 1 -- 172.21.15.107:0/3030061532 >> 172.21.15.107:0/3030061532 conn(0x561bd1007200 msgr2=0x561bd100cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 1 -- 172.21.15.107:0/3030061532 shutdown_connections 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 1 -- 172.21.15.107:0/3030061532 wait complete. 2024-09-19T04:26:06.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.002+0000 7f2855bee740 20 client.26258 trim_cache size 0 max 16384 2024-09-19T04:26:06.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.015+0000 7f2855bee740 1 Processor -- start 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.015+0000 7f2855bee740 1 -- start start 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2855bee740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100ac70 0x561bd100d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2855bee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 0x561bd1005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2855bee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd1015e50 0x561bd102d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2855bee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b3dd0 con 0x561bd100b590 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2855bee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b5a30 con 0x561bd1015e50 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2855bee740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b44a0 con 0x561bd100ac70 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2843400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 0x561bd1005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2843400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 0x561bd1005b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35876/0 (socket says 172.21.15.107:35876) 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2843400640 1 -- 172.21.15.107:0/1291889147 learned_addr learned my addr 172.21.15.107:0/1291889147 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f284be00640 1 --2- 172.21.15.107:0/1291889147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100ac70 0x561bd100d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2851000640 1 --2- 172.21.15.107:0/1291889147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd1015e50 0x561bd102d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2843400640 1 -- 172.21.15.107:0/1291889147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100ac70 msgr2=0x561bd100d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:06.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2843400640 1 --2- 172.21.15.107:0/1291889147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100ac70 0x561bd100d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2843400640 1 -- 172.21.15.107:0/1291889147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd1015e50 msgr2=0x561bd102d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:06.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2843400640 1 --2- 172.21.15.107:0/1291889147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd1015e50 0x561bd102d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f2843400640 1 -- 172.21.15.107:0/1291889147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd1007030 con 0x561bd100b590 2024-09-19T04:26:06.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.016+0000 7f284be00640 1 --2- 172.21.15.107:0/1291889147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100ac70 0x561bd100d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:06.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.017+0000 7f2843400640 1 --2- 172.21.15.107:0/1291889147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 0x561bd1005b60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2834000c00 tx=0x7f2834005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2b37f85622b7ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:06.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.017+0000 7f2840c00640 1 -- 172.21.15.107:0/1291889147 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834022020 con 0x561bd100b590 2024-09-19T04:26:06.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.017+0000 7f2840c00640 1 -- 172.21.15.107:0/1291889147 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2834010910 con 0x561bd100b590 2024-09-19T04:26:06.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.017+0000 7f2840c00640 1 -- 172.21.15.107:0/1291889147 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283401e960 con 0x561bd100b590 2024-09-19T04:26:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 -- 172.21.15.107:0/1291889147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 msgr2=0x561bd1005b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 --2- 172.21.15.107:0/1291889147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 0x561bd1005b60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2834000c00 tx=0x7f2834005f90 comp rx=0 tx=0).stop 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 -- 172.21.15.107:0/1291889147 shutdown_connections 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 --2- 172.21.15.107:0/1291889147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd1015e50 0x561bd102d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 --2- 172.21.15.107:0/1291889147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd100b590 0x561bd1005b60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 --2- 172.21.15.107:0/1291889147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100ac70 0x561bd100d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 -- 172.21.15.107:0/1291889147 >> 172.21.15.107:0/1291889147 conn(0x561bd1007200 msgr2=0x561bd1014d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 -- 172.21.15.107:0/1291889147 shutdown_connections 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 -- 172.21.15.107:0/1291889147 wait complete. 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 -- start start 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.018+0000 7f2855bee740 1 Processor -- start 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2855bee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 0x561bd1019ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2855bee740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100b590 0x561bd101a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2855bee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd1015e50 0x561bd101a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2855bee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b52a0 con 0x561bd1015e50 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2855bee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b4b70 con 0x561bd100ac70 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2855bee740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b1db0 con 0x561bd100b590 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2855bee740 10 client.0 _ll_register_callbacks cb 0x7f2844009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2851000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd1015e50 0x561bd101a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2851000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd1015e50 0x561bd101a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35878/0 (socket says 172.21.15.107:35878) 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2851000640 1 -- 172.21.15.107:0/818567158 learned_addr learned my addr 172.21.15.107:0/818567158 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f2843400640 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100b590 0x561bd101a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.019+0000 7f284be00640 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 0x561bd1019ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f2851000640 1 -- 172.21.15.107:0/818567158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100b590 msgr2=0x561bd101a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:06.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f2851000640 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100b590 0x561bd101a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f2851000640 1 -- 172.21.15.107:0/818567158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 msgr2=0x561bd1019ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f2851000640 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 0x561bd1019ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f2851000640 1 -- 172.21.15.107:0/818567158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd1002770 con 0x561bd1015e50 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f2851000640 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd1015e50 0x561bd101a580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2844011e80 tx=0x7f2844004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bade64cdd1f4d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f283a000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f283a000640 1 -- 172.21.15.107:0/818567158 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284402d060 con 0x561bd1015e50 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f2855bee740 1 -- 172.21.15.107:0/818567158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2834010910 con 0x561bd1015e50 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f283a000640 1 -- 172.21.15.107:0/818567158 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f2844010930 con 0x561bd1015e50 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f283a000640 1 -- 172.21.15.107:0/818567158 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2844010b30 con 0x561bd1015e50 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f2855bee740 10 client.16212 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.020+0000 7f2855bee740 1 -- 172.21.15.107:0/818567158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561bd1090b20 con 0x561bd1015e50 2024-09-19T04:26:06.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.021+0000 7f283a000640 1 -- 172.21.15.107:0/818567158 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2834010910 con 0x561bd1015e50 2024-09-19T04:26:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.021+0000 7f283a000640 1 -- 172.21.15.107:0/818567158 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f2844027070 con 0x561bd1015e50 2024-09-19T04:26:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.021+0000 7f2855bee740 10 client.16212 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.021+0000 7f2855bee740 10 client.16212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:26:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.021+0000 7f2855bee740 20 client.16212 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.021+0000 7f2855bee740 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x561bd10912a0 0x561bd10936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.021+0000 7f2855bee740 4 client.16212 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.021+0000 7f2855bee740 1 -- 172.21.15.107:0/818567158 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bd0f78e60 con 0x561bd10912a0 2024-09-19T04:26:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.021+0000 7f284be00640 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x561bd10912a0 0x561bd10936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f284be00640 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x561bd10912a0 0x561bd10936c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3e9ef3f37107330 server_cookie=d47d6f231c247391 in_seq=0 out_seq=0 2024-09-19T04:26:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f283a000640 10 client.16212 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:26:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f283a000640 1 -- 172.21.15.107:0/818567158 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bd0f78e60 con 0x561bd10912a0 2024-09-19T04:26:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f283a000640 10 client.16212 handle_command_reply: tid=0 2024-09-19T04:26:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f2855bee740 1 client.16212 shutdown 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f2855bee740 1 -- 172.21.15.107:0/818567158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd1015e50 msgr2=0x561bd101a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f2855bee740 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd1015e50 0x561bd101a580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2844011e80 tx=0x7f2844004ab0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f2855bee740 1 -- 172.21.15.107:0/818567158 shutdown_connections 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f2855bee740 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x561bd10912a0 0x561bd10936c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f2855bee740 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bd1015e50 0x561bd101a580 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f2855bee740 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bd100b590 0x561bd101a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f2855bee740 1 --2- 172.21.15.107:0/818567158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bd100ac70 0x561bd1019ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f2855bee740 1 -- 172.21.15.107:0/818567158 >> 172.21.15.107:0/818567158 conn(0x561bd1007200 msgr2=0x561bd1020930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.022+0000 7f283a000640 0 client.16212 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2855bee740 1 -- 172.21.15.107:0/818567158 shutdown_connections 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2855bee740 1 -- 172.21.15.107:0/818567158 wait complete. 2024-09-19T04:26:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2855bee740 20 client.16212 trim_cache size 0 max 16384 2024-09-19T04:26:06.029 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:26:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2853000640 1 -- 172.21.15.107:0/4007610764 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28300a33b0 msgr2=0x7f28300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2853000640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28300a33b0 0x7f28300a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2844009fd0 tx=0x7f2844009500 comp rx=0 tx=0).stop 2024-09-19T04:26:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2853000640 1 -- 172.21.15.107:0/4007610764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f284c104010 msgr2=0x7f284c110b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2853000640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f284c104010 0x7f284c110b60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f283c006010 tx=0x7f283c004040 comp rx=0 tx=0).stop 2024-09-19T04:26:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2853000640 1 -- 172.21.15.107:0/4007610764 shutdown_connections 2024-09-19T04:26:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2853000640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28300a33b0 0x7f28300a5870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2853000640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f284c109140 0x7f284c10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2853000640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c1049c0 0x7f284c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2853000640 1 --2- 172.21.15.107:0/4007610764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f284c104010 0x7f284c110b60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.023+0000 7f2853000640 1 -- 172.21.15.107:0/4007610764 >> 172.21.15.107:0/4007610764 conn(0x7f284c0ffa50 msgr2=0x7f284c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.024+0000 7f2853000640 1 -- 172.21.15.107:0/4007610764 shutdown_connections 2024-09-19T04:26:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:06.024+0000 7f2853000640 1 -- 172.21.15.107:0/4007610764 wait complete. 2024-09-19T04:26:06.672 INFO:tasks.workunit.client.0.smithi107.stdout: 1 204435 2.83 MB/sec execute 144 sec latency 1611.478 ms 2024-09-19T04:26:06.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:26:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:07.427 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:07 smithi107 ceph-mon[30659]: pgmap v176: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 822 KiB/s wr, 136 op/s 2024-09-19T04:26:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:07 smithi115 ceph-mon[35477]: pgmap v176: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 822 KiB/s wr, 136 op/s 2024-09-19T04:26:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:07 smithi145 ceph-mon[35096]: pgmap v176: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 822 KiB/s wr, 136 op/s 2024-09-19T04:26:07.672 INFO:tasks.workunit.client.0.smithi107.stdout: 1 206096 2.90 MB/sec execute 145 sec latency 11.120 ms 2024-09-19T04:26:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.878+0000 7f848fe00640 1 -- 172.21.15.107:0/1569450962 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8488106b80 msgr2=0x7f8488106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.878+0000 7f848fe00640 1 --2- 172.21.15.107:0/1569450962 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8488106b80 0x7f8488106f60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f847c009890 tx=0x7f847c02a380 comp rx=0 tx=0).stop 2024-09-19T04:26:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.878+0000 7f848fe00640 1 -- 172.21.15.107:0/1569450962 shutdown_connections 2024-09-19T04:26:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.878+0000 7f848fe00640 1 --2- 172.21.15.107:0/1569450962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8488100500 0x7f848810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.878+0000 7f848fe00640 1 --2- 172.21.15.107:0/1569450962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84880ffb60 0x7f84880fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.878+0000 7f848fe00640 1 --2- 172.21.15.107:0/1569450962 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8488106b80 0x7f8488106f60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.878+0000 7f848fe00640 1 -- 172.21.15.107:0/1569450962 >> 172.21.15.107:0/1569450962 conn(0x7f84880fba90 msgr2=0x7f84880fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.878+0000 7f848fe00640 1 -- 172.21.15.107:0/1569450962 shutdown_connections 2024-09-19T04:26:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.878+0000 7f848fe00640 1 -- 172.21.15.107:0/1569450962 wait complete. 2024-09-19T04:26:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.879+0000 7f848fe00640 1 Processor -- start 2024-09-19T04:26:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.879+0000 7f848fe00640 1 -- start start 2024-09-19T04:26:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.879+0000 7f848fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84880ffb60 0x7f848819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.879+0000 7f848fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8488100500 0x7f848819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.879+0000 7f848fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8488106b80 0x7f848819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.879+0000 7f848fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8488111f60 con 0x7f8488106b80 2024-09-19T04:26:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.879+0000 7f848fe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8488111de0 con 0x7f84880ffb60 2024-09-19T04:26:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.879+0000 7f848fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f84881120e0 con 0x7f8488100500 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8488100500 0x7f848819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84880ffb60 0x7f848819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8488100500 0x7f848819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42748/0 (socket says 172.21.15.107:42748) 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848ca00640 1 -- 172.21.15.107:0/644468880 learned_addr learned my addr 172.21.15.107:0/644468880 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848de00640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8488106b80 0x7f848819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848d400640 1 -- 172.21.15.107:0/644468880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8488100500 msgr2=0x7f848819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848d400640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8488100500 0x7f848819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848d400640 1 -- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8488106b80 msgr2=0x7f848819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848d400640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8488106b80 0x7f848819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848d400640 1 -- 172.21.15.107:0/644468880 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848819f5d0 con 0x7f84880ffb60 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848de00640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8488106b80 0x7f848819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.880+0000 7f848d400640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84880ffb60 0x7f848819a620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f847c0057c0 tx=0x7f847c033ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.881+0000 7f8486000640 1 -- 172.21.15.107:0/644468880 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c037040 con 0x7f84880ffb60 2024-09-19T04:26:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.881+0000 7f8486000640 1 -- 172.21.15.107:0/644468880 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847c033d90 con 0x7f84880ffb60 2024-09-19T04:26:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.881+0000 7f848fe00640 1 -- 172.21.15.107:0/644468880 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848819f860 con 0x7f84880ffb60 2024-09-19T04:26:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.881+0000 7f8486000640 1 -- 172.21.15.107:0/644468880 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c003970 con 0x7f84880ffb60 2024-09-19T04:26:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.881+0000 7f848fe00640 1 -- 172.21.15.107:0/644468880 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84881a71a0 con 0x7f84880ffb60 2024-09-19T04:26:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.884+0000 7f8486000640 1 -- 172.21.15.107:0/644468880 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f847c042020 con 0x7f84880ffb60 2024-09-19T04:26:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.884+0000 7f84927d6740 1 Processor -- start 2024-09-19T04:26:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.884+0000 7f84927d6740 1 -- start start 2024-09-19T04:26:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f84927d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 0x5570dd575050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f84927d6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd575590 0x5570dd57f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f84927d6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd57fe50 0x5570dd582240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f84927d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5570dd4db3e0 con 0x5570dd574c70 2024-09-19T04:26:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f84927d6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5570dd482d80 con 0x5570dd57fe50 2024-09-19T04:26:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f84927d6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5570dd4d73a0 con 0x5570dd575590 2024-09-19T04:26:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f848d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd575590 0x5570dd57f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f848d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd575590 0x5570dd57f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42758/0 (socket says 172.21.15.107:42758) 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f848d400640 1 -- 172.21.15.107:0/1523987086 learned_addr learned my addr 172.21.15.107:0/1523987086 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f848de00640 1 --2- 172.21.15.107:0/1523987086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 0x5570dd575050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.885+0000 7f848ca00640 1 --2- 172.21.15.107:0/1523987086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd57fe50 0x5570dd582240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f848d400640 1 -- 172.21.15.107:0/1523987086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd57fe50 msgr2=0x5570dd582240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f848d400640 1 --2- 172.21.15.107:0/1523987086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd57fe50 0x5570dd582240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f848d400640 1 -- 172.21.15.107:0/1523987086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 msgr2=0x5570dd575050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f848d400640 1 --2- 172.21.15.107:0/1523987086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 0x5570dd575050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f848d400640 1 -- 172.21.15.107:0/1523987086 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570dd4a1ae0 con 0x5570dd575590 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f848ca00640 1 --2- 172.21.15.107:0/1523987086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd57fe50 0x5570dd582240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f848d400640 1 --2- 172.21.15.107:0/1523987086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd575590 0x5570dd57f910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f847c0ec9d0 tx=0x7f847c0f7330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=605269b883eda3e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f8486000640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84700a3430 0x7f84700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f848de00640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84700a3430 0x7f84700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f847a600640 1 -- 172.21.15.107:0/1523987086 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0b78c0 con 0x5570dd575590 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f847a600640 1 -- 172.21.15.107:0/1523987086 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847c0fc030 con 0x5570dd575590 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f8486000640 1 -- 172.21.15.107:0/644468880 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f847c0ef240 con 0x7f84880ffb60 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.886+0000 7f847a600640 1 -- 172.21.15.107:0/1523987086 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0beeb0 con 0x5570dd575590 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.887+0000 7f84927d6740 1 -- 172.21.15.107:0/1523987086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd575590 msgr2=0x5570dd57f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.887+0000 7f84927d6740 1 --2- 172.21.15.107:0/1523987086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd575590 0x5570dd57f910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f847c0ec9d0 tx=0x7f847c0f7330 comp rx=0 tx=0).stop 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.887+0000 7f848de00640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84700a3430 0x7f84700a58f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f84881077c0 tx=0x7f8480011040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.887+0000 7f84927d6740 1 -- 172.21.15.107:0/1523987086 shutdown_connections 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.887+0000 7f84927d6740 1 --2- 172.21.15.107:0/1523987086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd57fe50 0x5570dd582240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.887+0000 7f84927d6740 1 --2- 172.21.15.107:0/1523987086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd575590 0x5570dd57f910 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.887+0000 7f84927d6740 1 --2- 172.21.15.107:0/1523987086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 0x5570dd575050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.887+0000 7f84927d6740 1 -- 172.21.15.107:0/1523987086 >> 172.21.15.107:0/1523987086 conn(0x5570dd571200 msgr2=0x5570dd5715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.887+0000 7f84927d6740 1 -- 172.21.15.107:0/1523987086 shutdown_connections 2024-09-19T04:26:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.887+0000 7f84927d6740 1 -- 172.21.15.107:0/1523987086 wait complete. 2024-09-19T04:26:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.888+0000 7f84927d6740 1 -- start start 2024-09-19T04:26:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.888+0000 7f84927d6740 1 Processor -- start 2024-09-19T04:26:07.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.888+0000 7f84927d6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd57cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.888+0000 7f84927d6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd57d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.888+0000 7f84927d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd57fe50 0x5570dd57d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.888+0000 7f84927d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5570dd5827e0 con 0x5570dd57fe50 2024-09-19T04:26:07.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.888+0000 7f84927d6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5570dd4960f0 con 0x5570dd575590 2024-09-19T04:26:07.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.888+0000 7f84927d6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5570dd582960 con 0x5570dd574c70 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f84927d6740 10 client.0 _ll_register_callbacks cb 0x5570dd47dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd57cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd57d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd57cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42774/0 (socket says 172.21.15.107:42774) 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848d400640 1 -- 172.21.15.107:0/2861241885 learned_addr learned my addr 172.21.15.107:0/2861241885 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848de00640 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd57fe50 0x5570dd57d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848d400640 1 -- 172.21.15.107:0/2861241885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 msgr2=0x5570dd57d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848d400640 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd57d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848d400640 1 -- 172.21.15.107:0/2861241885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd57fe50 msgr2=0x5570dd57d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848d400640 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd57fe50 0x5570dd57d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848d400640 1 -- 172.21.15.107:0/2861241885 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570dd24c3c0 con 0x5570dd574c70 2024-09-19T04:26:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848ca00640 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd57d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f848d400640 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd57cca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f847c0f7830 tx=0x7f847c030980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=544c45f365da784 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.889+0000 7f846ea00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f846ea00640 1 -- 172.21.15.107:0/2861241885 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0bf110 con 0x5570dd574c70 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f84927d6740 1 -- 172.21.15.107:0/2861241885 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f847c0fc030 con 0x5570dd574c70 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f846ea00640 1 -- 172.21.15.107:0/2861241885 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f847c038c50 con 0x5570dd574c70 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f846ea00640 1 -- 172.21.15.107:0/2861241885 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f847c0bf770 con 0x5570dd574c70 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f84927d6740 10 client.26236 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f84927d6740 1 -- 172.21.15.107:0/2861241885 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5570dd5827e0 con 0x5570dd574c70 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f846ea00640 1 -- 172.21.15.107:0/2861241885 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847c0fc030 con 0x5570dd574c70 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f846ea00640 1 -- 172.21.15.107:0/2861241885 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f847c03a070 con 0x5570dd574c70 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f84927d6740 10 client.26236 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f84927d6740 10 client.26236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:26:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.890+0000 7f84927d6740 20 client.26236 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.891+0000 7f84927d6740 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5570dd58f960 0x5570dd591db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.891+0000 7f84927d6740 4 client.26236 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.891+0000 7f84927d6740 1 -- 172.21.15.107:0/2861241885 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570dd4e8950 con 0x5570dd58f960 2024-09-19T04:26:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.891+0000 7f848ca00640 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5570dd58f960 0x5570dd591db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.891+0000 7f848ca00640 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5570dd58f960 0x5570dd591db0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef81aa289591b104 server_cookie=e5eb4cb2de474112 in_seq=0 out_seq=0 2024-09-19T04:26:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.891+0000 7f846ea00640 10 client.26236 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:26:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f846ea00640 1 -- 172.21.15.107:0/2861241885 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570dd4e8950 con 0x5570dd58f960 2024-09-19T04:26:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f846ea00640 10 client.26236 handle_command_reply: tid=0 2024-09-19T04:26:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f84927d6740 1 client.26236 shutdown 2024-09-19T04:26:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f84927d6740 1 -- 172.21.15.107:0/2861241885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 msgr2=0x5570dd57cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f84927d6740 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd57cca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f847c0f7830 tx=0x7f847c030980 comp rx=0 tx=0).stop 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f84927d6740 1 -- 172.21.15.107:0/2861241885 shutdown_connections 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f84927d6740 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5570dd58f960 0x5570dd591db0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f84927d6740 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd57fe50 0x5570dd57d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f846ea00640 0 client.26236 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f84927d6740 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd57d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f84927d6740 1 --2- 172.21.15.107:0/2861241885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd57cca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.892+0000 7f84927d6740 1 -- 172.21.15.107:0/2861241885 >> 172.21.15.107:0/2861241885 conn(0x5570dd571200 msgr2=0x5570dd576f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.893+0000 7f84927d6740 1 -- 172.21.15.107:0/2861241885 shutdown_connections 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.893+0000 7f84927d6740 1 -- 172.21.15.107:0/2861241885 wait complete. 2024-09-19T04:26:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.893+0000 7f84927d6740 20 client.26236 trim_cache size 0 max 16384 2024-09-19T04:26:07.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f84927d6740 1 Processor -- start 2024-09-19T04:26:07.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f84927d6740 1 -- start start 2024-09-19T04:26:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f84927d6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd597680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f84927d6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd597bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f84927d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd57fe50 0x5570dd56fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f84927d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84881afa00 con 0x5570dd57fe50 2024-09-19T04:26:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f84927d6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84881b1660 con 0x5570dd575590 2024-09-19T04:26:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f84927d6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f84881b00d0 con 0x5570dd574c70 2024-09-19T04:26:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f848d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd597680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f848d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd597680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42782/0 (socket says 172.21.15.107:42782) 2024-09-19T04:26:07.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f848d400640 1 -- 172.21.15.107:0/3098462766 learned_addr learned my addr 172.21.15.107:0/3098462766 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:07.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f848de00640 1 --2- 172.21.15.107:0/3098462766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd57fe50 0x5570dd56fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.908+0000 7f848ca00640 1 --2- 172.21.15.107:0/3098462766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd597bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.909+0000 7f848d400640 1 -- 172.21.15.107:0/3098462766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 msgr2=0x5570dd597bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.909+0000 7f848d400640 1 --2- 172.21.15.107:0/3098462766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd597bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.909+0000 7f848d400640 1 -- 172.21.15.107:0/3098462766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd57fe50 msgr2=0x5570dd56fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.909+0000 7f848d400640 1 --2- 172.21.15.107:0/3098462766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd57fe50 0x5570dd56fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.909+0000 7f848d400640 1 -- 172.21.15.107:0/3098462766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570dd571030 con 0x5570dd574c70 2024-09-19T04:26:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.909+0000 7f848d400640 1 --2- 172.21.15.107:0/3098462766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd597680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f847c0b7ff0 tx=0x7f847c0b9b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a5710c9eac2f788 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.909+0000 7f8479c00640 1 -- 172.21.15.107:0/3098462766 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0ba320 con 0x5570dd574c70 2024-09-19T04:26:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.909+0000 7f8479c00640 1 -- 172.21.15.107:0/3098462766 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847c0fc070 con 0x5570dd574c70 2024-09-19T04:26:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.909+0000 7f8479c00640 1 -- 172.21.15.107:0/3098462766 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0baf80 con 0x5570dd574c70 2024-09-19T04:26:07.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.910+0000 7f84927d6740 1 -- 172.21.15.107:0/3098462766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 msgr2=0x5570dd597680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:07.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.910+0000 7f84927d6740 1 --2- 172.21.15.107:0/3098462766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd597680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f847c0b7ff0 tx=0x7f847c0b9b30 comp rx=0 tx=0).stop 2024-09-19T04:26:07.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.910+0000 7f84927d6740 1 -- 172.21.15.107:0/3098462766 shutdown_connections 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.910+0000 7f84927d6740 1 --2- 172.21.15.107:0/3098462766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd57fe50 0x5570dd56fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.910+0000 7f84927d6740 1 --2- 172.21.15.107:0/3098462766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd597bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.910+0000 7f84927d6740 1 --2- 172.21.15.107:0/3098462766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd574c70 0x5570dd597680 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.910+0000 7f84927d6740 1 -- 172.21.15.107:0/3098462766 >> 172.21.15.107:0/3098462766 conn(0x5570dd571200 msgr2=0x5570dd57bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.910+0000 7f84927d6740 1 -- 172.21.15.107:0/3098462766 shutdown_connections 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.910+0000 7f84927d6740 1 -- 172.21.15.107:0/3098462766 wait complete. 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.911+0000 7f84927d6740 1 -- start start 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.911+0000 7f84927d6740 1 Processor -- start 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.911+0000 7f84927d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 0x5570dd587e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.911+0000 7f84927d6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd588360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.911+0000 7f84927d6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd57fe50 0x5570dd579690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.911+0000 7f84927d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84881b0ed0 con 0x5570dd574c70 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.911+0000 7f84927d6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84881b07a0 con 0x5570dd575590 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.911+0000 7f84927d6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f84881ad9e0 con 0x5570dd57fe50 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.911+0000 7f84927d6740 10 client.0 _ll_register_callbacks cb 0x7f8480009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 0x5570dd587e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd57fe50 0x5570dd579690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd57fe50 0x5570dd579690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42794/0 (socket says 172.21.15.107:42794) 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd588360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848de00640 1 -- 172.21.15.107:0/169891549 learned_addr learned my addr 172.21.15.107:0/169891549 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848de00640 1 -- 172.21.15.107:0/169891549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 msgr2=0x5570dd588360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848de00640 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd588360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848de00640 1 -- 172.21.15.107:0/169891549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 msgr2=0x5570dd587e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848de00640 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 0x5570dd587e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848de00640 1 -- 172.21.15.107:0/169891549 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570dd5723b0 con 0x5570dd57fe50 2024-09-19T04:26:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848d400640 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 0x5570dd587e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f848de00640 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd57fe50 0x5570dd579690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f848001b3f0 tx=0x7f8480004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=938d79660c06ac61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.912+0000 7f846ea00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:26:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f846ea00640 1 -- 172.21.15.107:0/169891549 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480004d20 con 0x5570dd57fe50 2024-09-19T04:26:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f846ea00640 1 -- 172.21.15.107:0/169891549 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f8480003970 con 0x5570dd57fe50 2024-09-19T04:26:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f84927d6740 1 -- 172.21.15.107:0/169891549 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f847c0fc070 con 0x5570dd57fe50 2024-09-19T04:26:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f846ea00640 1 -- 172.21.15.107:0/169891549 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8480035050 con 0x5570dd57fe50 2024-09-19T04:26:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f84927d6740 10 client.26242 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f84927d6740 1 -- 172.21.15.107:0/169891549 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5570dd56c770 con 0x5570dd57fe50 2024-09-19T04:26:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f846ea00640 1 -- 172.21.15.107:0/169891549 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847c0fc070 con 0x5570dd57fe50 2024-09-19T04:26:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f846ea00640 1 -- 172.21.15.107:0/169891549 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f848000b040 con 0x5570dd57fe50 2024-09-19T04:26:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f84927d6740 10 client.26242 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f84927d6740 10 client.26242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:26:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f84927d6740 20 client.26242 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f84927d6740 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5570dd5fa660 0x5570dd5fca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f84927d6740 4 client.26242 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.913+0000 7f84927d6740 1 -- 172.21.15.107:0/169891549 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570dd4e2e60 con 0x5570dd5fa660 2024-09-19T04:26:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.914+0000 7f848d400640 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5570dd5fa660 0x5570dd5fca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.914+0000 7f848d400640 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5570dd5fa660 0x5570dd5fca80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9dda280bc39024ee server_cookie=1420e64aa75d97e in_seq=0 out_seq=0 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.914+0000 7f846ea00640 10 client.26242 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.914+0000 7f846ea00640 1 -- 172.21.15.107:0/169891549 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570dd4e2e60 con 0x5570dd5fa660 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.914+0000 7f846ea00640 10 client.26242 handle_command_reply: tid=0 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.914+0000 7f84927d6740 1 client.26242 shutdown 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.914+0000 7f84927d6740 1 -- 172.21.15.107:0/169891549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd57fe50 msgr2=0x5570dd579690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.914+0000 7f84927d6740 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd57fe50 0x5570dd579690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f848001b3f0 tx=0x7f8480004ab0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.915+0000 7f84927d6740 1 -- 172.21.15.107:0/169891549 shutdown_connections 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.915+0000 7f84927d6740 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5570dd5fa660 0x5570dd5fca80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.915+0000 7f84927d6740 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5570dd57fe50 0x5570dd579690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.915+0000 7f84927d6740 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570dd575590 0x5570dd588360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.915+0000 7f846ea00640 0 client.26242 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:26:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.915+0000 7f84927d6740 1 --2- 172.21.15.107:0/169891549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570dd574c70 0x5570dd587e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.915+0000 7f84927d6740 1 -- 172.21.15.107:0/169891549 >> 172.21.15.107:0/169891549 conn(0x5570dd571200 msgr2=0x5570dd5766c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.915+0000 7f84927d6740 1 -- 172.21.15.107:0/169891549 shutdown_connections 2024-09-19T04:26:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.915+0000 7f84927d6740 1 -- 172.21.15.107:0/169891549 wait complete. 2024-09-19T04:26:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.915+0000 7f84927d6740 20 client.26242 trim_cache size 0 max 16384 2024-09-19T04:26:07.921 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:26:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 -- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84700a3430 msgr2=0x7f84700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84700a3430 0x7f84700a58f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f84881077c0 tx=0x7f8480011040 comp rx=0 tx=0).stop 2024-09-19T04:26:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 -- 172.21.15.107:0/644468880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84880ffb60 msgr2=0x7f848819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84880ffb60 0x7f848819a620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f847c0057c0 tx=0x7f847c033ae0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 -- 172.21.15.107:0/644468880 shutdown_connections 2024-09-19T04:26:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84700a3430 0x7f84700a58f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8488106b80 0x7f848819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8488100500 0x7f848819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 --2- 172.21.15.107:0/644468880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84880ffb60 0x7f848819a620 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 -- 172.21.15.107:0/644468880 >> 172.21.15.107:0/644468880 conn(0x7f84880fba90 msgr2=0x7f848810cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 -- 172.21.15.107:0/644468880 shutdown_connections 2024-09-19T04:26:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:07.916+0000 7f848fe00640 1 -- 172.21.15.107:0/644468880 wait complete. 2024-09-19T04:26:08.672 INFO:tasks.workunit.client.0.smithi107.stdout: 1 206452 2.88 MB/sec execute 146 sec latency 607.141 ms 2024-09-19T04:26:08.853 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:26:09.235 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:09.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:09 smithi107 ceph-mon[30659]: pgmap v177: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 658 KiB/s wr, 103 op/s 2024-09-19T04:26:09.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:09 smithi115 ceph-mon[35477]: pgmap v177: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 658 KiB/s wr, 103 op/s 2024-09-19T04:26:09.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:09 smithi145 ceph-mon[35096]: pgmap v177: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 658 KiB/s wr, 103 op/s 2024-09-19T04:26:09.672 INFO:tasks.workunit.client.0.smithi107.stdout: 1 207217 2.89 MB/sec execute 147 sec latency 1154.696 ms 2024-09-19T04:26:09.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.837+0000 7f50fcc00640 1 -- 172.21.15.107:0/4099808559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50f810e260 msgr2=0x7f50f8110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.837+0000 7f50fcc00640 1 --2- 172.21.15.107:0/4099808559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50f810e260 0x7f50f8110650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009ca0 tx=0x7f50f002a950 comp rx=0 tx=0).stop 2024-09-19T04:26:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.837+0000 7f50fcc00640 1 -- 172.21.15.107:0/4099808559 shutdown_connections 2024-09-19T04:26:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.837+0000 7f50fcc00640 1 --2- 172.21.15.107:0/4099808559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50f810e260 0x7f50f8110650 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.837+0000 7f50fcc00640 1 --2- 172.21.15.107:0/4099808559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50f8074ee0 0x7f50f810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.837+0000 7f50fcc00640 1 --2- 172.21.15.107:0/4099808559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50f80745c0 0x7f50f80749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.837+0000 7f50fcc00640 1 -- 172.21.15.107:0/4099808559 >> 172.21.15.107:0/4099808559 conn(0x7f50f80fd6e0 msgr2=0x7f50f80ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.837+0000 7f50fcc00640 1 -- 172.21.15.107:0/4099808559 shutdown_connections 2024-09-19T04:26:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.837+0000 7f50fcc00640 1 -- 172.21.15.107:0/4099808559 wait complete. 2024-09-19T04:26:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50fcc00640 1 Processor -- start 2024-09-19T04:26:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50fcc00640 1 -- start start 2024-09-19T04:26:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50fcc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50f80745c0 0x7f50f819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50fcc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50f8074ee0 0x7f50f819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50fcc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50f810e260 0x7f50f819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50fcc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8113fb0 con 0x7f50f80745c0 2024-09-19T04:26:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50fcc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8113e30 con 0x7f50f8074ee0 2024-09-19T04:26:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50fcc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8114130 con 0x7f50f810e260 2024-09-19T04:26:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50f6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50f8074ee0 0x7f50f819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50f6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50f8074ee0 0x7f50f819a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37552/0 (socket says 172.21.15.107:37552) 2024-09-19T04:26:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.838+0000 7f50f6a00640 1 -- 172.21.15.107:0/800840854 learned_addr learned my addr 172.21.15.107:0/800840854 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.839+0000 7f50f7400640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50f80745c0 0x7f50f819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.839+0000 7f50f7e00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50f810e260 0x7f50f819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.839+0000 7f50f6a00640 1 -- 172.21.15.107:0/800840854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50f810e260 msgr2=0x7f50f819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.839+0000 7f50f6a00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50f810e260 0x7f50f819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.839+0000 7f50f6a00640 1 -- 172.21.15.107:0/800840854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50f80745c0 msgr2=0x7f50f819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.839+0000 7f50f6a00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50f80745c0 0x7f50f819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.839+0000 7f50f6a00640 1 -- 172.21.15.107:0/800840854 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f819f3e0 con 0x7f50f8074ee0 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.839+0000 7f50f6a00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50f8074ee0 0x7f50f819a970 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f50e400e9f0 tx=0x7f50e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.840+0000 7f50efe00640 1 -- 172.21.15.107:0/800840854 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e400cd80 con 0x7f50f8074ee0 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.840+0000 7f50efe00640 1 -- 172.21.15.107:0/800840854 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e4010910 con 0x7f50f8074ee0 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.840+0000 7f50efe00640 1 -- 172.21.15.107:0/800840854 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4018990 con 0x7f50f8074ee0 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.840+0000 7f50fcc00640 1 -- 172.21.15.107:0/800840854 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f819f6d0 con 0x7f50f8074ee0 2024-09-19T04:26:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.840+0000 7f50fcc00640 1 -- 172.21.15.107:0/800840854 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f81a6fb0 con 0x7f50f8074ee0 2024-09-19T04:26:09.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.843+0000 7f5100c07740 1 Processor -- start 2024-09-19T04:26:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.843+0000 7f5100c07740 1 -- start start 2024-09-19T04:26:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.843+0000 7f5100c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.843+0000 7f5100c07740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 0x5578407e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.843+0000 7f5100c07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.843+0000 7f5100c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5578407423e0 con 0x5578407dbe40 2024-09-19T04:26:09.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.843+0000 7f5100c07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5578406e9d80 con 0x5578407e7020 2024-09-19T04:26:09.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.843+0000 7f5100c07740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55784073e3a0 con 0x5578407dc760 2024-09-19T04:26:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.843+0000 7f50efe00640 1 -- 172.21.15.107:0/800840854 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f50e4010ac0 con 0x7f50f8074ee0 2024-09-19T04:26:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 0x5578407e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 0x5578407e6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37328/0 (socket says 172.21.15.107:37328) 2024-09-19T04:26:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7400640 1 -- 172.21.15.107:0/1972650291 learned_addr learned my addr 172.21.15.107:0/1972650291 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407dc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43414/0 (socket says 172.21.15.107:43414) 2024-09-19T04:26:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f6a00640 1 --2- 172.21.15.107:0/1972650291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7400640 1 -- 172.21.15.107:0/1972650291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 msgr2=0x5578407e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7400640 1 --2- 172.21.15.107:0/1972650291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7400640 1 -- 172.21.15.107:0/1972650291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 msgr2=0x5578407dc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7400640 1 --2- 172.21.15.107:0/1972650291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407dc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7400640 1 -- 172.21.15.107:0/1972650291 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557840708ae0 con 0x5578407dc760 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.844+0000 7f50f7400640 1 --2- 172.21.15.107:0/1972650291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 0x5578407e6ae0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009d20 tx=0x7f50e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7549caf6aa6762f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.845+0000 7f50e3e00640 1 -- 172.21.15.107:0/1972650291 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8018640 con 0x5578407dc760 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.845+0000 7f50e3e00640 1 -- 172.21.15.107:0/1972650291 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e8009e10 con 0x5578407dc760 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.845+0000 7f50efe00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f50dc0a3430 0x7f50dc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.845+0000 7f50e3e00640 1 -- 172.21.15.107:0/1972650291 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8010cf0 con 0x5578407dc760 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.845+0000 7f50efe00640 1 -- 172.21.15.107:0/800840854 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f50e40cfff0 con 0x7f50f8074ee0 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.845+0000 7f50f7e00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f50dc0a3430 0x7f50dc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.846+0000 7f5100c07740 1 -- 172.21.15.107:0/1972650291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 msgr2=0x5578407e6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.846+0000 7f5100c07740 1 --2- 172.21.15.107:0/1972650291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 0x5578407e6ae0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009d20 tx=0x7f50e800f790 comp rx=0 tx=0).stop 2024-09-19T04:26:09.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.846+0000 7f5100c07740 1 -- 172.21.15.107:0/1972650291 shutdown_connections 2024-09-19T04:26:09.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.846+0000 7f5100c07740 1 --2- 172.21.15.107:0/1972650291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.846+0000 7f5100c07740 1 --2- 172.21.15.107:0/1972650291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 0x5578407e6ae0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.846+0000 7f5100c07740 1 --2- 172.21.15.107:0/1972650291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407dc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.846+0000 7f5100c07740 1 -- 172.21.15.107:0/1972650291 >> 172.21.15.107:0/1972650291 conn(0x5578407d83d0 msgr2=0x5578407d87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:09.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.846+0000 7f50f7e00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f50dc0a3430 0x7f50dc0a58f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f50f0005e30 tx=0x7f50f0005c50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:09.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.846+0000 7f5100c07740 1 -- 172.21.15.107:0/1972650291 shutdown_connections 2024-09-19T04:26:09.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.846+0000 7f5100c07740 1 -- 172.21.15.107:0/1972650291 wait complete. 2024-09-19T04:26:09.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.847+0000 7f5100c07740 1 -- start start 2024-09-19T04:26:09.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.847+0000 7f5100c07740 1 Processor -- start 2024-09-19T04:26:09.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.847+0000 7f5100c07740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.847+0000 7f5100c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dc760 0x5578407e4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.847+0000 7f5100c07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.847+0000 7f5100c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5578407e9bf0 con 0x5578407dc760 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.847+0000 7f5100c07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5578406fd0f0 con 0x5578407e7020 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.847+0000 7f5100c07740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5578407e9d70 con 0x5578407dbe40 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.847+0000 7f5100c07740 10 client.0 _ll_register_callbacks cb 0x5578406e4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dc760 0x5578407e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dc760 0x5578407e4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43430/0 (socket says 172.21.15.107:43430) 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f6a00640 1 -- 172.21.15.107:0/2524472516 learned_addr learned my addr 172.21.15.107:0/2524472516 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f6a00640 1 -- 172.21.15.107:0/2524472516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 msgr2=0x5578407e3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f7e00640 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f7400640 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f6a00640 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407e3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f6a00640 1 -- 172.21.15.107:0/2524472516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 msgr2=0x5578407e4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f6a00640 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f6a00640 1 -- 172.21.15.107:0/2524472516 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578404b33c0 con 0x5578407dc760 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.848+0000 7f50f7e00640 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.849+0000 7f50f7400640 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407e3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.849+0000 7f50f6a00640 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dc760 0x5578407e4440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f50e400e9c0 tx=0x7f50e409c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6521f8f19e373a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.849+0000 7f50e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.849+0000 7f50e0c00640 1 -- 172.21.15.107:0/2524472516 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40cdbf0 con 0x5578407dc760 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.849+0000 7f50e0c00640 1 -- 172.21.15.107:0/2524472516 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f50e4018b30 con 0x5578407dc760 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.849+0000 7f5100c07740 1 -- 172.21.15.107:0/2524472516 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e8009e10 con 0x5578407dc760 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.849+0000 7f50e0c00640 1 -- 172.21.15.107:0/2524472516 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f50e40d22e0 con 0x5578407dc760 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.849+0000 7f5100c07740 10 client.16242 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.849+0000 7f5100c07740 1 -- 172.21.15.107:0/2524472516 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5578407f5c60 con 0x5578407dc760 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.850+0000 7f50e0c00640 1 -- 172.21.15.107:0/2524472516 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e8009e10 con 0x5578407dc760 2024-09-19T04:26:09.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.850+0000 7f50e0c00640 1 -- 172.21.15.107:0/2524472516 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f50e409b800 con 0x5578407dc760 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.850+0000 7f5100c07740 10 client.16242 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.850+0000 7f5100c07740 10 client.16242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.850+0000 7f5100c07740 20 client.16242 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.850+0000 7f5100c07740 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5578407f7120 0x5578407f9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.850+0000 7f5100c07740 4 client.16242 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.850+0000 7f5100c07740 1 -- 172.21.15.107:0/2524472516 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55784074f950 con 0x5578407f7120 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.850+0000 7f50f7400640 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5578407f7120 0x5578407f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.851+0000 7f50f7400640 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5578407f7120 0x5578407f9570 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b237765f659dda1a server_cookie=b2caa0b9ded4685a in_seq=0 out_seq=0 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.851+0000 7f50e0c00640 10 client.16242 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.851+0000 7f50e0c00640 1 -- 172.21.15.107:0/2524472516 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55784074f950 con 0x5578407f7120 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f50e0c00640 10 client.16242 handle_command_reply: tid=0 2024-09-19T04:26:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 client.16242 shutdown 2024-09-19T04:26:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 -- 172.21.15.107:0/2524472516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dc760 msgr2=0x5578407e4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dc760 0x5578407e4440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f50e400e9c0 tx=0x7f50e409c200 comp rx=0 tx=0).stop 2024-09-19T04:26:09.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 -- 172.21.15.107:0/2524472516 shutdown_connections 2024-09-19T04:26:09.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5578407f7120 0x5578407f9570 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dc760 0x5578407e4440 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 --2- 172.21.15.107:0/2524472516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407e3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f50e0c00640 0 client.16242 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:26:09.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 -- 172.21.15.107:0/2524472516 >> 172.21.15.107:0/2524472516 conn(0x5578407d83d0 msgr2=0x5578407de100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:09.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 -- 172.21.15.107:0/2524472516 shutdown_connections 2024-09-19T04:26:09.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.852+0000 7f5100c07740 1 -- 172.21.15.107:0/2524472516 wait complete. 2024-09-19T04:26:09.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.853+0000 7f5100c07740 20 client.16242 trim_cache size 0 max 16384 2024-09-19T04:26:09.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.867+0000 7f5100c07740 1 Processor -- start 2024-09-19T04:26:09.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.867+0000 7f5100c07740 1 -- start start 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f5100c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407fee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f5100c07740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 0x5578407ff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f5100c07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f5100c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ab500 con 0x5578407dbe40 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f5100c07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ad160 con 0x5578407e7020 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f5100c07740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81abbd0 con 0x5578407dc760 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407fee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43436/0 (socket says 172.21.15.107:43436) 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7400640 1 -- 172.21.15.107:0/673183574 learned_addr learned my addr 172.21.15.107:0/673183574 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37580/0 (socket says 172.21.15.107:37580) 2024-09-19T04:26:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f6a00640 1 --2- 172.21.15.107:0/673183574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 0x5578407ff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7400640 1 -- 172.21.15.107:0/673183574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 msgr2=0x5578407ff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7400640 1 --2- 172.21.15.107:0/673183574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 0x5578407ff3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7400640 1 -- 172.21.15.107:0/673183574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 msgr2=0x5578407e1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7400640 1 --2- 172.21.15.107:0/673183574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7400640 1 -- 172.21.15.107:0/673183574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578407dbc70 con 0x5578407dbe40 2024-09-19T04:26:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.868+0000 7f50f7e00640 1 --2- 172.21.15.107:0/673183574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.869+0000 7f50f7400640 1 --2- 172.21.15.107:0/673183574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407fee80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f50e800fc90 tx=0x7f50e8009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857fe5d409963fc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.869+0000 7f50e3400640 1 -- 172.21.15.107:0/673183574 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8010940 con 0x5578407dbe40 2024-09-19T04:26:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.869+0000 7f50e3400640 1 -- 172.21.15.107:0/673183574 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e8038070 con 0x5578407dbe40 2024-09-19T04:26:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.869+0000 7f50e3400640 1 -- 172.21.15.107:0/673183574 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8003850 con 0x5578407dbe40 2024-09-19T04:26:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.870+0000 7f5100c07740 1 -- 172.21.15.107:0/673183574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 msgr2=0x5578407fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.870+0000 7f5100c07740 1 --2- 172.21.15.107:0/673183574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407fee80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f50e800fc90 tx=0x7f50e8009660 comp rx=0 tx=0).stop 2024-09-19T04:26:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.870+0000 7f5100c07740 1 -- 172.21.15.107:0/673183574 shutdown_connections 2024-09-19T04:26:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.870+0000 7f5100c07740 1 --2- 172.21.15.107:0/673183574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407e7020 0x5578407e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.870+0000 7f5100c07740 1 --2- 172.21.15.107:0/673183574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dc760 0x5578407ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.870+0000 7f5100c07740 1 --2- 172.21.15.107:0/673183574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407dbe40 0x5578407fee80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.870+0000 7f5100c07740 1 -- 172.21.15.107:0/673183574 >> 172.21.15.107:0/673183574 conn(0x5578407d83d0 msgr2=0x5578407d87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.870+0000 7f5100c07740 1 -- 172.21.15.107:0/673183574 shutdown_connections 2024-09-19T04:26:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.870+0000 7f5100c07740 1 -- 172.21.15.107:0/673183574 wait complete. 2024-09-19T04:26:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.870+0000 7f5100c07740 1 -- start start 2024-09-19T04:26:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f5100c07740 1 Processor -- start 2024-09-19T04:26:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f5100c07740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407ef860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f5100c07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407dc760 0x5578407eab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f5100c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407e7020 0x5578407eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f5100c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ac9d0 con 0x5578407e7020 2024-09-19T04:26:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f5100c07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ac2a0 con 0x5578407dc760 2024-09-19T04:26:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f5100c07740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81a94e0 con 0x5578407dbe40 2024-09-19T04:26:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f5100c07740 10 client.0 _ll_register_callbacks cb 0x7f50e400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f50f7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f50f7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407ef860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37342/0 (socket says 172.21.15.107:37342) 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f50f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407e7020 0x5578407eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f50f7400640 1 -- 172.21.15.107:0/1230655606 learned_addr learned my addr 172.21.15.107:0/1230655606 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.871+0000 7f50f6a00640 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407dc760 0x5578407eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f50f7400640 1 -- 172.21.15.107:0/1230655606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407dc760 msgr2=0x5578407eab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f50f7400640 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407dc760 0x5578407eab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f50f7400640 1 -- 172.21.15.107:0/1230655606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407e7020 msgr2=0x5578407eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f50f7400640 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407e7020 0x5578407eb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f50f7400640 1 -- 172.21.15.107:0/1230655606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578407eb9b0 con 0x5578407dbe40 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f50f6a00640 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407dc760 0x5578407eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f50f7400640 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407ef860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009250 tx=0x7f50e8009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d60eef6f84fd4f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f50e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f50e0c00640 1 -- 172.21.15.107:0/1230655606 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e80050f0 con 0x5578407dbe40 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f50e0c00640 1 -- 172.21.15.107:0/1230655606 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f50e8005290 con 0x5578407dbe40 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.872+0000 7f5100c07740 1 -- 172.21.15.107:0/1230655606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e8038070 con 0x5578407dbe40 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.873+0000 7f50e0c00640 1 -- 172.21.15.107:0/1230655606 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f50e8010940 con 0x5578407dbe40 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.873+0000 7f5100c07740 10 client.26257 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.873+0000 7f5100c07740 1 -- 172.21.15.107:0/1230655606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557840861f60 con 0x5578407dbe40 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.873+0000 7f50e0c00640 1 -- 172.21.15.107:0/1230655606 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e8038070 con 0x5578407dbe40 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.873+0000 7f50e0c00640 1 -- 172.21.15.107:0/1230655606 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f50e803c020 con 0x5578407dbe40 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.873+0000 7f5100c07740 10 client.26257 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.873+0000 7f5100c07740 10 client.26257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.874+0000 7f5100c07740 20 client.26257 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:09.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.874+0000 7f5100c07740 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5578408624a0 0x5578408648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:09.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.874+0000 7f5100c07740 4 client.26257 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:09.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.874+0000 7f5100c07740 1 -- 172.21.15.107:0/1230655606 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557840749e60 con 0x5578408624a0 2024-09-19T04:26:09.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.874+0000 7f50f6a00640 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5578408624a0 0x5578408648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:09.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.874+0000 7f50f6a00640 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5578408624a0 0x5578408648c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb3f415ddebfc29d server_cookie=60ac70bbaf8a341b in_seq=0 out_seq=0 2024-09-19T04:26:09.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.874+0000 7f50e0c00640 10 client.26257 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:26:09.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f50e0c00640 1 -- 172.21.15.107:0/1230655606 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557840749e60 con 0x5578408624a0 2024-09-19T04:26:09.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f50e0c00640 10 client.26257 handle_command_reply: tid=0 2024-09-19T04:26:09.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 client.26257 shutdown 2024-09-19T04:26:09.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 -- 172.21.15.107:0/1230655606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 msgr2=0x5578407ef860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407ef860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009250 tx=0x7f50e8009ed0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 -- 172.21.15.107:0/1230655606 shutdown_connections 2024-09-19T04:26:09.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5578408624a0 0x5578408648c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578407e7020 0x5578407eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578407dc760 0x5578407eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 --2- 172.21.15.107:0/1230655606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578407dbe40 0x5578407ef860 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f50e0c00640 0 client.26257 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:26:09.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 -- 172.21.15.107:0/1230655606 >> 172.21.15.107:0/1230655606 conn(0x5578407d83d0 msgr2=0x5578407dd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:09.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 -- 172.21.15.107:0/1230655606 shutdown_connections 2024-09-19T04:26:09.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 1 -- 172.21.15.107:0/1230655606 wait complete. 2024-09-19T04:26:09.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.875+0000 7f5100c07740 20 client.26257 trim_cache size 0 max 16384 2024-09-19T04:26:09.881 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:26:09.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 -- 172.21.15.107:0/800840854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f50dc0a3430 msgr2=0x7f50dc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f50dc0a3430 0x7f50dc0a58f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f50f0005e30 tx=0x7f50f0005c50 comp rx=0 tx=0).stop 2024-09-19T04:26:09.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 -- 172.21.15.107:0/800840854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50f8074ee0 msgr2=0x7f50f819a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:09.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50f8074ee0 0x7f50f819a970 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f50e400e9f0 tx=0x7f50e400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 -- 172.21.15.107:0/800840854 shutdown_connections 2024-09-19T04:26:09.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f50dc0a3430 0x7f50dc0a58f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50f810e260 0x7f50f819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50f8074ee0 0x7f50f819a970 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 --2- 172.21.15.107:0/800840854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50f80745c0 0x7f50f819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:09.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 -- 172.21.15.107:0/800840854 >> 172.21.15.107:0/800840854 conn(0x7f50f80fd6e0 msgr2=0x7f50f80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:09.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 -- 172.21.15.107:0/800840854 shutdown_connections 2024-09-19T04:26:09.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:09.876+0000 7f50fcc00640 1 -- 172.21.15.107:0/800840854 wait complete. 2024-09-19T04:26:10.672 INFO:tasks.workunit.client.0.smithi107.stdout: 1 207421 2.88 MB/sec execute 148 sec latency 832.312 ms 2024-09-19T04:26:10.879 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:26:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:11 smithi107 ceph-mon[30659]: pgmap v178: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 658 KiB/s wr, 103 op/s 2024-09-19T04:26:11.250 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:11.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:11 smithi115 ceph-mon[35477]: pgmap v178: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 658 KiB/s wr, 103 op/s 2024-09-19T04:26:11.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:11 smithi145 ceph-mon[35096]: pgmap v178: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 658 KiB/s wr, 103 op/s 2024-09-19T04:26:11.673 INFO:tasks.workunit.client.0.smithi107.stdout: 1 207421 2.86 MB/sec execute 149 sec latency 1832.376 ms 2024-09-19T04:26:11.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.809+0000 7fcdf5e00640 1 -- 172.21.15.107:0/4251425954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdf00745c0 msgr2=0x7fcdf00749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:11.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.809+0000 7fcdf5e00640 1 --2- 172.21.15.107:0/4251425954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdf00745c0 0x7fcdf00749a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009900 tx=0x7fcde002a480 comp rx=0 tx=0).stop 2024-09-19T04:26:11.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.809+0000 7fcdf5e00640 1 -- 172.21.15.107:0/4251425954 shutdown_connections 2024-09-19T04:26:11.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.809+0000 7fcdf5e00640 1 --2- 172.21.15.107:0/4251425954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf010e430 0x7fcdf0110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.809+0000 7fcdf5e00640 1 --2- 172.21.15.107:0/4251425954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdf0074ee0 0x7fcdf010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.809+0000 7fcdf5e00640 1 --2- 172.21.15.107:0/4251425954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdf00745c0 0x7fcdf00749a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.809+0000 7fcdf5e00640 1 -- 172.21.15.107:0/4251425954 >> 172.21.15.107:0/4251425954 conn(0x7fcdf00fd8b0 msgr2=0x7fcdf00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:11.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.809+0000 7fcdf5e00640 1 -- 172.21.15.107:0/4251425954 shutdown_connections 2024-09-19T04:26:11.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.810+0000 7fcdf5e00640 1 -- 172.21.15.107:0/4251425954 wait complete. 2024-09-19T04:26:11.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.811+0000 7fcdf5e00640 1 Processor -- start 2024-09-19T04:26:11.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.811+0000 7fcdf5e00640 1 -- start start 2024-09-19T04:26:11.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.811+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdf00745c0 0x7fcdf019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.811+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdf0074ee0 0x7fcdf019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.811+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf010e430 0x7fcdf019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.811+0000 7fcdf5e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf01141a0 con 0x7fcdf00745c0 2024-09-19T04:26:11.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.811+0000 7fcdf5e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0114020 con 0x7fcdf010e430 2024-09-19T04:26:11.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.811+0000 7fcdf5e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0114320 con 0x7fcdf0074ee0 2024-09-19T04:26:11.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcdef400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf010e430 0x7fcdf019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcde6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdf0074ee0 0x7fcdf019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcdef400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf010e430 0x7fcdf019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37608/0 (socket says 172.21.15.107:37608) 2024-09-19T04:26:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcdef400640 1 -- 172.21.15.107:0/3588040077 learned_addr learned my addr 172.21.15.107:0/3588040077 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcdeea00640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdf00745c0 0x7fcdf019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcdef400640 1 -- 172.21.15.107:0/3588040077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdf0074ee0 msgr2=0x7fcdf019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcdef400640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdf0074ee0 0x7fcdf019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcdef400640 1 -- 172.21.15.107:0/3588040077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdf00745c0 msgr2=0x7fcdf019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcdef400640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdf00745c0 0x7fcdf019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcdef400640 1 -- 172.21.15.107:0/3588040077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf019f690 con 0x7fcdf010e430 2024-09-19T04:26:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.812+0000 7fcde6000640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdf0074ee0 0x7fcdf019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.813+0000 7fcdef400640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf010e430 0x7fcdf019efb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcde800eb40 tx=0x7fcde800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.813+0000 7fcdf4a00640 1 -- 172.21.15.107:0/3588040077 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8004b20 con 0x7fcdf010e430 2024-09-19T04:26:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.813+0000 7fcdf5e00640 1 -- 172.21.15.107:0/3588040077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf019f980 con 0x7fcdf010e430 2024-09-19T04:26:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.813+0000 7fcdf4a00640 1 -- 172.21.15.107:0/3588040077 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde8004cc0 con 0x7fcdf010e430 2024-09-19T04:26:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.813+0000 7fcdf4a00640 1 -- 172.21.15.107:0/3588040077 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8010850 con 0x7fcdf010e430 2024-09-19T04:26:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.813+0000 7fcdf5e00640 1 -- 172.21.15.107:0/3588040077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf01a7260 con 0x7fcdf010e430 2024-09-19T04:26:11.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.816+0000 7fcdf88bb740 1 Processor -- start 2024-09-19T04:26:11.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.817+0000 7fcdf4a00640 1 -- 172.21.15.107:0/3588040077 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fcde8005020 con 0x7fcdf010e430 2024-09-19T04:26:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.817+0000 7fcdf88bb740 1 -- start start 2024-09-19T04:26:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.817+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093fec70 0x5565093ff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.817+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 0x556509409910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.817+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509409e50 0x55650940c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.817+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5565093653e0 con 0x5565093ff590 2024-09-19T04:26:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.817+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55650930cd80 con 0x556509409e50 2024-09-19T04:26:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.817+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5565093613a0 con 0x5565093fec70 2024-09-19T04:26:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcde6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093fec70 0x5565093ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcde6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093fec70 0x5565093ff050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37366/0 (socket says 172.21.15.107:37366) 2024-09-19T04:26:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcde6000640 1 -- 172.21.15.107:0/642844943 learned_addr learned my addr 172.21.15.107:0/642844943 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcdeea00640 1 --2- 172.21.15.107:0/642844943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 0x556509409910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcde6000640 1 --2- 172.21.15.107:0/642844943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509409e50 0x55650940c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcde6000640 1 -- 172.21.15.107:0/642844943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509409e50 msgr2=0x55650940c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcde6000640 1 --2- 172.21.15.107:0/642844943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509409e50 0x55650940c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcde6000640 1 -- 172.21.15.107:0/642844943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 msgr2=0x556509409910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcde6000640 1 --2- 172.21.15.107:0/642844943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 0x556509409910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcde6000640 1 -- 172.21.15.107:0/642844943 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650932bae0 con 0x5565093fec70 2024-09-19T04:26:11.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.818+0000 7fcde6000640 1 --2- 172.21.15.107:0/642844943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093fec70 0x5565093ff050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009fd0 tx=0x7fcdd8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f48a03694fa7c5f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.819+0000 7fcddfe00640 1 -- 172.21.15.107:0/642844943 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800fc20 con 0x5565093fec70 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.819+0000 7fcdf4a00640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcdd40a33b0 0x7fcdd40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.819+0000 7fcddfe00640 1 -- 172.21.15.107:0/642844943 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd800fdc0 con 0x5565093fec70 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.819+0000 7fcddfe00640 1 -- 172.21.15.107:0/642844943 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800e6d0 con 0x5565093fec70 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.819+0000 7fcdf4a00640 1 -- 172.21.15.107:0/3588040077 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fcde80d04c0 con 0x7fcdf010e430 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.819+0000 7fcdeea00640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcdd40a33b0 0x7fcdd40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdf88bb740 1 -- 172.21.15.107:0/642844943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093fec70 msgr2=0x5565093ff050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/642844943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093fec70 0x5565093ff050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009fd0 tx=0x7fcdd8009b80 comp rx=0 tx=0).stop 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdf88bb740 1 -- 172.21.15.107:0/642844943 shutdown_connections 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/642844943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509409e50 0x55650940c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/642844943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 0x556509409910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/642844943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093fec70 0x5565093ff050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdf88bb740 1 -- 172.21.15.107:0/642844943 >> 172.21.15.107:0/642844943 conn(0x5565093fb200 msgr2=0x5565093fb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdf88bb740 1 -- 172.21.15.107:0/642844943 shutdown_connections 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdf88bb740 1 -- 172.21.15.107:0/642844943 wait complete. 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdeea00640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcdd40a33b0 0x7fcdd40a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009230 tx=0x7fcde00058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.820+0000 7fcdf88bb740 1 -- start start 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.821+0000 7fcdf88bb740 1 Processor -- start 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.821+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 0x556509406d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.821+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093ff590 0x5565094072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.821+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 0x556509407800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.821+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55650940ca20 con 0x5565093fec70 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.821+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5565093200f0 con 0x5565093ff590 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.821+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55650940cba0 con 0x556509409e50 2024-09-19T04:26:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.821+0000 7fcdf88bb740 10 client.0 _ll_register_callbacks cb 0x556509307bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.821+0000 7fcde6000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093ff590 0x5565094072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.821+0000 7fcdef400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 0x556509407800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcde6000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093ff590 0x5565094072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37628/0 (socket says 172.21.15.107:37628) 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcde6000640 1 -- 172.21.15.107:0/1644440182 learned_addr learned my addr 172.21.15.107:0/1644440182 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcdef400640 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 0x556509407800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37376/0 (socket says 172.21.15.107:37376) 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcdeea00640 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 0x556509406d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcde6000640 1 -- 172.21.15.107:0/1644440182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 msgr2=0x556509407800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcde6000640 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 0x556509407800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcde6000640 1 -- 172.21.15.107:0/1644440182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 msgr2=0x556509406d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcde6000640 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 0x556509406d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcde6000640 1 -- 172.21.15.107:0/1644440182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565090d63c0 con 0x5565093ff590 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcdef400640 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 0x556509407800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcde6000640 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093ff590 0x5565094072c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcdd801ffd0 tx=0x7fcdd80044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d5b9b6f1d06c590 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.822+0000 7fcddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.823+0000 7fcddcc00640 1 -- 172.21.15.107:0/1644440182 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800fc20 con 0x5565093ff590 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.823+0000 7fcddcc00640 1 -- 172.21.15.107:0/1644440182 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fcdd8005240 con 0x5565093ff590 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.823+0000 7fcdf88bb740 1 -- 172.21.15.107:0/1644440182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd800fdc0 con 0x5565093ff590 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.823+0000 7fcddcc00640 1 -- 172.21.15.107:0/1644440182 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcdd8005440 con 0x5565093ff590 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.823+0000 7fcdf88bb740 10 client.26309 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.823+0000 7fcdf88bb740 1 -- 172.21.15.107:0/1644440182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556509418ab0 con 0x5565093ff590 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.823+0000 7fcddcc00640 1 -- 172.21.15.107:0/1644440182 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd800fdc0 con 0x5565093ff590 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.824+0000 7fcddcc00640 1 -- 172.21.15.107:0/1644440182 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7fcdd8026070 con 0x5565093ff590 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.824+0000 7fcdf88bb740 10 client.26309 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.824+0000 7fcdf88bb740 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.824+0000 7fcdf88bb740 20 client.26309 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.824+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556509419e00 0x55650941c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.824+0000 7fcdf88bb740 4 client.26309 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:11.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.824+0000 7fcdf88bb740 1 -- 172.21.15.107:0/1644440182 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556509372950 con 0x556509419e00 2024-09-19T04:26:11.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.824+0000 7fcdeea00640 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556509419e00 0x55650941c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.825+0000 7fcdeea00640 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556509419e00 0x55650941c1f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ecb940193962fd2 server_cookie=8e1c0be56e343e27 in_seq=0 out_seq=0 2024-09-19T04:26:11.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.825+0000 7fcddcc00640 10 client.26309 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:11.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.825+0000 7fcddcc00640 1 -- 172.21.15.107:0/1644440182 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556509372950 con 0x556509419e00 2024-09-19T04:26:11.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.825+0000 7fcddcc00640 10 client.26309 handle_command_reply: tid=0 2024-09-19T04:26:11.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 client.26309 shutdown 2024-09-19T04:26:11.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 -- 172.21.15.107:0/1644440182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093ff590 msgr2=0x5565094072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:11.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093ff590 0x5565094072c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcdd801ffd0 tx=0x7fcdd80044c0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 -- 172.21.15.107:0/1644440182 shutdown_connections 2024-09-19T04:26:11.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556509419e00 0x55650941c1f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 0x556509407800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093ff590 0x5565094072c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcddcc00640 0 client.26309 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/1644440182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 0x556509406d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 -- 172.21.15.107:0/1644440182 >> 172.21.15.107:0/1644440182 conn(0x5565093fb200 msgr2=0x5565093fa380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 -- 172.21.15.107:0/1644440182 shutdown_connections 2024-09-19T04:26:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 1 -- 172.21.15.107:0/1644440182 wait complete. 2024-09-19T04:26:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.826+0000 7fcdf88bb740 20 client.26309 trim_cache size 0 max 16384 2024-09-19T04:26:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.840+0000 7fcdf88bb740 1 Processor -- start 2024-09-19T04:26:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.840+0000 7fcdf88bb740 1 -- start start 2024-09-19T04:26:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.840+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093fec70 0x556509408cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.840+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 0x5565094219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.840+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 0x556509421f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.840+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf01af970 con 0x5565093ff590 2024-09-19T04:26:11.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.840+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf01b15d0 con 0x5565093fec70 2024-09-19T04:26:11.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.840+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf01b0040 con 0x556509409e50 2024-09-19T04:26:11.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.840+0000 7fcde6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 0x5565094219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.840+0000 7fcde6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 0x5565094219f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43486/0 (socket says 172.21.15.107:43486) 2024-09-19T04:26:11.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcde6000640 1 -- 172.21.15.107:0/2612390767 learned_addr learned my addr 172.21.15.107:0/2612390767 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:11.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcdeea00640 1 --2- 172.21.15.107:0/2612390767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093fec70 0x556509408cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcdef400640 1 --2- 172.21.15.107:0/2612390767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 0x556509421f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcde6000640 1 -- 172.21.15.107:0/2612390767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 msgr2=0x556509421f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:11.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcde6000640 1 --2- 172.21.15.107:0/2612390767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 0x556509421f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcde6000640 1 -- 172.21.15.107:0/2612390767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093fec70 msgr2=0x556509408cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcde6000640 1 --2- 172.21.15.107:0/2612390767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093fec70 0x556509408cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcde6000640 1 -- 172.21.15.107:0/2612390767 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565093fc2b0 con 0x5565093ff590 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcdeea00640 1 --2- 172.21.15.107:0/2612390767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093fec70 0x556509408cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcde6000640 1 --2- 172.21.15.107:0/2612390767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 0x5565094219f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8005c20 tx=0x7fcdd8011710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0f4598f90b2b73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcddf400640 1 -- 172.21.15.107:0/2612390767 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8009e90 con 0x5565093ff590 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcddf400640 1 -- 172.21.15.107:0/2612390767 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd801f440 con 0x5565093ff590 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.841+0000 7fcddf400640 1 -- 172.21.15.107:0/2612390767 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd801f930 con 0x5565093ff590 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.842+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2612390767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 msgr2=0x5565094219f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.842+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/2612390767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 0x5565094219f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8005c20 tx=0x7fcdd8011710 comp rx=0 tx=0).stop 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.842+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2612390767 shutdown_connections 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.842+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/2612390767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556509409e50 0x556509421f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.842+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/2612390767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093ff590 0x5565094219f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.842+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/2612390767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565093fec70 0x556509408cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.842+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2612390767 >> 172.21.15.107:0/2612390767 conn(0x5565093fb200 msgr2=0x5565094085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.842+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2612390767 shutdown_connections 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.842+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2612390767 wait complete. 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.843+0000 7fcdf88bb740 1 -- start start 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.843+0000 7fcdf88bb740 1 Processor -- start 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.843+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 0x556509412100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.843+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093ff590 0x556509412640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.843+0000 7fcdf88bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509409e50 0x55650940d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.843+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf01b0e40 con 0x5565093fec70 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.843+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf01b0710 con 0x556509409e50 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.843+0000 7fcdf88bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf01ad950 con 0x5565093ff590 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.843+0000 7fcdf88bb740 10 client.0 _ll_register_callbacks cb 0x7fcdd8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.843+0000 7fcdeea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 0x556509412100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcdeea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 0x556509412100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43492/0 (socket says 172.21.15.107:43492) 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcdeea00640 1 -- 172.21.15.107:0/2000486278 learned_addr learned my addr 172.21.15.107:0/2000486278 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcde6000640 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093ff590 0x556509412640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcdef400640 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509409e50 0x55650940d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcde6000640 1 -- 172.21.15.107:0/2000486278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509409e50 msgr2=0x55650940d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcde6000640 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509409e50 0x55650940d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcde6000640 1 -- 172.21.15.107:0/2000486278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 msgr2=0x556509412100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcde6000640 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 0x556509412100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcde6000640 1 -- 172.21.15.107:0/2000486278 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565093f6770 con 0x5565093ff590 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcdeea00640 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 0x556509412100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcde6000640 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093ff590 0x556509412640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x5565093efc50 tx=0x7fcdd8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82e3ccf867f204e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:26:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.844+0000 7fcddcc00640 1 -- 172.21.15.107:0/2000486278 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8035040 con 0x5565093ff590 2024-09-19T04:26:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2000486278 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd801f440 con 0x5565093ff590 2024-09-19T04:26:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcddcc00640 1 -- 172.21.15.107:0/2000486278 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fcdd8005240 con 0x5565093ff590 2024-09-19T04:26:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcddcc00640 1 -- 172.21.15.107:0/2000486278 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcdd8009e90 con 0x5565093ff590 2024-09-19T04:26:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcdf88bb740 10 client.26275 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2000486278 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5565094843a0 con 0x5565093ff590 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcddcc00640 1 -- 172.21.15.107:0/2000486278 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd801f440 con 0x5565093ff590 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcddcc00640 1 -- 172.21.15.107:0/2000486278 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7fcdd8038070 con 0x5565093ff590 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcdf88bb740 10 client.26275 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcdf88bb740 10 client.26275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcdf88bb740 20 client.26275 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5565094849e0 0x556509486e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcdf88bb740 4 client.26275 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.845+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2000486278 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55650936ce60 con 0x5565094849e0 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.846+0000 7fcdeea00640 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5565094849e0 0x556509486e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.846+0000 7fcdeea00640 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5565094849e0 0x556509486e00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2747bbd78fdc3e62 server_cookie=6e10b41bdd001d3d in_seq=0 out_seq=0 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.846+0000 7fcddcc00640 10 client.26275 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcddcc00640 1 -- 172.21.15.107:0/2000486278 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55650936ce60 con 0x5565094849e0 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcddcc00640 10 client.26275 handle_command_reply: tid=0 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 client.26275 shutdown 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2000486278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093ff590 msgr2=0x556509412640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093ff590 0x556509412640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x5565093efc50 tx=0x7fcdd8005850 comp rx=0 tx=0).stop 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2000486278 shutdown_connections 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5565094849e0 0x556509486e00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509409e50 0x55650940d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcddcc00640 0 client.26275 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5565093ff590 0x556509412640 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 --2- 172.21.15.107:0/2000486278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565093fec70 0x556509412100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2000486278 >> 172.21.15.107:0/2000486278 conn(0x5565093fb200 msgr2=0x556509408e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2000486278 shutdown_connections 2024-09-19T04:26:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 1 -- 172.21.15.107:0/2000486278 wait complete. 2024-09-19T04:26:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.847+0000 7fcdf88bb740 20 client.26275 trim_cache size 0 max 16384 2024-09-19T04:26:11.854 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:26:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 -- 172.21.15.107:0/3588040077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcdd40a33b0 msgr2=0x7fcdd40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcdd40a33b0 0x7fcdd40a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009230 tx=0x7fcde00058d0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 -- 172.21.15.107:0/3588040077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf010e430 msgr2=0x7fcdf019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf010e430 0x7fcdf019efb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcde800eb40 tx=0x7fcde800c2d0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 -- 172.21.15.107:0/3588040077 shutdown_connections 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcdd40a33b0 0x7fcdd40a5870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf010e430 0x7fcdf019efb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdf0074ee0 0x7fcdf019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 --2- 172.21.15.107:0/3588040077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdf00745c0 0x7fcdf019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 -- 172.21.15.107:0/3588040077 >> 172.21.15.107:0/3588040077 conn(0x7fcdf00fd8b0 msgr2=0x7fcdf00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 -- 172.21.15.107:0/3588040077 shutdown_connections 2024-09-19T04:26:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:11.848+0000 7fcdf5e00640 1 -- 172.21.15.107:0/3588040077 wait complete. 2024-09-19T04:26:12.672 INFO:tasks.workunit.client.0.smithi107.stdout: 1 207421 2.84 MB/sec execute 150 sec latency 2832.472 ms 2024-09-19T04:26:13.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:13 smithi115 ceph-mon[35477]: pgmap v179: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 977 KiB/s wr, 157 op/s 2024-09-19T04:26:13.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:13 smithi145 ceph-mon[35096]: pgmap v179: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 977 KiB/s wr, 157 op/s 2024-09-19T04:26:13.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:13 smithi107 ceph-mon[30659]: pgmap v179: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 977 KiB/s wr, 157 op/s 2024-09-19T04:26:13.672 INFO:tasks.workunit.client.0.smithi107.stdout: 1 207421 2.82 MB/sec execute 151 sec latency 3832.550 ms 2024-09-19T04:26:13.740 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:26:13.740 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:26:14.162 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:14.673 INFO:tasks.workunit.client.0.smithi107.stdout: 1 208164 2.81 MB/sec execute 152 sec latency 4117.000 ms 2024-09-19T04:26:14.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.740+0000 7f9217200640 1 -- 172.21.15.107:0/2674828412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92181079c0 msgr2=0x7f9218107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.740+0000 7f9217200640 1 --2- 172.21.15.107:0/2674828412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92181079c0 0x7f9218107da0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9208009930 tx=0x7f920802a420 comp rx=0 tx=0).stop 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.740+0000 7f9217200640 1 -- 172.21.15.107:0/2674828412 shutdown_connections 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.740+0000 7f9217200640 1 --2- 172.21.15.107:0/2674828412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9218102360 0x7f921810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.740+0000 7f9217200640 1 --2- 172.21.15.107:0/2674828412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92181019c0 0x7f9218101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.740+0000 7f9217200640 1 --2- 172.21.15.107:0/2674828412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92181079c0 0x7f9218107da0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.740+0000 7f9217200640 1 -- 172.21.15.107:0/2674828412 >> 172.21.15.107:0/2674828412 conn(0x7f92180fd740 msgr2=0x7f92180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.740+0000 7f9217200640 1 -- 172.21.15.107:0/2674828412 shutdown_connections 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.740+0000 7f9217200640 1 -- 172.21.15.107:0/2674828412 wait complete. 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.741+0000 7f9217200640 1 Processor -- start 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.741+0000 7f9217200640 1 -- start start 2024-09-19T04:26:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9217200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92181019c0 0x7f921819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9217200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102360 0x7f921819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9217200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92181079c0 0x7f921819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9217200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9218111e10 con 0x7f92181079c0 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9217200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9218111c90 con 0x7f9218102360 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9217200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9218111f90 con 0x7f92181019c0 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9215e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92181019c0 0x7f921819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9215e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92181019c0 0x7f921819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37414/0 (socket says 172.21.15.107:37414) 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9215e00640 1 -- 172.21.15.107:0/1571118906 learned_addr learned my addr 172.21.15.107:0/1571118906 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9215400640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102360 0x7f921819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9216800640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92181079c0 0x7f921819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9215e00640 1 -- 172.21.15.107:0/1571118906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102360 msgr2=0x7f921819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9215e00640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102360 0x7f921819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9215e00640 1 -- 172.21.15.107:0/1571118906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92181079c0 msgr2=0x7f921819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9215e00640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92181079c0 0x7f921819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.742+0000 7f9215e00640 1 -- 172.21.15.107:0/1571118906 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921819f420 con 0x7f92181019c0 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.743+0000 7f9215400640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102360 0x7f921819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.743+0000 7f9215e00640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92181019c0 0x7f921819a470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f920802a940 tx=0x7f9208003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.743+0000 7f920ea00640 1 -- 172.21.15.107:0/1571118906 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920803f070 con 0x7f92181019c0 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.743+0000 7f920ea00640 1 -- 172.21.15.107:0/1571118906 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9208031c90 con 0x7f92181019c0 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.743+0000 7f9217200640 1 -- 172.21.15.107:0/1571118906 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921819f6b0 con 0x7f92181019c0 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.743+0000 7f920ea00640 1 -- 172.21.15.107:0/1571118906 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208035030 con 0x7f92181019c0 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.743+0000 7f9217200640 1 -- 172.21.15.107:0/1571118906 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92181a6ff0 con 0x7f92181019c0 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.746+0000 7f920ea00640 1 -- 172.21.15.107:0/1571118906 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f9208042070 con 0x7f92181019c0 2024-09-19T04:26:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.746+0000 7f921f7e6740 1 Processor -- start 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.746+0000 7f921f7e6740 1 -- start start 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561887779cb0 0x56188777a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56188777a5d0 0x561887784950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887784e90 0x561887787280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f921f7e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618876e0290 con 0x56188777a5d0 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f921f7e6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561887687b40 con 0x561887784e90 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f921f7e6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5618876dc250 con 0x561887779cb0 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f9216800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561887779cb0 0x56188777a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f9216800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561887779cb0 0x56188777a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37422/0 (socket says 172.21.15.107:37422) 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f9216800640 1 -- 172.21.15.107:0/1792872547 learned_addr learned my addr 172.21.15.107:0/1792872547 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f9215400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887784e90 0x561887787280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f9215e00640 1 --2- 172.21.15.107:0/1792872547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56188777a5d0 0x561887784950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f9215400640 1 -- 172.21.15.107:0/1792872547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561887779cb0 msgr2=0x56188777a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f9215400640 1 --2- 172.21.15.107:0/1792872547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561887779cb0 0x56188777a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f9215400640 1 -- 172.21.15.107:0/1792872547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56188777a5d0 msgr2=0x561887784950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f9215400640 1 --2- 172.21.15.107:0/1792872547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56188777a5d0 0x561887784950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.747+0000 7f9215400640 1 -- 172.21.15.107:0/1792872547 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618874516a0 con 0x561887784e90 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.748+0000 7f9215400640 1 --2- 172.21.15.107:0/1792872547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887784e90 0x561887787280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9204009fa0 tx=0x7f92040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b4ac9c288df3425 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.748+0000 7f920ea00640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91fc0a3430 0x7f91fc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.748+0000 7f9203400640 1 -- 172.21.15.107:0/1792872547 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204004030 con 0x561887784e90 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.748+0000 7f920ea00640 1 -- 172.21.15.107:0/1571118906 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f92080f02b0 con 0x7f92181019c0 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.748+0000 7f9203400640 1 -- 172.21.15.107:0/1792872547 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9204005500 con 0x561887784e90 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.748+0000 7f9203400640 1 -- 172.21.15.107:0/1792872547 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204004e10 con 0x561887784e90 2024-09-19T04:26:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.748+0000 7f9216800640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91fc0a3430 0x7f91fc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.749+0000 7f921f7e6740 1 -- 172.21.15.107:0/1792872547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887784e90 msgr2=0x561887787280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.749+0000 7f921f7e6740 1 --2- 172.21.15.107:0/1792872547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887784e90 0x561887787280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9204009fa0 tx=0x7f92040118d0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.749+0000 7f9216800640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91fc0a3430 0x7f91fc0a58f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9210011630 tx=0x7f9210009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.749+0000 7f921f7e6740 1 -- 172.21.15.107:0/1792872547 shutdown_connections 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.749+0000 7f921f7e6740 1 --2- 172.21.15.107:0/1792872547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887784e90 0x561887787280 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.749+0000 7f921f7e6740 1 --2- 172.21.15.107:0/1792872547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56188777a5d0 0x561887784950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.749+0000 7f921f7e6740 1 --2- 172.21.15.107:0/1792872547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561887779cb0 0x56188777a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.749+0000 7f921f7e6740 1 -- 172.21.15.107:0/1792872547 >> 172.21.15.107:0/1792872547 conn(0x561887776240 msgr2=0x561887776620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.749+0000 7f921f7e6740 1 -- 172.21.15.107:0/1792872547 shutdown_connections 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.749+0000 7f921f7e6740 1 -- 172.21.15.107:0/1792872547 wait complete. 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.750+0000 7f921f7e6740 1 -- start start 2024-09-19T04:26:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.750+0000 7f921f7e6740 1 Processor -- start 2024-09-19T04:26:14.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.750+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x561887784780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.750+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887781a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.750+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x561887781f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.750+0000 7f921f7e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561887787b00 con 0x561887784e90 2024-09-19T04:26:14.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.750+0000 7f921f7e6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56188769afa0 con 0x561887779cb0 2024-09-19T04:26:14.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.750+0000 7f921f7e6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561887787c80 con 0x56188777a5d0 2024-09-19T04:26:14.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f921f7e6740 10 client.0 _ll_register_callbacks cb 0x5618873eeba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887781a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x561887784780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9216800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x561887781f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887781a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37432/0 (socket says 172.21.15.107:37432) 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x561887784780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37678/0 (socket says 172.21.15.107:37678) 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215400640 1 -- 172.21.15.107:0/2189046495 learned_addr learned my addr 172.21.15.107:0/2189046495 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215e00640 1 -- 172.21.15.107:0/2189046495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 msgr2=0x561887781a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215e00640 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887781a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215e00640 1 -- 172.21.15.107:0/2189046495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 msgr2=0x561887781f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215e00640 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x561887781f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215e00640 1 -- 172.21.15.107:0/2189046495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561887782750 con 0x561887779cb0 2024-09-19T04:26:14.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9216800640 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x561887781f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f9215e00640 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x561887784780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f92080ed9f0 tx=0x7f9208003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfed52f78c1207d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.751+0000 7f91f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f91f7e00640 1 -- 172.21.15.107:0/2189046495 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208049070 con 0x561887779cb0 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f91f7e00640 1 -- 172.21.15.107:0/2189046495 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f92080edc70 con 0x561887779cb0 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f921f7e6740 1 -- 172.21.15.107:0/2189046495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9204005500 con 0x561887779cb0 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f91f7e00640 1 -- 172.21.15.107:0/2189046495 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f92080ede70 con 0x561887779cb0 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f921f7e6740 10 client.26327 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f921f7e6740 1 -- 172.21.15.107:0/2189046495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5618877939c0 con 0x561887779cb0 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f91f7e00640 1 -- 172.21.15.107:0/2189046495 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9204005500 con 0x561887779cb0 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f91f7e00640 1 -- 172.21.15.107:0/2189046495 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f92080bc890 con 0x561887779cb0 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f921f7e6740 10 client.26327 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f921f7e6740 10 client.26327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:14.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f921f7e6740 20 client.26327 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:14.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f921f7e6740 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561887794de0 0x561887797280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f921f7e6740 4 client.26327 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:14.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.752+0000 7f921f7e6740 1 -- 172.21.15.107:0/2189046495 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618876ed800 con 0x561887794de0 2024-09-19T04:26:14.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.753+0000 7f9215400640 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561887794de0 0x561887797280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.753+0000 7f9215400640 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561887794de0 0x561887797280 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f9a6acda777cdc server_cookie=ca25236defe0e7a3 in_seq=0 out_seq=0 2024-09-19T04:26:14.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.753+0000 7f91f7e00640 10 client.26327 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:14.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f91f7e00640 1 -- 172.21.15.107:0/2189046495 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618876ed800 con 0x561887794de0 2024-09-19T04:26:14.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f91f7e00640 10 client.26327 handle_command_reply: tid=0 2024-09-19T04:26:14.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f921f7e6740 1 client.26327 shutdown 2024-09-19T04:26:14.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f921f7e6740 1 -- 172.21.15.107:0/2189046495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 msgr2=0x561887784780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f921f7e6740 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x561887784780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f92080ed9f0 tx=0x7f9208003f50 comp rx=0 tx=0).stop 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f921f7e6740 1 -- 172.21.15.107:0/2189046495 shutdown_connections 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f921f7e6740 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561887794de0 0x561887797280 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f921f7e6740 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x561887781f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f921f7e6740 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887781a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f921f7e6740 1 --2- 172.21.15.107:0/2189046495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x561887784780 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f921f7e6740 1 -- 172.21.15.107:0/2189046495 >> 172.21.15.107:0/2189046495 conn(0x561887776240 msgr2=0x56188777bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.754+0000 7f91f7e00640 0 client.26327 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.755+0000 7f921f7e6740 1 -- 172.21.15.107:0/2189046495 shutdown_connections 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.755+0000 7f921f7e6740 1 -- 172.21.15.107:0/2189046495 wait complete. 2024-09-19T04:26:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.755+0000 7f921f7e6740 20 client.26327 trim_cache size 0 max 16384 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.774+0000 7f921f7e6740 1 Processor -- start 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.774+0000 7f921f7e6740 1 -- start start 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.774+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x56188777d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.774+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x56188779cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.774+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x56188779d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f921f7e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f92181af790 con 0x561887784e90 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f921f7e6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b13f0 con 0x561887779cb0 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f921f7e6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f92181afe60 con 0x56188777a5d0 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9215400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x56188779cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9215e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x56188777d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9215400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x56188779cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37448/0 (socket says 172.21.15.107:37448) 2024-09-19T04:26:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9215400640 1 -- 172.21.15.107:0/1381202540 learned_addr learned my addr 172.21.15.107:0/1381202540 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9215e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x56188777d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:37684/0 (socket says 172.21.15.107:37684) 2024-09-19T04:26:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9216800640 1 --2- 172.21.15.107:0/1381202540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x56188779d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9215400640 1 -- 172.21.15.107:0/1381202540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 msgr2=0x56188777d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9215400640 1 --2- 172.21.15.107:0/1381202540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x56188777d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9215400640 1 -- 172.21.15.107:0/1381202540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 msgr2=0x56188779d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9215400640 1 --2- 172.21.15.107:0/1381202540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x56188779d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.775+0000 7f9215400640 1 -- 172.21.15.107:0/1381202540 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561887797900 con 0x56188777a5d0 2024-09-19T04:26:14.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.776+0000 7f9215400640 1 --2- 172.21.15.107:0/1381202540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x56188779cb50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9204009530 tx=0x7f92040118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eefb4f752e519e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:14.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.776+0000 7f9202a00640 1 -- 172.21.15.107:0/1381202540 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920401d040 con 0x56188777a5d0 2024-09-19T04:26:14.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.776+0000 7f9202a00640 1 -- 172.21.15.107:0/1381202540 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92040042d0 con 0x56188777a5d0 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.776+0000 7f9202a00640 1 -- 172.21.15.107:0/1381202540 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920400f5a0 con 0x56188777a5d0 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.777+0000 7f921f7e6740 1 -- 172.21.15.107:0/1381202540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 msgr2=0x56188779cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.777+0000 7f921f7e6740 1 --2- 172.21.15.107:0/1381202540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x56188779cb50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9204009530 tx=0x7f92040118d0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.777+0000 7f921f7e6740 1 -- 172.21.15.107:0/1381202540 shutdown_connections 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.777+0000 7f921f7e6740 1 --2- 172.21.15.107:0/1381202540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x56188779d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.777+0000 7f921f7e6740 1 --2- 172.21.15.107:0/1381202540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x56188779cb50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.777+0000 7f921f7e6740 1 --2- 172.21.15.107:0/1381202540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x56188777d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.777+0000 7f921f7e6740 1 -- 172.21.15.107:0/1381202540 >> 172.21.15.107:0/1381202540 conn(0x561887776240 msgr2=0x56188777c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.777+0000 7f921f7e6740 1 -- 172.21.15.107:0/1381202540 shutdown_connections 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.777+0000 7f921f7e6740 1 -- 172.21.15.107:0/1381202540 wait complete. 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.777+0000 7f921f7e6740 1 -- start start 2024-09-19T04:26:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f921f7e6740 1 Processor -- start 2024-09-19T04:26:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x56188778d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887788990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f921f7e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x561887788ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f921f7e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b0c60 con 0x561887784e90 2024-09-19T04:26:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f921f7e6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b0530 con 0x561887779cb0 2024-09-19T04:26:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f921f7e6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f92181ad770 con 0x56188777a5d0 2024-09-19T04:26:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f921f7e6740 10 client.0 _ll_register_callbacks cb 0x7f9208005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f9215400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887788990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f9215400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887788990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37458/0 (socket says 172.21.15.107:37458) 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.778+0000 7f9215400640 1 -- 172.21.15.107:0/4169639568 learned_addr learned my addr 172.21.15.107:0/4169639568 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f9215e00640 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x56188778d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f9216800640 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x561887788ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f9215400640 1 -- 172.21.15.107:0/4169639568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 msgr2=0x56188778d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f9215400640 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x56188778d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f9215400640 1 -- 172.21.15.107:0/4169639568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 msgr2=0x561887788ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f9215400640 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x561887788ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f9215400640 1 -- 172.21.15.107:0/4169639568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561887789700 con 0x56188777a5d0 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f9216800640 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x561887788ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f9215e00640 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x56188778d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f9215400640 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887788990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9204009d60 tx=0x7f9204009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d372e56ceaa9bd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f91f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f91f7e00640 1 -- 172.21.15.107:0/4169639568 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204003d60 con 0x56188777a5d0 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.779+0000 7f91f7e00640 1 -- 172.21.15.107:0/4169639568 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f9204004480 con 0x56188777a5d0 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.780+0000 7f921f7e6740 1 -- 172.21.15.107:0/4169639568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92040042d0 con 0x56188777a5d0 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.780+0000 7f91f7e00640 1 -- 172.21.15.107:0/4169639568 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9204041050 con 0x56188777a5d0 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.780+0000 7f921f7e6740 10 client.26293 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.780+0000 7f921f7e6740 1 -- 172.21.15.107:0/4169639568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5618877ffb50 con 0x56188777a5d0 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.780+0000 7f91f7e00640 1 -- 172.21.15.107:0/4169639568 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92040042d0 con 0x56188777a5d0 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.780+0000 7f91f7e00640 1 -- 172.21.15.107:0/4169639568 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f920400fcb0 con 0x56188777a5d0 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.780+0000 7f921f7e6740 10 client.26293 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.780+0000 7f921f7e6740 10 client.26293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.780+0000 7f921f7e6740 20 client.26293 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.780+0000 7f921f7e6740 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561887800150 0x561887802570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.781+0000 7f921f7e6740 4 client.26293 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:26:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.781+0000 7f921f7e6740 1 -- 172.21.15.107:0/4169639568 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5618876e7d10 con 0x561887800150 2024-09-19T04:26:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.781+0000 7f9215e00640 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561887800150 0x561887802570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.781+0000 7f9215e00640 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561887800150 0x561887802570 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234510a8897db1af server_cookie=9d3f3d1ed090f4cd in_seq=0 out_seq=0 2024-09-19T04:26:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.781+0000 7f91f7e00640 10 client.26293 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.782+0000 7f91f7e00640 1 -- 172.21.15.107:0/4169639568 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5618876e7d10 con 0x561887800150 2024-09-19T04:26:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.782+0000 7f91f7e00640 10 client.26293 handle_command_reply: tid=0 2024-09-19T04:26:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.782+0000 7f921f7e6740 1 client.26293 shutdown 2024-09-19T04:26:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 1 -- 172.21.15.107:0/4169639568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 msgr2=0x561887788990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887788990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9204009d60 tx=0x7f9204009ed0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 1 -- 172.21.15.107:0/4169639568 shutdown_connections 2024-09-19T04:26:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561887800150 0x561887802570 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561887784e90 0x561887788ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56188777a5d0 0x561887788990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 1 --2- 172.21.15.107:0/4169639568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561887779cb0 0x56188778d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 1 -- 172.21.15.107:0/4169639568 >> 172.21.15.107:0/4169639568 conn(0x561887776240 msgr2=0x56188777ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f91f7e00640 0 client.26293 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 1 -- 172.21.15.107:0/4169639568 shutdown_connections 2024-09-19T04:26:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 1 -- 172.21.15.107:0/4169639568 wait complete. 2024-09-19T04:26:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.783+0000 7f921f7e6740 20 client.26293 trim_cache size 0 max 16384 2024-09-19T04:26:14.788 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:26:14.789 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:26:14.789 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "d14bf3ab-d395-47ed-bb47-117df7cb3a8e", 2024-09-19T04:26:14.789 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:26:14.789 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:26:14.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 -- 172.21.15.107:0/1571118906 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91fc0a3430 msgr2=0x7f91fc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91fc0a3430 0x7f91fc0a58f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9210011630 tx=0x7f9210009500 comp rx=0 tx=0).stop 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 -- 172.21.15.107:0/1571118906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92181019c0 msgr2=0x7f921819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92181019c0 0x7f921819a470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f920802a940 tx=0x7f9208003b20 comp rx=0 tx=0).stop 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 -- 172.21.15.107:0/1571118906 shutdown_connections 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f91fc0a3430 0x7f91fc0a58f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92181079c0 0x7f921819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102360 0x7f921819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 --2- 172.21.15.107:0/1571118906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92181019c0 0x7f921819a470 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 -- 172.21.15.107:0/1571118906 >> 172.21.15.107:0/1571118906 conn(0x7f92180fd740 msgr2=0x7f92180fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 -- 172.21.15.107:0/1571118906 shutdown_connections 2024-09-19T04:26:14.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:14.784+0000 7f9217200640 1 -- 172.21.15.107:0/1571118906 wait complete. 2024-09-19T04:26:15.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:15 smithi107 ceph-mon[30659]: pgmap v180: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 636 KiB/s wr, 124 op/s 2024-09-19T04:26:15.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:15 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:26:15.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:15 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:15 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:26:15.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:15 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:15 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:26:15.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:15 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:15 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:26:15.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:15 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:15 smithi115 ceph-mon[35477]: pgmap v180: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 636 KiB/s wr, 124 op/s 2024-09-19T04:26:15.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:15 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:26:15.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:15 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:15 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:26:15.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:15 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:15 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:26:15.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:15 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:15 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:26:15.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:15 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:15 smithi145 ceph-mon[35096]: pgmap v180: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 636 KiB/s wr, 124 op/s 2024-09-19T04:26:15.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:15 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:26:15.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:15 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:15 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:26:15.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:15 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:15 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:26:15.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:15 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:15 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:26:15.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:15 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:15.673 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:26:15.673 INFO:tasks.workunit.client.0.smithi107.stdout: 1 209701 2.87 MB/sec execute 153 sec latency 31.266 ms 2024-09-19T04:26:16.067 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:16.355 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:26:16.355 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.355 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:26:16.355 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:26:16.355 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:26:16.355 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:26:16.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:26:16.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:26:16.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:16 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:26:16.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:16 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:26:16.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:16 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:26:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.624+0000 7effaf200640 1 -- 172.21.15.107:0/2895439085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb0103e30 msgr2=0x7effb0104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.624+0000 7effaf200640 1 --2- 172.21.15.107:0/2895439085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb0103e30 0x7effb0104210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7effa0009900 tx=0x7effa002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.624+0000 7effaf200640 1 -- 172.21.15.107:0/2895439085 shutdown_connections 2024-09-19T04:26:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.624+0000 7effaf200640 1 --2- 172.21.15.107:0/2895439085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb0108f60 0x7effb0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.624+0000 7effaf200640 1 --2- 172.21.15.107:0/2895439085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb01047e0 0x7effb0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.624+0000 7effaf200640 1 --2- 172.21.15.107:0/2895439085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb0103e30 0x7effb0104210 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.624+0000 7effaf200640 1 -- 172.21.15.107:0/2895439085 >> 172.21.15.107:0/2895439085 conn(0x7effb00ff8d0 msgr2=0x7effb0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.624+0000 7effaf200640 1 -- 172.21.15.107:0/2895439085 shutdown_connections 2024-09-19T04:26:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.624+0000 7effaf200640 1 -- 172.21.15.107:0/2895439085 wait complete. 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.625+0000 7effaf200640 1 Processor -- start 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.625+0000 7effaf200640 1 -- start start 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.625+0000 7effaf200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0103e30 0x7effb019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.625+0000 7effaf200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb01047e0 0x7effb019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.625+0000 7effaf200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb0108f60 0x7effb019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.625+0000 7effaf200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7effb0114010 con 0x7effb01047e0 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.625+0000 7effaf200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7effb0113e90 con 0x7effb0108f60 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.625+0000 7effaf200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7effb0114190 con 0x7effb0103e30 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effade00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0103e30 0x7effb019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effade00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0103e30 0x7effb019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37488/0 (socket says 172.21.15.107:37488) 2024-09-19T04:26:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effade00640 1 -- 172.21.15.107:0/919724184 learned_addr learned my addr 172.21.15.107:0/919724184 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:16.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effad400640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb01047e0 0x7effb019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effae800640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb0108f60 0x7effb019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effade00640 1 -- 172.21.15.107:0/919724184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb0108f60 msgr2=0x7effb019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:16.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effade00640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb0108f60 0x7effb019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effade00640 1 -- 172.21.15.107:0/919724184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb01047e0 msgr2=0x7effb019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:16.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effade00640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb01047e0 0x7effb019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effade00640 1 -- 172.21.15.107:0/919724184 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb019f450 con 0x7effb0103e30 2024-09-19T04:26:16.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.626+0000 7effade00640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0103e30 0x7effb019a4a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7effa0004770 tx=0x7effa0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:16.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.627+0000 7effa6a00640 1 -- 172.21.15.107:0/919724184 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa003f070 con 0x7effb0103e30 2024-09-19T04:26:16.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.627+0000 7effaf200640 1 -- 172.21.15.107:0/919724184 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb019f6e0 con 0x7effb0103e30 2024-09-19T04:26:16.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.627+0000 7effa6a00640 1 -- 172.21.15.107:0/919724184 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa0031ca0 con 0x7effb0103e30 2024-09-19T04:26:16.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.627+0000 7effa6a00640 1 -- 172.21.15.107:0/919724184 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0035030 con 0x7effb0103e30 2024-09-19T04:26:16.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.627+0000 7effaf200640 1 -- 172.21.15.107:0/919724184 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb01a7020 con 0x7effb0103e30 2024-09-19T04:26:16.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.630+0000 7effa6a00640 1 -- 172.21.15.107:0/919724184 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7effa0042070 con 0x7effb0103e30 2024-09-19T04:26:16.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.630+0000 7effb77f3740 1 Processor -- start 2024-09-19T04:26:16.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.630+0000 7effb77f3740 1 -- start start 2024-09-19T04:26:16.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effb77f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c26b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effb77f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 0x55f21c27e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effb77f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c280b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effb77f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f21c1d13e0 con 0x55f21c26ac70 2024-09-19T04:26:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effb77f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f21c178d80 con 0x55f21c27e740 2024-09-19T04:26:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effb77f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f21c1cd3a0 con 0x55f21c26b590 2024-09-19T04:26:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c26b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c26b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43566/0 (socket says 172.21.15.107:43566) 2024-09-19T04:26:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effae800640 1 -- 172.21.15.107:0/2190678350 learned_addr learned my addr 172.21.15.107:0/2190678350 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effade00640 1 --2- 172.21.15.107:0/2190678350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 0x55f21c27e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effad400640 1 --2- 172.21.15.107:0/2190678350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c280b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effae800640 1 -- 172.21.15.107:0/2190678350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 msgr2=0x55f21c27e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effae800640 1 --2- 172.21.15.107:0/2190678350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 0x55f21c27e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effae800640 1 -- 172.21.15.107:0/2190678350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 msgr2=0x55f21c280b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effae800640 1 --2- 172.21.15.107:0/2190678350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c280b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.631+0000 7effae800640 1 -- 172.21.15.107:0/2190678350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21c197ae0 con 0x55f21c26ac70 2024-09-19T04:26:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.632+0000 7effa6a00640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7eff940a33b0 0x7eff940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.632+0000 7effae800640 1 --2- 172.21.15.107:0/2190678350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c26b050 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7effa8009f50 tx=0x7effa80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad17fe03cfee37e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.632+0000 7effa6a00640 1 -- 172.21.15.107:0/919724184 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7effa00eff40 con 0x7effb0103e30 2024-09-19T04:26:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.632+0000 7effade00640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7eff940a33b0 0x7eff940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.632+0000 7eff9b000640 1 -- 172.21.15.107:0/2190678350 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8004d20 con 0x55f21c26ac70 2024-09-19T04:26:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.632+0000 7eff9b000640 1 -- 172.21.15.107:0/2190678350 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8004050 con 0x55f21c26ac70 2024-09-19T04:26:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.632+0000 7eff9b000640 1 -- 172.21.15.107:0/2190678350 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8005280 con 0x55f21c26ac70 2024-09-19T04:26:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.633+0000 7effb77f3740 1 -- 172.21.15.107:0/2190678350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 msgr2=0x55f21c26b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.633+0000 7effade00640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7eff940a33b0 0x7eff940a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7effa00ed790 tx=0x7effa00f3a90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.633+0000 7effb77f3740 1 --2- 172.21.15.107:0/2190678350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c26b050 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7effa8009f50 tx=0x7effa80118d0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.633+0000 7effb77f3740 1 -- 172.21.15.107:0/2190678350 shutdown_connections 2024-09-19T04:26:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.633+0000 7effb77f3740 1 --2- 172.21.15.107:0/2190678350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c280b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.633+0000 7effb77f3740 1 --2- 172.21.15.107:0/2190678350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 0x55f21c27e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.633+0000 7effb77f3740 1 --2- 172.21.15.107:0/2190678350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c26b050 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.633+0000 7effb77f3740 1 -- 172.21.15.107:0/2190678350 >> 172.21.15.107:0/2190678350 conn(0x55f21c2672e0 msgr2=0x55f21c2676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.633+0000 7effb77f3740 1 -- 172.21.15.107:0/2190678350 shutdown_connections 2024-09-19T04:26:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.633+0000 7effb77f3740 1 -- 172.21.15.107:0/2190678350 wait complete. 2024-09-19T04:26:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.634+0000 7effb77f3740 1 -- start start 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.634+0000 7effb77f3740 1 Processor -- start 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effb77f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c27b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effb77f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 0x55f21c27bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effb77f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c27c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effb77f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f21c281350 con 0x55f21c26ac70 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effb77f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f21c18c0f0 con 0x55f21c27e740 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effb77f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f21c2814d0 con 0x55f21c26b590 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effb77f3740 10 client.0 _ll_register_callbacks cb 0x55f21c17d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c27b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effad400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 0x55f21c27bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effad400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 0x55f21c27bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37508/0 (socket says 172.21.15.107:37508) 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effad400640 1 -- 172.21.15.107:0/452298299 learned_addr learned my addr 172.21.15.107:0/452298299 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effae800640 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c27c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effad400640 1 -- 172.21.15.107:0/452298299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 msgr2=0x55f21c27c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effad400640 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c27c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effad400640 1 -- 172.21.15.107:0/452298299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 msgr2=0x55f21c27b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effad400640 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c27b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effad400640 1 -- 172.21.15.107:0/452298299 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21bf423c0 con 0x55f21c26b590 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.635+0000 7effae800640 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c27b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.636+0000 7effad400640 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 0x55f21c27bc80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff9c012710 tx=0x7eff9c012bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a78f9a34ef1045 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.636+0000 7eff8f400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:26:16.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.636+0000 7eff8f400640 1 -- 172.21.15.107:0/452298299 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c004020 con 0x55f21c26b590 2024-09-19T04:26:16.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.636+0000 7effb77f3740 1 -- 172.21.15.107:0/452298299 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa8004050 con 0x55f21c26b590 2024-09-19T04:26:16.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.636+0000 7eff8f400640 1 -- 172.21.15.107:0/452298299 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7eff9c0041c0 con 0x55f21c26b590 2024-09-19T04:26:16.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.636+0000 7eff8f400640 1 -- 172.21.15.107:0/452298299 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7eff9c0043c0 con 0x55f21c26b590 2024-09-19T04:26:16.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.636+0000 7effb77f3740 10 client.26305 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:16.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.636+0000 7effb77f3740 1 -- 172.21.15.107:0/452298299 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f21c28d1c0 con 0x55f21c26b590 2024-09-19T04:26:16.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.637+0000 7eff8f400640 1 -- 172.21.15.107:0/452298299 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8004050 con 0x55f21c26b590 2024-09-19T04:26:16.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.637+0000 7eff8f400640 1 -- 172.21.15.107:0/452298299 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7eff9c003aa0 con 0x55f21c26b590 2024-09-19T04:26:16.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.637+0000 7effb77f3740 10 client.26305 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:16.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.637+0000 7effb77f3740 10 client.26305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:16.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.637+0000 7effb77f3740 20 client.26305 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:16.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.637+0000 7effb77f3740 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f21c28e640 0x55f21c290a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.637+0000 7effb77f3740 4 client.26305 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:16.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.637+0000 7effb77f3740 1 -- 172.21.15.107:0/452298299 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f21c1de950 con 0x55f21c28e640 2024-09-19T04:26:16.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.637+0000 7effae800640 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f21c28e640 0x55f21c290a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.638+0000 7effae800640 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f21c28e640 0x55f21c290a90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e327cac60647aa60 server_cookie=4031e12314a14590 in_seq=0 out_seq=0 2024-09-19T04:26:16.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.638+0000 7eff8f400640 10 client.26305 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7eff8f400640 1 -- 172.21.15.107:0/452298299 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f21c1de950 con 0x55f21c28e640 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7eff8f400640 10 client.26305 handle_command_reply: tid=0 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 client.26305 shutdown 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 -- 172.21.15.107:0/452298299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 msgr2=0x55f21c27bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 0x55f21c27bc80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff9c012710 tx=0x7eff9c012bd0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 -- 172.21.15.107:0/452298299 shutdown_connections 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f21c28e640 0x55f21c290a90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c27c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26b590 0x55f21c27bc80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 --2- 172.21.15.107:0/452298299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26ac70 0x55f21c27b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7eff8f400640 0 client.26305 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:16.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 -- 172.21.15.107:0/452298299 >> 172.21.15.107:0/452298299 conn(0x55f21c2672e0 msgr2=0x55f21c26cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:16.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 -- 172.21.15.107:0/452298299 shutdown_connections 2024-09-19T04:26:16.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 1 -- 172.21.15.107:0/452298299 wait complete. 2024-09-19T04:26:16.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.639+0000 7effb77f3740 20 client.26305 trim_cache size 0 max 16384 2024-09-19T04:26:16.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effb77f3740 1 Processor -- start 2024-09-19T04:26:16.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effb77f3740 1 -- start start 2024-09-19T04:26:16.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effb77f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c262060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effb77f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 0x55f21c2963f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effb77f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c296930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effb77f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ab420 con 0x55f21c26b590 2024-09-19T04:26:16.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effb77f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ad080 con 0x55f21c27e740 2024-09-19T04:26:16.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effb77f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7effb01abaf0 con 0x55f21c26ac70 2024-09-19T04:26:16.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 0x55f21c2963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effae800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c262060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 0x55f21c2963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43594/0 (socket says 172.21.15.107:43594) 2024-09-19T04:26:16.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effad400640 1 -- 172.21.15.107:0/785304642 learned_addr learned my addr 172.21.15.107:0/785304642 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:16.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.657+0000 7effae800640 1 --2- 172.21.15.107:0/785304642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c296930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.658+0000 7effae800640 1 -- 172.21.15.107:0/785304642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 msgr2=0x55f21c296930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:16.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.658+0000 7effae800640 1 --2- 172.21.15.107:0/785304642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c296930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.658+0000 7effae800640 1 -- 172.21.15.107:0/785304642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 msgr2=0x55f21c2963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.658+0000 7effae800640 1 --2- 172.21.15.107:0/785304642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 0x55f21c2963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.658+0000 7effae800640 1 -- 172.21.15.107:0/785304642 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21c279c90 con 0x55f21c26ac70 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.658+0000 7effae800640 1 --2- 172.21.15.107:0/785304642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c262060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7effa801b040 tx=0x7effa80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38239218ee0bc31c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.658+0000 7eff9a600640 1 -- 172.21.15.107:0/785304642 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa80045a0 con 0x55f21c26ac70 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.658+0000 7eff9a600640 1 -- 172.21.15.107:0/785304642 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8027440 con 0x55f21c26ac70 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.658+0000 7eff9a600640 1 -- 172.21.15.107:0/785304642 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa800f6e0 con 0x55f21c26ac70 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.659+0000 7effb77f3740 1 -- 172.21.15.107:0/785304642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 msgr2=0x55f21c262060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.659+0000 7effb77f3740 1 --2- 172.21.15.107:0/785304642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c262060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7effa801b040 tx=0x7effa80118d0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.659+0000 7effb77f3740 1 -- 172.21.15.107:0/785304642 shutdown_connections 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.659+0000 7effb77f3740 1 --2- 172.21.15.107:0/785304642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c296930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.659+0000 7effb77f3740 1 --2- 172.21.15.107:0/785304642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 0x55f21c2963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.659+0000 7effb77f3740 1 --2- 172.21.15.107:0/785304642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c262060 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.659+0000 7effb77f3740 1 -- 172.21.15.107:0/785304642 >> 172.21.15.107:0/785304642 conn(0x55f21c2672e0 msgr2=0x55f21c26cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:16.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.659+0000 7effb77f3740 1 -- 172.21.15.107:0/785304642 shutdown_connections 2024-09-19T04:26:16.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.659+0000 7effb77f3740 1 -- 172.21.15.107:0/785304642 wait complete. 2024-09-19T04:26:16.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.660+0000 7effb77f3740 1 -- start start 2024-09-19T04:26:16.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.660+0000 7effb77f3740 1 Processor -- start 2024-09-19T04:26:16.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.660+0000 7effb77f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c286eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.660+0000 7effb77f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 0x55f21c2821e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.660+0000 7effb77f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c282720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.660+0000 7effb77f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ac8f0 con 0x55f21c26b590 2024-09-19T04:26:16.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.660+0000 7effb77f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ac1c0 con 0x55f21c27e740 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.660+0000 7effb77f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7effb01a9400 con 0x55f21c26ac70 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.660+0000 7effb77f3740 10 client.0 _ll_register_callbacks cb 0x7eff9c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effae800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c286eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effae800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c286eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37522/0 (socket says 172.21.15.107:37522) 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effae800640 1 -- 172.21.15.107:0/990399522 learned_addr learned my addr 172.21.15.107:0/990399522 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effad400640 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 0x55f21c2821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effae800640 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c282720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effad400640 1 -- 172.21.15.107:0/990399522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 msgr2=0x55f21c286eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effad400640 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c286eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effad400640 1 -- 172.21.15.107:0/990399522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 msgr2=0x55f21c282720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:16.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effad400640 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c282720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effad400640 1 -- 172.21.15.107:0/990399522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21c282f00 con 0x55f21c26b590 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effae800640 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c286eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7effad400640 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 0x55f21c2821e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7eff9c000c00 tx=0x7eff9c004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafc115ae9c2587a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.661+0000 7eff8f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7eff8f400640 1 -- 172.21.15.107:0/990399522 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c026760 con 0x55f21c26b590 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7eff8f400640 1 -- 172.21.15.107:0/990399522 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7eff9c0269f0 con 0x55f21c26b590 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7effb77f3740 1 -- 172.21.15.107:0/990399522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa8027440 con 0x55f21c26b590 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7eff8f400640 1 -- 172.21.15.107:0/990399522 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7eff9c037050 con 0x55f21c26b590 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7effb77f3740 10 client.16302 fetch_fsmap learned FSMap version 37 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7effb77f3740 1 -- 172.21.15.107:0/990399522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f21c2f93e0 con 0x55f21c26b590 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7eff8f400640 1 -- 172.21.15.107:0/990399522 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8027440 con 0x55f21c26b590 2024-09-19T04:26:16.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7eff8f400640 1 -- 172.21.15.107:0/990399522 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7eff9c0035f0 con 0x55f21c26b590 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7effb77f3740 10 client.16302 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7effb77f3740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.662+0000 7effb77f3740 20 client.16302 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.663+0000 7effb77f3740 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f21c2f9c50 0x55f21c2fc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.663+0000 7effb77f3740 4 client.16302 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.663+0000 7effb77f3740 1 -- 172.21.15.107:0/990399522 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f21c1d8e60 con 0x55f21c2f9c50 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.663+0000 7effae800640 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f21c2f9c50 0x55f21c2fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.663+0000 7effae800640 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f21c2f9c50 0x55f21c2fc070 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f29139006e7155 server_cookie=2f813b375ca93892 in_seq=0 out_seq=0 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.663+0000 7eff8f400640 10 client.16302 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.663+0000 7eff8f400640 1 -- 172.21.15.107:0/990399522 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f21c1d8e60 con 0x55f21c2f9c50 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.663+0000 7eff8f400640 10 client.16302 handle_command_reply: tid=0 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 client.16302 shutdown 2024-09-19T04:26:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 -- 172.21.15.107:0/990399522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 msgr2=0x55f21c2821e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 0x55f21c2821e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7eff9c000c00 tx=0x7eff9c004250 comp rx=0 tx=0).stop 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 -- 172.21.15.107:0/990399522 shutdown_connections 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f21c2f9c50 0x55f21c2fc070 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f21c27e740 0x55f21c282720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f21c26b590 0x55f21c2821e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 --2- 172.21.15.107:0/990399522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f21c26ac70 0x55f21c286eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 -- 172.21.15.107:0/990399522 >> 172.21.15.107:0/990399522 conn(0x55f21c2672e0 msgr2=0x55f21c27a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7eff8f400640 0 client.16302 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 -- 172.21.15.107:0/990399522 shutdown_connections 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 1 -- 172.21.15.107:0/990399522 wait complete. 2024-09-19T04:26:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.664+0000 7effb77f3740 20 client.16302 trim_cache size 0 max 16384 2024-09-19T04:26:16.671 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:26:16.671 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:26:16.671 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:26:16.671 INFO:teuthology.orchestra.run.smithi107.stdout: "d14bf3ab-d395-47ed-bb47-117df7cb3a8e": { 2024-09-19T04:26:16.671 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:26:16.671 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d14bf3ab-d395-47ed-bb47-117df7cb3a8e", 2024-09-19T04:26:16.671 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:26:16.671 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:26:16.671 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:26:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:26:16.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 -- 172.21.15.107:0/919724184 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7eff940a33b0 msgr2=0x7eff940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:16.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7eff940a33b0 0x7eff940a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7effa00ed790 tx=0x7effa00f3a90 comp rx=0 tx=0).stop 2024-09-19T04:26:16.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 -- 172.21.15.107:0/919724184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0103e30 msgr2=0x7effb019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:16.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0103e30 0x7effb019a4a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7effa0004770 tx=0x7effa0003c10 comp rx=0 tx=0).stop 2024-09-19T04:26:16.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 -- 172.21.15.107:0/919724184 shutdown_connections 2024-09-19T04:26:16.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7eff940a33b0 0x7eff940a5870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb0108f60 0x7effb019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb01047e0 0x7effb019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 --2- 172.21.15.107:0/919724184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0103e30 0x7effb019a4a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:16.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 -- 172.21.15.107:0/919724184 >> 172.21.15.107:0/919724184 conn(0x7effb00ff8d0 msgr2=0x7effb0100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:16.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 -- 172.21.15.107:0/919724184 shutdown_connections 2024-09-19T04:26:16.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:16.665+0000 7effaf200640 1 -- 172.21.15.107:0/919724184 wait complete. 2024-09-19T04:26:16.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 210001 2.86 MB/sec execute 154 sec latency 731.756 ms 2024-09-19T04:26:17.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:17 smithi107 ceph-mon[30659]: pgmap v181: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-09-19T04:26:17.375 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:17 smithi145 ceph-mon[35096]: pgmap v181: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-09-19T04:26:17.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:17 smithi115 ceph-mon[35477]: pgmap v181: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-09-19T04:26:17.568 INFO:tasks.cephfs.filesystem:scrub status for tag:d14bf3ab-d395-47ed-bb47-117df7cb3a8e - {'path': '/', 'tag': 'd14bf3ab-d395-47ed-bb47-117df7cb3a8e', 'options': 'recursive,force'} 2024-09-19T04:26:17.673 INFO:tasks.workunit.client.0.smithi107.stdout: 1 210001 2.84 MB/sec execute 155 sec latency 1731.885 ms 2024-09-19T04:26:18.673 INFO:tasks.workunit.client.0.smithi107.stdout: 1 210001 2.82 MB/sec execute 156 sec latency 2731.987 ms 2024-09-19T04:26:19.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:19 smithi115 ceph-mon[35477]: pgmap v182: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 894 KiB/s wr, 91 op/s 2024-09-19T04:26:19.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:19 smithi145 ceph-mon[35096]: pgmap v182: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 894 KiB/s wr, 91 op/s 2024-09-19T04:26:19.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:19 smithi107 ceph-mon[30659]: pgmap v182: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 894 KiB/s wr, 91 op/s 2024-09-19T04:26:19.673 INFO:tasks.workunit.client.0.smithi107.stdout: 1 210688 2.83 MB/sec execute 157 sec latency 3354.472 ms 2024-09-19T04:26:20.673 INFO:tasks.workunit.client.0.smithi107.stdout: 1 210970 2.81 MB/sec execute 158 sec latency 788.541 ms 2024-09-19T04:26:21.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:21 smithi115 ceph-mon[35477]: pgmap v183: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 894 KiB/s wr, 91 op/s 2024-09-19T04:26:21.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:21 smithi145 ceph-mon[35096]: pgmap v183: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 894 KiB/s wr, 91 op/s 2024-09-19T04:26:21.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:21 smithi107 ceph-mon[30659]: pgmap v183: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 894 KiB/s wr, 91 op/s 2024-09-19T04:26:21.673 INFO:tasks.workunit.client.0.smithi107.stdout: 1 211480 2.80 MB/sec execute 159 sec latency 1590.671 ms 2024-09-19T04:26:22.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:22 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:26:22.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:22 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:26:22.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:22 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:26:22.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:22 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:26:22.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:22 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:26:22.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:22 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:26:22.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:22 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:26:22.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:22 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:26:22.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:22 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:26:22.673 INFO:tasks.workunit.client.0.smithi107.stdout: 1 213207 2.87 MB/sec execute 160 sec latency 10.139 ms 2024-09-19T04:26:23.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:23 smithi115 ceph-mon[35477]: pgmap v184: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-09-19T04:26:23.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:23 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:26:23.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:23 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:26:23.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:23 smithi145 ceph-mon[35096]: pgmap v184: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-09-19T04:26:23.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:23 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:26:23.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:23 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:26:23.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:23 smithi107 ceph-mon[30659]: pgmap v184: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-09-19T04:26:23.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:23 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:26:23.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:23 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:26:23.673 INFO:tasks.workunit.client.0.smithi107.stdout: 1 213541 2.85 MB/sec execute 161 sec latency 641.371 ms 2024-09-19T04:26:24.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 213721 2.84 MB/sec execute 162 sec latency 1184.798 ms 2024-09-19T04:26:25.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:25 smithi115 ceph-mon[35477]: pgmap v185: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 844 KiB/s wr, 77 op/s 2024-09-19T04:26:25.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:25 smithi145 ceph-mon[35096]: pgmap v185: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 844 KiB/s wr, 77 op/s 2024-09-19T04:26:25.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:25 smithi107 ceph-mon[30659]: pgmap v185: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 844 KiB/s wr, 77 op/s 2024-09-19T04:26:25.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 214512 2.85 MB/sec execute 163 sec latency 496.394 ms 2024-09-19T04:26:26.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 214512 2.83 MB/sec execute 164 sec latency 1496.510 ms 2024-09-19T04:26:27.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:27 smithi115 ceph-mon[35477]: pgmap v186: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-19T04:26:27.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:27 smithi145 ceph-mon[35096]: pgmap v186: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-19T04:26:27.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:27 smithi107 ceph-mon[30659]: pgmap v186: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-19T04:26:27.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 214512 2.81 MB/sec execute 165 sec latency 2496.581 ms 2024-09-19T04:26:28.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 214512 2.79 MB/sec execute 166 sec latency 3496.668 ms 2024-09-19T04:26:29.367 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:29 smithi107 ceph-mon[30659]: pgmap v187: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 810 KiB/s wr, 97 op/s 2024-09-19T04:26:29.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:29 smithi115 ceph-mon[35477]: pgmap v187: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 810 KiB/s wr, 97 op/s 2024-09-19T04:26:29.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:29 smithi145 ceph-mon[35096]: pgmap v187: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 810 KiB/s wr, 97 op/s 2024-09-19T04:26:29.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 215803 2.82 MB/sec execute 167 sec latency 3775.317 ms 2024-09-19T04:26:30.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 217089 2.84 MB/sec execute 168 sec latency 52.602 ms 2024-09-19T04:26:31.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:31 smithi115 ceph-mon[35477]: pgmap v188: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 810 KiB/s wr, 97 op/s 2024-09-19T04:26:31.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:31 smithi145 ceph-mon[35096]: pgmap v188: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 810 KiB/s wr, 97 op/s 2024-09-19T04:26:31.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:31 smithi107 ceph-mon[30659]: pgmap v188: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 810 KiB/s wr, 97 op/s 2024-09-19T04:26:31.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 217089 2.83 MB/sec execute 169 sec latency 1052.665 ms 2024-09-19T04:26:32.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 217089 2.81 MB/sec execute 170 sec latency 2052.780 ms 2024-09-19T04:26:33.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:33 smithi115 ceph-mon[35477]: pgmap v189: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 962 KiB/s wr, 129 op/s 2024-09-19T04:26:33.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:33 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:26:33.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:33 smithi145 ceph-mon[35096]: pgmap v189: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 962 KiB/s wr, 129 op/s 2024-09-19T04:26:33.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:33 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:26:33.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:33 smithi107 ceph-mon[30659]: pgmap v189: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 962 KiB/s wr, 129 op/s 2024-09-19T04:26:33.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:33 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:26:33.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 217089 2.79 MB/sec execute 171 sec latency 3052.852 ms 2024-09-19T04:26:34.674 INFO:tasks.workunit.client.0.smithi107.stdout: 1 217213 2.78 MB/sec execute 172 sec latency 3596.909 ms 2024-09-19T04:26:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:35 smithi115 ceph-mon[35477]: pgmap v190: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 694 KiB/s wr, 89 op/s 2024-09-19T04:26:35.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:35 smithi145 ceph-mon[35096]: pgmap v190: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 694 KiB/s wr, 89 op/s 2024-09-19T04:26:35.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:35 smithi107 ceph-mon[30659]: pgmap v190: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 694 KiB/s wr, 89 op/s 2024-09-19T04:26:35.675 INFO:tasks.workunit.client.0.smithi107.stdout: 1 217213 2.76 MB/sec execute 173 sec latency 1358.196 ms 2024-09-19T04:26:36.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:36 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:26:36.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:36 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:26:36.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:36 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:26:36.675 INFO:tasks.workunit.client.0.smithi107.stdout: 1 217213 2.75 MB/sec execute 174 sec latency 2358.254 ms 2024-09-19T04:26:37.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:37 smithi115 ceph-mon[35477]: pgmap v191: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 112 op/s 2024-09-19T04:26:37.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:37 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:26:37.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:37 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:26:37.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:37 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:26:37.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:37 smithi145 ceph-mon[35096]: pgmap v191: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 112 op/s 2024-09-19T04:26:37.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:37 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:26:37.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:37 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:26:37.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:37 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:26:37.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:37 smithi107 ceph-mon[30659]: pgmap v191: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 112 op/s 2024-09-19T04:26:37.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:37 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:26:37.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:37 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:26:37.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:37 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:26:37.675 INFO:tasks.workunit.client.0.smithi107.stdout: 1 217213 2.73 MB/sec execute 175 sec latency 3358.369 ms 2024-09-19T04:26:38.675 INFO:tasks.workunit.client.0.smithi107.stdout: 1 217213 2.72 MB/sec execute 176 sec latency 4358.516 ms 2024-09-19T04:26:39.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:39 smithi115 ceph-mon[35477]: pgmap v192: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 278 KiB/s wr, 55 op/s 2024-09-19T04:26:39.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:39 smithi145 ceph-mon[35096]: pgmap v192: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 278 KiB/s wr, 55 op/s 2024-09-19T04:26:39.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:39 smithi107 ceph-mon[30659]: pgmap v192: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 278 KiB/s wr, 55 op/s 2024-09-19T04:26:39.678 INFO:tasks.workunit.client.0.smithi107.stdout: 1 217874 2.73 MB/sec execute 177 sec latency 4901.066 ms 2024-09-19T04:26:40.678 INFO:tasks.workunit.client.0.smithi107.stdout: 1 218114 2.71 MB/sec execute 178 sec latency 457.943 ms 2024-09-19T04:26:41.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:41 smithi115 ceph-mon[35477]: pgmap v193: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 278 KiB/s wr, 55 op/s 2024-09-19T04:26:41.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:41 smithi145 ceph-mon[35096]: pgmap v193: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 278 KiB/s wr, 55 op/s 2024-09-19T04:26:41.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:41 smithi107 ceph-mon[30659]: pgmap v193: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 278 KiB/s wr, 55 op/s 2024-09-19T04:26:41.678 INFO:tasks.workunit.client.0.smithi107.stdout: 1 218114 2.70 MB/sec execute 179 sec latency 1312.114 ms 2024-09-19T04:26:42.679 INFO:tasks.workunit.client.0.smithi107.stdout: 1 218114 2.68 MB/sec execute 180 sec latency 2312.224 ms 2024-09-19T04:26:43.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:43 smithi115 ceph-mon[35477]: pgmap v194: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 609 KiB/s wr, 78 op/s 2024-09-19T04:26:43.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:43 smithi145 ceph-mon[35096]: pgmap v194: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 609 KiB/s wr, 78 op/s 2024-09-19T04:26:43.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:43 smithi107 ceph-mon[30659]: pgmap v194: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 609 KiB/s wr, 78 op/s 2024-09-19T04:26:43.679 INFO:tasks.workunit.client.0.smithi107.stdout: 1 218114 2.67 MB/sec execute 181 sec latency 3312.314 ms 2024-09-19T04:26:44.679 INFO:tasks.workunit.client.0.smithi107.stdout: 1 219173 2.69 MB/sec execute 182 sec latency 3851.035 ms 2024-09-19T04:26:45.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:45 smithi115 ceph-mon[35477]: pgmap v195: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 456 KiB/s wr, 45 op/s 2024-09-19T04:26:45.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:45 smithi145 ceph-mon[35096]: pgmap v195: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 456 KiB/s wr, 45 op/s 2024-09-19T04:26:45.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:45 smithi107 ceph-mon[30659]: pgmap v195: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 456 KiB/s wr, 45 op/s 2024-09-19T04:26:45.679 INFO:tasks.workunit.client.0.smithi107.stdout: 1 220375 2.72 MB/sec execute 183 sec latency 154.742 ms 2024-09-19T04:26:46.679 INFO:tasks.workunit.client.0.smithi107.stdout: 1 220632 2.70 MB/sec execute 184 sec latency 662.850 ms 2024-09-19T04:26:47.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:47 smithi115 ceph-mon[35477]: pgmap v196: 97 pgs: 97 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 603 KiB/s wr, 77 op/s 2024-09-19T04:26:47.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:47 smithi145 ceph-mon[35096]: pgmap v196: 97 pgs: 97 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 603 KiB/s wr, 77 op/s 2024-09-19T04:26:47.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:47 smithi107 ceph-mon[30659]: pgmap v196: 97 pgs: 97 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 603 KiB/s wr, 77 op/s 2024-09-19T04:26:47.569 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:26:47.679 INFO:tasks.workunit.client.0.smithi107.stdout: 1 220632 2.69 MB/sec execute 185 sec latency 1662.919 ms 2024-09-19T04:26:47.951 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:48.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:48 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:26:48.447 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:48 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:26:48.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:48 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:26:48.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.558+0000 7f0a0c800640 1 -- 172.21.15.107:0/4231590184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a04074620 msgr2=0x7f0a04074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.558+0000 7f0a0c800640 1 --2- 172.21.15.107:0/4231590184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a04074620 0x7f0a04074a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009900 tx=0x7f09f802a480 comp rx=0 tx=0).stop 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.559+0000 7f0a0c800640 1 -- 172.21.15.107:0/4231590184 shutdown_connections 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.559+0000 7f0a0c800640 1 --2- 172.21.15.107:0/4231590184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a0410e3c0 0x7f0a04110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.559+0000 7f0a0c800640 1 --2- 172.21.15.107:0/4231590184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a04074f40 0x7f0a0410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.559+0000 7f0a0c800640 1 --2- 172.21.15.107:0/4231590184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a04074620 0x7f0a04074a00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.559+0000 7f0a0c800640 1 -- 172.21.15.107:0/4231590184 >> 172.21.15.107:0/4231590184 conn(0x7f0a040fd8c0 msgr2=0x7f0a040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.559+0000 7f0a0c800640 1 -- 172.21.15.107:0/4231590184 shutdown_connections 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.559+0000 7f0a0c800640 1 -- 172.21.15.107:0/4231590184 wait complete. 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.560+0000 7f0a0c800640 1 Processor -- start 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.560+0000 7f0a0c800640 1 -- start start 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.560+0000 7f0a0c800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a04074620 0x7f0a0419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.560+0000 7f0a0c800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a04074f40 0x7f0a0419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.560+0000 7f0a0c800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a0410e3c0 0x7f0a0419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.560+0000 7f0a0c800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0a04114150 con 0x7f0a04074f40 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.560+0000 7f0a0c800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a04113fd0 con 0x7f0a04074620 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.560+0000 7f0a0c800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041142d0 con 0x7f0a0410e3c0 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a09e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a04074620 0x7f0a0419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a09e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a04074620 0x7f0a0419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:40218/0 (socket says 172.21.15.107:40218) 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a09e00640 1 -- 172.21.15.107:0/3418431318 learned_addr learned my addr 172.21.15.107:0/3418431318 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a09400640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a04074f40 0x7f0a0419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a0a800640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a0410e3c0 0x7f0a0419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a09e00640 1 -- 172.21.15.107:0/3418431318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a0410e3c0 msgr2=0x7f0a0419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a09e00640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a0410e3c0 0x7f0a0419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a09e00640 1 -- 172.21.15.107:0/3418431318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a04074f40 msgr2=0x7f0a0419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a09e00640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a04074f40 0x7f0a0419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a09e00640 1 -- 172.21.15.107:0/3418431318 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0419f640 con 0x7f0a04074620 2024-09-19T04:26:48.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.561+0000 7f0a09e00640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a04074620 0x7f0a0419a690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f09f8006010 tx=0x7f09f80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.562+0000 7f09fea00640 1 -- 172.21.15.107:0/3418431318 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f803f030 con 0x7f0a04074620 2024-09-19T04:26:48.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.562+0000 7f0a0c800640 1 -- 172.21.15.107:0/3418431318 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0419f8d0 con 0x7f0a04074620 2024-09-19T04:26:48.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.562+0000 7f09fea00640 1 -- 172.21.15.107:0/3418431318 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f8003ce0 con 0x7f0a04074620 2024-09-19T04:26:48.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.562+0000 7f09fea00640 1 -- 172.21.15.107:0/3418431318 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8035070 con 0x7f0a04074620 2024-09-19T04:26:48.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.562+0000 7f0a0c800640 1 -- 172.21.15.107:0/3418431318 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a041a7210 con 0x7f0a04074620 2024-09-19T04:26:48.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.565+0000 7f0a0f449740 1 Processor -- start 2024-09-19T04:26:48.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.565+0000 7f09fea00640 1 -- 172.21.15.107:0/3418431318 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f09f8042030 con 0x7f0a04074620 2024-09-19T04:26:48.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.565+0000 7f0a0f449740 1 -- start start 2024-09-19T04:26:48.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 0x561bbdcdcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 0x561bbdcea160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a0f449740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561bbdc433e0 con 0x561bbdce7d70 2024-09-19T04:26:48.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a0f449740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561bbdbead80 con 0x561bbdcdd4b0 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a0f449740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561bbdc3f3a0 con 0x561bbdcdcb90 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a09e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a0a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 0x561bbdcdcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a09e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:40234/0 (socket says 172.21.15.107:40234) 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a09e00640 1 -- 172.21.15.107:0/435496736 learned_addr learned my addr 172.21.15.107:0/435496736 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.566+0000 7f0a09400640 1 --2- 172.21.15.107:0/435496736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 0x561bbdcea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f0a09e00640 1 -- 172.21.15.107:0/435496736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 msgr2=0x561bbdcdcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f0a09e00640 1 --2- 172.21.15.107:0/435496736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 0x561bbdcdcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f0a09e00640 1 -- 172.21.15.107:0/435496736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 msgr2=0x561bbdcea160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f0a09e00640 1 --2- 172.21.15.107:0/435496736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 0x561bbdcea160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f0a09e00640 1 -- 172.21.15.107:0/435496736 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bbdc09ae0 con 0x561bbdcdd4b0 2024-09-19T04:26:48.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f0a0a800640 1 --2- 172.21.15.107:0/435496736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 0x561bbdcdcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:48.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f09fea00640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09ec0a33b0 0x7f09ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f0a09e00640 1 --2- 172.21.15.107:0/435496736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce7830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f09f80ed9c0 tx=0x7f09f80f8500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d487e6bcfe4a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f0a0a800640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09ec0a33b0 0x7f09ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f09fea00640 1 -- 172.21.15.107:0/3418431318 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f09f80f0340 con 0x7f0a04074620 2024-09-19T04:26:48.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f09f3000640 1 -- 172.21.15.107:0/435496736 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80b8a20 con 0x561bbdcdd4b0 2024-09-19T04:26:48.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f09f3000640 1 -- 172.21.15.107:0/435496736 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f803b580 con 0x561bbdcdd4b0 2024-09-19T04:26:48.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.567+0000 7f09f3000640 1 -- 172.21.15.107:0/435496736 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f803bcb0 con 0x561bbdcdd4b0 2024-09-19T04:26:48.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.568+0000 7f0a0f449740 1 -- 172.21.15.107:0/435496736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 msgr2=0x561bbdce7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.568+0000 7f0a0a800640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09ec0a33b0 0x7f09ec0a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0a00005fd0 tx=0x7f0a000096a0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.568+0000 7f0a0f449740 1 --2- 172.21.15.107:0/435496736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce7830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f09f80ed9c0 tx=0x7f09f80f8500 comp rx=0 tx=0).stop 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.568+0000 7f0a0f449740 1 -- 172.21.15.107:0/435496736 shutdown_connections 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.568+0000 7f0a0f449740 1 --2- 172.21.15.107:0/435496736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 0x561bbdcea160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.568+0000 7f0a0f449740 1 --2- 172.21.15.107:0/435496736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce7830 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.568+0000 7f0a0f449740 1 --2- 172.21.15.107:0/435496736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 0x561bbdcdcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.568+0000 7f0a0f449740 1 -- 172.21.15.107:0/435496736 >> 172.21.15.107:0/435496736 conn(0x561bbdcd9120 msgr2=0x561bbdcd9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.569+0000 7f0a0f449740 1 -- 172.21.15.107:0/435496736 shutdown_connections 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.569+0000 7f0a0f449740 1 -- 172.21.15.107:0/435496736 wait complete. 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.569+0000 7f0a0f449740 1 -- start start 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.569+0000 7f0a0f449740 1 Processor -- start 2024-09-19T04:26:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 0x561bbdce4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 0x561bbdce5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a0f449740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561bbdcea700 con 0x561bbdce7d70 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a0f449740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561bbdbfe0f0 con 0x561bbdcdd4b0 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a0f449740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561bbdcea880 con 0x561bbdcdcb90 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a0f449740 10 client.0 _ll_register_callbacks cb 0x561bbdbef9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a0a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 0x561bbdce5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a0a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 0x561bbdce5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49080/0 (socket says 172.21.15.107:49080) 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a0a800640 1 -- 172.21.15.107:0/2022265632 learned_addr learned my addr 172.21.15.107:0/2022265632 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a09e00640 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 0x561bbdce4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.570+0000 7f0a09400640 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.571+0000 7f0a0a800640 1 -- 172.21.15.107:0/2022265632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 msgr2=0x561bbdce4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.571+0000 7f0a0a800640 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 0x561bbdce4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.571+0000 7f0a0a800640 1 -- 172.21.15.107:0/2022265632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 msgr2=0x561bbdce5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.571+0000 7f0a0a800640 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.571+0000 7f0a0a800640 1 -- 172.21.15.107:0/2022265632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bbd9b43c0 con 0x561bbdce7d70 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.571+0000 7f0a0a800640 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 0x561bbdce5650 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0a000107d0 tx=0x7f0a00004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28f869e4f636dca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.571+0000 7f09e7400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.571+0000 7f09e7400640 1 -- 172.21.15.107:0/2022265632 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0001f880 con 0x561bbdce7d70 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.571+0000 7f09e7400640 1 -- 172.21.15.107:0/2022265632 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f0a0001fa20 con 0x561bbdce7d70 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.571+0000 7f0a0f449740 1 -- 172.21.15.107:0/2022265632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561bbdc50950 con 0x561bbdce7d70 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f09e7400640 1 -- 172.21.15.107:0/2022265632 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0a0001fc20 con 0x561bbdce7d70 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f0a0f449740 10 client.16314 fetch_fsmap learned FSMap version 39 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f0a0f449740 1 -- 172.21.15.107:0/2022265632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561bbdcf63c0 con 0x561bbdce7d70 2024-09-19T04:26:48.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f09e7400640 1 -- 172.21.15.107:0/2022265632 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561bbdc50950 con 0x561bbdce7d70 2024-09-19T04:26:48.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f09e7400640 1 -- 172.21.15.107:0/2022265632 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f0a00026070 con 0x561bbdce7d70 2024-09-19T04:26:48.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f0a0f449740 10 client.16314 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:26:48.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f0a0f449740 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:48.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f0a0f449740 20 client.16314 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:48.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f0a0f449740 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561bbdcf7990 0x561bbdcf9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f0a0f449740 4 client.16314 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:48.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.572+0000 7f0a0f449740 1 -- 172.21.15.107:0/2022265632 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bbdc4ae60 con 0x561bbdcf7990 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.573+0000 7f0a09e00640 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561bbdcf7990 0x561bbdcf9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.573+0000 7f0a09e00640 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561bbdcf7990 0x561bbdcf9df0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94bbc5ba9477fb84 server_cookie=419dc035b0f639de in_seq=0 out_seq=0 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.573+0000 7f09e7400640 10 client.16314 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f09e7400640 1 -- 172.21.15.107:0/2022265632 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561bbdc4ae60 con 0x561bbdcf7990 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f09e7400640 10 client.16314 handle_command_reply: tid=0 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f0a0f449740 1 client.16314 shutdown 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f0a0f449740 1 -- 172.21.15.107:0/2022265632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 msgr2=0x561bbdce5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f0a0f449740 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 0x561bbdce5650 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0a000107d0 tx=0x7f0a00004ab0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f0a0f449740 1 -- 172.21.15.107:0/2022265632 shutdown_connections 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f0a0f449740 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561bbdcf7990 0x561bbdcf9df0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f0a0f449740 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdce7d70 0x561bbdce5650 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f0a0f449740 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f0a0f449740 1 --2- 172.21.15.107:0/2022265632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdcdcb90 0x561bbdce4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f0a0f449740 1 -- 172.21.15.107:0/2022265632 >> 172.21.15.107:0/2022265632 conn(0x561bbdcd9120 msgr2=0x561bbdcdee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.574+0000 7f09e7400640 0 client.16314 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:48.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.575+0000 7f0a0f449740 1 -- 172.21.15.107:0/2022265632 shutdown_connections 2024-09-19T04:26:48.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.575+0000 7f0a0f449740 1 -- 172.21.15.107:0/2022265632 wait complete. 2024-09-19T04:26:48.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.575+0000 7f0a0f449740 20 client.16314 trim_cache size 0 max 16384 2024-09-19T04:26:48.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.592+0000 7f0a0f449740 1 Processor -- start 2024-09-19T04:26:48.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.592+0000 7f0a0f449740 1 -- start start 2024-09-19T04:26:48.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.592+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdcffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 0x561bbdce6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a0f449740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041afa40 con 0x561bbdcdcb90 2024-09-19T04:26:48.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a0f449740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041b16a0 con 0x561bbdcdd4b0 2024-09-19T04:26:48.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a0f449740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041b0110 con 0x561bbdce7d70 2024-09-19T04:26:48.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a09e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a09e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49092/0 (socket says 172.21.15.107:49092) 2024-09-19T04:26:48.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a09e00640 1 -- 172.21.15.107:0/4075216518 learned_addr learned my addr 172.21.15.107:0/4075216518 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:48.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a09400640 1 --2- 172.21.15.107:0/4075216518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdcffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a0a800640 1 --2- 172.21.15.107:0/4075216518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 0x561bbdce6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a09e00640 1 -- 172.21.15.107:0/4075216518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 msgr2=0x561bbdce6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a09e00640 1 --2- 172.21.15.107:0/4075216518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 0x561bbdce6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a09e00640 1 -- 172.21.15.107:0/4075216518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 msgr2=0x561bbdcffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a09e00640 1 --2- 172.21.15.107:0/4075216518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdcffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a09e00640 1 -- 172.21.15.107:0/4075216518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bbdcd9db0 con 0x561bbdcdcb90 2024-09-19T04:26:48.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.593+0000 7f0a0a800640 1 --2- 172.21.15.107:0/4075216518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 0x561bbdce6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:48.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.594+0000 7f0a09e00640 1 --2- 172.21.15.107:0/4075216518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcff700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f09f80f8940 tx=0x7f09f802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41b52b26b39b2f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.594+0000 7f09f2600640 1 -- 172.21.15.107:0/4075216518 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8049070 con 0x561bbdcdcb90 2024-09-19T04:26:48.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.594+0000 7f09f2600640 1 -- 172.21.15.107:0/4075216518 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f80bc780 con 0x561bbdcdcb90 2024-09-19T04:26:48.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.594+0000 7f09f2600640 1 -- 172.21.15.107:0/4075216518 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80bc0e0 con 0x561bbdcdcb90 2024-09-19T04:26:48.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.594+0000 7f0a0f449740 1 -- 172.21.15.107:0/4075216518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 msgr2=0x561bbdcff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.594+0000 7f0a0f449740 1 --2- 172.21.15.107:0/4075216518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcff700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f09f80f8940 tx=0x7f09f802a480 comp rx=0 tx=0).stop 2024-09-19T04:26:48.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.595+0000 7f0a0f449740 1 -- 172.21.15.107:0/4075216518 shutdown_connections 2024-09-19T04:26:48.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.595+0000 7f0a0f449740 1 --2- 172.21.15.107:0/4075216518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 0x561bbdce6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.595+0000 7f0a0f449740 1 --2- 172.21.15.107:0/4075216518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdcffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.595+0000 7f0a0f449740 1 --2- 172.21.15.107:0/4075216518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcff700 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.595+0000 7f0a0f449740 1 -- 172.21.15.107:0/4075216518 >> 172.21.15.107:0/4075216518 conn(0x561bbdcd9120 msgr2=0x561bbdce3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.595+0000 7f0a0f449740 1 -- 172.21.15.107:0/4075216518 shutdown_connections 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.595+0000 7f0a0f449740 1 -- 172.21.15.107:0/4075216518 wait complete. 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.595+0000 7f0a0f449740 1 -- start start 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.595+0000 7f0a0f449740 1 Processor -- start 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcf0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a0f449740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 0x561bbdce9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a0f449740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041b0f10 con 0x561bbdcdcb90 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a0f449740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041b07e0 con 0x561bbdcdd4b0 2024-09-19T04:26:48.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a0f449740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041ada20 con 0x561bbdce7d70 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a0f449740 10 client.0 _ll_register_callbacks cb 0x7f0a00010690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a09400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a09e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcf0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a09400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:40254/0 (socket says 172.21.15.107:40254) 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a09400640 1 -- 172.21.15.107:0/1346523323 learned_addr learned my addr 172.21.15.107:0/1346523323 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a0a800640 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 0x561bbdce9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a09400640 1 -- 172.21.15.107:0/1346523323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 msgr2=0x561bbdce9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a09400640 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 0x561bbdce9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a09400640 1 -- 172.21.15.107:0/1346523323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 msgr2=0x561bbdcf0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a09400640 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcf0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a09400640 1 -- 172.21.15.107:0/1346523323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bbdce98a0 con 0x561bbdcdd4b0 2024-09-19T04:26:48.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.596+0000 7f0a09e00640 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcf0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:48.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.597+0000 7f0a09400640 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce8bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005d00 tx=0x7f09f4012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c806467f87e6fd05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:48.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.597+0000 7f09e7400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:26:48.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.597+0000 7f09e7400640 1 -- 172.21.15.107:0/1346523323 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4021030 con 0x561bbdcdd4b0 2024-09-19T04:26:48.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.597+0000 7f0a0f449740 1 -- 172.21.15.107:0/1346523323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09f80bc780 con 0x561bbdcdd4b0 2024-09-19T04:26:48.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.597+0000 7f09e7400640 1 -- 172.21.15.107:0/1346523323 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f09f4010c30 con 0x561bbdcdd4b0 2024-09-19T04:26:48.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.597+0000 7f09e7400640 1 -- 172.21.15.107:0/1346523323 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f09f402a050 con 0x561bbdcdd4b0 2024-09-19T04:26:48.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.597+0000 7f0a0f449740 10 client.26369 fetch_fsmap learned FSMap version 39 2024-09-19T04:26:48.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.597+0000 7f0a0f449740 1 -- 172.21.15.107:0/1346523323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561bbdd62760 con 0x561bbdcdd4b0 2024-09-19T04:26:48.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.598+0000 7f09e7400640 1 -- 172.21.15.107:0/1346523323 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f80bc780 con 0x561bbdcdd4b0 2024-09-19T04:26:48.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.598+0000 7f09e7400640 1 -- 172.21.15.107:0/1346523323 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f09f40047a0 con 0x561bbdcdd4b0 2024-09-19T04:26:48.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.598+0000 7f0a0f449740 10 client.26369 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:26:48.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.598+0000 7f0a0f449740 10 client.26369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:26:48.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.598+0000 7f0a0f449740 20 client.26369 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:48.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.598+0000 7f0a0f449740 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561bbdd63050 0x561bbdd65470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:48.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.598+0000 7f0a0f449740 4 client.26369 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.598+0000 7f0a0f449740 1 -- 172.21.15.107:0/1346523323 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561bbdbc70c0 con 0x561bbdd63050 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.598+0000 7f0a09e00640 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561bbdd63050 0x561bbdd65470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a09e00640 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561bbdd63050 0x561bbdd65470 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5fa5e093880d31a server_cookie=c51e3152edfb15ad in_seq=0 out_seq=0 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f09e7400640 10 client.26369 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f09e7400640 1 -- 172.21.15.107:0/1346523323 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561bbdbc70c0 con 0x561bbdd63050 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f09e7400640 10 client.26369 handle_command_reply: tid=0 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 client.26369 shutdown 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 -- 172.21.15.107:0/1346523323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 msgr2=0x561bbdce8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce8bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005d00 tx=0x7f09f4012b60 comp rx=0 tx=0).stop 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 -- 172.21.15.107:0/1346523323 shutdown_connections 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561bbdd63050 0x561bbdd65470 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561bbdce7d70 0x561bbdce9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f09e7400640 0 client.26369 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bbdcdd4b0 0x561bbdce8bf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 --2- 172.21.15.107:0/1346523323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbdcdcb90 0x561bbdcf0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 -- 172.21.15.107:0/1346523323 >> 172.21.15.107:0/1346523323 conn(0x561bbdcd9120 msgr2=0x561bbdcde5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 -- 172.21.15.107:0/1346523323 shutdown_connections 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 1 -- 172.21.15.107:0/1346523323 wait complete. 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.599+0000 7f0a0f449740 20 client.26369 trim_cache size 0 max 16384 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:26:48.606 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:26:48.607 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:26:48.607 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 -- 172.21.15.107:0/3418431318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09ec0a33b0 msgr2=0x7f09ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09ec0a33b0 0x7f09ec0a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0a00005fd0 tx=0x7f0a000096a0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 -- 172.21.15.107:0/3418431318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a04074620 msgr2=0x7f0a0419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a04074620 0x7f0a0419a690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f09f8006010 tx=0x7f09f80042f0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 -- 172.21.15.107:0/3418431318 shutdown_connections 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09ec0a33b0 0x7f09ec0a5870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a0410e3c0 0x7f0a0419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a04074f40 0x7f0a0419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 --2- 172.21.15.107:0/3418431318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a04074620 0x7f0a0419a690 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:48.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 -- 172.21.15.107:0/3418431318 >> 172.21.15.107:0/3418431318 conn(0x7f0a040fd8c0 msgr2=0x7f0a040fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 -- 172.21.15.107:0/3418431318 shutdown_connections 2024-09-19T04:26:48.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:48.600+0000 7f0a0c800640 1 -- 172.21.15.107:0/3418431318 wait complete. 2024-09-19T04:26:48.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 220632 2.67 MB/sec execute 186 sec latency 2663.074 ms 2024-09-19T04:26:49.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:49 smithi115 ceph-mon[35477]: pgmap v197: 97 pgs: 97 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 478 KiB/s wr, 54 op/s 2024-09-19T04:26:49.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:49 smithi145 ceph-mon[35096]: pgmap v197: 97 pgs: 97 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 478 KiB/s wr, 54 op/s 2024-09-19T04:26:49.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:49 smithi107 ceph-mon[30659]: pgmap v197: 97 pgs: 97 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 478 KiB/s wr, 54 op/s 2024-09-19T04:26:49.550 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:26:49.551 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:26:49.679 INFO:tasks.workunit.client.0.smithi107.stdout: 1 220756 2.66 MB/sec execute 187 sec latency 3202.298 ms 2024-09-19T04:26:49.935 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:50.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.551+0000 7fbcc5400640 1 -- 172.21.15.107:0/2621358184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc00745c0 msgr2=0x7fbcc00749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.551+0000 7fbcc5400640 1 --2- 172.21.15.107:0/2621358184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc00745c0 0x7fbcc00749a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009900 tx=0x7fbcb002a480 comp rx=0 tx=0).stop 2024-09-19T04:26:50.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.551+0000 7fbcc5400640 1 -- 172.21.15.107:0/2621358184 shutdown_connections 2024-09-19T04:26:50.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.551+0000 7fbcc5400640 1 --2- 172.21.15.107:0/2621358184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc010e430 0x7fbcc0110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.551+0000 7fbcc5400640 1 --2- 172.21.15.107:0/2621358184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc0074ee0 0x7fbcc010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.551+0000 7fbcc5400640 1 --2- 172.21.15.107:0/2621358184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc00745c0 0x7fbcc00749a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.551+0000 7fbcc5400640 1 -- 172.21.15.107:0/2621358184 >> 172.21.15.107:0/2621358184 conn(0x7fbcc00fd8b0 msgr2=0x7fbcc00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:50.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.551+0000 7fbcc5400640 1 -- 172.21.15.107:0/2621358184 shutdown_connections 2024-09-19T04:26:50.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.551+0000 7fbcc5400640 1 -- 172.21.15.107:0/2621358184 wait complete. 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.552+0000 7fbcc5400640 1 Processor -- start 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.552+0000 7fbcc5400640 1 -- start start 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcc5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc00745c0 0x7fbcc019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcc5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc0074ee0 0x7fbcc019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcc5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc010e430 0x7fbcc019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcc5400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0114180 con 0x7fbcc0074ee0 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcc5400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0114000 con 0x7fbcc010e430 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcc5400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0114300 con 0x7fbcc00745c0 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcb7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc0074ee0 0x7fbcc019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcb7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc0074ee0 0x7fbcc019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39086/0 (socket says 172.21.15.107:39086) 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcb7e00640 1 -- 172.21.15.107:0/1500439425 learned_addr learned my addr 172.21.15.107:0/1500439425 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcbe800640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc00745c0 0x7fbcc019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.553+0000 7fbcbf200640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc010e430 0x7fbcc019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcb7e00640 1 -- 172.21.15.107:0/1500439425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc00745c0 msgr2=0x7fbcc019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcb7e00640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc00745c0 0x7fbcc019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcb7e00640 1 -- 172.21.15.107:0/1500439425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc010e430 msgr2=0x7fbcc019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcb7e00640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc010e430 0x7fbcc019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcb7e00640 1 -- 172.21.15.107:0/1500439425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc019f6d0 con 0x7fbcc0074ee0 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcb7e00640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc0074ee0 0x7fbcc019ac60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbca800e9f0 tx=0x7fbca800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcb7400640 1 -- 172.21.15.107:0/1500439425 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca800cd80 con 0x7fbcc0074ee0 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcb7400640 1 -- 172.21.15.107:0/1500439425 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca8010910 con 0x7fbcc0074ee0 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcc5400640 1 -- 172.21.15.107:0/1500439425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc019f9c0 con 0x7fbcc0074ee0 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcb7400640 1 -- 172.21.15.107:0/1500439425 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8018990 con 0x7fbcc0074ee0 2024-09-19T04:26:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.554+0000 7fbcc5400640 1 -- 172.21.15.107:0/1500439425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc01a72a0 con 0x7fbcc0074ee0 2024-09-19T04:26:50.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.555+0000 7fbcc5400640 1 -- 172.21.15.107:0/1500439425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc00749a0 con 0x7fbcc0074ee0 2024-09-19T04:26:50.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.557+0000 7fbcb7400640 1 -- 172.21.15.107:0/1500439425 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fbca8010ac0 con 0x7fbcc0074ee0 2024-09-19T04:26:50.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.559+0000 7fbcb7400640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbca40a3430 0x7fbca40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:50.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.560+0000 7fbcb7400640 1 -- 172.21.15.107:0/1500439425 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7fbca80cffa0 con 0x7fbcc0074ee0 2024-09-19T04:26:50.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.560+0000 7fbcbe800640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbca40a3430 0x7fbca40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:50.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.560+0000 7fbcbe800640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbca40a3430 0x7fbca40a58f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009230 tx=0x7fbcb00058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:50.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.564+0000 7fbcb7400640 1 -- 172.21.15.107:0/1500439425 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbca809c320 con 0x7fbcc0074ee0 2024-09-19T04:26:50.679 INFO:tasks.workunit.client.0.smithi107.stdout: 1 220756 2.65 MB/sec execute 188 sec latency 1370.189 ms 2024-09-19T04:26:50.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.776+0000 7fbcc5400640 1 -- 172.21.15.107:0/1500439425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcc01095e0 con 0x7fbcc0074ee0 2024-09-19T04:26:50.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.776+0000 7fbcb7400640 1 -- 172.21.15.107:0/1500439425 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12005 (secure 0 0 0) 0x7fbca80a0620 con 0x7fbcc0074ee0 2024-09-19T04:26:50.778 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:26:50.779 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":39,"btime":"2024-09-19T04:26:47:139640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:26:46.463362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:26:50.779 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 39 2024-09-19T04:26:50.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 -- 172.21.15.107:0/1500439425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbca40a3430 msgr2=0x7fbca40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbca40a3430 0x7fbca40a58f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009230 tx=0x7fbcb00058d0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 -- 172.21.15.107:0/1500439425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc0074ee0 msgr2=0x7fbcc019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:50.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc0074ee0 0x7fbcc019ac60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbca800e9f0 tx=0x7fbca800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 -- 172.21.15.107:0/1500439425 shutdown_connections 2024-09-19T04:26:50.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbca40a3430 0x7fbca40a58f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc010e430 0x7fbcc019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc0074ee0 0x7fbcc019ac60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 --2- 172.21.15.107:0/1500439425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbcc00745c0 0x7fbcc019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:50.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 -- 172.21.15.107:0/1500439425 >> 172.21.15.107:0/1500439425 conn(0x7fbcc00fd8b0 msgr2=0x7fbcc00fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:50.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 -- 172.21.15.107:0/1500439425 shutdown_connections 2024-09-19T04:26:50.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:50.779+0000 7fbcc5400640 1 -- 172.21.15.107:0/1500439425 wait complete. 2024-09-19T04:26:51.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:51 smithi115 ceph-mon[35477]: pgmap v198: 97 pgs: 97 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 478 KiB/s wr, 54 op/s 2024-09-19T04:26:51.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:51 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1500439425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:51.457 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:51 smithi107 ceph-mon[30659]: pgmap v198: 97 pgs: 97 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 478 KiB/s wr, 54 op/s 2024-09-19T04:26:51.457 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:51 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1500439425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:51.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:51 smithi145 ceph-mon[35096]: pgmap v198: 97 pgs: 97 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 478 KiB/s wr, 54 op/s 2024-09-19T04:26:51.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:51 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1500439425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:26:51.495 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:26:51.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 220756 2.63 MB/sec execute 189 sec latency 2370.272 ms 2024-09-19T04:26:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:52.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.422+0000 7f722ae00640 1 -- 172.21.15.107:0/214426364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241047e0 msgr2=0x7f7224108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.422+0000 7f722ae00640 1 --2- 172.21.15.107:0/214426364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241047e0 0x7f7224108830 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7210009900 tx=0x7f721002a410 comp rx=0 tx=0).stop 2024-09-19T04:26:52.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.422+0000 7f722ae00640 1 -- 172.21.15.107:0/214426364 shutdown_connections 2024-09-19T04:26:52.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.422+0000 7f722ae00640 1 --2- 172.21.15.107:0/214426364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224108f60 0x7f7224110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.422+0000 7f722ae00640 1 --2- 172.21.15.107:0/214426364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241047e0 0x7f7224108830 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.422+0000 7f722ae00640 1 --2- 172.21.15.107:0/214426364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7224103e30 0x7f7224104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.422+0000 7f722ae00640 1 -- 172.21.15.107:0/214426364 >> 172.21.15.107:0/214426364 conn(0x7f72240ff8d0 msgr2=0x7f7224101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.422+0000 7f722ae00640 1 -- 172.21.15.107:0/214426364 shutdown_connections 2024-09-19T04:26:52.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.422+0000 7f722ae00640 1 -- 172.21.15.107:0/214426364 wait complete. 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.423+0000 7f722ae00640 1 Processor -- start 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.423+0000 7f722ae00640 1 -- start start 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.423+0000 7f722ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224103e30 0x7f722419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.423+0000 7f722ae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241047e0 0x7f722419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.423+0000 7f722ae00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7224108f60 0x7f722419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.423+0000 7f722ae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7224114010 con 0x7f72241047e0 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.423+0000 7f722ae00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7224113e90 con 0x7f7224108f60 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.423+0000 7f722ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7224114190 con 0x7f7224103e30 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f7229000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241047e0 0x7f722419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f722a400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7224108f60 0x7f722419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f7229a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224103e30 0x7f722419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f7229000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241047e0 0x7f722419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39094/0 (socket says 172.21.15.107:39094) 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f7229000640 1 -- 172.21.15.107:0/2221486560 learned_addr learned my addr 172.21.15.107:0/2221486560 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f722a400640 1 -- 172.21.15.107:0/2221486560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224103e30 msgr2=0x7f722419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f722a400640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224103e30 0x7f722419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f722a400640 1 -- 172.21.15.107:0/2221486560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241047e0 msgr2=0x7f722419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f722a400640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241047e0 0x7f722419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f722a400640 1 -- 172.21.15.107:0/2221486560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722419f420 con 0x7f7224108f60 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f7229000640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241047e0 0x7f722419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.424+0000 7f722a400640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7224108f60 0x7f722419ed40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f721c00da10 tx=0x7f721c00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.425+0000 7f7222000640 1 -- 172.21.15.107:0/2221486560 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c00bd70 con 0x7f7224108f60 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.425+0000 7f7222000640 1 -- 172.21.15.107:0/2221486560 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f721c003e50 con 0x7f7224108f60 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.425+0000 7f7222000640 1 -- 172.21.15.107:0/2221486560 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c0106a0 con 0x7f7224108f60 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.425+0000 7f722ae00640 1 -- 172.21.15.107:0/2221486560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f722419f710 con 0x7f7224108f60 2024-09-19T04:26:52.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.425+0000 7f722ae00640 1 -- 172.21.15.107:0/2221486560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72241a6ff0 con 0x7f7224108f60 2024-09-19T04:26:52.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.428+0000 7f722f04a740 1 Processor -- start 2024-09-19T04:26:52.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.428+0000 7f7222000640 1 -- 172.21.15.107:0/2221486560 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f721c003970 con 0x7f7224108f60 2024-09-19T04:26:52.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.428+0000 7f722f04a740 1 -- start start 2024-09-19T04:26:52.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.428+0000 7f722f04a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 0x562621d71220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f722f04a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d71760 0x562621d7bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f722f04a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d7c020 0x562621d7e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f722f04a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562621cd73e0 con 0x562621d7c020 2024-09-19T04:26:52.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f722f04a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562621c7ed80 con 0x562621d71760 2024-09-19T04:26:52.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f722f04a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562621cd33a0 con 0x562621d70e40 2024-09-19T04:26:52.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 0x562621d71220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d71760 0x562621d7bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d71760 0x562621d7bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57708/0 (socket says 172.21.15.107:57708) 2024-09-19T04:26:52.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229a00640 1 -- 172.21.15.107:0/30167249 learned_addr learned my addr 172.21.15.107:0/30167249 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:52.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229000640 1 --2- 172.21.15.107:0/30167249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d7c020 0x562621d7e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229a00640 1 -- 172.21.15.107:0/30167249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 msgr2=0x562621d71220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229a00640 1 --2- 172.21.15.107:0/30167249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 0x562621d71220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229a00640 1 -- 172.21.15.107:0/30167249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d7c020 msgr2=0x562621d7e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229a00640 1 --2- 172.21.15.107:0/30167249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d7c020 0x562621d7e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229a00640 1 -- 172.21.15.107:0/30167249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562621c9dae0 con 0x562621d71760 2024-09-19T04:26:52.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7229000640 1 --2- 172.21.15.107:0/30167249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d7c020 0x562621d7e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:52.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.429+0000 7f7222000640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f720c0a3430 0x7f720c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.430+0000 7f7229a00640 1 --2- 172.21.15.107:0/30167249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d71760 0x562621d7bae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7218009cf0 tx=0x7f7218011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d462c479ef266f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.430+0000 7f7222000640 1 -- 172.21.15.107:0/2221486560 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f721c0cffb0 con 0x7f7224108f60 2024-09-19T04:26:52.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.430+0000 7f7229000640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f720c0a3430 0x7f720c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.430+0000 7f7216a00640 1 -- 172.21.15.107:0/30167249 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218004230 con 0x562621d71760 2024-09-19T04:26:52.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.430+0000 7f7216a00640 1 -- 172.21.15.107:0/30167249 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72180043d0 con 0x562621d71760 2024-09-19T04:26:52.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.430+0000 7f7216a00640 1 -- 172.21.15.107:0/30167249 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218019960 con 0x562621d71760 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.431+0000 7f722f04a740 1 -- 172.21.15.107:0/30167249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d71760 msgr2=0x562621d7bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.431+0000 7f7229000640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f720c0a3430 0x7f720c0a58f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7210031520 tx=0x7f72100046c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.431+0000 7f722f04a740 1 --2- 172.21.15.107:0/30167249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d71760 0x562621d7bae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7218009cf0 tx=0x7f7218011810 comp rx=0 tx=0).stop 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.431+0000 7f722f04a740 1 -- 172.21.15.107:0/30167249 shutdown_connections 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.431+0000 7f722f04a740 1 --2- 172.21.15.107:0/30167249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d7c020 0x562621d7e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.431+0000 7f722f04a740 1 --2- 172.21.15.107:0/30167249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d71760 0x562621d7bae0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.431+0000 7f722f04a740 1 --2- 172.21.15.107:0/30167249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 0x562621d71220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.431+0000 7f722f04a740 1 -- 172.21.15.107:0/30167249 >> 172.21.15.107:0/30167249 conn(0x562621d6d3d0 msgr2=0x562621d6d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.431+0000 7f722f04a740 1 -- 172.21.15.107:0/30167249 shutdown_connections 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.431+0000 7f722f04a740 1 -- 172.21.15.107:0/30167249 wait complete. 2024-09-19T04:26:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.432+0000 7f722f04a740 1 -- start start 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.432+0000 7f722f04a740 1 Processor -- start 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.432+0000 7f722f04a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d70e40 0x562621d78f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.432+0000 7f722f04a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 0x562621d79490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.432+0000 7f722f04a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d7c020 0x562621d799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.432+0000 7f722f04a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562621d7ebf0 con 0x562621d71760 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.432+0000 7f722f04a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562621c920f0 con 0x562621d70e40 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.432+0000 7f722f04a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562621d7ed70 con 0x562621d7c020 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.432+0000 7f722f04a740 10 client.0 _ll_register_callbacks cb 0x562621c79bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f7229a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d70e40 0x562621d78f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f7229000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 0x562621d79490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f7229a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d70e40 0x562621d78f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57712/0 (socket says 172.21.15.107:57712) 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f7229a00640 1 -- 172.21.15.107:0/2838776429 learned_addr learned my addr 172.21.15.107:0/2838776429 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f722a400640 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d7c020 0x562621d799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f7229a00640 1 -- 172.21.15.107:0/2838776429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d7c020 msgr2=0x562621d799d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f7229a00640 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d7c020 0x562621d799d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f7229a00640 1 -- 172.21.15.107:0/2838776429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 msgr2=0x562621d79490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f7229a00640 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 0x562621d79490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f7229a00640 1 -- 172.21.15.107:0/2838776429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562621a483c0 con 0x562621d70e40 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f7229a00640 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d70e40 0x562621d78f50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f72180195c0 tx=0x7f7218003dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33a964bc1d840aa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.433+0000 7f720b400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.434+0000 7f720b400640 1 -- 172.21.15.107:0/2838776429 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218027070 con 0x562621d70e40 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.434+0000 7f720b400640 1 -- 172.21.15.107:0/2838776429 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f7218005040 con 0x562621d70e40 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.434+0000 7f722f04a740 1 -- 172.21.15.107:0/2838776429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72180043d0 con 0x562621d70e40 2024-09-19T04:26:52.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.434+0000 7f720b400640 1 -- 172.21.15.107:0/2838776429 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7218033050 con 0x562621d70e40 2024-09-19T04:26:52.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.434+0000 7f722f04a740 10 client.26387 fetch_fsmap learned FSMap version 39 2024-09-19T04:26:52.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.434+0000 7f722f04a740 1 -- 172.21.15.107:0/2838776429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562621d8ac80 con 0x562621d70e40 2024-09-19T04:26:52.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.434+0000 7f720b400640 1 -- 172.21.15.107:0/2838776429 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72180043d0 con 0x562621d70e40 2024-09-19T04:26:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.435+0000 7f720b400640 1 -- 172.21.15.107:0/2838776429 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f7218004bb0 con 0x562621d70e40 2024-09-19T04:26:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.435+0000 7f722f04a740 10 client.26387 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.435+0000 7f722f04a740 10 client.26387 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.435+0000 7f722f04a740 20 client.26387 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.435+0000 7f722f04a740 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x562621d8c030 0x562621d8e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.435+0000 7f722f04a740 4 client.26387 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.435+0000 7f722f04a740 1 -- 172.21.15.107:0/2838776429 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562621ce4950 con 0x562621d8c030 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.435+0000 7f7229000640 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x562621d8c030 0x562621d8e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.436+0000 7f7229000640 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x562621d8c030 0x562621d8e420 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=379a4c8408017d server_cookie=69d774ed591f09f7 in_seq=0 out_seq=0 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.436+0000 7f720b400640 10 client.26387 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f720b400640 1 -- 172.21.15.107:0/2838776429 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562621ce4950 con 0x562621d8c030 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f720b400640 10 client.26387 handle_command_reply: tid=0 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 client.26387 shutdown 2024-09-19T04:26:52.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 -- 172.21.15.107:0/2838776429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d70e40 msgr2=0x562621d78f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d70e40 0x562621d78f50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f72180195c0 tx=0x7f7218003dc0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 -- 172.21.15.107:0/2838776429 shutdown_connections 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x562621d8c030 0x562621d8e420 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d7c020 0x562621d799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 0x562621d79490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 --2- 172.21.15.107:0/2838776429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d70e40 0x562621d78f50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 -- 172.21.15.107:0/2838776429 >> 172.21.15.107:0/2838776429 conn(0x562621d6d3d0 msgr2=0x562621d730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f720b400640 0 client.26387 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 -- 172.21.15.107:0/2838776429 shutdown_connections 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 1 -- 172.21.15.107:0/2838776429 wait complete. 2024-09-19T04:26:52.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.437+0000 7f722f04a740 20 client.26387 trim_cache size 0 max 16384 2024-09-19T04:26:52.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f722f04a740 1 Processor -- start 2024-09-19T04:26:52.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f722f04a740 1 -- start start 2024-09-19T04:26:52.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f722f04a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 0x562621d78830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f722f04a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 0x562621d7a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f722f04a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d7c020 0x562621d93bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f722f04a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72241af830 con 0x562621d71760 2024-09-19T04:26:52.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f722f04a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f72241b1490 con 0x562621d7c020 2024-09-19T04:26:52.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f722f04a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f72241aff00 con 0x562621d70e40 2024-09-19T04:26:52.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f7229000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 0x562621d7a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f7229000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 0x562621d7a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39120/0 (socket says 172.21.15.107:39120) 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f7229000640 1 -- 172.21.15.107:0/492564654 learned_addr learned my addr 172.21.15.107:0/492564654 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f722a400640 1 --2- 172.21.15.107:0/492564654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d7c020 0x562621d93bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.451+0000 7f7229a00640 1 --2- 172.21.15.107:0/492564654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 0x562621d78830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.452+0000 7f7229000640 1 -- 172.21.15.107:0/492564654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 msgr2=0x562621d78830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.452+0000 7f7229000640 1 --2- 172.21.15.107:0/492564654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 0x562621d78830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.452+0000 7f7229000640 1 -- 172.21.15.107:0/492564654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d7c020 msgr2=0x562621d93bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.452+0000 7f7229000640 1 --2- 172.21.15.107:0/492564654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d7c020 0x562621d93bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.452+0000 7f7229000640 1 -- 172.21.15.107:0/492564654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562621d71590 con 0x562621d71760 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.452+0000 7f7229a00640 1 --2- 172.21.15.107:0/492564654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 0x562621d78830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:52.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.452+0000 7f7229000640 1 --2- 172.21.15.107:0/492564654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 0x562621d7a7b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7210054fd0 tx=0x7f72100314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb04ae07547ddb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.452+0000 7f7216000640 1 -- 172.21.15.107:0/492564654 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721005a070 con 0x562621d71760 2024-09-19T04:26:52.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.452+0000 7f7216000640 1 -- 172.21.15.107:0/492564654 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7210003cd0 con 0x562621d71760 2024-09-19T04:26:52.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.452+0000 7f7216000640 1 -- 172.21.15.107:0/492564654 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210054c80 con 0x562621d71760 2024-09-19T04:26:52.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.453+0000 7f722f04a740 1 -- 172.21.15.107:0/492564654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 msgr2=0x562621d7a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.453+0000 7f722f04a740 1 --2- 172.21.15.107:0/492564654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 0x562621d7a7b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7210054fd0 tx=0x7f72100314b0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.453+0000 7f722f04a740 1 -- 172.21.15.107:0/492564654 shutdown_connections 2024-09-19T04:26:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.453+0000 7f722f04a740 1 --2- 172.21.15.107:0/492564654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d7c020 0x562621d93bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.453+0000 7f722f04a740 1 --2- 172.21.15.107:0/492564654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d71760 0x562621d7a7b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.453+0000 7f722f04a740 1 --2- 172.21.15.107:0/492564654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d70e40 0x562621d78830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.453+0000 7f722f04a740 1 -- 172.21.15.107:0/492564654 >> 172.21.15.107:0/492564654 conn(0x562621d6d3d0 msgr2=0x562621d730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.453+0000 7f722f04a740 1 -- 172.21.15.107:0/492564654 shutdown_connections 2024-09-19T04:26:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.453+0000 7f722f04a740 1 -- 172.21.15.107:0/492564654 wait complete. 2024-09-19T04:26:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.453+0000 7f722f04a740 1 -- start start 2024-09-19T04:26:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f722f04a740 1 Processor -- start 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f722f04a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d70e40 0x562621d7ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f722f04a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d71760 0x562621d80460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f722f04a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d7c020 0x562621d809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f722f04a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72241b0d00 con 0x562621d70e40 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f722f04a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f72241b05d0 con 0x562621d7c020 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f722f04a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f72241ad810 con 0x562621d71760 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f722f04a740 10 client.0 _ll_register_callbacks cb 0x7f7210009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f7229a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d70e40 0x562621d7ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f7229a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d70e40 0x562621d7ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39132/0 (socket says 172.21.15.107:39132) 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.454+0000 7f7229a00640 1 -- 172.21.15.107:0/2812190937 learned_addr learned my addr 172.21.15.107:0/2812190937 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f722a400640 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d7c020 0x562621d809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f7229000640 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d71760 0x562621d80460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f7229a00640 1 -- 172.21.15.107:0/2812190937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d71760 msgr2=0x562621d80460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f7229a00640 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d71760 0x562621d80460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f7229a00640 1 -- 172.21.15.107:0/2812190937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d7c020 msgr2=0x562621d809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f7229a00640 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d7c020 0x562621d809a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f7229a00640 1 -- 172.21.15.107:0/2812190937 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562621df5d80 con 0x562621d70e40 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f7229a00640 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d70e40 0x562621d7ff20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7218000c00 tx=0x7f7218003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdd77e82a64293d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f720b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f720b400640 1 -- 172.21.15.107:0/2812190937 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721803c060 con 0x562621d70e40 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f720b400640 1 -- 172.21.15.107:0/2812190937 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f7218005000 con 0x562621d70e40 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.455+0000 7f722f04a740 1 -- 172.21.15.107:0/2812190937 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7210003cd0 con 0x562621d70e40 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f720b400640 1 -- 172.21.15.107:0/2812190937 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7218027070 con 0x562621d70e40 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f722f04a740 10 client.16344 fetch_fsmap learned FSMap version 39 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f722f04a740 1 -- 172.21.15.107:0/2812190937 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562621df7060 con 0x562621d70e40 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f720b400640 1 -- 172.21.15.107:0/2812190937 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7210003cd0 con 0x562621d70e40 2024-09-19T04:26:52.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f720b400640 1 -- 172.21.15.107:0/2812190937 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f7218041460 con 0x562621d70e40 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f722f04a740 10 client.16344 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f722f04a740 10 client.16344 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f722f04a740 20 client.16344 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f722f04a740 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x562621df7720 0x562621df9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f722f04a740 4 client.16344 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f722f04a740 1 -- 172.21.15.107:0/2812190937 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562621cdee60 con 0x562621df7720 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.456+0000 7f7229000640 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x562621df7720 0x562621df9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f7229000640 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x562621df7720 0x562621df9b10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=591e4a7342dc01e6 server_cookie=21bcd819e32c29c6 in_seq=0 out_seq=0 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f720b400640 10 client.16344 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f720b400640 1 -- 172.21.15.107:0/2812190937 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562621cdee60 con 0x562621df7720 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f720b400640 10 client.16344 handle_command_reply: tid=0 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f722f04a740 1 client.16344 shutdown 2024-09-19T04:26:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f722f04a740 1 -- 172.21.15.107:0/2812190937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d70e40 msgr2=0x562621d7ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f722f04a740 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d70e40 0x562621d7ff20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7218000c00 tx=0x7f7218003cc0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f722f04a740 1 -- 172.21.15.107:0/2812190937 shutdown_connections 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f722f04a740 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x562621df7720 0x562621df9b10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f722f04a740 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562621d7c020 0x562621d809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f722f04a740 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562621d71760 0x562621d80460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f722f04a740 1 --2- 172.21.15.107:0/2812190937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621d70e40 0x562621d7ff20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f720b400640 0 client.16344 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.457+0000 7f722f04a740 1 -- 172.21.15.107:0/2812190937 >> 172.21.15.107:0/2812190937 conn(0x562621d6d3d0 msgr2=0x562621d6bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.458+0000 7f722f04a740 1 -- 172.21.15.107:0/2812190937 shutdown_connections 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.458+0000 7f722f04a740 1 -- 172.21.15.107:0/2812190937 wait complete. 2024-09-19T04:26:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.458+0000 7f722f04a740 20 client.16344 trim_cache size 0 max 16384 2024-09-19T04:26:52.465 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:26:52.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.458+0000 7f722ae00640 1 -- 172.21.15.107:0/2221486560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f720c0a3430 msgr2=0x7f720c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.458+0000 7f722ae00640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f720c0a3430 0x7f720c0a58f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7210031520 tx=0x7f72100046c0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.458+0000 7f722ae00640 1 -- 172.21.15.107:0/2221486560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7224108f60 msgr2=0x7f722419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.458+0000 7f722ae00640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7224108f60 0x7f722419ed40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f721c00da10 tx=0x7f721c00ded0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.459+0000 7f722ae00640 1 -- 172.21.15.107:0/2221486560 shutdown_connections 2024-09-19T04:26:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.459+0000 7f722ae00640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f720c0a3430 0x7f720c0a58f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.459+0000 7f722ae00640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7224108f60 0x7f722419ed40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.459+0000 7f722ae00640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241047e0 0x7f722419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.459+0000 7f722ae00640 1 --2- 172.21.15.107:0/2221486560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224103e30 0x7f722419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.459+0000 7f722ae00640 1 -- 172.21.15.107:0/2221486560 >> 172.21.15.107:0/2221486560 conn(0x7f72240ff8d0 msgr2=0x7f7224100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.459+0000 7f722ae00640 1 -- 172.21.15.107:0/2221486560 shutdown_connections 2024-09-19T04:26:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:52.459+0000 7f722ae00640 1 -- 172.21.15.107:0/2221486560 wait complete. 2024-09-19T04:26:52.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 220756 2.62 MB/sec execute 190 sec latency 3370.328 ms 2024-09-19T04:26:53.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:53 smithi107 ceph-mon[30659]: pgmap v199: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 895 KiB/s wr, 86 op/s 2024-09-19T04:26:53.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:53 smithi115 ceph-mon[35477]: pgmap v199: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 895 KiB/s wr, 86 op/s 2024-09-19T04:26:53.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:53 smithi145 ceph-mon[35096]: pgmap v199: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 895 KiB/s wr, 86 op/s 2024-09-19T04:26:53.482 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:26:53.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 220756 2.60 MB/sec execute 191 sec latency 4370.385 ms 2024-09-19T04:26:53.853 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:54.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.452+0000 7f5bcce00640 1 -- 172.21.15.107:0/869197237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bc8102360 msgr2=0x7f5bc810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:54.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.452+0000 7f5bcce00640 1 --2- 172.21.15.107:0/869197237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bc8102360 0x7f5bc810e890 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009cd0 tx=0x7f5bc002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.452+0000 7f5bcce00640 1 -- 172.21.15.107:0/869197237 shutdown_connections 2024-09-19T04:26:54.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.452+0000 7f5bcce00640 1 --2- 172.21.15.107:0/869197237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bc8102360 0x7f5bc810e890 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.452+0000 7f5bcce00640 1 --2- 172.21.15.107:0/869197237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bc81019c0 0x7f5bc8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.452+0000 7f5bcce00640 1 --2- 172.21.15.107:0/869197237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc81079c0 0x7f5bc8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.452+0000 7f5bcce00640 1 -- 172.21.15.107:0/869197237 >> 172.21.15.107:0/869197237 conn(0x7f5bc80fd740 msgr2=0x7f5bc80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:54.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.452+0000 7f5bcce00640 1 -- 172.21.15.107:0/869197237 shutdown_connections 2024-09-19T04:26:54.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.453+0000 7f5bcce00640 1 -- 172.21.15.107:0/869197237 wait complete. 2024-09-19T04:26:54.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.454+0000 7f5bcce00640 1 Processor -- start 2024-09-19T04:26:54.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.454+0000 7f5bcce00640 1 -- start start 2024-09-19T04:26:54.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.454+0000 7f5bcce00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc81019c0 0x7f5bc819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.454+0000 7f5bcce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bc8102360 0x7f5bc819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.454+0000 7f5bcce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bc81079c0 0x7f5bc819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.454+0000 7f5bcce00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8111e10 con 0x7f5bc81079c0 2024-09-19T04:26:54.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.454+0000 7f5bcce00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8111c90 con 0x7f5bc81019c0 2024-09-19T04:26:54.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.454+0000 7f5bcce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8111f90 con 0x7f5bc8102360 2024-09-19T04:26:54.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc81019c0 0x7f5bc819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bc8102360 0x7f5bc819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc81019c0 0x7f5bc819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57758/0 (socket says 172.21.15.107:57758) 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc7400640 1 -- 172.21.15.107:0/1151891008 learned_addr learned my addr 172.21.15.107:0/1151891008 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc7e00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bc81079c0 0x7f5bc819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc6a00640 1 -- 172.21.15.107:0/1151891008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc81019c0 msgr2=0x7f5bc819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc81019c0 0x7f5bc819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc6a00640 1 -- 172.21.15.107:0/1151891008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bc81079c0 msgr2=0x7f5bc819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bc81079c0 0x7f5bc819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc6a00640 1 -- 172.21.15.107:0/1151891008 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc819f440 con 0x7f5bc8102360 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc7400640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc81019c0 0x7f5bc819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.455+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bc8102360 0x7f5bc819a9d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400e9f0 tx=0x7f5bb400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.456+0000 7f5bbfe00640 1 -- 172.21.15.107:0/1151891008 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400ccc0 con 0x7f5bc8102360 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.456+0000 7f5bbfe00640 1 -- 172.21.15.107:0/1151891008 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb4010910 con 0x7f5bc8102360 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.456+0000 7f5bcce00640 1 -- 172.21.15.107:0/1151891008 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc819f730 con 0x7f5bc8102360 2024-09-19T04:26:54.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.456+0000 7f5bbfe00640 1 -- 172.21.15.107:0/1151891008 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4018960 con 0x7f5bc8102360 2024-09-19T04:26:54.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.456+0000 7f5bcce00640 1 -- 172.21.15.107:0/1151891008 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81a7010 con 0x7f5bc8102360 2024-09-19T04:26:54.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.459+0000 7f5bd0dd9740 1 Processor -- start 2024-09-19T04:26:54.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.459+0000 7f5bd0dd9740 1 -- start start 2024-09-19T04:26:54.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.459+0000 7f5bbfe00640 1 -- 172.21.15.107:0/1151891008 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f5bb4010ac0 con 0x7f5bc8102360 2024-09-19T04:26:54.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 0x55d2419ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d2419f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d2419fb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d2419543e0 con 0x55d2419f9020 2024-09-19T04:26:54.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d2418fbd80 con 0x55d2419ede40 2024-09-19T04:26:54.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d2419503a0 con 0x55d2419ee760 2024-09-19T04:26:54.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 0x55d2419ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d2419f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d2419fb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 0x55d2419ee220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57774/0 (socket says 172.21.15.107:57774) 2024-09-19T04:26:54.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc7e00640 1 -- 172.21.15.107:0/4070949245 learned_addr learned my addr 172.21.15.107:0/4070949245 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d2419f8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55868/0 (socket says 172.21.15.107:55868) 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc7e00640 1 -- 172.21.15.107:0/4070949245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 msgr2=0x55d2419f8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc7e00640 1 --2- 172.21.15.107:0/4070949245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d2419f8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc7e00640 1 -- 172.21.15.107:0/4070949245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 msgr2=0x55d2419fb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc7e00640 1 --2- 172.21.15.107:0/4070949245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d2419fb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.460+0000 7f5bc7e00640 1 -- 172.21.15.107:0/4070949245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d24191aae0 con 0x55d2419ede40 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.461+0000 7f5bc7400640 1 --2- 172.21.15.107:0/4070949245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d2419f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.461+0000 7f5bc7e00640 1 --2- 172.21.15.107:0/4070949245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 0x55d2419ee220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009cd0 tx=0x7f5bc0009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8e3bc8a9b59275 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.461+0000 7f5bbfe00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5bac0a3430 0x7f5bac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.461+0000 7f5bbcc00640 1 -- 172.21.15.107:0/4070949245 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0049070 con 0x55d2419ede40 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.461+0000 7f5bbcc00640 1 -- 172.21.15.107:0/4070949245 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc0036040 con 0x55d2419ede40 2024-09-19T04:26:54.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.461+0000 7f5bbcc00640 1 -- 172.21.15.107:0/4070949245 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0004040 con 0x55d2419ede40 2024-09-19T04:26:54.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.461+0000 7f5bbfe00640 1 -- 172.21.15.107:0/1151891008 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f5bb40cffe0 con 0x7f5bc8102360 2024-09-19T04:26:54.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.462+0000 7f5bc7400640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5bac0a3430 0x7f5bac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.462+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/4070949245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 msgr2=0x55d2419ee220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.462+0000 7f5bc7400640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5bac0a3430 0x7f5bac0a58f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55d2419eb230 tx=0x7f5bb8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.462+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/4070949245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 0x55d2419ee220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009cd0 tx=0x7f5bc0009a20 comp rx=0 tx=0).stop 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.463+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/4070949245 shutdown_connections 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.463+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/4070949245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d2419fb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.463+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/4070949245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d2419f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.463+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/4070949245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 0x55d2419ee220 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.463+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/4070949245 >> 172.21.15.107:0/4070949245 conn(0x55d2419ea3d0 msgr2=0x55d2419ea7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.463+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/4070949245 shutdown_connections 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.463+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/4070949245 wait complete. 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.463+0000 7f5bd0dd9740 1 -- start start 2024-09-19T04:26:54.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.463+0000 7f5bd0dd9740 1 Processor -- start 2024-09-19T04:26:54.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.464+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419ede40 0x55d2419f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.464+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d2419f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.464+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419f9020 0x55d2419f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.464+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d2419fbbf0 con 0x55d2419ede40 2024-09-19T04:26:54.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.464+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d24190f0f0 con 0x55d2419ee760 2024-09-19T04:26:54.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.464+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d2419fbd70 con 0x55d2419f9020 2024-09-19T04:26:54.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.464+0000 7f5bd0dd9740 10 client.0 _ll_register_callbacks cb 0x55d2418f6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:54.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.464+0000 7f5bc7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419ede40 0x55d2419f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.464+0000 7f5bc7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419ede40 0x55d2419f5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39176/0 (socket says 172.21.15.107:39176) 2024-09-19T04:26:54.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.464+0000 7f5bc7400640 1 -- 172.21.15.107:0/1853299185 learned_addr learned my addr 172.21.15.107:0/1853299185 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bc7e00640 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419f9020 0x55d2419f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d2419f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bc7400640 1 -- 172.21.15.107:0/1853299185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419f9020 msgr2=0x55d2419f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bc7400640 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419f9020 0x55d2419f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bc7400640 1 -- 172.21.15.107:0/1853299185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 msgr2=0x55d2419f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bc7400640 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d2419f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bc7400640 1 -- 172.21.15.107:0/1853299185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2416c53c0 con 0x55d2419ede40 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bc7e00640 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419f9020 0x55d2419f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d2419f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bc7400640 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419ede40 0x55d2419f5f40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009db0 tx=0x7f5bb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee22460adfc3a695 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.465+0000 7f5bb1600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:26:54.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.466+0000 7f5bb1600640 1 -- 172.21.15.107:0/1853299185 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb802e060 con 0x55d2419ede40 2024-09-19T04:26:54.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.466+0000 7f5bb1600640 1 -- 172.21.15.107:0/1853299185 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f5bb8018030 con 0x55d2419ede40 2024-09-19T04:26:54.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.466+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/1853299185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc0036040 con 0x55d2419ede40 2024-09-19T04:26:54.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.466+0000 7f5bb1600640 1 -- 172.21.15.107:0/1853299185 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5bb8012e30 con 0x55d2419ede40 2024-09-19T04:26:54.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.466+0000 7f5bd0dd9740 10 client.16356 fetch_fsmap learned FSMap version 39 2024-09-19T04:26:54.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.466+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/1853299185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d241a07d40 con 0x55d2419ede40 2024-09-19T04:26:54.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.467+0000 7f5bb1600640 1 -- 172.21.15.107:0/1853299185 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc0036040 con 0x55d2419ede40 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.467+0000 7f5bb1600640 1 -- 172.21.15.107:0/1853299185 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f5bb8029ad0 con 0x55d2419ede40 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.467+0000 7f5bd0dd9740 10 client.16356 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.467+0000 7f5bd0dd9740 10 client.16356 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.467+0000 7f5bd0dd9740 20 client.16356 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.467+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55d241a09200 0x55d241a0b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.467+0000 7f5bd0dd9740 4 client.16356 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.467+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/1853299185 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d241961950 con 0x55d241a09200 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.468+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55d241a09200 0x55d241a0b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.468+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55d241a09200 0x55d241a0b610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e88611d5d1eee78 server_cookie=2a053604d0269933 in_seq=0 out_seq=0 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.468+0000 7f5bb1600640 10 client.16356 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:26:54.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bb1600640 1 -- 172.21.15.107:0/1853299185 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d241961950 con 0x55d241a09200 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bb1600640 10 client.16356 handle_command_reply: tid=0 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bd0dd9740 1 client.16356 shutdown 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/1853299185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419ede40 msgr2=0x55d2419f5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419ede40 0x55d2419f5f40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009db0 tx=0x7f5bb8004730 comp rx=0 tx=0).stop 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/1853299185 shutdown_connections 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55d241a09200 0x55d241a0b610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419f9020 0x55d2419f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d2419f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/1853299185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419ede40 0x55d2419f5f40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/1853299185 >> 172.21.15.107:0/1853299185 conn(0x55d2419ea3d0 msgr2=0x55d2419f00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.469+0000 7f5bb1600640 0 client.16356 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.470+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/1853299185 shutdown_connections 2024-09-19T04:26:54.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.470+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/1853299185 wait complete. 2024-09-19T04:26:54.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.470+0000 7f5bd0dd9740 20 client.16356 trim_cache size 0 max 16384 2024-09-19T04:26:54.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.485+0000 7f5bd0dd9740 1 Processor -- start 2024-09-19T04:26:54.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.485+0000 7f5bd0dd9740 1 -- start start 2024-09-19T04:26:54.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ede40 0x55d2419f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d241a10ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d241a11420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81af740 con 0x55d2419f9020 2024-09-19T04:26:54.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81b13a0 con 0x55d2419ee760 2024-09-19T04:26:54.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81afe10 con 0x55d2419ede40 2024-09-19T04:26:54.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d241a10ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ede40 0x55d2419f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d241a10ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57804/0 (socket says 172.21.15.107:57804) 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc6a00640 1 -- 172.21.15.107:0/2493881494 learned_addr learned my addr 172.21.15.107:0/2493881494 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc7e00640 1 --2- 172.21.15.107:0/2493881494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d241a11420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc6a00640 1 -- 172.21.15.107:0/2493881494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ede40 msgr2=0x55d2419f08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/2493881494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ede40 0x55d2419f08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc6a00640 1 -- 172.21.15.107:0/2493881494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 msgr2=0x55d241a11420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/2493881494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d241a11420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc6a00640 1 -- 172.21.15.107:0/2493881494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2419eaf80 con 0x55d2419ee760 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.486+0000 7f5bc7400640 1 --2- 172.21.15.107:0/2493881494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ede40 0x55d2419f08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.487+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/2493881494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d241a10ee0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4010f50 tx=0x7f5bb40d2040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67a80755f254d36e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:54.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.487+0000 7f5bb3e00640 1 -- 172.21.15.107:0/2493881494 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb409bd00 con 0x55d2419ee760 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.487+0000 7f5bb3e00640 1 -- 172.21.15.107:0/2493881494 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb409c3a0 con 0x55d2419ee760 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.487+0000 7f5bb3e00640 1 -- 172.21.15.107:0/2493881494 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4098ba0 con 0x55d2419ee760 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.488+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/2493881494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 msgr2=0x55d241a10ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.488+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/2493881494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d241a10ee0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4010f50 tx=0x7f5bb40d2040 comp rx=0 tx=0).stop 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.488+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/2493881494 shutdown_connections 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.488+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/2493881494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d241a11420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.488+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/2493881494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ee760 0x55d241a10ee0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.488+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/2493881494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ede40 0x55d2419f08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.488+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/2493881494 >> 172.21.15.107:0/2493881494 conn(0x55d2419ea3d0 msgr2=0x55d2419f1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.488+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/2493881494 shutdown_connections 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.488+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/2493881494 wait complete. 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.489+0000 7f5bd0dd9740 1 -- start start 2024-09-19T04:26:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.489+0000 7f5bd0dd9740 1 Processor -- start 2024-09-19T04:26:54.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.489+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 0x55d241a00ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.489+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d241a01530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.489+0000 7f5bd0dd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d2419f2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.489+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81b0c10 con 0x55d2419f9020 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.489+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81b04e0 con 0x55d2419ede40 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.489+0000 7f5bd0dd9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81ad720 con 0x55d2419ee760 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.489+0000 7f5bd0dd9740 10 client.0 _ll_register_callbacks cb 0x7f5bb800fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d241a01530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d241a01530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55890/0 (socket says 172.21.15.107:55890) 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc6a00640 1 -- 172.21.15.107:0/3747913548 learned_addr learned my addr 172.21.15.107:0/3747913548 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc7400640 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 0x55d241a00ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc7e00640 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d2419f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc6a00640 1 -- 172.21.15.107:0/3747913548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 msgr2=0x55d241a00ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 0x55d241a00ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc6a00640 1 -- 172.21.15.107:0/3747913548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 msgr2=0x55d2419f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d2419f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc6a00640 1 -- 172.21.15.107:0/3747913548 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2419f2ab0 con 0x55d2419ee760 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc7e00640 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d2419f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bc6a00640 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d241a01530 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400e9c0 tx=0x7f5bb4098a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=824c68c7c3b6b15d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bb1600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bb1600640 1 -- 172.21.15.107:0/3747913548 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb409bd00 con 0x55d2419ee760 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.490+0000 7f5bb1600640 1 -- 172.21.15.107:0/3747913548 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f5bb40d20b0 con 0x55d2419ee760 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/3747913548 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb409c3a0 con 0x55d2419ee760 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bb1600640 1 -- 172.21.15.107:0/3747913548 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5bb40a06a0 con 0x55d2419ee760 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bd0dd9740 10 client.26371 fetch_fsmap learned FSMap version 39 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/3747913548 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d241a736b0 con 0x55d2419ee760 2024-09-19T04:26:54.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bb1600640 1 -- 172.21.15.107:0/3747913548 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb409c3a0 con 0x55d2419ee760 2024-09-19T04:26:54.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bb1600640 1 -- 172.21.15.107:0/3747913548 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f5bb409b7f0 con 0x55d2419ee760 2024-09-19T04:26:54.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bd0dd9740 10 client.26371 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:26:54.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bd0dd9740 10 client.26371 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:26:54.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bd0dd9740 20 client.26371 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:54.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55d241a73d20 0x55d241a76140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:54.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bd0dd9740 4 client.26371 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:54.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.491+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/3747913548 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d24195be60 con 0x55d241a73d20 2024-09-19T04:26:54.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.492+0000 7f5bc7400640 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55d241a73d20 0x55d241a76140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:54.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.492+0000 7f5bc7400640 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55d241a73d20 0x55d241a76140 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=328b8374009912ad server_cookie=2c65315e1d841002 in_seq=0 out_seq=0 2024-09-19T04:26:54.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.492+0000 7f5bb1600640 10 client.26371 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:26:54.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.492+0000 7f5bb1600640 1 -- 172.21.15.107:0/3747913548 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d24195be60 con 0x55d241a73d20 2024-09-19T04:26:54.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.492+0000 7f5bb1600640 10 client.26371 handle_command_reply: tid=0 2024-09-19T04:26:54.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.492+0000 7f5bd0dd9740 1 client.26371 shutdown 2024-09-19T04:26:54.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/3747913548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 msgr2=0x55d241a01530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d241a01530 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400e9c0 tx=0x7f5bb4098a10 comp rx=0 tx=0).stop 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/3747913548 shutdown_connections 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55d241a73d20 0x55d241a76140 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2419f9020 0x55d2419f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2419ee760 0x55d241a01530 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bb1600640 0 client.26371 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 1 --2- 172.21.15.107:0/3747913548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2419ede40 0x55d241a00ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/3747913548 >> 172.21.15.107:0/3747913548 conn(0x55d2419ea3d0 msgr2=0x55d2419e8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/3747913548 shutdown_connections 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 1 -- 172.21.15.107:0/3747913548 wait complete. 2024-09-19T04:26:54.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.493+0000 7f5bd0dd9740 20 client.26371 trim_cache size 0 max 16384 2024-09-19T04:26:54.499 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:26:54.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.494+0000 7f5bcce00640 1 -- 172.21.15.107:0/1151891008 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5bac0a3430 msgr2=0x7f5bac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:54.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.494+0000 7f5bcce00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5bac0a3430 0x7f5bac0a58f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55d2419eb230 tx=0x7f5bb8005f90 comp rx=0 tx=0).stop 2024-09-19T04:26:54.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.494+0000 7f5bcce00640 1 -- 172.21.15.107:0/1151891008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bc8102360 msgr2=0x7f5bc819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:54.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.494+0000 7f5bcce00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bc8102360 0x7f5bc819a9d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400e9f0 tx=0x7f5bb400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.495+0000 7f5bcce00640 1 -- 172.21.15.107:0/1151891008 shutdown_connections 2024-09-19T04:26:54.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.495+0000 7f5bcce00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5bac0a3430 0x7f5bac0a58f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.495+0000 7f5bcce00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bc81079c0 0x7f5bc819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.495+0000 7f5bcce00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bc8102360 0x7f5bc819a9d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.495+0000 7f5bcce00640 1 --2- 172.21.15.107:0/1151891008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc81019c0 0x7f5bc819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:54.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.495+0000 7f5bcce00640 1 -- 172.21.15.107:0/1151891008 >> 172.21.15.107:0/1151891008 conn(0x7f5bc80fd740 msgr2=0x7f5bc80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:54.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.495+0000 7f5bcce00640 1 -- 172.21.15.107:0/1151891008 shutdown_connections 2024-09-19T04:26:54.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:54.495+0000 7f5bcce00640 1 -- 172.21.15.107:0/1151891008 wait complete. 2024-09-19T04:26:54.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 221434 2.61 MB/sec execute 192 sec latency 4907.949 ms 2024-09-19T04:26:55.345 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:26:55.432 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:55 smithi107 ceph-mon[30659]: pgmap v200: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 564 KiB/s wr, 63 op/s 2024-09-19T04:26:55.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:55 smithi115 ceph-mon[35477]: pgmap v200: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 564 KiB/s wr, 63 op/s 2024-09-19T04:26:55.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:55 smithi145 ceph-mon[35096]: pgmap v200: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 564 KiB/s wr, 63 op/s 2024-09-19T04:26:55.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 221657 2.60 MB/sec execute 193 sec latency 498.417 ms 2024-09-19T04:26:55.718 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.328+0000 7f1936600640 1 -- 172.21.15.107:0/2976655812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19301049c0 msgr2=0x7f1930108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.328+0000 7f1936600640 1 --2- 172.21.15.107:0/2976655812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19301049c0 0x7f1930108a10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f191c009900 tx=0x7f191c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.328+0000 7f1936600640 1 -- 172.21.15.107:0/2976655812 shutdown_connections 2024-09-19T04:26:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.328+0000 7f1936600640 1 --2- 172.21.15.107:0/2976655812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1930109140 0x7f1930110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.328+0000 7f1936600640 1 --2- 172.21.15.107:0/2976655812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19301049c0 0x7f1930108a10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.328+0000 7f1936600640 1 --2- 172.21.15.107:0/2976655812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1930104010 0x7f19301043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.328+0000 7f1936600640 1 -- 172.21.15.107:0/2976655812 >> 172.21.15.107:0/2976655812 conn(0x7f19300ffa50 msgr2=0x7f1930101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.328+0000 7f1936600640 1 -- 172.21.15.107:0/2976655812 shutdown_connections 2024-09-19T04:26:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.328+0000 7f1936600640 1 -- 172.21.15.107:0/2976655812 wait complete. 2024-09-19T04:26:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.329+0000 7f1936600640 1 Processor -- start 2024-09-19T04:26:56.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.329+0000 7f1936600640 1 -- start start 2024-09-19T04:26:56.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.329+0000 7f1936600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1930104010 0x7f193019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.329+0000 7f1936600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19301049c0 0x7f193019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.329+0000 7f1936600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1930109140 0x7f193019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.329+0000 7f1936600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1930114190 con 0x7f1930104010 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.329+0000 7f1936600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1930114010 con 0x7f19301049c0 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.329+0000 7f1936600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1930114310 con 0x7f1930109140 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1930104010 0x7f193019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1930104010 0x7f193019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39216/0 (socket says 172.21.15.107:39216) 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19301049c0 0x7f193019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192f400640 1 -- 172.21.15.107:0/3962398704 learned_addr learned my addr 172.21.15.107:0/3962398704 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192fe00640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1930109140 0x7f193019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192f400640 1 -- 172.21.15.107:0/3962398704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1930109140 msgr2=0x7f193019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192f400640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1930109140 0x7f193019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192f400640 1 -- 172.21.15.107:0/3962398704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19301049c0 msgr2=0x7f193019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192f400640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19301049c0 0x7f193019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192f400640 1 -- 172.21.15.107:0/3962398704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193019f6c0 con 0x7f1930104010 2024-09-19T04:26:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192ea00640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19301049c0 0x7f193019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.330+0000 7f192f400640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1930104010 0x7f193019a710 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f192000e9f0 tx=0x7f192000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.331+0000 7f1927e00640 1 -- 172.21.15.107:0/3962398704 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192000ccc0 con 0x7f1930104010 2024-09-19T04:26:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.331+0000 7f1927e00640 1 -- 172.21.15.107:0/3962398704 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1920010910 con 0x7f1930104010 2024-09-19T04:26:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.331+0000 7f1927e00640 1 -- 172.21.15.107:0/3962398704 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19200188a0 con 0x7f1930104010 2024-09-19T04:26:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.331+0000 7f1936600640 1 -- 172.21.15.107:0/3962398704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193019f9b0 con 0x7f1930104010 2024-09-19T04:26:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.331+0000 7f1936600640 1 -- 172.21.15.107:0/3962398704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19301a7290 con 0x7f1930104010 2024-09-19T04:26:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.334+0000 7f19390bd740 1 Processor -- start 2024-09-19T04:26:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.334+0000 7f19390bd740 1 -- start start 2024-09-19T04:26:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.334+0000 7f1927e00640 1 -- 172.21.15.107:0/3962398704 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f1920010ac0 con 0x7f1930104010 2024-09-19T04:26:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.334+0000 7f19390bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387fc70 0x5640f3880050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.334+0000 7f19390bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f3880590 0x5640f388a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.334+0000 7f19390bd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f388ae50 0x5640f388d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.334+0000 7f19390bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5640f37e63e0 con 0x5640f3880590 2024-09-19T04:26:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.334+0000 7f19390bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5640f378dd80 con 0x5640f387fc70 2024-09-19T04:26:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.334+0000 7f19390bd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5640f37e23a0 con 0x5640f388ae50 2024-09-19T04:26:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387fc70 0x5640f3880050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387fc70 0x5640f3880050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57834/0 (socket says 172.21.15.107:57834) 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192fe00640 1 -- 172.21.15.107:0/1974067241 learned_addr learned my addr 172.21.15.107:0/1974067241 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192f400640 1 --2- 172.21.15.107:0/1974067241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f3880590 0x5640f388a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192ea00640 1 --2- 172.21.15.107:0/1974067241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f388ae50 0x5640f388d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192fe00640 1 -- 172.21.15.107:0/1974067241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f388ae50 msgr2=0x5640f388d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192fe00640 1 --2- 172.21.15.107:0/1974067241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f388ae50 0x5640f388d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192fe00640 1 -- 172.21.15.107:0/1974067241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f3880590 msgr2=0x5640f388a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192fe00640 1 --2- 172.21.15.107:0/1974067241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f3880590 0x5640f388a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192fe00640 1 -- 172.21.15.107:0/1974067241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f37acae0 con 0x5640f387fc70 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.335+0000 7f192fe00640 1 --2- 172.21.15.107:0/1974067241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387fc70 0x5640f3880050 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1928009f60 tx=0x7f1928009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ecf34980fd9af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.336+0000 7f1924c00640 1 -- 172.21.15.107:0/1974067241 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192801e070 con 0x5640f387fc70 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.336+0000 7f1924c00640 1 -- 172.21.15.107:0/1974067241 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1928003c30 con 0x5640f387fc70 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.336+0000 7f1924c00640 1 -- 172.21.15.107:0/1974067241 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192800f980 con 0x5640f387fc70 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.336+0000 7f1927e00640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f19140a33b0 0x7f19140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.336+0000 7f1927e00640 1 -- 172.21.15.107:0/3962398704 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f19200d0120 con 0x7f1930104010 2024-09-19T04:26:56.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.336+0000 7f192f400640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f19140a33b0 0x7f19140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.336+0000 7f19390bd740 1 -- 172.21.15.107:0/1974067241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387fc70 msgr2=0x5640f3880050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.336+0000 7f19390bd740 1 --2- 172.21.15.107:0/1974067241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387fc70 0x5640f3880050 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1928009f60 tx=0x7f1928009f90 comp rx=0 tx=0).stop 2024-09-19T04:26:56.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.337+0000 7f19390bd740 1 -- 172.21.15.107:0/1974067241 shutdown_connections 2024-09-19T04:26:56.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.337+0000 7f19390bd740 1 --2- 172.21.15.107:0/1974067241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f388ae50 0x5640f388d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.337+0000 7f19390bd740 1 --2- 172.21.15.107:0/1974067241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f3880590 0x5640f388a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.337+0000 7f19390bd740 1 --2- 172.21.15.107:0/1974067241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387fc70 0x5640f3880050 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.337+0000 7f19390bd740 1 -- 172.21.15.107:0/1974067241 >> 172.21.15.107:0/1974067241 conn(0x5640f387c200 msgr2=0x5640f387c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.337+0000 7f19390bd740 1 -- 172.21.15.107:0/1974067241 shutdown_connections 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.337+0000 7f19390bd740 1 -- 172.21.15.107:0/1974067241 wait complete. 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.337+0000 7f192f400640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f19140a33b0 0x7f19140a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5640f387d060 tx=0x7f1920009700 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.337+0000 7f19390bd740 1 -- start start 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f19390bd740 1 Processor -- start 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f19390bd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 0x5640f3887db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f19390bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 0x5640f38882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f19390bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f388ae50 0x5640f3888830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f19390bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5640f388da20 con 0x5640f388ae50 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f19390bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5640f37a10f0 con 0x5640f3880590 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f19390bd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5640f388dba0 con 0x5640f387fc70 2024-09-19T04:26:56.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f19390bd740 10 client.0 _ll_register_callbacks cb 0x5640f3788bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f192fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f388ae50 0x5640f3888830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f192fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f388ae50 0x5640f3888830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39246/0 (socket says 172.21.15.107:39246) 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f192fe00640 1 -- 172.21.15.107:0/2699020690 learned_addr learned my addr 172.21.15.107:0/2699020690 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.338+0000 7f192ea00640 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 0x5640f38882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.339+0000 7f192fe00640 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 0x5640f3887db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.339+0000 7f192ea00640 1 -- 172.21.15.107:0/2699020690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 msgr2=0x5640f3887db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.339+0000 7f192ea00640 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 0x5640f3887db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.339+0000 7f192ea00640 1 -- 172.21.15.107:0/2699020690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f388ae50 msgr2=0x5640f3888830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.339+0000 7f192ea00640 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f388ae50 0x5640f3888830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.339+0000 7f192ea00640 1 -- 172.21.15.107:0/2699020690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f35573c0 con 0x5640f3880590 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.339+0000 7f192fe00640 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f388ae50 0x5640f3888830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:56.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.339+0000 7f192ea00640 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 0x5640f38882f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f191c02a910 tx=0x7f191c005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e52b6024f0c076e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.339+0000 7f1919600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f1919600640 1 -- 172.21.15.107:0/2699020690 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c035040 con 0x5640f3880590 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f1919600640 1 -- 172.21.15.107:0/2699020690 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f191c004760 con 0x5640f3880590 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f19390bd740 1 -- 172.21.15.107:0/2699020690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1928003c30 con 0x5640f3880590 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f1919600640 1 -- 172.21.15.107:0/2699020690 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f191c04a020 con 0x5640f3880590 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f19390bd740 10 client.26423 fetch_fsmap learned FSMap version 39 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f19390bd740 1 -- 172.21.15.107:0/2699020690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5640f3899ae0 con 0x5640f3880590 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f1919600640 1 -- 172.21.15.107:0/2699020690 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1928003c30 con 0x5640f3880590 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f1919600640 1 -- 172.21.15.107:0/2699020690 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f191c031b40 con 0x5640f3880590 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f19390bd740 10 client.26423 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f19390bd740 10 client.26423 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f19390bd740 20 client.26423 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:56.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f19390bd740 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5640f389af40 0x5640f389d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f19390bd740 4 client.26423 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.340+0000 7f19390bd740 1 -- 172.21.15.107:0/2699020690 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640f37f3950 con 0x5640f389af40 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.341+0000 7f192fe00640 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5640f389af40 0x5640f389d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.341+0000 7f192fe00640 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5640f389af40 0x5640f389d330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f9447f921b5429 server_cookie=b1d6170175ab2e8a in_seq=0 out_seq=0 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.341+0000 7f1919600640 10 client.26423 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f1919600640 1 -- 172.21.15.107:0/2699020690 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640f37f3950 con 0x5640f389af40 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f1919600640 10 client.26423 handle_command_reply: tid=0 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f19390bd740 1 client.26423 shutdown 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f19390bd740 1 -- 172.21.15.107:0/2699020690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 msgr2=0x5640f38882f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f19390bd740 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 0x5640f38882f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f191c02a910 tx=0x7f191c005040 comp rx=0 tx=0).stop 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f19390bd740 1 -- 172.21.15.107:0/2699020690 shutdown_connections 2024-09-19T04:26:56.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f19390bd740 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5640f389af40 0x5640f389d330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f19390bd740 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f388ae50 0x5640f3888830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f19390bd740 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 0x5640f38882f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f19390bd740 1 --2- 172.21.15.107:0/2699020690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 0x5640f3887db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.342+0000 7f19390bd740 1 -- 172.21.15.107:0/2699020690 >> 172.21.15.107:0/2699020690 conn(0x5640f387c200 msgr2=0x5640f3881f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.343+0000 7f1919600640 0 client.26423 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:26:56.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.343+0000 7f19390bd740 1 -- 172.21.15.107:0/2699020690 shutdown_connections 2024-09-19T04:26:56.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.343+0000 7f19390bd740 1 -- 172.21.15.107:0/2699020690 wait complete. 2024-09-19T04:26:56.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.343+0000 7f19390bd740 20 client.26423 trim_cache size 0 max 16384 2024-09-19T04:26:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.356+0000 7f19390bd740 1 Processor -- start 2024-09-19T04:26:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.356+0000 7f19390bd740 1 -- start start 2024-09-19T04:26:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f19390bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387c200 0x5640f38826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f19390bd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 0x5640f3886680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f19390bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 0x5640f38a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f19390bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19301afac0 con 0x5640f387c200 2024-09-19T04:26:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f19390bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f19301b1720 con 0x5640f3880590 2024-09-19T04:26:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f19390bd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f19301b0190 con 0x5640f387fc70 2024-09-19T04:26:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f192fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387c200 0x5640f38826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f192fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387c200 0x5640f38826d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39250/0 (socket says 172.21.15.107:39250) 2024-09-19T04:26:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f192fe00640 1 -- 172.21.15.107:0/3365751662 learned_addr learned my addr 172.21.15.107:0/3365751662 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f192ea00640 1 --2- 172.21.15.107:0/3365751662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 0x5640f3886680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f192fe00640 1 --2- 172.21.15.107:0/3365751662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 0x5640f38a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f192ea00640 1 -- 172.21.15.107:0/3365751662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 msgr2=0x5640f38a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f192ea00640 1 --2- 172.21.15.107:0/3365751662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 0x5640f38a2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f192ea00640 1 -- 172.21.15.107:0/3365751662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387c200 msgr2=0x5640f38826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f192ea00640 1 --2- 172.21.15.107:0/3365751662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387c200 0x5640f38826d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.357+0000 7f192ea00640 1 -- 172.21.15.107:0/3365751662 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f3877770 con 0x5640f387fc70 2024-09-19T04:26:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.358+0000 7f192ea00640 1 --2- 172.21.15.107:0/3365751662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 0x5640f3886680 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f191c0036f0 tx=0x7f191c02ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50029ec3f8018953 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.358+0000 7f191be00640 1 -- 172.21.15.107:0/3365751662 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c035030 con 0x5640f387fc70 2024-09-19T04:26:56.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.358+0000 7f191be00640 1 -- 172.21.15.107:0/3365751662 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f191c041030 con 0x5640f387fc70 2024-09-19T04:26:56.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.358+0000 7f191be00640 1 -- 172.21.15.107:0/3365751662 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c0049c0 con 0x5640f387fc70 2024-09-19T04:26:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 -- 172.21.15.107:0/3365751662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 msgr2=0x5640f3886680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 --2- 172.21.15.107:0/3365751662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 0x5640f3886680 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f191c0036f0 tx=0x7f191c02ab70 comp rx=0 tx=0).stop 2024-09-19T04:26:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 -- 172.21.15.107:0/3365751662 shutdown_connections 2024-09-19T04:26:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 --2- 172.21.15.107:0/3365751662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f3880590 0x5640f38a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 --2- 172.21.15.107:0/3365751662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f387fc70 0x5640f3886680 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 --2- 172.21.15.107:0/3365751662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387c200 0x5640f38826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 -- 172.21.15.107:0/3365751662 >> 172.21.15.107:0/3365751662 conn(0x5640f387cc80 msgr2=0x5640f3889e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 -- 172.21.15.107:0/3365751662 shutdown_connections 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 -- 172.21.15.107:0/3365751662 wait complete. 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 -- start start 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.359+0000 7f19390bd740 1 Processor -- start 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f19390bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387c200 0x5640f38931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f19390bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387fc70 0x5640f3893700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f19390bd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f3880590 0x5640f388e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f19390bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19301b0f90 con 0x5640f387fc70 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f19390bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f19301b0860 con 0x5640f387c200 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f19390bd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f19301adaa0 con 0x5640f3880590 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f19390bd740 10 client.0 _ll_register_callbacks cb 0x7f1920005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f192fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387c200 0x5640f38931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f192fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387c200 0x5640f38931c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57862/0 (socket says 172.21.15.107:57862) 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f192fe00640 1 -- 172.21.15.107:0/1561569811 learned_addr learned my addr 172.21.15.107:0/1561569811 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f192ea00640 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387fc70 0x5640f3893700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.360+0000 7f192fe00640 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f3880590 0x5640f388e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f192fe00640 1 -- 172.21.15.107:0/1561569811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f3880590 msgr2=0x5640f388e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f192fe00640 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f3880590 0x5640f388e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f192fe00640 1 -- 172.21.15.107:0/1561569811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387fc70 msgr2=0x5640f3893700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f192fe00640 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387fc70 0x5640f3893700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f192fe00640 1 -- 172.21.15.107:0/1561569811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f388ef50 con 0x5640f387c200 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f192fe00640 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387c200 0x5640f38931c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1928009f30 tx=0x7f192802b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0b2793067b0d7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f1919600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:26:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f1919600640 1 -- 172.21.15.107:0/1561569811 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192801e040 con 0x5640f387c200 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f19390bd740 1 -- 172.21.15.107:0/1561569811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f191c041030 con 0x5640f387c200 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f1919600640 1 -- 172.21.15.107:0/1561569811 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f1928004580 con 0x5640f387c200 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f1919600640 1 -- 172.21.15.107:0/1561569811 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f192803f050 con 0x5640f387c200 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f19390bd740 10 client.26429 fetch_fsmap learned FSMap version 39 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.361+0000 7f19390bd740 1 -- 172.21.15.107:0/1561569811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5640f3905860 con 0x5640f387c200 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.362+0000 7f1919600640 1 -- 172.21.15.107:0/1561569811 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f191c041030 con 0x5640f387c200 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.362+0000 7f1919600640 1 -- 172.21.15.107:0/1561569811 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f19280194a0 con 0x5640f387c200 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.362+0000 7f19390bd740 10 client.26429 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.362+0000 7f19390bd740 10 client.26429 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.362+0000 7f19390bd740 20 client.26429 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.362+0000 7f19390bd740 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5640f388ae50 0x5640f3907d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.362+0000 7f19390bd740 4 client.26429 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.362+0000 7f19390bd740 1 -- 172.21.15.107:0/1561569811 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640f37ede60 con 0x5640f388ae50 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.362+0000 7f192ea00640 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5640f388ae50 0x5640f3907d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f192ea00640 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5640f388ae50 0x5640f3907d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef698245322cbea8 server_cookie=a13f772396524d5a in_seq=0 out_seq=0 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f1919600640 10 client.26429 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f1919600640 1 -- 172.21.15.107:0/1561569811 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640f37ede60 con 0x5640f388ae50 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f1919600640 10 client.26429 handle_command_reply: tid=0 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 client.26429 shutdown 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 -- 172.21.15.107:0/1561569811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387c200 msgr2=0x5640f38931c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387c200 0x5640f38931c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1928009f30 tx=0x7f192802b040 comp rx=0 tx=0).stop 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 -- 172.21.15.107:0/1561569811 shutdown_connections 2024-09-19T04:26:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5640f388ae50 0x5640f3907d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640f3880590 0x5640f388e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640f387fc70 0x5640f3893700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f1919600640 0 client.26429 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:26:56.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 --2- 172.21.15.107:0/1561569811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5640f387c200 0x5640f38931c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 -- 172.21.15.107:0/1561569811 >> 172.21.15.107:0/1561569811 conn(0x5640f387cc80 msgr2=0x5640f3889610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 -- 172.21.15.107:0/1561569811 shutdown_connections 2024-09-19T04:26:56.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 1 -- 172.21.15.107:0/1561569811 wait complete. 2024-09-19T04:26:56.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.363+0000 7f19390bd740 20 client.26429 trim_cache size 0 max 16384 2024-09-19T04:26:56.372 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:26:56.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.364+0000 7f1936600640 1 -- 172.21.15.107:0/3962398704 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f19140a33b0 msgr2=0x7f19140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.364+0000 7f1936600640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f19140a33b0 0x7f19140a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5640f387d060 tx=0x7f1920009700 comp rx=0 tx=0).stop 2024-09-19T04:26:56.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.364+0000 7f1936600640 1 -- 172.21.15.107:0/3962398704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1930104010 msgr2=0x7f193019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.364+0000 7f1936600640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1930104010 0x7f193019a710 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f192000e9f0 tx=0x7f192000edf0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.364+0000 7f1936600640 1 -- 172.21.15.107:0/3962398704 shutdown_connections 2024-09-19T04:26:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.364+0000 7f1936600640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f19140a33b0 0x7f19140a5870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.364+0000 7f1936600640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1930109140 0x7f193019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.364+0000 7f1936600640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19301049c0 0x7f193019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.364+0000 7f1936600640 1 --2- 172.21.15.107:0/3962398704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1930104010 0x7f193019a710 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.364+0000 7f1936600640 1 -- 172.21.15.107:0/3962398704 >> 172.21.15.107:0/3962398704 conn(0x7f19300ffa50 msgr2=0x7f1930100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.365+0000 7f1936600640 1 -- 172.21.15.107:0/3962398704 shutdown_connections 2024-09-19T04:26:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:56.365+0000 7f1936600640 1 -- 172.21.15.107:0/3962398704 wait complete. 2024-09-19T04:26:56.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 221657 2.59 MB/sec execute 194 sec latency 1241.357 ms 2024-09-19T04:26:57.137 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:26:57.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:57 smithi115 ceph-mon[35477]: pgmap v201: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 900 KiB/s wr, 90 op/s 2024-09-19T04:26:57.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:57 smithi145 ceph-mon[35096]: pgmap v201: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 900 KiB/s wr, 90 op/s 2024-09-19T04:26:57.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:57 smithi107 ceph-mon[30659]: pgmap v201: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 900 KiB/s wr, 90 op/s 2024-09-19T04:26:57.505 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:57.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 221657 2.57 MB/sec execute 195 sec latency 2241.445 ms 2024-09-19T04:26:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.090+0000 7f5c3ee00640 1 -- 172.21.15.107:0/2414425437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c381079c0 msgr2=0x7f5c38107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.090+0000 7f5c3ee00640 1 --2- 172.21.15.107:0/2414425437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c381079c0 0x7f5c38107da0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009900 tx=0x7f5c2c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.090+0000 7f5c3ee00640 1 -- 172.21.15.107:0/2414425437 shutdown_connections 2024-09-19T04:26:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.090+0000 7f5c3ee00640 1 --2- 172.21.15.107:0/2414425437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c38102360 0x7f5c3810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.090+0000 7f5c3ee00640 1 --2- 172.21.15.107:0/2414425437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c381019c0 0x7f5c38101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.090+0000 7f5c3ee00640 1 --2- 172.21.15.107:0/2414425437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c381079c0 0x7f5c38107da0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.090+0000 7f5c3ee00640 1 -- 172.21.15.107:0/2414425437 >> 172.21.15.107:0/2414425437 conn(0x7f5c380fd740 msgr2=0x7f5c380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.090+0000 7f5c3ee00640 1 -- 172.21.15.107:0/2414425437 shutdown_connections 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.090+0000 7f5c3ee00640 1 -- 172.21.15.107:0/2414425437 wait complete. 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.091+0000 7f5c3ee00640 1 Processor -- start 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.091+0000 7f5c3ee00640 1 -- start start 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3ee00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c381019c0 0x7f5c3819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3ee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c38102360 0x7f5c3819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3ee00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c381079c0 0x7f5c3819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3ee00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c38111e10 con 0x7f5c381079c0 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3ee00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5c38111c90 con 0x7f5c381019c0 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3ee00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5c38111f90 con 0x7f5c38102360 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c381079c0 0x7f5c3819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c381019c0 0x7f5c3819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c381079c0 0x7f5c3819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39276/0 (socket says 172.21.15.107:39276) 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3e400640 1 -- 172.21.15.107:0/323376136 learned_addr learned my addr 172.21.15.107:0/323376136 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3d000640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c38102360 0x7f5c3819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3da00640 1 -- 172.21.15.107:0/323376136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c38102360 msgr2=0x7f5c3819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3da00640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c38102360 0x7f5c3819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3da00640 1 -- 172.21.15.107:0/323376136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c381079c0 msgr2=0x7f5c3819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3da00640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c381079c0 0x7f5c3819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3da00640 1 -- 172.21.15.107:0/323376136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3819f440 con 0x7f5c381019c0 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.092+0000 7f5c3e400640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c381079c0 0x7f5c3819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.093+0000 7f5c3da00640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c381019c0 0x7f5c3819a490 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c02a910 tx=0x7f5c2c003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.093+0000 7f5c36000640 1 -- 172.21.15.107:0/323376136 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c03f070 con 0x7f5c381019c0 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.093+0000 7f5c3ee00640 1 -- 172.21.15.107:0/323376136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3819f6d0 con 0x7f5c381019c0 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.093+0000 7f5c36000640 1 -- 172.21.15.107:0/323376136 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c2c031c20 con 0x7f5c381019c0 2024-09-19T04:26:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.093+0000 7f5c36000640 1 -- 172.21.15.107:0/323376136 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c035030 con 0x7f5c381019c0 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.093+0000 7f5c3ee00640 1 -- 172.21.15.107:0/323376136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c381a7010 con 0x7f5c381019c0 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.097+0000 7f5c42e07740 1 Processor -- start 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.097+0000 7f5c42e07740 1 -- start start 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.097+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.097+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 0x55e0a0ccaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.097+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0ccb020 0x55e0a0ccd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.097+0000 7f5c42e07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0c263e0 con 0x55e0a0cc0760 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c42e07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0bcdd80 con 0x55e0a0ccb020 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c42e07740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0c223a0 con 0x55e0a0cbfe40 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c36000640 1 -- 172.21.15.107:0/323376136 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f5c2c042070 con 0x7f5c381019c0 2024-09-19T04:26:58.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c3e400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c3e400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55962/0 (socket says 172.21.15.107:55962) 2024-09-19T04:26:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c3e400640 1 -- 172.21.15.107:0/92785145 learned_addr learned my addr 172.21.15.107:0/92785145 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c3d000640 1 --2- 172.21.15.107:0/92785145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0ccb020 0x55e0a0ccd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c3da00640 1 --2- 172.21.15.107:0/92785145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 0x55e0a0ccaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c3e400640 1 -- 172.21.15.107:0/92785145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0ccb020 msgr2=0x55e0a0ccd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c3e400640 1 --2- 172.21.15.107:0/92785145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0ccb020 0x55e0a0ccd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c3e400640 1 -- 172.21.15.107:0/92785145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 msgr2=0x55e0a0ccaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c3e400640 1 --2- 172.21.15.107:0/92785145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 0x55e0a0ccaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.098+0000 7f5c3e400640 1 -- 172.21.15.107:0/92785145 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a0becae0 con 0x55e0a0cbfe40 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.099+0000 7f5c3e400640 1 --2- 172.21.15.107:0/92785145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc0220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5c30011850 tx=0x7f5c30011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f91e195a69d1486 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.099+0000 7f5c2aa00640 1 -- 172.21.15.107:0/92785145 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30011330 con 0x55e0a0cbfe40 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.099+0000 7f5c2aa00640 1 -- 172.21.15.107:0/92785145 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c3000fe80 con 0x55e0a0cbfe40 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.099+0000 7f5c2aa00640 1 -- 172.21.15.107:0/92785145 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300193e0 con 0x55e0a0cbfe40 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c42e07740 1 -- 172.21.15.107:0/92785145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 msgr2=0x55e0a0cc0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c42e07740 1 --2- 172.21.15.107:0/92785145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc0220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5c30011850 tx=0x7f5c30011d10 comp rx=0 tx=0).stop 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c36000640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5c200a3430 0x7f5c200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c42e07740 1 -- 172.21.15.107:0/92785145 shutdown_connections 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c42e07740 1 --2- 172.21.15.107:0/92785145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0ccb020 0x55e0a0ccd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c42e07740 1 --2- 172.21.15.107:0/92785145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 0x55e0a0ccaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c42e07740 1 --2- 172.21.15.107:0/92785145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc0220 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c42e07740 1 -- 172.21.15.107:0/92785145 >> 172.21.15.107:0/92785145 conn(0x55e0a0cbc3d0 msgr2=0x55e0a0cbc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c36000640 1 -- 172.21.15.107:0/323376136 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f5c2c0f0200 con 0x7f5c381019c0 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c3e400640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5c200a3430 0x7f5c200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c42e07740 1 -- 172.21.15.107:0/92785145 shutdown_connections 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c42e07740 1 -- 172.21.15.107:0/92785145 wait complete. 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.100+0000 7f5c42e07740 1 -- start start 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c3e400640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5c200a3430 0x7f5c200a58f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5c30011660 tx=0x7f5c30010a90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c42e07740 1 Processor -- start 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cc0760 0x55e0a0cc8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0cc8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c42e07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0ccdbf0 con 0x55e0a0ccb020 2024-09-19T04:26:58.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c42e07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0be10f0 con 0x55e0a0cbfe40 2024-09-19T04:26:58.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c42e07740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0ccdd70 con 0x55e0a0cc0760 2024-09-19T04:26:58.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c42e07740 10 client.0 _ll_register_callbacks cb 0x55e0a0bc8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:58.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c3e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0cc8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c3da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c3e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0cc8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39298/0 (socket says 172.21.15.107:39298) 2024-09-19T04:26:58.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.101+0000 7f5c3e400640 1 -- 172.21.15.107:0/1823226393 learned_addr learned my addr 172.21.15.107:0/1823226393 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:58.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c3d000640 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cc0760 0x55e0a0cc8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c3da00640 1 -- 172.21.15.107:0/1823226393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cc0760 msgr2=0x55e0a0cc8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c3da00640 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cc0760 0x55e0a0cc8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c3da00640 1 -- 172.21.15.107:0/1823226393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 msgr2=0x55e0a0cc8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c3da00640 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0cc8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c3da00640 1 -- 172.21.15.107:0/1823226393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a09973c0 con 0x55e0a0cbfe40 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c3da00640 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc7e10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0ed940 tx=0x7f5c2c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3872e5f51d1af92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c1f400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c1f400640 1 -- 172.21.15.107:0/1823226393 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c049070 con 0x55e0a0cbfe40 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c42e07740 1 -- 172.21.15.107:0/1823226393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e0a0c33950 con 0x55e0a0cbfe40 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c1f400640 1 -- 172.21.15.107:0/1823226393 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f5c2c0b8bf0 con 0x55e0a0cbfe40 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.102+0000 7f5c1f400640 1 -- 172.21.15.107:0/1823226393 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5c2c0bc0c0 con 0x55e0a0cbfe40 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.103+0000 7f5c42e07740 10 client.26441 fetch_fsmap learned FSMap version 39 2024-09-19T04:26:58.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.103+0000 7f5c42e07740 1 -- 172.21.15.107:0/1823226393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e0a0cd9b60 con 0x55e0a0cbfe40 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.103+0000 7f5c1f400640 1 -- 172.21.15.107:0/1823226393 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e0a0c33950 con 0x55e0a0cbfe40 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.103+0000 7f5c1f400640 1 -- 172.21.15.107:0/1823226393 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f5c2c0edbc0 con 0x55e0a0cbfe40 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.103+0000 7f5c42e07740 10 client.26441 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.103+0000 7f5c42e07740 10 client.26441 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.103+0000 7f5c42e07740 20 client.26441 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.103+0000 7f5c42e07740 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55e0a0cdafc0 0x55e0a0cdd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.103+0000 7f5c42e07740 4 client.26441 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.103+0000 7f5c42e07740 1 -- 172.21.15.107:0/1823226393 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0a0c2de60 con 0x55e0a0cdafc0 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.104+0000 7f5c3d000640 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55e0a0cdafc0 0x55e0a0cdd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.104+0000 7f5c3d000640 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55e0a0cdafc0 0x55e0a0cdd440 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba2feef45924fc59 server_cookie=9c083af85090a0ed in_seq=0 out_seq=0 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.104+0000 7f5c1f400640 10 client.26441 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:26:58.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c1f400640 1 -- 172.21.15.107:0/1823226393 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0a0c2de60 con 0x55e0a0cdafc0 2024-09-19T04:26:58.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c1f400640 10 client.26441 handle_command_reply: tid=0 2024-09-19T04:26:58.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 client.26441 shutdown 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 -- 172.21.15.107:0/1823226393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 msgr2=0x55e0a0cc7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc7e10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0ed940 tx=0x7f5c2c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 -- 172.21.15.107:0/1823226393 shutdown_connections 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55e0a0cdafc0 0x55e0a0cdd440 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0cc8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cc0760 0x55e0a0cc8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 --2- 172.21.15.107:0/1823226393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc7e10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c1f400640 0 client.26441 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 -- 172.21.15.107:0/1823226393 >> 172.21.15.107:0/1823226393 conn(0x55e0a0cbc3d0 msgr2=0x55e0a0cc2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 -- 172.21.15.107:0/1823226393 shutdown_connections 2024-09-19T04:26:58.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 1 -- 172.21.15.107:0/1823226393 wait complete. 2024-09-19T04:26:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.105+0000 7f5c42e07740 20 client.26441 trim_cache size 0 max 16384 2024-09-19T04:26:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.119+0000 7f5c42e07740 1 Processor -- start 2024-09-19T04:26:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.119+0000 7f5c42e07740 1 -- start start 2024-09-19T04:26:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.119+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.119+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cc0760 0x55e0a0ce2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.119+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0ce3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.119+0000 7f5c42e07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381af7b0 con 0x55e0a0ccb020 2024-09-19T04:26:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.119+0000 7f5c42e07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381b1410 con 0x55e0a0cc0760 2024-09-19T04:26:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c42e07740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381afe80 con 0x55e0a0cbfe40 2024-09-19T04:26:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0ce3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3d000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cc0760 0x55e0a0ce2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3da00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0ce3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39304/0 (socket says 172.21.15.107:39304) 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3e400640 1 -- 172.21.15.107:0/1157018584 learned_addr learned my addr 172.21.15.107:0/1157018584 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3e400640 1 -- 172.21.15.107:0/1157018584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 msgr2=0x55e0a0cc6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3e400640 1 --2- 172.21.15.107:0/1157018584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3e400640 1 -- 172.21.15.107:0/1157018584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cc0760 msgr2=0x55e0a0ce2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3e400640 1 --2- 172.21.15.107:0/1157018584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cc0760 0x55e0a0ce2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3e400640 1 -- 172.21.15.107:0/1157018584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a0ccdad0 con 0x55e0a0ccb020 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3d000640 1 --2- 172.21.15.107:0/1157018584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cc0760 0x55e0a0ce2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.120+0000 7f5c3da00640 1 --2- 172.21.15.107:0/1157018584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.121+0000 7f5c3e400640 1 --2- 172.21.15.107:0/1157018584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0ce3250 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5c30011520 tx=0x7f5c30023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75ae98d25ce06bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.121+0000 7f5c2a000640 1 -- 172.21.15.107:0/1157018584 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3002c050 con 0x55e0a0ccb020 2024-09-19T04:26:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.121+0000 7f5c2a000640 1 -- 172.21.15.107:0/1157018584 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c30037070 con 0x55e0a0ccb020 2024-09-19T04:26:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.121+0000 7f5c2a000640 1 -- 172.21.15.107:0/1157018584 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3001f040 con 0x55e0a0ccb020 2024-09-19T04:26:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 -- 172.21.15.107:0/1157018584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 msgr2=0x55e0a0ce3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 --2- 172.21.15.107:0/1157018584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0ce3250 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5c30011520 tx=0x7f5c30023000 comp rx=0 tx=0).stop 2024-09-19T04:26:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 -- 172.21.15.107:0/1157018584 shutdown_connections 2024-09-19T04:26:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 --2- 172.21.15.107:0/1157018584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0ccb020 0x55e0a0ce3250 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 --2- 172.21.15.107:0/1157018584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cc0760 0x55e0a0ce2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 --2- 172.21.15.107:0/1157018584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 -- 172.21.15.107:0/1157018584 >> 172.21.15.107:0/1157018584 conn(0x55e0a0cbc3d0 msgr2=0x55e0a0cc66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 -- 172.21.15.107:0/1157018584 shutdown_connections 2024-09-19T04:26:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 -- 172.21.15.107:0/1157018584 wait complete. 2024-09-19T04:26:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 -- start start 2024-09-19T04:26:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.122+0000 7f5c42e07740 1 Processor -- start 2024-09-19T04:26:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 0x55e0a0cc5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c42e07740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0ccb020 0x55e0a0cc5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c42e07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381b0c80 con 0x55e0a0cc0760 2024-09-19T04:26:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c42e07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381b0550 con 0x55e0a0cbfe40 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c42e07740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381ad790 con 0x55e0a0ccb020 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c42e07740 10 client.0 _ll_register_callbacks cb 0x7f5c2c0b8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 0x55e0a0cc5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 0x55e0a0cc5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39318/0 (socket says 172.21.15.107:39318) 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3d000640 1 -- 172.21.15.107:0/3087545630 learned_addr learned my addr 172.21.15.107:0/3087545630 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3e400640 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0ccb020 0x55e0a0cc5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3da00640 1 -- 172.21.15.107:0/3087545630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0ccb020 msgr2=0x55e0a0cc5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3da00640 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0ccb020 0x55e0a0cc5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3da00640 1 -- 172.21.15.107:0/3087545630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 msgr2=0x55e0a0cc5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3da00640 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 0x55e0a0cc5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3da00640 1 -- 172.21.15.107:0/3087545630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a0cd14d0 con 0x55e0a0cbfe40 2024-09-19T04:26:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.123+0000 7f5c3d000640 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 0x55e0a0cc5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c3da00640 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc4e00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0ba830 tx=0x7f5c2c0b8e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6114ecfdc58b49b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c1f400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c1f400640 1 -- 172.21.15.107:0/3087545630 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c049040 con 0x55e0a0cbfe40 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c1f400640 1 -- 172.21.15.107:0/3087545630 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f5c2c031740 con 0x55e0a0cbfe40 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c42e07740 1 -- 172.21.15.107:0/3087545630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c30037070 con 0x55e0a0cbfe40 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c1f400640 1 -- 172.21.15.107:0/3087545630 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5c2c0bbee0 con 0x55e0a0cbfe40 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c42e07740 10 client.26447 fetch_fsmap learned FSMap version 39 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c42e07740 1 -- 172.21.15.107:0/3087545630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e0a0d45f90 con 0x55e0a0cbfe40 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c1f400640 1 -- 172.21.15.107:0/3087545630 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c30037070 con 0x55e0a0cbfe40 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c1f400640 1 -- 172.21.15.107:0/3087545630 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f5c2c0bb440 con 0x55e0a0cbfe40 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c42e07740 10 client.26447 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c42e07740 10 client.26447 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:26:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c42e07740 20 client.26447 populate_metadata read hostname 'smithi107' 2024-09-19T04:26:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.124+0000 7f5c42e07740 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55e0a0d46550 0x55e0a0d48970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:26:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.125+0000 7f5c42e07740 4 client.26447 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:26:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.125+0000 7f5c42e07740 1 -- 172.21.15.107:0/3087545630 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0a0baa010 con 0x55e0a0d46550 2024-09-19T04:26:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.125+0000 7f5c3d000640 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55e0a0d46550 0x55e0a0d48970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:26:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.125+0000 7f5c3d000640 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55e0a0d46550 0x55e0a0d48970 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6c69ad136a561db server_cookie=aaa5624c751e4bd8 in_seq=0 out_seq=0 2024-09-19T04:26:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.125+0000 7f5c1f400640 10 client.26447 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:26:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c1f400640 1 -- 172.21.15.107:0/3087545630 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0a0baa010 con 0x55e0a0d46550 2024-09-19T04:26:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c1f400640 10 client.26447 handle_command_reply: tid=0 2024-09-19T04:26:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 client.26447 shutdown 2024-09-19T04:26:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 -- 172.21.15.107:0/3087545630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 msgr2=0x55e0a0cc4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc4e00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0ba830 tx=0x7f5c2c0b8e20 comp rx=0 tx=0).stop 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 -- 172.21.15.107:0/3087545630 shutdown_connections 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55e0a0d46550 0x55e0a0d48970 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0a0ccb020 0x55e0a0cc5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c1f400640 0 client.26447 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0a0cc0760 0x55e0a0cc5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 --2- 172.21.15.107:0/3087545630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0a0cbfe40 0x55e0a0cc4e00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 -- 172.21.15.107:0/3087545630 >> 172.21.15.107:0/3087545630 conn(0x55e0a0cbc3d0 msgr2=0x55e0a0cc6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 -- 172.21.15.107:0/3087545630 shutdown_connections 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 1 -- 172.21.15.107:0/3087545630 wait complete. 2024-09-19T04:26:58.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.126+0000 7f5c42e07740 20 client.26447 trim_cache size 0 max 16384 2024-09-19T04:26:58.170 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:26:58.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 -- 172.21.15.107:0/323376136 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5c200a3430 msgr2=0x7f5c200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5c200a3430 0x7f5c200a58f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5c30011660 tx=0x7f5c30010a90 comp rx=0 tx=0).stop 2024-09-19T04:26:58.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 -- 172.21.15.107:0/323376136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c381019c0 msgr2=0x7f5c3819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:26:58.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c381019c0 0x7f5c3819a490 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c02a910 tx=0x7f5c2c003b20 comp rx=0 tx=0).stop 2024-09-19T04:26:58.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 -- 172.21.15.107:0/323376136 shutdown_connections 2024-09-19T04:26:58.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5c200a3430 0x7f5c200a58f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c381079c0 0x7f5c3819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c38102360 0x7f5c3819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 --2- 172.21.15.107:0/323376136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c381019c0 0x7f5c3819a490 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:26:58.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 -- 172.21.15.107:0/323376136 >> 172.21.15.107:0/323376136 conn(0x7f5c380fd740 msgr2=0x7f5c380fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:26:58.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 -- 172.21.15.107:0/323376136 shutdown_connections 2024-09-19T04:26:58.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:26:58.127+0000 7f5c3ee00640 1 -- 172.21.15.107:0/323376136 wait complete. 2024-09-19T04:26:58.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 221657 2.56 MB/sec execute 196 sec latency 3241.531 ms 2024-09-19T04:26:59.267 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:26:59.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:26:59 smithi115 ceph-mon[35477]: pgmap v202: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 753 KiB/s wr, 58 op/s 2024-09-19T04:26:59.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:26:59 smithi145 ceph-mon[35096]: pgmap v202: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 753 KiB/s wr, 58 op/s 2024-09-19T04:26:59.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:26:59 smithi107 ceph-mon[30659]: pgmap v202: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 753 KiB/s wr, 58 op/s 2024-09-19T04:26:59.646 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:26:59.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 222355 2.56 MB/sec execute 197 sec latency 3514.695 ms 2024-09-19T04:27:00.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.247+0000 7f3c65800640 1 -- 172.21.15.107:0/712795350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c60101df0 msgr2=0x7f3c601021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:00.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.247+0000 7f3c65800640 1 --2- 172.21.15.107:0/712795350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c60101df0 0x7f3c601021d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009860 tx=0x7f3c5002a340 comp rx=0 tx=0).stop 2024-09-19T04:27:00.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.247+0000 7f3c65800640 1 -- 172.21.15.107:0/712795350 shutdown_connections 2024-09-19T04:27:00.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.247+0000 7f3c65800640 1 --2- 172.21.15.107:0/712795350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c60106f20 0x7f3c6010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.247+0000 7f3c65800640 1 --2- 172.21.15.107:0/712795350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c601027a0 0x7f3c601067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.247+0000 7f3c65800640 1 --2- 172.21.15.107:0/712795350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c60101df0 0x7f3c601021d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.247+0000 7f3c65800640 1 -- 172.21.15.107:0/712795350 >> 172.21.15.107:0/712795350 conn(0x7f3c600fd8b0 msgr2=0x7f3c600ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:00.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.247+0000 7f3c65800640 1 -- 172.21.15.107:0/712795350 shutdown_connections 2024-09-19T04:27:00.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.247+0000 7f3c65800640 1 -- 172.21.15.107:0/712795350 wait complete. 2024-09-19T04:27:00.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.248+0000 7f3c65800640 1 Processor -- start 2024-09-19T04:27:00.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.248+0000 7f3c65800640 1 -- start start 2024-09-19T04:27:00.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.249+0000 7f3c65800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c60101df0 0x7f3c601984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.249+0000 7f3c65800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c601027a0 0x7f3c60198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.249+0000 7f3c65800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c60106f20 0x7f3c6019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.249+0000 7f3c65800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60111f80 con 0x7f3c601027a0 2024-09-19T04:27:00.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.249+0000 7f3c65800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60111e00 con 0x7f3c60101df0 2024-09-19T04:27:00.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.249+0000 7f3c65800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60112100 con 0x7f3c60106f20 2024-09-19T04:27:00.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.249+0000 7f3c5e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c601027a0 0x7f3c60198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.249+0000 7f3c5e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c601027a0 0x7f3c60198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39076/0 (socket says 172.21.15.107:39076) 2024-09-19T04:27:00.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.249+0000 7f3c5e000640 1 -- 172.21.15.107:0/2878199662 learned_addr learned my addr 172.21.15.107:0/2878199662 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:00.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.249+0000 7f3c5f400640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c60106f20 0x7f3c6019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c5e000640 1 -- 172.21.15.107:0/2878199662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c60106f20 msgr2=0x7f3c6019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:00.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c5e000640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c60106f20 0x7f3c6019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c5e000640 1 -- 172.21.15.107:0/2878199662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c60101df0 msgr2=0x7f3c601984e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:27:00.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c5e000640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c60101df0 0x7f3c601984e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c5e000640 1 -- 172.21.15.107:0/2878199662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6019d490 con 0x7f3c601027a0 2024-09-19T04:27:00.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c5e000640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c601027a0 0x7f3c60198a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00e9c0 tx=0x7f3c4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:00.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c57400640 1 -- 172.21.15.107:0/2878199662 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00cde0 con 0x7f3c601027a0 2024-09-19T04:27:00.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c57400640 1 -- 172.21.15.107:0/2878199662 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4c0035f0 con 0x7f3c601027a0 2024-09-19T04:27:00.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c65800640 1 -- 172.21.15.107:0/2878199662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6019d780 con 0x7f3c601027a0 2024-09-19T04:27:00.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c57400640 1 -- 172.21.15.107:0/2878199662 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c010690 con 0x7f3c601027a0 2024-09-19T04:27:00.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.250+0000 7f3c65800640 1 -- 172.21.15.107:0/2878199662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c601a5060 con 0x7f3c601027a0 2024-09-19T04:27:00.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.253+0000 7f3c6823d740 1 Processor -- start 2024-09-19T04:27:00.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.254+0000 7f3c6823d740 1 -- start start 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.254+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 0x55b4219f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.254+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219fa490 0x55b421a08d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.254+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 0x55b421a0b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.254+0000 7f3c6823d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b4219603e0 con 0x55b421a09260 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.254+0000 7f3c6823d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b421907d80 con 0x55b4219f9b70 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.254+0000 7f3c6823d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b42195c3a0 con 0x55b4219fa490 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c57400640 1 -- 172.21.15.107:0/2878199662 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f3c4c004020 con 0x7f3c601027a0 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219fa490 0x55b421a08d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219fa490 0x55b421a08d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55642/0 (socket says 172.21.15.107:55642) 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5ea00640 1 -- 172.21.15.107:0/2796615058 learned_addr learned my addr 172.21.15.107:0/2796615058 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5f400640 1 --2- 172.21.15.107:0/2796615058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 0x55b4219f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5ea00640 1 -- 172.21.15.107:0/2796615058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 msgr2=0x55b4219f9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:00.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5ea00640 1 --2- 172.21.15.107:0/2796615058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 0x55b4219f9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5e000640 1 --2- 172.21.15.107:0/2796615058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 0x55b421a0b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5ea00640 1 -- 172.21.15.107:0/2796615058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 msgr2=0x55b421a0b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5ea00640 1 --2- 172.21.15.107:0/2796615058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 0x55b421a0b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5ea00640 1 -- 172.21.15.107:0/2796615058 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b421926ae0 con 0x55b4219fa490 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.255+0000 7f3c5e000640 1 --2- 172.21.15.107:0/2796615058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 0x55b421a0b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.256+0000 7f3c5ea00640 1 --2- 172.21.15.107:0/2796615058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219fa490 0x55b421a08d20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3c50000c00 tx=0x7f3c50009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c6421ea1277ce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.256+0000 7f3c4be00640 1 -- 172.21.15.107:0/2796615058 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5003f020 con 0x55b4219fa490 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.256+0000 7f3c4be00640 1 -- 172.21.15.107:0/2796615058 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c50030e50 con 0x55b4219fa490 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.256+0000 7f3c4be00640 1 -- 172.21.15.107:0/2796615058 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50037040 con 0x55b4219fa490 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.256+0000 7f3c57400640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3c440a3430 0x7f3c440a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c57400640 1 -- 172.21.15.107:0/2878199662 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f3c4c0d0050 con 0x7f3c601027a0 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c5f400640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3c440a3430 0x7f3c440a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c6823d740 1 -- 172.21.15.107:0/2796615058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219fa490 msgr2=0x55b421a08d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c6823d740 1 --2- 172.21.15.107:0/2796615058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219fa490 0x55b421a08d20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3c50000c00 tx=0x7f3c50009450 comp rx=0 tx=0).stop 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c6823d740 1 -- 172.21.15.107:0/2796615058 shutdown_connections 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c6823d740 1 --2- 172.21.15.107:0/2796615058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 0x55b421a0b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c6823d740 1 --2- 172.21.15.107:0/2796615058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219fa490 0x55b421a08d20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c6823d740 1 --2- 172.21.15.107:0/2796615058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 0x55b4219f9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c6823d740 1 -- 172.21.15.107:0/2796615058 >> 172.21.15.107:0/2796615058 conn(0x55b4219f6200 msgr2=0x55b4219f65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c5f400640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3c440a3430 0x7f3c440a58f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3c58005fd0 tx=0x7f3c58009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c6823d740 1 -- 172.21.15.107:0/2796615058 shutdown_connections 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.257+0000 7f3c6823d740 1 -- 172.21.15.107:0/2796615058 wait complete. 2024-09-19T04:27:00.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.258+0000 7f3c6823d740 1 -- start start 2024-09-19T04:27:00.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.258+0000 7f3c6823d740 1 Processor -- start 2024-09-19T04:27:00.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 0x55b421a06160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b421a09260 0x55b421a06be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c6823d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b421a0be30 con 0x55b4219fa490 2024-09-19T04:27:00.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c6823d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b42191b0f0 con 0x55b421a09260 2024-09-19T04:27:00.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c6823d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b421a0bfb0 con 0x55b4219f9b70 2024-09-19T04:27:00.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c6823d740 10 client.0 _ll_register_callbacks cb 0x55b421902bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:00.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c5e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c5ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 0x55b421a06160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c5e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39092/0 (socket says 172.21.15.107:39092) 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c5e000640 1 -- 172.21.15.107:0/1334643278 learned_addr learned my addr 172.21.15.107:0/1334643278 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c5e000640 1 -- 172.21.15.107:0/1334643278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 msgr2=0x55b421a06160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c5e000640 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 0x55b421a06160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c5e000640 1 -- 172.21.15.107:0/1334643278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b421a09260 msgr2=0x55b421a06be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c5e000640 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b421a09260 0x55b421a06be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c5e000640 1 -- 172.21.15.107:0/1334643278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4216d13c0 con 0x55b4219fa490 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.259+0000 7f3c5ea00640 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 0x55b421a06160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.260+0000 7f3c5e000640 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a066a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00c6a0 tx=0x7f3c4c09bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f4be42161eb46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:00.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.260+0000 7f3c48c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.260+0000 7f3c48c00640 1 -- 172.21.15.107:0/1334643278 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c09c380 con 0x55b4219fa490 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.260+0000 7f3c48c00640 1 -- 172.21.15.107:0/1334643278 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f3c4c0d2070 con 0x55b4219fa490 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.260+0000 7f3c6823d740 1 -- 172.21.15.107:0/1334643278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c50030e50 con 0x55b4219fa490 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.260+0000 7f3c48c00640 1 -- 172.21.15.107:0/1334643278 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3c4c0b4070 con 0x55b4219fa490 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.260+0000 7f3c6823d740 10 client.16404 fetch_fsmap learned FSMap version 39 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.260+0000 7f3c6823d740 1 -- 172.21.15.107:0/1334643278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b421a17e80 con 0x55b4219fa490 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.260+0000 7f3c48c00640 1 -- 172.21.15.107:0/1334643278 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c50030e50 con 0x55b4219fa490 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.261+0000 7f3c48c00640 1 -- 172.21.15.107:0/1334643278 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f3c4c09b7d0 con 0x55b4219fa490 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.261+0000 7f3c6823d740 10 client.16404 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.261+0000 7f3c6823d740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.261+0000 7f3c6823d740 20 client.16404 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:00.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.261+0000 7f3c6823d740 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b421a192e0 0x55b421a1b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.261+0000 7f3c6823d740 4 client.16404 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.261+0000 7f3c6823d740 1 -- 172.21.15.107:0/1334643278 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b42196d950 con 0x55b421a192e0 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.261+0000 7f3c5ea00640 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b421a192e0 0x55b421a1b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.262+0000 7f3c5ea00640 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b421a192e0 0x55b421a1b780 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2426dcb4566477d server_cookie=3a9873f2f3679ca5 in_seq=0 out_seq=0 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.262+0000 7f3c48c00640 10 client.16404 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.262+0000 7f3c48c00640 1 -- 172.21.15.107:0/1334643278 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b42196d950 con 0x55b421a192e0 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.262+0000 7f3c48c00640 10 client.16404 handle_command_reply: tid=0 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 client.16404 shutdown 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 -- 172.21.15.107:0/1334643278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 msgr2=0x55b421a066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a066a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00c6a0 tx=0x7f3c4c09bce0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 -- 172.21.15.107:0/1334643278 shutdown_connections 2024-09-19T04:27:00.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b421a192e0 0x55b421a1b780 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b421a09260 0x55b421a06be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a066a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 --2- 172.21.15.107:0/1334643278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 0x55b421a06160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 -- 172.21.15.107:0/1334643278 >> 172.21.15.107:0/1334643278 conn(0x55b4219f6200 msgr2=0x55b4219fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c48c00640 0 client.16404 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 -- 172.21.15.107:0/1334643278 shutdown_connections 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 1 -- 172.21.15.107:0/1334643278 wait complete. 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.263+0000 7f3c6823d740 20 client.16404 trim_cache size 0 max 16384 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.277+0000 7f3c6823d740 1 Processor -- start 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.277+0000 7f3c6823d740 1 -- start start 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.277+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 0x55b421a21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.277+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219fa490 0x55b421a215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 0x55b4219f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c6823d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601ab4b0 con 0x55b421a09260 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c6823d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601ad110 con 0x55b4219fa490 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c6823d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601abb80 con 0x55b4219f9b70 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 0x55b4219f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219fa490 0x55b421a215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 0x55b4219f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39094/0 (socket says 172.21.15.107:39094) 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5f400640 1 -- 172.21.15.107:0/618856263 learned_addr learned my addr 172.21.15.107:0/618856263 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5ea00640 1 --2- 172.21.15.107:0/618856263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 0x55b421a21090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5e000640 1 -- 172.21.15.107:0/618856263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 msgr2=0x55b421a21090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5e000640 1 --2- 172.21.15.107:0/618856263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 0x55b421a21090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5e000640 1 -- 172.21.15.107:0/618856263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 msgr2=0x55b4219f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5e000640 1 --2- 172.21.15.107:0/618856263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 0x55b4219f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5e000640 1 -- 172.21.15.107:0/618856263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4219f1770 con 0x55b4219fa490 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.278+0000 7f3c5ea00640 1 --2- 172.21.15.107:0/618856263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 0x55b421a21090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.279+0000 7f3c5e000640 1 --2- 172.21.15.107:0/618856263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219fa490 0x55b421a215d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c09a630 tx=0x7f3c4c09be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15c6f69f1beea4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.279+0000 7f3c4b400640 1 -- 172.21.15.107:0/618856263 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c09c380 con 0x55b4219fa490 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.279+0000 7f3c4b400640 1 -- 172.21.15.107:0/618856263 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4c0a0730 con 0x55b4219fa490 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.279+0000 7f3c4b400640 1 -- 172.21.15.107:0/618856263 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c09b7d0 con 0x55b4219fa490 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.279+0000 7f3c6823d740 1 -- 172.21.15.107:0/618856263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219fa490 msgr2=0x55b421a215d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.279+0000 7f3c6823d740 1 --2- 172.21.15.107:0/618856263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219fa490 0x55b421a215d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c09a630 tx=0x7f3c4c09be10 comp rx=0 tx=0).stop 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.280+0000 7f3c6823d740 1 -- 172.21.15.107:0/618856263 shutdown_connections 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.280+0000 7f3c6823d740 1 --2- 172.21.15.107:0/618856263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b421a09260 0x55b4219f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.280+0000 7f3c6823d740 1 --2- 172.21.15.107:0/618856263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219fa490 0x55b421a215d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.280+0000 7f3c6823d740 1 --2- 172.21.15.107:0/618856263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b4219f9b70 0x55b421a21090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.280+0000 7f3c6823d740 1 -- 172.21.15.107:0/618856263 >> 172.21.15.107:0/618856263 conn(0x55b4219f6200 msgr2=0x55b421a05250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.280+0000 7f3c6823d740 1 -- 172.21.15.107:0/618856263 shutdown_connections 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.280+0000 7f3c6823d740 1 -- 172.21.15.107:0/618856263 wait complete. 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.280+0000 7f3c6823d740 1 -- start start 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.280+0000 7f3c6823d740 1 Processor -- start 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 0x55b421a0c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a0cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c6823d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b421a09260 0x55b421a82050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c6823d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601ac980 con 0x55b4219fa490 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c6823d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601ac250 con 0x55b4219f9b70 2024-09-19T04:27:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c6823d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601a94b0 con 0x55b421a09260 2024-09-19T04:27:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c6823d740 10 client.0 _ll_register_callbacks cb 0x7f3c4c004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c5ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 0x55b421a0c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c5e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a0cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c5ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 0x55b421a0c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:56874/0 (socket says 172.21.15.107:56874) 2024-09-19T04:27:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c5ea00640 1 -- 172.21.15.107:0/469230794 learned_addr learned my addr 172.21.15.107:0/469230794 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c5f400640 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b421a09260 0x55b421a82050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c5e000640 1 -- 172.21.15.107:0/469230794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b421a09260 msgr2=0x55b421a82050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c5e000640 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b421a09260 0x55b421a82050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c5e000640 1 -- 172.21.15.107:0/469230794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 msgr2=0x55b421a0c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c5e000640 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 0x55b421a0c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.281+0000 7f3c5e000640 1 -- 172.21.15.107:0/469230794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4219f7390 con 0x55b4219fa490 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.282+0000 7f3c5e000640 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a0cb20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55b421a07250 tx=0x7f3c4c09bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790dde9c16edf182 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.282+0000 7f3c48c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.282+0000 7f3c48c00640 1 -- 172.21.15.107:0/469230794 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c09c380 con 0x55b4219fa490 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.282+0000 7f3c48c00640 1 -- 172.21.15.107:0/469230794 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8440+0+0 (secure 0 0 0) 0x7f3c4c09b240 con 0x55b4219fa490 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.282+0000 7f3c6823d740 1 -- 172.21.15.107:0/469230794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c4c0a0730 con 0x55b4219fa490 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.282+0000 7f3c48c00640 1 -- 172.21.15.107:0/469230794 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3c4c0037a0 con 0x55b4219fa490 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.282+0000 7f3c6823d740 10 client.16410 fetch_fsmap learned FSMap version 39 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.282+0000 7f3c6823d740 1 -- 172.21.15.107:0/469230794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b421a835c0 con 0x55b4219fa490 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.282+0000 7f3c48c00640 1 -- 172.21.15.107:0/469230794 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4c0a0730 con 0x55b4219fa490 2024-09-19T04:27:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.283+0000 7f3c48c00640 1 -- 172.21.15.107:0/469230794 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f3c4c0a18c0 con 0x55b4219fa490 2024-09-19T04:27:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.283+0000 7f3c6823d740 10 client.16410 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:27:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.283+0000 7f3c6823d740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.283+0000 7f3c6823d740 20 client.16410 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.283+0000 7f3c6823d740 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b421a83a90 0x55b421a85eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.283+0000 7f3c6823d740 4 client.16410 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:27:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.283+0000 7f3c6823d740 1 -- 172.21.15.107:0/469230794 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b421967e60 con 0x55b421a83a90 2024-09-19T04:27:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.283+0000 7f3c5ea00640 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b421a83a90 0x55b421a85eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.283+0000 7f3c5ea00640 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b421a83a90 0x55b421a85eb0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10499e98e1707596 server_cookie=e8d4a5da1e5a3ffa in_seq=0 out_seq=0 2024-09-19T04:27:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c48c00640 10 client.16410 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c48c00640 1 -- 172.21.15.107:0/469230794 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b421967e60 con 0x55b421a83a90 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c48c00640 10 client.16410 handle_command_reply: tid=0 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 client.16410 shutdown 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 -- 172.21.15.107:0/469230794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 msgr2=0x55b421a0cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a0cb20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55b421a07250 tx=0x7f3c4c09bce0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 -- 172.21.15.107:0/469230794 shutdown_connections 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b421a83a90 0x55b421a85eb0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b421a09260 0x55b421a82050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4219fa490 0x55b421a0cb20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 --2- 172.21.15.107:0/469230794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4219f9b70 0x55b421a0c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 -- 172.21.15.107:0/469230794 >> 172.21.15.107:0/469230794 conn(0x55b4219f6200 msgr2=0x55b421a079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:00.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c48c00640 0 client.16410 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:00.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 -- 172.21.15.107:0/469230794 shutdown_connections 2024-09-19T04:27:00.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 1 -- 172.21.15.107:0/469230794 wait complete. 2024-09-19T04:27:00.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.284+0000 7f3c6823d740 20 client.16410 trim_cache size 0 max 16384 2024-09-19T04:27:00.374 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:27:00.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.285+0000 7f3c65800640 1 -- 172.21.15.107:0/2878199662 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3c440a3430 msgr2=0x7f3c440a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:00.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.285+0000 7f3c65800640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3c440a3430 0x7f3c440a58f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3c58005fd0 tx=0x7f3c58009500 comp rx=0 tx=0).stop 2024-09-19T04:27:00.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.285+0000 7f3c65800640 1 -- 172.21.15.107:0/2878199662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c601027a0 msgr2=0x7f3c60198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:00.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.285+0000 7f3c65800640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c601027a0 0x7f3c60198a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00e9c0 tx=0x7f3c4c00ee80 comp rx=0 tx=0).stop 2024-09-19T04:27:00.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.285+0000 7f3c65800640 1 -- 172.21.15.107:0/2878199662 shutdown_connections 2024-09-19T04:27:00.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.285+0000 7f3c65800640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3c440a3430 0x7f3c440a58f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.285+0000 7f3c65800640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c60106f20 0x7f3c6019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.285+0000 7f3c65800640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c601027a0 0x7f3c60198a20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.285+0000 7f3c65800640 1 --2- 172.21.15.107:0/2878199662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c60101df0 0x7f3c601984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:00.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.285+0000 7f3c65800640 1 -- 172.21.15.107:0/2878199662 >> 172.21.15.107:0/2878199662 conn(0x7f3c600fd8b0 msgr2=0x7f3c600feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:00.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.286+0000 7f3c65800640 1 -- 172.21.15.107:0/2878199662 shutdown_connections 2024-09-19T04:27:00.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:00.286+0000 7f3c65800640 1 -- 172.21.15.107:0/2878199662 wait complete. 2024-09-19T04:27:00.515 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:00 smithi107 ceph-mon[30659]: pgmap v203: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 753 KiB/s wr, 58 op/s 2024-09-19T04:27:00.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 222395 2.55 MB/sec execute 198 sec latency 1311.674 ms 2024-09-19T04:27:00.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:00 smithi115 ceph-mon[35477]: pgmap v203: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 753 KiB/s wr, 58 op/s 2024-09-19T04:27:00.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:00 smithi145 ceph-mon[35096]: pgmap v203: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 753 KiB/s wr, 58 op/s 2024-09-19T04:27:01.680 INFO:tasks.workunit.client.0.smithi107.stdout: 1 222395 2.53 MB/sec execute 199 sec latency 1117.111 ms 2024-09-19T04:27:02.395 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:27:02.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:27:02.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 222524 2.52 MB/sec execute 200 sec latency 1303.943 ms 2024-09-19T04:27:02.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:02 smithi115 ceph-mon[35477]: pgmap v204: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1021 KiB/s wr, 81 op/s 2024-09-19T04:27:02.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:02 smithi145 ceph-mon[35096]: pgmap v204: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1021 KiB/s wr, 81 op/s 2024-09-19T04:27:02.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:02 smithi107 ceph-mon[30659]: pgmap v204: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1021 KiB/s wr, 81 op/s 2024-09-19T04:27:02.770 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:03.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.377+0000 7fdd4b200640 1 -- 172.21.15.107:0/3721102028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd4c108f60 msgr2=0x7fdd4c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:03.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.377+0000 7fdd4b200640 1 --2- 172.21.15.107:0/3721102028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd4c108f60 0x7fdd4c110ae0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009ca0 tx=0x7fdd4402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.377+0000 7fdd4b200640 1 -- 172.21.15.107:0/3721102028 shutdown_connections 2024-09-19T04:27:03.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.377+0000 7fdd4b200640 1 --2- 172.21.15.107:0/3721102028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd4c108f60 0x7fdd4c110ae0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.377+0000 7fdd4b200640 1 --2- 172.21.15.107:0/3721102028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd4c1047e0 0x7fdd4c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.377+0000 7fdd4b200640 1 --2- 172.21.15.107:0/3721102028 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd4c103e30 0x7fdd4c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.377+0000 7fdd4b200640 1 -- 172.21.15.107:0/3721102028 >> 172.21.15.107:0/3721102028 conn(0x7fdd4c0ff910 msgr2=0x7fdd4c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:03.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.377+0000 7fdd4b200640 1 -- 172.21.15.107:0/3721102028 shutdown_connections 2024-09-19T04:27:03.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.377+0000 7fdd4b200640 1 -- 172.21.15.107:0/3721102028 wait complete. 2024-09-19T04:27:03.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.378+0000 7fdd4b200640 1 Processor -- start 2024-09-19T04:27:03.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.378+0000 7fdd4b200640 1 -- start start 2024-09-19T04:27:03.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd4c103e30 0x7fdd4c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd4c1047e0 0x7fdd4c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd4c108f60 0x7fdd4c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c113fc0 con 0x7fdd4c108f60 2024-09-19T04:27:03.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c113e40 con 0x7fdd4c103e30 2024-09-19T04:27:03.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c114140 con 0x7fdd4c1047e0 2024-09-19T04:27:03.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd4c108f60 0x7fdd4c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd4c108f60 0x7fdd4c19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39118/0 (socket says 172.21.15.107:39118) 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4a800640 1 -- 172.21.15.107:0/1277477471 learned_addr learned my addr 172.21.15.107:0/1277477471 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd49e00640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd4c103e30 0x7fdd4c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd43e00640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd4c1047e0 0x7fdd4c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4a800640 1 -- 172.21.15.107:0/1277477471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd4c1047e0 msgr2=0x7fdd4c19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4a800640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd4c1047e0 0x7fdd4c19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4a800640 1 -- 172.21.15.107:0/1277477471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd4c103e30 msgr2=0x7fdd4c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4a800640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd4c103e30 0x7fdd4c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.379+0000 7fdd4a800640 1 -- 172.21.15.107:0/1277477471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c19f3b0 con 0x7fdd4c108f60 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.380+0000 7fdd4a800640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd4c108f60 0x7fdd4c19ecd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdd44031a20 tx=0x7fdd44031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.380+0000 7fdd42a00640 1 -- 172.21.15.107:0/1277477471 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44003c30 con 0x7fdd4c108f60 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.380+0000 7fdd42a00640 1 -- 172.21.15.107:0/1277477471 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd44003dd0 con 0x7fdd4c108f60 2024-09-19T04:27:03.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.380+0000 7fdd4b200640 1 -- 172.21.15.107:0/1277477471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4c19f640 con 0x7fdd4c108f60 2024-09-19T04:27:03.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.380+0000 7fdd42a00640 1 -- 172.21.15.107:0/1277477471 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4402fc40 con 0x7fdd4c108f60 2024-09-19T04:27:03.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.380+0000 7fdd4b200640 1 -- 172.21.15.107:0/1277477471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4c1a6f80 con 0x7fdd4c108f60 2024-09-19T04:27:03.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.383+0000 7fdd42a00640 1 -- 172.21.15.107:0/1277477471 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fdd44004050 con 0x7fdd4c108f60 2024-09-19T04:27:03.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.383+0000 7fdd53671740 1 Processor -- start 2024-09-19T04:27:03.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd53671740 1 -- start start 2024-09-19T04:27:03.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd53671740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 0x5560d4d44090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd53671740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 0x5560d4d4e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd53671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d51280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd53671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5560d4caa290 con 0x5560d4d4ee90 2024-09-19T04:27:03.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd53671740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5560d4c51b40 con 0x5560d4d43cb0 2024-09-19T04:27:03.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd53671740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5560d4ca6250 con 0x5560d4d445d0 2024-09-19T04:27:03.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 0x5560d4d4e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 0x5560d4d4e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55714/0 (socket says 172.21.15.107:55714) 2024-09-19T04:27:03.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd49e00640 1 -- 172.21.15.107:0/2952505674 learned_addr learned my addr 172.21.15.107:0/2952505674 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:03.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd43e00640 1 --2- 172.21.15.107:0/2952505674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d51280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.384+0000 7fdd43e00640 1 --2- 172.21.15.107:0/2952505674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 0x5560d4d44090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd49e00640 1 -- 172.21.15.107:0/2952505674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 msgr2=0x5560d4d44090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:03.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd49e00640 1 --2- 172.21.15.107:0/2952505674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 0x5560d4d44090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd49e00640 1 -- 172.21.15.107:0/2952505674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 msgr2=0x5560d4d51280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd49e00640 1 --2- 172.21.15.107:0/2952505674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d51280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd49e00640 1 -- 172.21.15.107:0/2952505674 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d4a1b6a0 con 0x5560d4d445d0 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd42a00640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdd300a3430 0x7fdd300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd49e00640 1 --2- 172.21.15.107:0/2952505674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 0x5560d4d4e950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009d20 tx=0x7fdd3c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=333dad295f71025b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd42a00640 1 -- 172.21.15.107:0/1277477471 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fdd440f0d60 con 0x7fdd4c108f60 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd43e00640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdd300a3430 0x7fdd300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd3b000640 1 -- 172.21.15.107:0/2952505674 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c01e070 con 0x5560d4d445d0 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd3b000640 1 -- 172.21.15.107:0/2952505674 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3c019b10 con 0x5560d4d445d0 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.385+0000 7fdd3b000640 1 -- 172.21.15.107:0/2952505674 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00ebf0 con 0x5560d4d445d0 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.386+0000 7fdd53671740 1 -- 172.21.15.107:0/2952505674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 msgr2=0x5560d4d4e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:03.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.386+0000 7fdd43e00640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdd300a3430 0x7fdd300a58f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5560d4d415c0 tx=0x7fdd34014040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:03.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.386+0000 7fdd53671740 1 --2- 172.21.15.107:0/2952505674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 0x5560d4d4e950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009d20 tx=0x7fdd3c011810 comp rx=0 tx=0).stop 2024-09-19T04:27:03.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.386+0000 7fdd53671740 1 -- 172.21.15.107:0/2952505674 shutdown_connections 2024-09-19T04:27:03.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.386+0000 7fdd53671740 1 --2- 172.21.15.107:0/2952505674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d51280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.386+0000 7fdd53671740 1 --2- 172.21.15.107:0/2952505674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 0x5560d4d4e950 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.386+0000 7fdd53671740 1 --2- 172.21.15.107:0/2952505674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 0x5560d4d44090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.386+0000 7fdd53671740 1 -- 172.21.15.107:0/2952505674 >> 172.21.15.107:0/2952505674 conn(0x5560d4d40240 msgr2=0x5560d4d40620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:03.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.386+0000 7fdd53671740 1 -- 172.21.15.107:0/2952505674 shutdown_connections 2024-09-19T04:27:03.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.386+0000 7fdd53671740 1 -- 172.21.15.107:0/2952505674 wait complete. 2024-09-19T04:27:03.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.387+0000 7fdd53671740 1 -- start start 2024-09-19T04:27:03.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.387+0000 7fdd53671740 1 Processor -- start 2024-09-19T04:27:03.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.387+0000 7fdd53671740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d4e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.387+0000 7fdd53671740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 0x5560d4d4ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.387+0000 7fdd53671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d4bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.387+0000 7fdd53671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5560d4d51b00 con 0x5560d4d4ee90 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.387+0000 7fdd53671740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5560d4c64fa0 con 0x5560d4d445d0 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.387+0000 7fdd53671740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5560d4d51c80 con 0x5560d4d43cb0 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.387+0000 7fdd53671740 10 client.0 _ll_register_callbacks cb 0x5560d49b8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d4e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd43e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 0x5560d4d4ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d4e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55722/0 (socket says 172.21.15.107:55722) 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd49e00640 1 -- 172.21.15.107:0/3332469006 learned_addr learned my addr 172.21.15.107:0/3332469006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd4a800640 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d4bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd49e00640 1 -- 172.21.15.107:0/3332469006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 msgr2=0x5560d4d4ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd49e00640 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 0x5560d4d4ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd49e00640 1 -- 172.21.15.107:0/3332469006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 msgr2=0x5560d4d4bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd49e00640 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d4bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd49e00640 1 -- 172.21.15.107:0/3332469006 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d4d4c7d0 con 0x5560d4d43cb0 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd49e00640 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d4e790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c005e00 tx=0x7fdd3c019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cddcb687f337bef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.388+0000 7fdd2f400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.389+0000 7fdd2f400640 1 -- 172.21.15.107:0/3332469006 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c01e070 con 0x5560d4d43cb0 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.389+0000 7fdd2f400640 1 -- 172.21.15.107:0/3332469006 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fdd3c00eaf0 con 0x5560d4d43cb0 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.389+0000 7fdd53671740 1 -- 172.21.15.107:0/3332469006 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3c019b10 con 0x5560d4d43cb0 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.389+0000 7fdd2f400640 1 -- 172.21.15.107:0/3332469006 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdd3c031050 con 0x5560d4d43cb0 2024-09-19T04:27:03.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.389+0000 7fdd53671740 10 client.26437 fetch_fsmap learned FSMap version 39 2024-09-19T04:27:03.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.389+0000 7fdd53671740 1 -- 172.21.15.107:0/3332469006 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5560d4d5d9e0 con 0x5560d4d43cb0 2024-09-19T04:27:03.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.389+0000 7fdd2f400640 1 -- 172.21.15.107:0/3332469006 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3c019b10 con 0x5560d4d43cb0 2024-09-19T04:27:03.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.390+0000 7fdd2f400640 1 -- 172.21.15.107:0/3332469006 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fdd3c0046c0 con 0x5560d4d43cb0 2024-09-19T04:27:03.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.390+0000 7fdd53671740 10 client.26437 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:27:03.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.390+0000 7fdd53671740 10 client.26437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:03.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.390+0000 7fdd53671740 20 client.26437 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:03.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.390+0000 7fdd53671740 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5560d4d5ee40 0x5560d4d61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.390+0000 7fdd53671740 4 client.26437 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:03.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.390+0000 7fdd53671740 1 -- 172.21.15.107:0/3332469006 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560d4cb7800 con 0x5560d4d5ee40 2024-09-19T04:27:03.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.390+0000 7fdd43e00640 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5560d4d5ee40 0x5560d4d61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.391+0000 7fdd43e00640 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5560d4d5ee40 0x5560d4d61230 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc11240222647e53 server_cookie=8e64db45c55e5031 in_seq=0 out_seq=0 2024-09-19T04:27:03.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.391+0000 7fdd2f400640 10 client.26437 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.391+0000 7fdd2f400640 1 -- 172.21.15.107:0/3332469006 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560d4cb7800 con 0x5560d4d5ee40 2024-09-19T04:27:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.391+0000 7fdd2f400640 10 client.26437 handle_command_reply: tid=0 2024-09-19T04:27:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 client.26437 shutdown 2024-09-19T04:27:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 -- 172.21.15.107:0/3332469006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 msgr2=0x5560d4d4e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d4e790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c005e00 tx=0x7fdd3c019660 comp rx=0 tx=0).stop 2024-09-19T04:27:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 -- 172.21.15.107:0/3332469006 shutdown_connections 2024-09-19T04:27:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5560d4d5ee40 0x5560d4d61230 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d4bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 0x5560d4d4ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 --2- 172.21.15.107:0/3332469006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d4e790 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd2f400640 0 client.26437 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:03.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 -- 172.21.15.107:0/3332469006 >> 172.21.15.107:0/3332469006 conn(0x5560d4d40240 msgr2=0x5560d4d45f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:03.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 -- 172.21.15.107:0/3332469006 shutdown_connections 2024-09-19T04:27:03.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 1 -- 172.21.15.107:0/3332469006 wait complete. 2024-09-19T04:27:03.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.392+0000 7fdd53671740 20 client.26437 trim_cache size 0 max 16384 2024-09-19T04:27:03.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.410+0000 7fdd53671740 1 Processor -- start 2024-09-19T04:27:03.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.410+0000 7fdd53671740 1 -- start start 2024-09-19T04:27:03.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.410+0000 7fdd53671740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 0x5560d4d46eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.410+0000 7fdd53671740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 0x5560d4d3f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.410+0000 7fdd53671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d66b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.410+0000 7fdd53671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1af8a0 con 0x5560d4d4ee90 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.410+0000 7fdd53671740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1b1500 con 0x5560d4d43cb0 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.410+0000 7fdd53671740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1aff70 con 0x5560d4d445d0 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd4a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d66b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 0x5560d4d46eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd4a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d66b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39152/0 (socket says 172.21.15.107:39152) 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd4a800640 1 -- 172.21.15.107:0/3638599368 learned_addr learned my addr 172.21.15.107:0/3638599368 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd43e00640 1 --2- 172.21.15.107:0/3638599368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 0x5560d4d3f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd4a800640 1 -- 172.21.15.107:0/3638599368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 msgr2=0x5560d4d3f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd4a800640 1 --2- 172.21.15.107:0/3638599368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 0x5560d4d3f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd4a800640 1 -- 172.21.15.107:0/3638599368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 msgr2=0x5560d4d46eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:03.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd4a800640 1 --2- 172.21.15.107:0/3638599368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 0x5560d4d46eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd4a800640 1 -- 172.21.15.107:0/3638599368 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d4d413b0 con 0x5560d4d4ee90 2024-09-19T04:27:03.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd49e00640 1 --2- 172.21.15.107:0/3638599368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 0x5560d4d46eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:03.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.411+0000 7fdd4a800640 1 --2- 172.21.15.107:0/3638599368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d66b00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdd440b9720 tx=0x7fdd44009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f16d4e4cbb3afa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:03.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd3a600640 1 -- 172.21.15.107:0/3638599368 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd440bd2c0 con 0x5560d4d4ee90 2024-09-19T04:27:03.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd3a600640 1 -- 172.21.15.107:0/3638599368 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd44036040 con 0x5560d4d4ee90 2024-09-19T04:27:03.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd3a600640 1 -- 172.21.15.107:0/3638599368 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44004730 con 0x5560d4d4ee90 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd53671740 1 -- 172.21.15.107:0/3638599368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 msgr2=0x5560d4d66b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd53671740 1 --2- 172.21.15.107:0/3638599368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d66b00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdd440b9720 tx=0x7fdd44009980 comp rx=0 tx=0).stop 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd53671740 1 -- 172.21.15.107:0/3638599368 shutdown_connections 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd53671740 1 --2- 172.21.15.107:0/3638599368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d66b00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd53671740 1 --2- 172.21.15.107:0/3638599368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d445d0 0x5560d4d3f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd53671740 1 --2- 172.21.15.107:0/3638599368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d43cb0 0x5560d4d46eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd53671740 1 -- 172.21.15.107:0/3638599368 >> 172.21.15.107:0/3638599368 conn(0x5560d4d40240 msgr2=0x5560d4d3ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd53671740 1 -- 172.21.15.107:0/3638599368 shutdown_connections 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.412+0000 7fdd53671740 1 -- 172.21.15.107:0/3638599368 wait complete. 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.413+0000 7fdd53671740 1 -- start start 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.413+0000 7fdd53671740 1 Processor -- start 2024-09-19T04:27:03.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.413+0000 7fdd53671740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d57440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.413+0000 7fdd53671740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 0x5560d4d53780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.413+0000 7fdd53671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d53cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.413+0000 7fdd53671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1b0d70 con 0x5560d4d4ee90 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.413+0000 7fdd53671740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1b0640 con 0x5560d4d445d0 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.413+0000 7fdd53671740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1ad880 con 0x5560d4d43cb0 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.413+0000 7fdd53671740 10 client.0 _ll_register_callbacks cb 0x5560d4d4c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d57440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd4a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d53cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d57440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55738/0 (socket says 172.21.15.107:55738) 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd43e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 0x5560d4d53780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd49e00640 1 -- 172.21.15.107:0/1027754055 learned_addr learned my addr 172.21.15.107:0/1027754055 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd4a800640 1 -- 172.21.15.107:0/1027754055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 msgr2=0x5560d4d57440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd4a800640 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d57440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd4a800640 1 -- 172.21.15.107:0/1027754055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 msgr2=0x5560d4d53780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd4a800640 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 0x5560d4d53780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd4a800640 1 -- 172.21.15.107:0/1027754055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d4d54300 con 0x5560d4d4ee90 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd43e00640 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 0x5560d4d53780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd49e00640 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d57440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd4a800640 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d53cc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5560d4d47430 tx=0x7fdd4402f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81147c06cb8f8bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.414+0000 7fdd2f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd2f400640 1 -- 172.21.15.107:0/1027754055 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd440da070 con 0x5560d4d4ee90 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd2f400640 1 -- 172.21.15.107:0/1027754055 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fdd440b9930 con 0x5560d4d4ee90 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd53671740 1 -- 172.21.15.107:0/1027754055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd44036040 con 0x5560d4d4ee90 2024-09-19T04:27:03.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd2f400640 1 -- 172.21.15.107:0/1027754055 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdd440bb5c0 con 0x5560d4d4ee90 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd53671740 10 client.16428 fetch_fsmap learned FSMap version 39 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd53671740 1 -- 172.21.15.107:0/1027754055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5560d4dca930 con 0x5560d4d4ee90 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd2f400640 1 -- 172.21.15.107:0/1027754055 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd44036040 con 0x5560d4d4ee90 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd2f400640 1 -- 172.21.15.107:0/1027754055 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fdd440c5a10 con 0x5560d4d4ee90 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd53671740 10 client.16428 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd53671740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd53671740 20 client.16428 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd53671740 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5560d4dcb000 0x5560d4dcd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd53671740 4 client.16428 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.415+0000 7fdd53671740 1 -- 172.21.15.107:0/1027754055 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5560d4d4c310 con 0x5560d4dcb000 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.416+0000 7fdd49e00640 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5560d4dcb000 0x5560d4dcd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.416+0000 7fdd49e00640 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5560d4dcb000 0x5560d4dcd420 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8380f0effe92c7cf server_cookie=4399e5d0cb176bb5 in_seq=0 out_seq=0 2024-09-19T04:27:03.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.416+0000 7fdd2f400640 10 client.16428 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:03.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.417+0000 7fdd2f400640 1 -- 172.21.15.107:0/1027754055 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5560d4d4c310 con 0x5560d4dcb000 2024-09-19T04:27:03.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.417+0000 7fdd2f400640 10 client.16428 handle_command_reply: tid=0 2024-09-19T04:27:03.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.417+0000 7fdd53671740 1 client.16428 shutdown 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.417+0000 7fdd53671740 1 -- 172.21.15.107:0/1027754055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 msgr2=0x5560d4d53cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.417+0000 7fdd53671740 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d53cc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5560d4d47430 tx=0x7fdd4402f460 comp rx=0 tx=0).stop 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.418+0000 7fdd53671740 1 -- 172.21.15.107:0/1027754055 shutdown_connections 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.418+0000 7fdd53671740 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5560d4dcb000 0x5560d4dcd420 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.418+0000 7fdd53671740 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d4d4ee90 0x5560d4d53cc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.418+0000 7fdd53671740 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560d4d445d0 0x5560d4d53780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.418+0000 7fdd53671740 1 --2- 172.21.15.107:0/1027754055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5560d4d43cb0 0x5560d4d57440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.418+0000 7fdd53671740 1 -- 172.21.15.107:0/1027754055 >> 172.21.15.107:0/1027754055 conn(0x5560d4d40240 msgr2=0x5560d4d3f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.418+0000 7fdd2f400640 0 client.16428 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.418+0000 7fdd53671740 1 -- 172.21.15.107:0/1027754055 shutdown_connections 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.418+0000 7fdd53671740 1 -- 172.21.15.107:0/1027754055 wait complete. 2024-09-19T04:27:03.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.418+0000 7fdd53671740 20 client.16428 trim_cache size 0 max 16384 2024-09-19T04:27:03.424 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:27:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:27:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "ea376188-090c-4aa3-89a4-1a0483a9ce00", 2024-09-19T04:27:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:27:03.424 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:27:03.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 -- 172.21.15.107:0/1277477471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdd300a3430 msgr2=0x7fdd300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdd300a3430 0x7fdd300a58f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5560d4d415c0 tx=0x7fdd34014040 comp rx=0 tx=0).stop 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 -- 172.21.15.107:0/1277477471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd4c108f60 msgr2=0x7fdd4c19ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd4c108f60 0x7fdd4c19ecd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdd44031a20 tx=0x7fdd44031a50 comp rx=0 tx=0).stop 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 -- 172.21.15.107:0/1277477471 shutdown_connections 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdd300a3430 0x7fdd300a58f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd4c108f60 0x7fdd4c19ecd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd4c1047e0 0x7fdd4c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 --2- 172.21.15.107:0/1277477471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd4c103e30 0x7fdd4c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 -- 172.21.15.107:0/1277477471 >> 172.21.15.107:0/1277477471 conn(0x7fdd4c0ff910 msgr2=0x7fdd4c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 -- 172.21.15.107:0/1277477471 shutdown_connections 2024-09-19T04:27:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:03.419+0000 7fdd4b200640 1 -- 172.21.15.107:0/1277477471 wait complete. 2024-09-19T04:27:03.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 222985 2.53 MB/sec execute 201 sec latency 1464.166 ms 2024-09-19T04:27:03.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:03 smithi115 ceph-mon[35477]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:27:03.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:03 smithi145 ceph-mon[35096]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:27:03.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:03 smithi107 ceph-mon[30659]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:27:04.468 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:27:04.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 223959 2.55 MB/sec execute 202 sec latency 24.423 ms 2024-09-19T04:27:04.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: pgmap v206: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 724 KiB/s wr, 59 op/s 2024-09-19T04:27:04.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:27:04.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:04.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:04.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:04.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:04.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:04 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: pgmap v206: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 724 KiB/s wr, 59 op/s 2024-09-19T04:27:04.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:04.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:04.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:04.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:04.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:04.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:04 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: pgmap v206: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 724 KiB/s wr, 59 op/s 2024-09-19T04:27:04.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:27:04.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:04.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:04.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:04.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:04.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:04.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:04.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.733 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.733 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:04 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:04.889 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.493+0000 7f1874c00640 1 -- 172.21.15.107:0/4264323172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1870103e30 msgr2=0x7f1870104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.493+0000 7f1874c00640 1 --2- 172.21.15.107:0/4264323172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1870103e30 0x7f1870104210 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1860009900 tx=0x7f186002a410 comp rx=0 tx=0).stop 2024-09-19T04:27:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.493+0000 7f1874c00640 1 -- 172.21.15.107:0/4264323172 shutdown_connections 2024-09-19T04:27:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.493+0000 7f1874c00640 1 --2- 172.21.15.107:0/4264323172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1870108f60 0x7f1870110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.493+0000 7f1874c00640 1 --2- 172.21.15.107:0/4264323172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18701047e0 0x7f1870108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.493+0000 7f1874c00640 1 --2- 172.21.15.107:0/4264323172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1870103e30 0x7f1870104210 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.493+0000 7f1874c00640 1 -- 172.21.15.107:0/4264323172 >> 172.21.15.107:0/4264323172 conn(0x7f18700ff910 msgr2=0x7f1870101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.493+0000 7f1874c00640 1 -- 172.21.15.107:0/4264323172 shutdown_connections 2024-09-19T04:27:05.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.493+0000 7f1874c00640 1 -- 172.21.15.107:0/4264323172 wait complete. 2024-09-19T04:27:05.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.494+0000 7f1874c00640 1 Processor -- start 2024-09-19T04:27:05.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.494+0000 7f1874c00640 1 -- start start 2024-09-19T04:27:05.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.495+0000 7f1874c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1870103e30 0x7f187019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.495+0000 7f1874c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18701047e0 0x7f187019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.495+0000 7f1874c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1870108f60 0x7f187019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.495+0000 7f1874c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1870113fd0 con 0x7f1870103e30 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.495+0000 7f1874c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1870113e50 con 0x7f1870108f60 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.495+0000 7f1874c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1870114150 con 0x7f18701047e0 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.495+0000 7f186f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1870103e30 0x7f187019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.495+0000 7f186f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1870103e30 0x7f187019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39178/0 (socket says 172.21.15.107:39178) 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.495+0000 7f186f400640 1 -- 172.21.15.107:0/2792555110 learned_addr learned my addr 172.21.15.107:0/2792555110 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.495+0000 7f186ea00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18701047e0 0x7f187019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f186fe00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1870108f60 0x7f187019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f186f400640 1 -- 172.21.15.107:0/2792555110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18701047e0 msgr2=0x7f187019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f186f400640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18701047e0 0x7f187019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f186f400640 1 -- 172.21.15.107:0/2792555110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1870108f60 msgr2=0x7f187019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f186f400640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1870108f60 0x7f187019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f186f400640 1 -- 172.21.15.107:0/2792555110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187019f460 con 0x7f1870103e30 2024-09-19T04:27:05.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f186ea00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18701047e0 0x7f187019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:05.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f186f400640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1870103e30 0x7f187019a4b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1860004770 tx=0x7f1860003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:05.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f1867e00640 1 -- 172.21.15.107:0/2792555110 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186003f070 con 0x7f1870103e30 2024-09-19T04:27:05.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f1867e00640 1 -- 172.21.15.107:0/2792555110 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1860031ca0 con 0x7f1870103e30 2024-09-19T04:27:05.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f1867e00640 1 -- 172.21.15.107:0/2792555110 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860035030 con 0x7f1870103e30 2024-09-19T04:27:05.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.496+0000 7f1874c00640 1 -- 172.21.15.107:0/2792555110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187019f6f0 con 0x7f1870103e30 2024-09-19T04:27:05.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.497+0000 7f1874c00640 1 -- 172.21.15.107:0/2792555110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187019fb80 con 0x7f1870103e30 2024-09-19T04:27:05.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.499+0000 7f1878e0d740 1 Processor -- start 2024-09-19T04:27:05.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f1867e00640 1 -- 172.21.15.107:0/2792555110 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f1860042070 con 0x7f1870103e30 2024-09-19T04:27:05.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f1878e0d740 1 -- start start 2024-09-19T04:27:05.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101892d50 0x557101893130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101893670 0x55710189d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 0x5571018a0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f1878e0d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5571017f93e0 con 0x557101893670 2024-09-19T04:27:05.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f1878e0d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5571017a0d80 con 0x55710189df30 2024-09-19T04:27:05.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f1878e0d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5571017f53a0 con 0x557101892d50 2024-09-19T04:27:05.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f186ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 0x5571018a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f186f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101893670 0x55710189d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f186ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 0x5571018a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:56954/0 (socket says 172.21.15.107:56954) 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.500+0000 7f186ea00640 1 -- 172.21.15.107:0/1656026390 learned_addr learned my addr 172.21.15.107:0/1656026390 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.501+0000 7f186fe00640 1 --2- 172.21.15.107:0/1656026390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101892d50 0x557101893130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.501+0000 7f186ea00640 1 -- 172.21.15.107:0/1656026390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101892d50 msgr2=0x557101893130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.501+0000 7f186ea00640 1 --2- 172.21.15.107:0/1656026390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101892d50 0x557101893130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.501+0000 7f186ea00640 1 -- 172.21.15.107:0/1656026390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101893670 msgr2=0x55710189d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.501+0000 7f186ea00640 1 --2- 172.21.15.107:0/1656026390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101893670 0x55710189d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.501+0000 7f186ea00640 1 -- 172.21.15.107:0/1656026390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571017bfae0 con 0x55710189df30 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.501+0000 7f186ea00640 1 --2- 172.21.15.107:0/1656026390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 0x5571018a0320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f185c0114f0 tx=0x7f185c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b84331ffa644d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.501+0000 7f185be00640 1 -- 172.21.15.107:0/1656026390 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c01e070 con 0x55710189df30 2024-09-19T04:27:05.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.501+0000 7f185be00640 1 -- 172.21.15.107:0/1656026390 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f185c0035f0 con 0x55710189df30 2024-09-19T04:27:05.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.501+0000 7f185be00640 1 -- 172.21.15.107:0/1656026390 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c00f820 con 0x55710189df30 2024-09-19T04:27:05.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f1867e00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f18540a3430 0x7f18540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f1867e00640 1 -- 172.21.15.107:0/2792555110 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f18600effd0 con 0x7f1870103e30 2024-09-19T04:27:05.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f1878e0d740 1 -- 172.21.15.107:0/1656026390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 msgr2=0x5571018a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:05.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1656026390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 0x5571018a0320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f185c0114f0 tx=0x7f185c0119b0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f186fe00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f18540a3430 0x7f18540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f1878e0d740 1 -- 172.21.15.107:0/1656026390 shutdown_connections 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1656026390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 0x5571018a0320 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1656026390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101893670 0x55710189d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1656026390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101892d50 0x557101893130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f1878e0d740 1 -- 172.21.15.107:0/1656026390 >> 172.21.15.107:0/1656026390 conn(0x55710188f2e0 msgr2=0x55710188f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.502+0000 7f1878e0d740 1 -- 172.21.15.107:0/1656026390 shutdown_connections 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.503+0000 7f1878e0d740 1 -- 172.21.15.107:0/1656026390 wait complete. 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.503+0000 7f186fe00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f18540a3430 0x7f18540a58f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1868005fd0 tx=0x7f18680096a0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.503+0000 7f1878e0d740 1 -- start start 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.503+0000 7f1878e0d740 1 Processor -- start 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557101892d50 0x55710189ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x55710189b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 0x55710189b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f1878e0d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5571018a0ba0 con 0x55710189df30 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f1878e0d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5571017b40f0 con 0x557101892d50 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f1878e0d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5571018a0d20 con 0x557101893670 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f1878e0d740 10 client.0 _ll_register_callbacks cb 0x5571017a59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f186fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 0x55710189b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f186f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557101892d50 0x55710189ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f186fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 0x55710189b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39200/0 (socket says 172.21.15.107:39200) 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f186f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557101892d50 0x55710189ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:56968/0 (socket says 172.21.15.107:56968) 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f186fe00640 1 -- 172.21.15.107:0/2475140004 learned_addr learned my addr 172.21.15.107:0/2475140004 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.504+0000 7f186ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x55710189b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.505+0000 7f186ea00640 1 -- 172.21.15.107:0/2475140004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557101892d50 msgr2=0x55710189ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.505+0000 7f186ea00640 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557101892d50 0x55710189ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.505+0000 7f186ea00640 1 -- 172.21.15.107:0/2475140004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 msgr2=0x55710189b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.505+0000 7f186ea00640 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 0x55710189b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.505+0000 7f186ea00640 1 -- 172.21.15.107:0/2475140004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55710156a3c0 con 0x557101893670 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.505+0000 7f186f400640 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557101892d50 0x55710189ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.505+0000 7f186ea00640 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x55710189b2e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f185c011eb0 tx=0x7f185c004240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9284d90055745d7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.505+0000 7f1858c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.505+0000 7f1858c00640 1 -- 172.21.15.107:0/2475140004 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c01e040 con 0x557101893670 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.505+0000 7f1878e0d740 1 -- 172.21.15.107:0/2475140004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185c0035f0 con 0x557101893670 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1858c00640 1 -- 172.21.15.107:0/2475140004 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f185c005040 con 0x557101893670 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1858c00640 1 -- 172.21.15.107:0/2475140004 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f185c005280 con 0x557101893670 2024-09-19T04:27:05.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1878e0d740 10 client.26455 fetch_fsmap learned FSMap version 39 2024-09-19T04:27:05.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1878e0d740 1 -- 172.21.15.107:0/2475140004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5571018ac7d0 con 0x557101893670 2024-09-19T04:27:05.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1858c00640 1 -- 172.21.15.107:0/2475140004 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f185c0035f0 con 0x557101893670 2024-09-19T04:27:05.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1858c00640 1 -- 172.21.15.107:0/2475140004 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f185c021030 con 0x557101893670 2024-09-19T04:27:05.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1878e0d740 10 client.26455 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:27:05.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1878e0d740 10 client.26455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:05.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1878e0d740 20 client.26455 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:05.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1878e0d740 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5571018adc30 0x5571018b0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1878e0d740 4 client.26455 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:05.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.506+0000 7f1878e0d740 1 -- 172.21.15.107:0/2475140004 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557101806950 con 0x5571018adc30 2024-09-19T04:27:05.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.507+0000 7f186f400640 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5571018adc30 0x5571018b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.507+0000 7f186f400640 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5571018adc30 0x5571018b0020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31759d17fb6366f8 server_cookie=606b2b19b60a2d04 in_seq=0 out_seq=0 2024-09-19T04:27:05.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.507+0000 7f1858c00640 10 client.26455 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:05.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.508+0000 7f1858c00640 1 -- 172.21.15.107:0/2475140004 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557101806950 con 0x5571018adc30 2024-09-19T04:27:05.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.508+0000 7f1858c00640 10 client.26455 handle_command_reply: tid=0 2024-09-19T04:27:05.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.508+0000 7f1878e0d740 1 client.26455 shutdown 2024-09-19T04:27:05.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 1 -- 172.21.15.107:0/2475140004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 msgr2=0x55710189b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x55710189b2e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f185c011eb0 tx=0x7f185c004240 comp rx=0 tx=0).stop 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 1 -- 172.21.15.107:0/2475140004 shutdown_connections 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5571018adc30 0x5571018b0020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 0x55710189b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x55710189b2e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 1 --2- 172.21.15.107:0/2475140004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557101892d50 0x55710189ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1858c00640 0 client.26455 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 1 -- 172.21.15.107:0/2475140004 >> 172.21.15.107:0/2475140004 conn(0x55710188f2e0 msgr2=0x557101895010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 1 -- 172.21.15.107:0/2475140004 shutdown_connections 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 1 -- 172.21.15.107:0/2475140004 wait complete. 2024-09-19T04:27:05.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.509+0000 7f1878e0d740 20 client.26455 trim_cache size 0 max 16384 2024-09-19T04:27:05.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f1878e0d740 1 Processor -- start 2024-09-19T04:27:05.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f1878e0d740 1 -- start start 2024-09-19T04:27:05.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101892d50 0x5571018b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x5571018b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 0x557101898040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f1878e0d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18701af9c0 con 0x557101892d50 2024-09-19T04:27:05.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f1878e0d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18701b1620 con 0x55710189df30 2024-09-19T04:27:05.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f1878e0d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f18701b0090 con 0x557101893670 2024-09-19T04:27:05.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f186f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101892d50 0x5571018b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f186f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101892d50 0x5571018b5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39208/0 (socket says 172.21.15.107:39208) 2024-09-19T04:27:05.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f186f400640 1 -- 172.21.15.107:0/1335222273 learned_addr learned my addr 172.21.15.107:0/1335222273 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:05.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f186fe00640 1 --2- 172.21.15.107:0/1335222273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 0x557101898040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.527+0000 7f186ea00640 1 --2- 172.21.15.107:0/1335222273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x5571018b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.528+0000 7f186f400640 1 -- 172.21.15.107:0/1335222273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 msgr2=0x5571018b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:05.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.528+0000 7f186f400640 1 --2- 172.21.15.107:0/1335222273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x5571018b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.528+0000 7f186f400640 1 -- 172.21.15.107:0/1335222273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 msgr2=0x557101898040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:05.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.528+0000 7f186f400640 1 --2- 172.21.15.107:0/1335222273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 0x557101898040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.528+0000 7f186f400640 1 -- 172.21.15.107:0/1335222273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571018b0390 con 0x557101892d50 2024-09-19T04:27:05.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.528+0000 7f186f400640 1 --2- 172.21.15.107:0/1335222273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101892d50 0x5571018b5840 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f18600ed740 tx=0x7f18600f3a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11465f2401090b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:05.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.528+0000 7f185b400640 1 -- 172.21.15.107:0/1335222273 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860049040 con 0x557101892d50 2024-09-19T04:27:05.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.528+0000 7f185b400640 1 -- 172.21.15.107:0/1335222273 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18600bc320 con 0x557101892d50 2024-09-19T04:27:05.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.528+0000 7f185b400640 1 -- 172.21.15.107:0/1335222273 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18600b8990 con 0x557101892d50 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.529+0000 7f1878e0d740 1 -- 172.21.15.107:0/1335222273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101892d50 msgr2=0x5571018b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.529+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1335222273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101892d50 0x5571018b5840 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f18600ed740 tx=0x7f18600f3a80 comp rx=0 tx=0).stop 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.529+0000 7f1878e0d740 1 -- 172.21.15.107:0/1335222273 shutdown_connections 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.529+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1335222273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55710189df30 0x557101898040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.529+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1335222273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x5571018b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.529+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1335222273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557101892d50 0x5571018b5840 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.529+0000 7f1878e0d740 1 -- 172.21.15.107:0/1335222273 >> 172.21.15.107:0/1335222273 conn(0x55710188f2e0 msgr2=0x55710189ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.529+0000 7f1878e0d740 1 -- 172.21.15.107:0/1335222273 shutdown_connections 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.529+0000 7f1878e0d740 1 -- 172.21.15.107:0/1335222273 wait complete. 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.530+0000 7f1878e0d740 1 -- start start 2024-09-19T04:27:05.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.530+0000 7f1878e0d740 1 Processor -- start 2024-09-19T04:27:05.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.530+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x5571018a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.530+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 0x5571018a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.530+0000 7f1878e0d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571018a69f0 0x5571018a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.530+0000 7f1878e0d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18701b0e90 con 0x55710189df30 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.530+0000 7f1878e0d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18701b0760 con 0x5571018a69f0 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.530+0000 7f1878e0d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f18701ad9a0 con 0x557101893670 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.530+0000 7f1878e0d740 10 client.0 _ll_register_callbacks cb 0x7f1868009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x5571018a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 0x5571018a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x5571018a5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55778/0 (socket says 172.21.15.107:55778) 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186f400640 1 -- 172.21.15.107:0/1118417655 learned_addr learned my addr 172.21.15.107:0/1118417655 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186fe00640 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571018a69f0 0x5571018a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186ea00640 1 -- 172.21.15.107:0/1118417655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 msgr2=0x5571018a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:05.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186ea00640 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x5571018a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186ea00640 1 -- 172.21.15.107:0/1118417655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571018a69f0 msgr2=0x5571018a2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186ea00640 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571018a69f0 0x5571018a2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186ea00640 1 -- 172.21.15.107:0/1118417655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571018a3630 con 0x55710189df30 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f186ea00640 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 0x5571018a64b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f185c004d60 tx=0x7f185c005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f0282145f88d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f1858c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.531+0000 7f1858c00640 1 -- 172.21.15.107:0/1118417655 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c01e040 con 0x55710189df30 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1878e0d740 1 -- 172.21.15.107:0/1118417655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18600bc320 con 0x55710189df30 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1858c00640 1 -- 172.21.15.107:0/1118417655 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f185c003cb0 con 0x55710189df30 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1858c00640 1 -- 172.21.15.107:0/1118417655 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f185c0042b0 con 0x55710189df30 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1878e0d740 10 client.16446 fetch_fsmap learned FSMap version 39 2024-09-19T04:27:05.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1878e0d740 1 -- 172.21.15.107:0/1118417655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5571019191f0 con 0x55710189df30 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1858c00640 1 -- 172.21.15.107:0/1118417655 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18600bc320 con 0x55710189df30 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1858c00640 1 -- 172.21.15.107:0/1118417655 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f185c021030 con 0x55710189df30 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1878e0d740 10 client.16446 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1878e0d740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1878e0d740 20 client.16446 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557101919940 0x55710191bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1878e0d740 4 client.16446 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.532+0000 7f1878e0d740 1 -- 172.21.15.107:0/1118417655 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557101800e60 con 0x557101919940 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.533+0000 7f186f400640 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557101919940 0x55710191bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.533+0000 7f186f400640 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557101919940 0x55710191bd60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60abab8b4a696295 server_cookie=95ad1677d68a4bc3 in_seq=0 out_seq=0 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.533+0000 7f1858c00640 10 client.16446 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:05.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.533+0000 7f1858c00640 1 -- 172.21.15.107:0/1118417655 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557101800e60 con 0x557101919940 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.533+0000 7f1858c00640 10 client.16446 handle_command_reply: tid=0 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.533+0000 7f1878e0d740 1 client.16446 shutdown 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 1 -- 172.21.15.107:0/1118417655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 msgr2=0x5571018a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 0x5571018a64b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f185c004d60 tx=0x7f185c005fc0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 1 -- 172.21.15.107:0/1118417655 shutdown_connections 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x557101919940 0x55710191bd60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571018a69f0 0x5571018a2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55710189df30 0x5571018a64b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 1 --2- 172.21.15.107:0/1118417655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557101893670 0x5571018a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 1 -- 172.21.15.107:0/1118417655 >> 172.21.15.107:0/1118417655 conn(0x55710188f2e0 msgr2=0x55710188db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1858c00640 0 client.16446 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:05.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 1 -- 172.21.15.107:0/1118417655 shutdown_connections 2024-09-19T04:27:05.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 1 -- 172.21.15.107:0/1118417655 wait complete. 2024-09-19T04:27:05.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.534+0000 7f1878e0d740 20 client.16446 trim_cache size 0 max 16384 2024-09-19T04:27:05.541 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:27:05.541 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:27:05.541 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:27:05.541 INFO:teuthology.orchestra.run.smithi107.stdout: "ea376188-090c-4aa3-89a4-1a0483a9ce00": { 2024-09-19T04:27:05.541 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:27:05.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "ea376188-090c-4aa3-89a4-1a0483a9ce00", 2024-09-19T04:27:05.542 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:27:05.542 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:27:05.542 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:27:05.542 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:27:05.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 -- 172.21.15.107:0/2792555110 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f18540a3430 msgr2=0x7f18540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f18540a3430 0x7f18540a58f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1868005fd0 tx=0x7f18680096a0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 -- 172.21.15.107:0/2792555110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1870103e30 msgr2=0x7f187019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1870103e30 0x7f187019a4b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1860004770 tx=0x7f1860003c10 comp rx=0 tx=0).stop 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 -- 172.21.15.107:0/2792555110 shutdown_connections 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f18540a3430 0x7f18540a58f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1870108f60 0x7f187019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18701047e0 0x7f187019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 --2- 172.21.15.107:0/2792555110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1870103e30 0x7f187019a4b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 -- 172.21.15.107:0/2792555110 >> 172.21.15.107:0/2792555110 conn(0x7f18700ff910 msgr2=0x7f1870100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 -- 172.21.15.107:0/2792555110 shutdown_connections 2024-09-19T04:27:05.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:05.535+0000 7f1874c00640 1 -- 172.21.15.107:0/2792555110 wait complete. 2024-09-19T04:27:05.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 224180 2.54 MB/sec execute 203 sec latency 790.470 ms 2024-09-19T04:27:06.251 INFO:tasks.cephfs.filesystem:scrub status for tag:ea376188-090c-4aa3-89a4-1a0483a9ce00 - {'path': '/', 'tag': 'ea376188-090c-4aa3-89a4-1a0483a9ce00', 'options': 'recursive,force'} 2024-09-19T04:27:06.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:06 smithi107 ceph-mon[30659]: pgmap v207: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 719 KiB/s wr, 113 op/s 2024-09-19T04:27:06.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 224180 2.53 MB/sec execute 204 sec latency 1790.538 ms 2024-09-19T04:27:06.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:06 smithi115 ceph-mon[35477]: pgmap v207: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 719 KiB/s wr, 113 op/s 2024-09-19T04:27:06.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:06 smithi145 ceph-mon[35096]: pgmap v207: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 719 KiB/s wr, 113 op/s 2024-09-19T04:27:07.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 224180 2.52 MB/sec execute 205 sec latency 2790.618 ms 2024-09-19T04:27:07.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:07 smithi115 ceph-mon[35477]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:27:07.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:07 smithi145 ceph-mon[35096]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:27:07.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:07 smithi107 ceph-mon[30659]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:27:08.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 224180 2.51 MB/sec execute 206 sec latency 3790.740 ms 2024-09-19T04:27:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:08 smithi115 ceph-mon[35477]: pgmap v209: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 899 KiB/s wr, 142 op/s 2024-09-19T04:27:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:08 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:08 smithi145 ceph-mon[35096]: pgmap v209: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 899 KiB/s wr, 142 op/s 2024-09-19T04:27:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:08 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:08.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:08 smithi107 ceph-mon[30659]: pgmap v209: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 899 KiB/s wr, 142 op/s 2024-09-19T04:27:08.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:08 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:09.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 224939 2.52 MB/sec execute 207 sec latency 4328.489 ms 2024-09-19T04:27:10.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 225205 2.51 MB/sec execute 208 sec latency 817.128 ms 2024-09-19T04:27:10.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:10 smithi115 ceph-mon[35477]: pgmap v210: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 497 KiB/s wr, 108 op/s 2024-09-19T04:27:10.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:10 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:10.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:10 smithi145 ceph-mon[35096]: pgmap v210: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 497 KiB/s wr, 108 op/s 2024-09-19T04:27:10.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:10 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:10.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:10 smithi107 ceph-mon[30659]: pgmap v210: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 497 KiB/s wr, 108 op/s 2024-09-19T04:27:10.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:10 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:11.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 225205 2.49 MB/sec execute 209 sec latency 1817.197 ms 2024-09-19T04:27:12.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 225205 2.48 MB/sec execute 210 sec latency 2817.303 ms 2024-09-19T04:27:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:12 smithi115 ceph-mon[35477]: pgmap v211: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 615 KiB/s wr, 128 op/s 2024-09-19T04:27:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:27:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:27:12.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:27:12.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:27:12.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:27:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:12 smithi145 ceph-mon[35096]: pgmap v211: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 615 KiB/s wr, 128 op/s 2024-09-19T04:27:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:27:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:27:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:27:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:27:12.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:27:12.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:12 smithi107 ceph-mon[30659]: pgmap v211: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 615 KiB/s wr, 128 op/s 2024-09-19T04:27:12.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:27:12.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:27:12.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:27:12.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:27:12.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:27:13.681 INFO:tasks.workunit.client.0.smithi107.stdout: 1 225205 2.47 MB/sec execute 211 sec latency 3817.426 ms 2024-09-19T04:27:14.682 INFO:tasks.workunit.client.0.smithi107.stdout: 1 226863 2.50 MB/sec execute 212 sec latency 4090.854 ms 2024-09-19T04:27:14.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:14 smithi115 ceph-mon[35477]: pgmap v212: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 550 KiB/s wr, 114 op/s 2024-09-19T04:27:14.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:14 smithi145 ceph-mon[35096]: pgmap v212: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 550 KiB/s wr, 114 op/s 2024-09-19T04:27:14.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:14 smithi107 ceph-mon[30659]: pgmap v212: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 550 KiB/s wr, 114 op/s 2024-09-19T04:27:15.682 INFO:tasks.workunit.client.0.smithi107.stdout: 1 227658 2.51 MB/sec execute 213 sec latency 259.656 ms 2024-09-19T04:27:16.682 INFO:tasks.workunit.client.0.smithi107.stdout: 1 227778 2.50 MB/sec execute 214 sec latency 867.938 ms 2024-09-19T04:27:16.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:16 smithi115 ceph-mon[35477]: pgmap v213: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 386 KiB/s wr, 70 op/s 2024-09-19T04:27:16.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:16 smithi145 ceph-mon[35096]: pgmap v213: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 386 KiB/s wr, 70 op/s 2024-09-19T04:27:16.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:16 smithi107 ceph-mon[30659]: pgmap v213: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 386 KiB/s wr, 70 op/s 2024-09-19T04:27:17.682 INFO:tasks.workunit.client.0.smithi107.stdout: 1 227778 2.49 MB/sec execute 215 sec latency 1868.059 ms 2024-09-19T04:27:18.682 INFO:tasks.workunit.client.0.smithi107.stdout: 1 227778 2.48 MB/sec execute 216 sec latency 2868.185 ms 2024-09-19T04:27:18.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:18 smithi115 ceph-mon[35477]: pgmap v214: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 361 KiB/s wr, 65 op/s 2024-09-19T04:27:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:18 smithi107 ceph-mon[30659]: pgmap v214: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 361 KiB/s wr, 65 op/s 2024-09-19T04:27:18.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:18 smithi145 ceph-mon[35096]: pgmap v214: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 361 KiB/s wr, 65 op/s 2024-09-19T04:27:19.682 INFO:tasks.workunit.client.0.smithi107.stdout: 1 227846 2.47 MB/sec execute 217 sec latency 3426.140 ms 2024-09-19T04:27:20.682 INFO:tasks.workunit.client.0.smithi107.stdout: 1 227846 2.46 MB/sec execute 218 sec latency 1376.839 ms 2024-09-19T04:27:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:20 smithi115 ceph-mon[35477]: pgmap v215: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 321 KiB/s wr, 58 op/s 2024-09-19T04:27:20.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:20 smithi145 ceph-mon[35096]: pgmap v215: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 321 KiB/s wr, 58 op/s 2024-09-19T04:27:20.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:20 smithi107 ceph-mon[30659]: pgmap v215: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 321 KiB/s wr, 58 op/s 2024-09-19T04:27:21.682 INFO:tasks.workunit.client.0.smithi107.stdout: 1 227846 2.45 MB/sec execute 219 sec latency 2376.945 ms 2024-09-19T04:27:22.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 227846 2.43 MB/sec execute 220 sec latency 3377.085 ms 2024-09-19T04:27:22.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:22 smithi115 ceph-mon[35477]: pgmap v216: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 611 KiB/s wr, 83 op/s 2024-09-19T04:27:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:22 smithi145 ceph-mon[35096]: pgmap v216: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 611 KiB/s wr, 83 op/s 2024-09-19T04:27:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:22 smithi107 ceph-mon[30659]: pgmap v216: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 611 KiB/s wr, 83 op/s 2024-09-19T04:27:23.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 227846 2.42 MB/sec execute 221 sec latency 4377.211 ms 2024-09-19T04:27:24.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 228696 2.43 MB/sec execute 222 sec latency 4911.554 ms 2024-09-19T04:27:24.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:24 smithi115 ceph-mon[35477]: pgmap v217: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 483 KiB/s wr, 60 op/s 2024-09-19T04:27:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:24 smithi145 ceph-mon[35096]: pgmap v217: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 483 KiB/s wr, 60 op/s 2024-09-19T04:27:24.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:24 smithi107 ceph-mon[30659]: pgmap v217: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 483 KiB/s wr, 60 op/s 2024-09-19T04:27:25.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 228748 2.42 MB/sec execute 223 sec latency 954.430 ms 2024-09-19T04:27:26.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 228748 2.41 MB/sec execute 224 sec latency 1954.503 ms 2024-09-19T04:27:26.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:26 smithi115 ceph-mon[35477]: pgmap v218: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 609 KiB/s wr, 79 op/s 2024-09-19T04:27:26.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:26 smithi145 ceph-mon[35096]: pgmap v218: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 609 KiB/s wr, 79 op/s 2024-09-19T04:27:26.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:26 smithi107 ceph-mon[30659]: pgmap v218: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 609 KiB/s wr, 79 op/s 2024-09-19T04:27:27.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 228748 2.40 MB/sec execute 225 sec latency 2954.630 ms 2024-09-19T04:27:28.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 228748 2.39 MB/sec execute 226 sec latency 3954.808 ms 2024-09-19T04:27:28.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:28 smithi115 ceph-mon[35477]: pgmap v219: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 415 KiB/s wr, 44 op/s 2024-09-19T04:27:28.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:28 smithi145 ceph-mon[35096]: pgmap v219: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 415 KiB/s wr, 44 op/s 2024-09-19T04:27:28.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:28 smithi107 ceph-mon[30659]: pgmap v219: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 415 KiB/s wr, 44 op/s 2024-09-19T04:27:29.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 229760 2.40 MB/sec execute 227 sec latency 4492.713 ms 2024-09-19T04:27:30.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 231198 2.43 MB/sec execute 228 sec latency 12.476 ms 2024-09-19T04:27:30.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:30 smithi107 ceph-mon[30659]: pgmap v220: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 415 KiB/s wr, 44 op/s 2024-09-19T04:27:30.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:30 smithi115 ceph-mon[35477]: pgmap v220: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 415 KiB/s wr, 44 op/s 2024-09-19T04:27:30.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:30 smithi145 ceph-mon[35096]: pgmap v220: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 415 KiB/s wr, 44 op/s 2024-09-19T04:27:31.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 231271 2.42 MB/sec execute 229 sec latency 963.196 ms 2024-09-19T04:27:32.683 INFO:tasks.workunit.client.0.smithi107.stdout: 1 231271 2.41 MB/sec execute 230 sec latency 1963.277 ms 2024-09-19T04:27:32.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:32 smithi115 ceph-mon[35477]: pgmap v221: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 647 KiB/s wr, 73 op/s 2024-09-19T04:27:32.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:32 smithi145 ceph-mon[35096]: pgmap v221: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 647 KiB/s wr, 73 op/s 2024-09-19T04:27:32.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:32 smithi107 ceph-mon[30659]: pgmap v221: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 647 KiB/s wr, 73 op/s 2024-09-19T04:27:33.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 231271 2.40 MB/sec execute 231 sec latency 2963.401 ms 2024-09-19T04:27:33.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:33 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:33.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:33 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:33.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:33 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:34.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 232057 2.41 MB/sec execute 232 sec latency 3499.063 ms 2024-09-19T04:27:34.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:34 smithi115 ceph-mon[35477]: pgmap v222: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 358 KiB/s wr, 48 op/s 2024-09-19T04:27:34.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:34 smithi145 ceph-mon[35096]: pgmap v222: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 358 KiB/s wr, 48 op/s 2024-09-19T04:27:34.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:34 smithi107 ceph-mon[30659]: pgmap v222: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 358 KiB/s wr, 48 op/s 2024-09-19T04:27:35.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 232296 2.40 MB/sec execute 233 sec latency 842.908 ms 2024-09-19T04:27:36.252 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:27:36.644 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:36.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 232806 2.39 MB/sec execute 234 sec latency 1636.486 ms 2024-09-19T04:27:36.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:36 smithi115 ceph-mon[35477]: pgmap v223: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 541 KiB/s wr, 80 op/s 2024-09-19T04:27:36.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:36 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:27:36.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:36 smithi107 ceph-mon[30659]: pgmap v223: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 541 KiB/s wr, 80 op/s 2024-09-19T04:27:36.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:36 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:27:36.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:36 smithi145 ceph-mon[35096]: pgmap v223: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 541 KiB/s wr, 80 op/s 2024-09-19T04:27:36.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:36 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:27:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.249+0000 7f95f3e00640 1 -- 172.21.15.107:0/575219701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f41079c0 msgr2=0x7f95f4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.249+0000 7f95f3e00640 1 --2- 172.21.15.107:0/575219701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f41079c0 0x7f95f4107da0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009930 tx=0x7f95e402a440 comp rx=0 tx=0).stop 2024-09-19T04:27:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.249+0000 7f95f3e00640 1 -- 172.21.15.107:0/575219701 shutdown_connections 2024-09-19T04:27:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.249+0000 7f95f3e00640 1 --2- 172.21.15.107:0/575219701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f4102360 0x7f95f410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.249+0000 7f95f3e00640 1 --2- 172.21.15.107:0/575219701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f41019c0 0x7f95f4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.249+0000 7f95f3e00640 1 --2- 172.21.15.107:0/575219701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f41079c0 0x7f95f4107da0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.249+0000 7f95f3e00640 1 -- 172.21.15.107:0/575219701 >> 172.21.15.107:0/575219701 conn(0x7f95f40fd740 msgr2=0x7f95f40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.249+0000 7f95f3e00640 1 -- 172.21.15.107:0/575219701 shutdown_connections 2024-09-19T04:27:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.249+0000 7f95f3e00640 1 -- 172.21.15.107:0/575219701 wait complete. 2024-09-19T04:27:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.250+0000 7f95f3e00640 1 Processor -- start 2024-09-19T04:27:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f3e00640 1 -- start start 2024-09-19T04:27:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f41019c0 0x7f95f419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f4102360 0x7f95f419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f41079c0 0x7f95f419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4111d90 con 0x7f95f4102360 2024-09-19T04:27:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f3e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4111c10 con 0x7f95f41079c0 2024-09-19T04:27:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4111f10 con 0x7f95f41019c0 2024-09-19T04:27:37.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f41079c0 0x7f95f419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f41019c0 0x7f95f419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f4102360 0x7f95f419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f41079c0 0x7f95f419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:33264/0 (socket says 172.21.15.107:33264) 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.251+0000 7f95f3400640 1 -- 172.21.15.107:0/2663351553 learned_addr learned my addr 172.21.15.107:0/2663351553 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.252+0000 7f95f3400640 1 -- 172.21.15.107:0/2663351553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f41019c0 msgr2=0x7f95f419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.252+0000 7f95f3400640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f41019c0 0x7f95f419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.252+0000 7f95f3400640 1 -- 172.21.15.107:0/2663351553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f4102360 msgr2=0x7f95f419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.252+0000 7f95f3400640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f4102360 0x7f95f419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.252+0000 7f95f3400640 1 -- 172.21.15.107:0/2663351553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f419f480 con 0x7f95f41079c0 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.252+0000 7f95f3400640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f41079c0 0x7f95f419eda0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00eb10 tx=0x7f95ec00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.252+0000 7f95eb400640 1 -- 172.21.15.107:0/2663351553 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec004b20 con 0x7f95f41079c0 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.252+0000 7f95f3e00640 1 -- 172.21.15.107:0/2663351553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f419f770 con 0x7f95f41079c0 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.252+0000 7f95eb400640 1 -- 172.21.15.107:0/2663351553 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ec004cc0 con 0x7f95f41079c0 2024-09-19T04:27:37.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.252+0000 7f95eb400640 1 -- 172.21.15.107:0/2663351553 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec010760 con 0x7f95f41079c0 2024-09-19T04:27:37.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.253+0000 7f95f3e00640 1 -- 172.21.15.107:0/2663351553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f41a7050 con 0x7f95f41079c0 2024-09-19T04:27:37.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.256+0000 7f95eb400640 1 -- 172.21.15.107:0/2663351553 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f95ec005020 con 0x7f95f41079c0 2024-09-19T04:27:37.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.257+0000 7f95eb400640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d80a33b0 0x7f95d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.257+0000 7f95eb400640 1 -- 172.21.15.107:0/2663351553 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f95ec0d0250 con 0x7f95f41079c0 2024-09-19T04:27:37.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.258+0000 7f95fbff8740 1 Processor -- start 2024-09-19T04:27:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.258+0000 7f95fbff8740 1 -- start start 2024-09-19T04:27:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.258+0000 7f95f2a00640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d80a33b0 0x7f95d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.258+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55fa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.258+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc560d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.258+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc560fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.258+0000 7f95fbff8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563fc55603e0 con 0x563fc55fa550 2024-09-19T04:27:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.258+0000 7f95fbff8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563fc5507d80 con 0x563fc55f9c30 2024-09-19T04:27:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.258+0000 7f95fbff8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563fc555c3a0 con 0x563fc560d740 2024-09-19T04:27:37.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc560fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc560d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc560fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46290/0 (socket says 172.21.15.107:46290) 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f3400640 1 -- 172.21.15.107:0/4134944973 learned_addr learned my addr 172.21.15.107:0/4134944973 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55fa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:33274/0 (socket says 172.21.15.107:33274) 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f2a00640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d80a33b0 0x7f95d80a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009900 tx=0x7f95e40058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f2000640 1 -- 172.21.15.107:0/4134944973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 msgr2=0x563fc560fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f2000640 1 --2- 172.21.15.107:0/4134944973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc560fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f2000640 1 -- 172.21.15.107:0/4134944973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 msgr2=0x563fc55fa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f2000640 1 --2- 172.21.15.107:0/4134944973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55fa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.259+0000 7f95f2000640 1 -- 172.21.15.107:0/4134944973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc5526ae0 con 0x563fc55fa550 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.260+0000 7f95f3400640 1 --2- 172.21.15.107:0/4134944973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc560fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.260+0000 7f95f2a00640 1 --2- 172.21.15.107:0/4134944973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:37.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.260+0000 7f95f2000640 1 --2- 172.21.15.107:0/4134944973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc560d200 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009bf0 tx=0x7f95e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f858a6f5a713f5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.260+0000 7f95df400640 1 -- 172.21.15.107:0/4134944973 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0009da0 con 0x563fc55fa550 2024-09-19T04:27:37.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.260+0000 7f95df400640 1 -- 172.21.15.107:0/4134944973 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e0005020 con 0x563fc55fa550 2024-09-19T04:27:37.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.260+0000 7f95df400640 1 -- 172.21.15.107:0/4134944973 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0004c60 con 0x563fc55fa550 2024-09-19T04:27:37.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.261+0000 7f95fbff8740 1 -- 172.21.15.107:0/4134944973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 msgr2=0x563fc560d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.261+0000 7f95fbff8740 1 --2- 172.21.15.107:0/4134944973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc560d200 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009bf0 tx=0x7f95e0011810 comp rx=0 tx=0).stop 2024-09-19T04:27:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.261+0000 7f95fbff8740 1 -- 172.21.15.107:0/4134944973 shutdown_connections 2024-09-19T04:27:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.261+0000 7f95fbff8740 1 --2- 172.21.15.107:0/4134944973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc560fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.261+0000 7f95fbff8740 1 --2- 172.21.15.107:0/4134944973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc560d200 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.261+0000 7f95fbff8740 1 --2- 172.21.15.107:0/4134944973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.261+0000 7f95fbff8740 1 -- 172.21.15.107:0/4134944973 >> 172.21.15.107:0/4134944973 conn(0x563fc55f62e0 msgr2=0x563fc55f66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.262+0000 7f95fbff8740 1 -- 172.21.15.107:0/4134944973 shutdown_connections 2024-09-19T04:27:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.262+0000 7f95fbff8740 1 -- 172.21.15.107:0/4134944973 wait complete. 2024-09-19T04:27:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.262+0000 7f95fbff8740 1 -- start start 2024-09-19T04:27:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.262+0000 7f95fbff8740 1 Processor -- start 2024-09-19T04:27:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc560a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc560ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc560b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95fbff8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563fc56103b0 con 0x563fc55fa550 2024-09-19T04:27:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95fbff8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563fc551b0f0 con 0x563fc55f9c30 2024-09-19T04:27:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95fbff8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563fc5610530 con 0x563fc560d740 2024-09-19T04:27:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95fbff8740 10 client.0 _ll_register_callbacks cb 0x563fc550c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95f2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc560a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95f2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc560a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:33284/0 (socket says 172.21.15.107:33284) 2024-09-19T04:27:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95f2a00640 1 -- 172.21.15.107:0/2825190841 learned_addr learned my addr 172.21.15.107:0/2825190841 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.263+0000 7f95f3400640 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc560b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.264+0000 7f95f2000640 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc560ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.264+0000 7f95f2a00640 1 -- 172.21.15.107:0/2825190841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 msgr2=0x563fc560b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.264+0000 7f95f2a00640 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc560b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.264+0000 7f95f2a00640 1 -- 172.21.15.107:0/2825190841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 msgr2=0x563fc560ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.264+0000 7f95f2a00640 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc560ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.264+0000 7f95f2a00640 1 -- 172.21.15.107:0/2825190841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc52d13c0 con 0x563fc55f9c30 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.264+0000 7f95f2000640 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc560ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.265+0000 7f95f2a00640 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc560a620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f95e4006010 tx=0x7f95e4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d92b87cb8d15d293 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.265+0000 7f95d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.265+0000 7f95d3e00640 1 -- 172.21.15.107:0/2825190841 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4035040 con 0x563fc55f9c30 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.265+0000 7f95fbff8740 1 -- 172.21.15.107:0/2825190841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e0005020 con 0x563fc55f9c30 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.265+0000 7f95d3e00640 1 -- 172.21.15.107:0/2825190841 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f95e4003a00 con 0x563fc55f9c30 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.265+0000 7f95d3e00640 1 -- 172.21.15.107:0/2825190841 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f95e4003c00 con 0x563fc55f9c30 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.265+0000 7f95fbff8740 10 client.26507 fetch_fsmap learned FSMap version 42 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.265+0000 7f95fbff8740 1 -- 172.21.15.107:0/2825190841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563fc561c0a0 con 0x563fc55f9c30 2024-09-19T04:27:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.265+0000 7f95d3e00640 1 -- 172.21.15.107:0/2825190841 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e0005020 con 0x563fc55f9c30 2024-09-19T04:27:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.266+0000 7f95d3e00640 1 -- 172.21.15.107:0/2825190841 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7f95e404a430 con 0x563fc55f9c30 2024-09-19T04:27:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.266+0000 7f95fbff8740 10 client.26507 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:27:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.266+0000 7f95fbff8740 10 client.26507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.266+0000 7f95fbff8740 20 client.26507 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.266+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563fc561d5d0 0x563fc561fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.266+0000 7f95fbff8740 4 client.26507 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.266+0000 7f95fbff8740 1 -- 172.21.15.107:0/2825190841 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc556d950 con 0x563fc561d5d0 2024-09-19T04:27:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.266+0000 7f95f2000640 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563fc561d5d0 0x563fc561fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.267+0000 7f95f2000640 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563fc561d5d0 0x563fc561fa70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc5e084d318bdad server_cookie=685949fe5aae10b3 in_seq=0 out_seq=0 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.267+0000 7f95d3e00640 10 client.26507 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.267+0000 7f95d3e00640 1 -- 172.21.15.107:0/2825190841 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fc556d950 con 0x563fc561d5d0 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.267+0000 7f95d3e00640 10 client.26507 handle_command_reply: tid=0 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 client.26507 shutdown 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 -- 172.21.15.107:0/2825190841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 msgr2=0x563fc560a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc560a620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f95e4006010 tx=0x7f95e4009260 comp rx=0 tx=0).stop 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 -- 172.21.15.107:0/2825190841 shutdown_connections 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563fc561d5d0 0x563fc561fa70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc560b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc560ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2825190841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc560a620 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95d3e00640 0 client.26507 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 -- 172.21.15.107:0/2825190841 >> 172.21.15.107:0/2825190841 conn(0x563fc55f62e0 msgr2=0x563fc55fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 -- 172.21.15.107:0/2825190841 shutdown_connections 2024-09-19T04:27:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 1 -- 172.21.15.107:0/2825190841 wait complete. 2024-09-19T04:27:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.268+0000 7f95fbff8740 20 client.26507 trim_cache size 0 max 16384 2024-09-19T04:27:37.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.287+0000 7f95fbff8740 1 Processor -- start 2024-09-19T04:27:37.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.287+0000 7f95fbff8740 1 -- start start 2024-09-19T04:27:37.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc55f5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc56253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95fbff8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41ab510 con 0x563fc55fa550 2024-09-19T04:27:37.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95fbff8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41ad140 con 0x563fc55f9c30 2024-09-19T04:27:37.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95fbff8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41abc10 con 0x563fc560d740 2024-09-19T04:27:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95f2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95f2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:33288/0 (socket says 172.21.15.107:33288) 2024-09-19T04:27:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95f2a00640 1 -- 172.21.15.107:0/306295507 learned_addr learned my addr 172.21.15.107:0/306295507 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95f3400640 1 --2- 172.21.15.107:0/306295507 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc56253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95f2000640 1 --2- 172.21.15.107:0/306295507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc55f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95f2a00640 1 -- 172.21.15.107:0/306295507 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 msgr2=0x563fc56253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95f2a00640 1 --2- 172.21.15.107:0/306295507 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc56253d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95f2a00640 1 -- 172.21.15.107:0/306295507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 msgr2=0x563fc55f5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95f2a00640 1 --2- 172.21.15.107:0/306295507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc55f5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.288+0000 7f95f2a00640 1 -- 172.21.15.107:0/306295507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc55f6110 con 0x563fc55f9c30 2024-09-19T04:27:37.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.289+0000 7f95f2a00640 1 --2- 172.21.15.107:0/306295507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55f1060 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f95e404a820 tx=0x7f95e4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b845db088349742a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.289+0000 7f95dea00640 1 -- 172.21.15.107:0/306295507 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4060030 con 0x563fc55f9c30 2024-09-19T04:27:37.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.289+0000 7f95dea00640 1 -- 172.21.15.107:0/306295507 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e4062070 con 0x563fc55f9c30 2024-09-19T04:27:37.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.289+0000 7f95dea00640 1 -- 172.21.15.107:0/306295507 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4035050 con 0x563fc55f9c30 2024-09-19T04:27:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.290+0000 7f95fbff8740 1 -- 172.21.15.107:0/306295507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 msgr2=0x563fc55f1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.290+0000 7f95fbff8740 1 --2- 172.21.15.107:0/306295507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55f1060 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f95e404a820 tx=0x7f95e4000bc0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.290+0000 7f95fbff8740 1 -- 172.21.15.107:0/306295507 shutdown_connections 2024-09-19T04:27:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.290+0000 7f95fbff8740 1 --2- 172.21.15.107:0/306295507 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc560d740 0x563fc56253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.290+0000 7f95fbff8740 1 --2- 172.21.15.107:0/306295507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc55f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.290+0000 7f95fbff8740 1 --2- 172.21.15.107:0/306295507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc55f9c30 0x563fc55f1060 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.290+0000 7f95fbff8740 1 -- 172.21.15.107:0/306295507 >> 172.21.15.107:0/306295507 conn(0x563fc55f62e0 msgr2=0x563fc55f4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.290+0000 7f95fbff8740 1 -- 172.21.15.107:0/306295507 shutdown_connections 2024-09-19T04:27:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.290+0000 7f95fbff8740 1 -- 172.21.15.107:0/306295507 wait complete. 2024-09-19T04:27:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.291+0000 7f95fbff8740 1 -- start start 2024-09-19T04:27:37.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.291+0000 7f95fbff8740 1 Processor -- start 2024-09-19T04:27:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.291+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc55f9c30 0x563fc56160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.291+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc56113d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.291+0000 7f95fbff8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc560d740 0x563fc5611b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95fbff8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41ac9b0 con 0x563fc55fa550 2024-09-19T04:27:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95fbff8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41ac2e0 con 0x563fc560d740 2024-09-19T04:27:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95fbff8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41a9590 con 0x563fc55f9c30 2024-09-19T04:27:37.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95fbff8740 10 client.0 _ll_register_callbacks cb 0x7f95e4031700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc55f9c30 0x563fc56160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc55f9c30 0x563fc56160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46324/0 (socket says 172.21.15.107:46324) 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f2a00640 1 -- 172.21.15.107:0/2473117128 learned_addr learned my addr 172.21.15.107:0/2473117128 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f3400640 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc560d740 0x563fc5611b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f2000640 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc56113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f2a00640 1 -- 172.21.15.107:0/2473117128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc560d740 msgr2=0x563fc5611b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f2a00640 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc560d740 0x563fc5611b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f2a00640 1 -- 172.21.15.107:0/2473117128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 msgr2=0x563fc56113d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f2a00640 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc56113d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f2a00640 1 -- 172.21.15.107:0/2473117128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc56873a0 con 0x563fc55f9c30 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95f2a00640 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc55f9c30 0x563fc56160a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f95e4004770 tx=0x7f95e4009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bbfece73bb75cb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.292+0000 7f95d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:27:37.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.293+0000 7f95d3e00640 1 -- 172.21.15.107:0/2473117128 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4060040 con 0x563fc55f9c30 2024-09-19T04:27:37.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.293+0000 7f95d3e00640 1 -- 172.21.15.107:0/2473117128 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f95e4003800 con 0x563fc55f9c30 2024-09-19T04:27:37.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.293+0000 7f95fbff8740 1 -- 172.21.15.107:0/2473117128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e4062070 con 0x563fc55f9c30 2024-09-19T04:27:37.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.293+0000 7f95d3e00640 1 -- 172.21.15.107:0/2473117128 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f95e4035040 con 0x563fc55f9c30 2024-09-19T04:27:37.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.293+0000 7f95fbff8740 10 client.26479 fetch_fsmap learned FSMap version 42 2024-09-19T04:27:37.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.293+0000 7f95fbff8740 1 -- 172.21.15.107:0/2473117128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563fc5688550 con 0x563fc55f9c30 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.294+0000 7f95d3e00640 1 -- 172.21.15.107:0/2473117128 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e4062070 con 0x563fc55f9c30 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.294+0000 7f95d3e00640 1 -- 172.21.15.107:0/2473117128 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7f95e4076020 con 0x563fc55f9c30 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.294+0000 7f95fbff8740 10 client.26479 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.294+0000 7f95fbff8740 10 client.26479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.295+0000 7f95fbff8740 20 client.26479 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.295+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563fc5688bb0 0x563fc568afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.295+0000 7f95fbff8740 4 client.26479 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.295+0000 7f95fbff8740 1 -- 172.21.15.107:0/2473117128 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fc5567e60 con 0x563fc5688bb0 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.295+0000 7f95f2000640 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563fc5688bb0 0x563fc568afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95f2000640 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563fc5688bb0 0x563fc568afd0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c49c4830ce28798 server_cookie=d69c447780385dab in_seq=0 out_seq=0 2024-09-19T04:27:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95d3e00640 10 client.26479 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95d3e00640 1 -- 172.21.15.107:0/2473117128 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563fc5567e60 con 0x563fc5688bb0 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95d3e00640 10 client.26479 handle_command_reply: tid=0 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95fbff8740 1 client.26479 shutdown 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95fbff8740 1 -- 172.21.15.107:0/2473117128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc55f9c30 msgr2=0x563fc56160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc55f9c30 0x563fc56160a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f95e4004770 tx=0x7f95e4009880 comp rx=0 tx=0).stop 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95fbff8740 1 -- 172.21.15.107:0/2473117128 shutdown_connections 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563fc5688bb0 0x563fc568afd0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc560d740 0x563fc5611b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc55fa550 0x563fc56113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95fbff8740 1 --2- 172.21.15.107:0/2473117128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563fc55f9c30 0x563fc56160a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95fbff8740 1 -- 172.21.15.107:0/2473117128 >> 172.21.15.107:0/2473117128 conn(0x563fc55f62e0 msgr2=0x563fc55fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.296+0000 7f95d3e00640 0 client.26479 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:37.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.297+0000 7f95fbff8740 1 -- 172.21.15.107:0/2473117128 shutdown_connections 2024-09-19T04:27:37.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.297+0000 7f95fbff8740 1 -- 172.21.15.107:0/2473117128 wait complete. 2024-09-19T04:27:37.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.297+0000 7f95fbff8740 20 client.26479 trim_cache size 0 max 16384 2024-09-19T04:27:37.300 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:27:37.300 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:27:37.300 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:27:37.300 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:27:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 -- 172.21.15.107:0/2663351553 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d80a33b0 msgr2=0x7f95d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d80a33b0 0x7f95d80a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009900 tx=0x7f95e40058d0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 -- 172.21.15.107:0/2663351553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f41079c0 msgr2=0x7f95f419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f41079c0 0x7f95f419eda0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00eb10 tx=0x7f95ec00c2d0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 -- 172.21.15.107:0/2663351553 shutdown_connections 2024-09-19T04:27:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d80a33b0 0x7f95d80a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f41079c0 0x7f95f419eda0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f4102360 0x7f95f419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 --2- 172.21.15.107:0/2663351553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f41019c0 0x7f95f419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:37.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 -- 172.21.15.107:0/2663351553 >> 172.21.15.107:0/2663351553 conn(0x7f95f40fd740 msgr2=0x7f95f410cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:37.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 -- 172.21.15.107:0/2663351553 shutdown_connections 2024-09-19T04:27:37.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:37.298+0000 7f95f3e00640 1 -- 172.21.15.107:0/2663351553 wait complete. 2024-09-19T04:27:37.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 234462 2.44 MB/sec execute 235 sec latency 15.112 ms 2024-09-19T04:27:37.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:37 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:27:37.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:37 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:27:37.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:37 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:27:37.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:37 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:27:37.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:37 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:27:37.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:37 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:27:37.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:37 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:27:37.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:37 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:27:37.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:37 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:27:38.186 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:27:38.187 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:27:38.557 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:38.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 234870 2.43 MB/sec execute 236 sec latency 558.040 ms 2024-09-19T04:27:38.911 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:38 smithi107 ceph-mon[30659]: pgmap v224: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 415 KiB/s wr, 61 op/s 2024-09-19T04:27:38.911 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:38 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:38.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:38 smithi115 ceph-mon[35477]: pgmap v224: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 415 KiB/s wr, 61 op/s 2024-09-19T04:27:38.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:38 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:38.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:38 smithi145 ceph-mon[35096]: pgmap v224: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 415 KiB/s wr, 61 op/s 2024-09-19T04:27:38.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:38 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.280+0000 7fa927e00640 1 -- 172.21.15.107:0/1738925595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9281047e0 msgr2=0x7fa928108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.280+0000 7fa927e00640 1 --2- 172.21.15.107:0/1738925595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9281047e0 0x7fa928108830 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa910009900 tx=0x7fa91002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:27:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.280+0000 7fa927e00640 1 -- 172.21.15.107:0/1738925595 shutdown_connections 2024-09-19T04:27:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.280+0000 7fa927e00640 1 --2- 172.21.15.107:0/1738925595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa928108f60 0x7fa928110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.280+0000 7fa927e00640 1 --2- 172.21.15.107:0/1738925595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9281047e0 0x7fa928108830 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.280+0000 7fa927e00640 1 --2- 172.21.15.107:0/1738925595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa928103e30 0x7fa928104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.280+0000 7fa927e00640 1 -- 172.21.15.107:0/1738925595 >> 172.21.15.107:0/1738925595 conn(0x7fa9280ff8d0 msgr2=0x7fa928101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.280+0000 7fa927e00640 1 -- 172.21.15.107:0/1738925595 shutdown_connections 2024-09-19T04:27:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.280+0000 7fa927e00640 1 -- 172.21.15.107:0/1738925595 wait complete. 2024-09-19T04:27:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.281+0000 7fa927e00640 1 Processor -- start 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.281+0000 7fa927e00640 1 -- start start 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.281+0000 7fa927e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa928103e30 0x7fa92819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.281+0000 7fa927e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9281047e0 0x7fa92819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.281+0000 7fa927e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa928108f60 0x7fa92819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.281+0000 7fa927e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa928113fd0 con 0x7fa928103e30 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa927e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa928113e50 con 0x7fa928108f60 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa927e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa928114150 con 0x7fa9281047e0 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa926a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa928103e30 0x7fa92819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa91e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9281047e0 0x7fa92819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa926a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa928103e30 0x7fa92819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36904/0 (socket says 172.21.15.107:36904) 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa926a00640 1 -- 172.21.15.107:0/3683280014 learned_addr learned my addr 172.21.15.107:0/3683280014 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa927400640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa928108f60 0x7fa92819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa91e000640 1 -- 172.21.15.107:0/3683280014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa928108f60 msgr2=0x7fa92819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa91e000640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa928108f60 0x7fa92819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa91e000640 1 -- 172.21.15.107:0/3683280014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa928103e30 msgr2=0x7fa92819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa91e000640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa928103e30 0x7fa92819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa91e000640 1 -- 172.21.15.107:0/3683280014 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92819f490 con 0x7fa9281047e0 2024-09-19T04:27:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.282+0000 7fa926a00640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa928103e30 0x7fa92819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.283+0000 7fa91e000640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9281047e0 0x7fa92819aa20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa91002a910 tx=0x7fa910003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.283+0000 7fa91fe00640 1 -- 172.21.15.107:0/3683280014 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91003f070 con 0x7fa9281047e0 2024-09-19T04:27:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.283+0000 7fa91fe00640 1 -- 172.21.15.107:0/3683280014 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa910031c90 con 0x7fa9281047e0 2024-09-19T04:27:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.283+0000 7fa927e00640 1 -- 172.21.15.107:0/3683280014 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92819f720 con 0x7fa9281047e0 2024-09-19T04:27:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.283+0000 7fa927e00640 1 -- 172.21.15.107:0/3683280014 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9281a7060 con 0x7fa9281047e0 2024-09-19T04:27:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.283+0000 7fa91fe00640 1 -- 172.21.15.107:0/3683280014 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910035030 con 0x7fa9281047e0 2024-09-19T04:27:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.284+0000 7fa927e00640 1 -- 172.21.15.107:0/3683280014 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa904005290 con 0x7fa9281047e0 2024-09-19T04:27:39.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.287+0000 7fa91fe00640 1 -- 172.21.15.107:0/3683280014 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fa910042070 con 0x7fa9281047e0 2024-09-19T04:27:39.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.288+0000 7fa91fe00640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa90c0a3430 0x7fa90c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:39.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.288+0000 7fa91fe00640 1 -- 172.21.15.107:0/3683280014 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7fa9100f0420 con 0x7fa9281047e0 2024-09-19T04:27:39.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.288+0000 7fa926a00640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa90c0a3430 0x7fa90c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:39.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.289+0000 7fa926a00640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa90c0a3430 0x7fa90c0a58f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa91800e810 tx=0x7fa918009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:39.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.292+0000 7fa91fe00640 1 -- 172.21.15.107:0/3683280014 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa9100bcdc0 con 0x7fa9281047e0 2024-09-19T04:27:39.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.531+0000 7fa927e00640 1 -- 172.21.15.107:0/3683280014 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa904005850 con 0x7fa9281047e0 2024-09-19T04:27:39.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.532+0000 7fa91fe00640 1 -- 172.21.15.107:0/3683280014 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+12001 (secure 0 0 0) 0x7fa9100bc6c0 con 0x7fa9281047e0 2024-09-19T04:27:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:27:39.534 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":43,"btime":"2024-09-19T04:27:37:653937+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:27:36.706357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:27:39.535 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 43 2024-09-19T04:27:39.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.534+0000 7fa927e00640 1 -- 172.21.15.107:0/3683280014 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa90c0a3430 msgr2=0x7fa90c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:39.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.534+0000 7fa927e00640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa90c0a3430 0x7fa90c0a58f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa91800e810 tx=0x7fa918009500 comp rx=0 tx=0).stop 2024-09-19T04:27:39.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.534+0000 7fa927e00640 1 -- 172.21.15.107:0/3683280014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9281047e0 msgr2=0x7fa92819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:39.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.534+0000 7fa927e00640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9281047e0 0x7fa92819aa20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa91002a910 tx=0x7fa910003b20 comp rx=0 tx=0).stop 2024-09-19T04:27:39.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.535+0000 7fa927e00640 1 -- 172.21.15.107:0/3683280014 shutdown_connections 2024-09-19T04:27:39.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.535+0000 7fa927e00640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa90c0a3430 0x7fa90c0a58f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:39.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.535+0000 7fa927e00640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa928108f60 0x7fa92819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:39.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.535+0000 7fa927e00640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9281047e0 0x7fa92819aa20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:39.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.535+0000 7fa927e00640 1 --2- 172.21.15.107:0/3683280014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa928103e30 0x7fa92819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:39.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.535+0000 7fa927e00640 1 -- 172.21.15.107:0/3683280014 >> 172.21.15.107:0/3683280014 conn(0x7fa9280ff8d0 msgr2=0x7fa928100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:39.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.535+0000 7fa927e00640 1 -- 172.21.15.107:0/3683280014 shutdown_connections 2024-09-19T04:27:39.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:39.535+0000 7fa927e00640 1 -- 172.21.15.107:0/3683280014 wait complete. 2024-09-19T04:27:39.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 235666 2.44 MB/sec execute 237 sec latency 1093.057 ms 2024-09-19T04:27:39.917 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:39 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3683280014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:27:39.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:39 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3683280014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:27:39.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:39 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3683280014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:27:40.316 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:27:40.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 235839 2.43 MB/sec execute 238 sec latency 887.251 ms 2024-09-19T04:27:40.687 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:40.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:40 smithi115 ceph-mon[35477]: pgmap v225: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 415 KiB/s wr, 61 op/s 2024-09-19T04:27:40.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:40 smithi145 ceph-mon[35096]: pgmap v225: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 415 KiB/s wr, 61 op/s 2024-09-19T04:27:40.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:40 smithi107 ceph-mon[30659]: pgmap v225: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 415 KiB/s wr, 61 op/s 2024-09-19T04:27:41.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.321+0000 7f75c2200640 1 -- 172.21.15.107:0/3515959383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75bc106f10 msgr2=0x7f75bc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.321+0000 7f75c2200640 1 --2- 172.21.15.107:0/3515959383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75bc106f10 0x7f75bc10ea90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009ca0 tx=0x7f75b402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.322+0000 7f75c2200640 1 -- 172.21.15.107:0/3515959383 shutdown_connections 2024-09-19T04:27:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.322+0000 7f75c2200640 1 --2- 172.21.15.107:0/3515959383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75bc106f10 0x7f75bc10ea90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.322+0000 7f75c2200640 1 --2- 172.21.15.107:0/3515959383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75bc102790 0x7f75bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.322+0000 7f75c2200640 1 --2- 172.21.15.107:0/3515959383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75bc101de0 0x7f75bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.322+0000 7f75c2200640 1 -- 172.21.15.107:0/3515959383 >> 172.21.15.107:0/3515959383 conn(0x7f75bc0fd8c0 msgr2=0x7f75bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.322+0000 7f75c2200640 1 -- 172.21.15.107:0/3515959383 shutdown_connections 2024-09-19T04:27:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.322+0000 7f75c2200640 1 -- 172.21.15.107:0/3515959383 wait complete. 2024-09-19T04:27:41.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.323+0000 7f75c2200640 1 Processor -- start 2024-09-19T04:27:41.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.323+0000 7f75c2200640 1 -- start start 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.323+0000 7f75c2200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75bc101de0 0x7f75bc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.323+0000 7f75c2200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75bc102790 0x7f75bc19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.323+0000 7f75c2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75bc106f10 0x7f75bc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.323+0000 7f75c2200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc111f70 con 0x7f75bc106f10 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.323+0000 7f75c2200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc111df0 con 0x7f75bc101de0 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.323+0000 7f75c2200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1120f0 con 0x7f75bc102790 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.323+0000 7f75bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75bc106f10 0x7f75bc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.323+0000 7f75bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75bc106f10 0x7f75bc19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36918/0 (socket says 172.21.15.107:36918) 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.324+0000 7f75bbe00640 1 -- 172.21.15.107:0/845988175 learned_addr learned my addr 172.21.15.107:0/845988175 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.324+0000 7f75baa00640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75bc102790 0x7f75bc19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.324+0000 7f75bb400640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75bc101de0 0x7f75bc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.324+0000 7f75bbe00640 1 -- 172.21.15.107:0/845988175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75bc102790 msgr2=0x7f75bc19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:41.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.324+0000 7f75bbe00640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75bc102790 0x7f75bc19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.324+0000 7f75bbe00640 1 -- 172.21.15.107:0/845988175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75bc101de0 msgr2=0x7f75bc19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:41.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.324+0000 7f75bbe00640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75bc101de0 0x7f75bc19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.324+0000 7f75bbe00640 1 -- 172.21.15.107:0/845988175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc19f5f0 con 0x7f75bc106f10 2024-09-19T04:27:41.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.324+0000 7f75bbe00640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75bc106f10 0x7f75bc19ef10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f75b4031a20 tx=0x7f75b4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:41.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.325+0000 7f75b3e00640 1 -- 172.21.15.107:0/845988175 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4003c30 con 0x7f75bc106f10 2024-09-19T04:27:41.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.325+0000 7f75b3e00640 1 -- 172.21.15.107:0/845988175 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b4003dd0 con 0x7f75bc106f10 2024-09-19T04:27:41.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.325+0000 7f75c2200640 1 -- 172.21.15.107:0/845988175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75bc19f880 con 0x7f75bc106f10 2024-09-19T04:27:41.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.325+0000 7f75b3e00640 1 -- 172.21.15.107:0/845988175 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b402fc40 con 0x7f75bc106f10 2024-09-19T04:27:41.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.325+0000 7f75c2200640 1 -- 172.21.15.107:0/845988175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75bc1a71c0 con 0x7f75bc106f10 2024-09-19T04:27:41.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.328+0000 7f75b3e00640 1 -- 172.21.15.107:0/845988175 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f75b4004050 con 0x7f75bc106f10 2024-09-19T04:27:41.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.328+0000 7f75c4e7c740 1 Processor -- start 2024-09-19T04:27:41.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.328+0000 7f75c4e7c740 1 -- start start 2024-09-19T04:27:41.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.328+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 0x56170fff6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710000910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 0x561710003240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56170ff5c3e0 con 0x56170fff5c70 2024-09-19T04:27:41.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56170ff03d80 con 0x561710000e50 2024-09-19T04:27:41.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56170ff583a0 con 0x56170fff6590 2024-09-19T04:27:41.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 0x56170fff6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75bb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710000910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75bb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710000910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:60636/0 (socket says 172.21.15.107:60636) 2024-09-19T04:27:41.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75bb400640 1 -- 172.21.15.107:0/4123224803 learned_addr learned my addr 172.21.15.107:0/4123224803 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75baa00640 1 --2- 172.21.15.107:0/4123224803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 0x561710003240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75bb400640 1 -- 172.21.15.107:0/4123224803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 msgr2=0x561710003240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75bb400640 1 --2- 172.21.15.107:0/4123224803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 0x561710003240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75bb400640 1 -- 172.21.15.107:0/4123224803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 msgr2=0x56170fff6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75bb400640 1 --2- 172.21.15.107:0/4123224803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 0x56170fff6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75bb400640 1 -- 172.21.15.107:0/4123224803 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56170ff22ae0 con 0x56170fff6590 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.329+0000 7f75baa00640 1 --2- 172.21.15.107:0/4123224803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 0x561710003240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.330+0000 7f75b3e00640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75a00a3430 0x7f75a00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.330+0000 7f75b3e00640 1 -- 172.21.15.107:0/845988175 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f75b40f0d40 con 0x7f75bc106f10 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.330+0000 7f75bb400640 1 --2- 172.21.15.107:0/4123224803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710000910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009cf0 tx=0x7f75ac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fbb7e16bdb9b0db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:41.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.330+0000 7f75baa00640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75a00a3430 0x7f75a00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.330+0000 7f75a7e00640 1 -- 172.21.15.107:0/4123224803 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac004230 con 0x56170fff6590 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.330+0000 7f75a7e00640 1 -- 172.21.15.107:0/4123224803 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ac0043d0 con 0x56170fff6590 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.330+0000 7f75a7e00640 1 -- 172.21.15.107:0/4123224803 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac019960 con 0x56170fff6590 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.331+0000 7f75c4e7c740 1 -- 172.21.15.107:0/4123224803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 msgr2=0x561710000910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.331+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/4123224803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710000910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009cf0 tx=0x7f75ac011810 comp rx=0 tx=0).stop 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.331+0000 7f75baa00640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75a00a3430 0x7f75a00a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f75a80058a0 tx=0x7f75a800b740 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.331+0000 7f75c4e7c740 1 -- 172.21.15.107:0/4123224803 shutdown_connections 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.331+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/4123224803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 0x561710003240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.331+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/4123224803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710000910 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.331+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/4123224803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 0x56170fff6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.331+0000 7f75c4e7c740 1 -- 172.21.15.107:0/4123224803 >> 172.21.15.107:0/4123224803 conn(0x56170fff2200 msgr2=0x56170fff25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.331+0000 7f75c4e7c740 1 -- 172.21.15.107:0/4123224803 shutdown_connections 2024-09-19T04:27:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.331+0000 7f75c4e7c740 1 -- 172.21.15.107:0/4123224803 wait complete. 2024-09-19T04:27:41.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.332+0000 7f75c4e7c740 1 -- start start 2024-09-19T04:27:41.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.332+0000 7f75c4e7c740 1 Processor -- start 2024-09-19T04:27:41.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.332+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fffdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.332+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 0x56170fffe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.332+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 0x56170fffe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.332+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561710003a20 con 0x56170fff6590 2024-09-19T04:27:41.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.332+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56170ff170f0 con 0x56170fff5c70 2024-09-19T04:27:41.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.332+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561710003ba0 con 0x561710000e50 2024-09-19T04:27:41.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.332+0000 7f75c4e7c740 10 client.0 _ll_register_callbacks cb 0x56170fefebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:41.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 0x56170fffe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75bb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fffdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75bbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 0x56170fffe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75bb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fffdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39376/0 (socket says 172.21.15.107:39376) 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75bb400640 1 -- 172.21.15.107:0/3571707814 learned_addr learned my addr 172.21.15.107:0/3571707814 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75baa00640 1 -- 172.21.15.107:0/3571707814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 msgr2=0x56170fffe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75baa00640 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 0x56170fffe7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75baa00640 1 -- 172.21.15.107:0/3571707814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 msgr2=0x56170fffdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75baa00640 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fffdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75baa00640 1 -- 172.21.15.107:0/3571707814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56170fccd3c0 con 0x56170fff6590 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75bb400640 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fffdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75bbe00640 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 0x56170fffe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75baa00640 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 0x56170fffe270 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f75a8003e50 tx=0x7f75a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a67f54b54b0e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.333+0000 7f75a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.334+0000 7f75a4c00640 1 -- 172.21.15.107:0/3571707814 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a801c880 con 0x56170fff6590 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.334+0000 7f75c4e7c740 1 -- 172.21.15.107:0/3571707814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75ac0043d0 con 0x56170fff6590 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.334+0000 7f75a4c00640 1 -- 172.21.15.107:0/3571707814 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f75a801ca20 con 0x56170fff6590 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.334+0000 7f75a4c00640 1 -- 172.21.15.107:0/3571707814 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f75a801cc20 con 0x56170fff6590 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.334+0000 7f75c4e7c740 10 client.16479 fetch_fsmap learned FSMap version 43 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.334+0000 7f75c4e7c740 1 -- 172.21.15.107:0/3571707814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56171000fa80 con 0x56170fff6590 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.334+0000 7f75a4c00640 1 -- 172.21.15.107:0/3571707814 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ac0043d0 con 0x56170fff6590 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.334+0000 7f75a4c00640 1 -- 172.21.15.107:0/3571707814 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f75a8025070 con 0x56170fff6590 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.335+0000 7f75c4e7c740 10 client.16479 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.335+0000 7f75c4e7c740 10 client.16479 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.335+0000 7f75c4e7c740 20 client.16479 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.335+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x561710010f50 0x561710013380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.335+0000 7f75c4e7c740 4 client.16479 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:41.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.335+0000 7f75c4e7c740 1 -- 172.21.15.107:0/3571707814 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56170ff69950 con 0x561710010f50 2024-09-19T04:27:41.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.335+0000 7f75bb400640 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x561710010f50 0x561710013380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.336+0000 7f75bb400640 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x561710010f50 0x561710013380 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3ce8ccdfbc229a87 server_cookie=beb4881101d2d8b8 in_seq=0 out_seq=0 2024-09-19T04:27:41.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.336+0000 7f75a4c00640 10 client.16479 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:27:41.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.336+0000 7f75a4c00640 1 -- 172.21.15.107:0/3571707814 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56170ff69950 con 0x561710010f50 2024-09-19T04:27:41.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.336+0000 7f75a4c00640 10 client.16479 handle_command_reply: tid=0 2024-09-19T04:27:41.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.336+0000 7f75c4e7c740 1 client.16479 shutdown 2024-09-19T04:27:41.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 1 -- 172.21.15.107:0/3571707814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 msgr2=0x56170fffe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:41.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 0x56170fffe270 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f75a8003e50 tx=0x7f75a8004730 comp rx=0 tx=0).stop 2024-09-19T04:27:41.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 1 -- 172.21.15.107:0/3571707814 shutdown_connections 2024-09-19T04:27:41.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x561710010f50 0x561710013380 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 0x56170fffe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 0x56170fffe270 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/3571707814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fffdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75a4c00640 0 client.16479 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:27:41.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 1 -- 172.21.15.107:0/3571707814 >> 172.21.15.107:0/3571707814 conn(0x56170fff2200 msgr2=0x56170fff7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:41.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 1 -- 172.21.15.107:0/3571707814 shutdown_connections 2024-09-19T04:27:41.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 1 -- 172.21.15.107:0/3571707814 wait complete. 2024-09-19T04:27:41.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.337+0000 7f75c4e7c740 20 client.16479 trim_cache size 0 max 16384 2024-09-19T04:27:41.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.350+0000 7f75c4e7c740 1 Processor -- start 2024-09-19T04:27:41.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.350+0000 7f75c4e7c740 1 -- start start 2024-09-19T04:27:41.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fff86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 0x561710018c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 0x5617100191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1afab0 con 0x56170fff6590 2024-09-19T04:27:41.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1b16b0 con 0x56170fff5c70 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1b01b0 con 0x561710000e50 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 0x561710018c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75bb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fff86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 0x561710018c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36942/0 (socket says 172.21.15.107:36942) 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75baa00640 1 -- 172.21.15.107:0/2159323368 learned_addr learned my addr 172.21.15.107:0/2159323368 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75bbe00640 1 --2- 172.21.15.107:0/2159323368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 0x5617100191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75bb400640 1 -- 172.21.15.107:0/2159323368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 msgr2=0x5617100191d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75bb400640 1 --2- 172.21.15.107:0/2159323368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 0x5617100191d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75bb400640 1 -- 172.21.15.107:0/2159323368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 msgr2=0x561710018c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75bb400640 1 --2- 172.21.15.107:0/2159323368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 0x561710018c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.351+0000 7f75bb400640 1 -- 172.21.15.107:0/2159323368 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56170ffed770 con 0x56170fff5c70 2024-09-19T04:27:41.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.352+0000 7f75bb400640 1 --2- 172.21.15.107:0/2159323368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fff86d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f75ac019500 tx=0x7f75ac019560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62cd972243c5c36b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:41.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.352+0000 7f75a7400640 1 -- 172.21.15.107:0/2159323368 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac005040 con 0x56170fff5c70 2024-09-19T04:27:41.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.352+0000 7f75a7400640 1 -- 172.21.15.107:0/2159323368 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ac0035f0 con 0x56170fff5c70 2024-09-19T04:27:41.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.352+0000 7f75a7400640 1 -- 172.21.15.107:0/2159323368 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac019810 con 0x56170fff5c70 2024-09-19T04:27:41.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 -- 172.21.15.107:0/2159323368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 msgr2=0x56170fff86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:41.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/2159323368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fff86d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f75ac019500 tx=0x7f75ac019560 comp rx=0 tx=0).stop 2024-09-19T04:27:41.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 -- 172.21.15.107:0/2159323368 shutdown_connections 2024-09-19T04:27:41.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/2159323368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561710000e50 0x5617100191d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/2159323368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff6590 0x561710018c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/2159323368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170fff5c70 0x56170fff86d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 -- 172.21.15.107:0/2159323368 >> 172.21.15.107:0/2159323368 conn(0x56170fff2200 msgr2=0x56170fffce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 -- 172.21.15.107:0/2159323368 shutdown_connections 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 -- 172.21.15.107:0/2159323368 wait complete. 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 -- start start 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.353+0000 7f75c4e7c740 1 Processor -- start 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 0x561710004350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75c4e7c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 0x561710082460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1b0f50 con 0x56170fff5c70 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1b0880 con 0x561710000e50 2024-09-19T04:27:41.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75c4e7c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1adab0 con 0x56170fff6590 2024-09-19T04:27:41.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75c4e7c740 10 client.0 _ll_register_callbacks cb 0x7f75a800bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:41.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 0x561710004350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75bbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 0x561710082460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 0x561710004350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36946/0 (socket says 172.21.15.107:36946) 2024-09-19T04:27:41.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75bb400640 1 -- 172.21.15.107:0/1481105852 learned_addr learned my addr 172.21.15.107:0/1481105852 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:41.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.354+0000 7f75baa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75baa00640 1 -- 172.21.15.107:0/1481105852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 msgr2=0x561710082460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:41.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75baa00640 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 0x561710082460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75baa00640 1 -- 172.21.15.107:0/1481105852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 msgr2=0x561710004350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75baa00640 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 0x561710004350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75baa00640 1 -- 172.21.15.107:0/1481105852 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617100829a0 con 0x56170fff6590 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75bb400640 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 0x561710004350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75baa00640 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710004890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x561710003750 tx=0x7f75a8003f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d49b118c4ed162af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75a4c00640 1 -- 172.21.15.107:0/1481105852 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8033070 con 0x56170fff6590 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75c4e7c740 1 -- 172.21.15.107:0/1481105852 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75ac0035f0 con 0x56170fff6590 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.355+0000 7f75a4c00640 1 -- 172.21.15.107:0/1481105852 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f75a8005450 con 0x56170fff6590 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75a4c00640 1 -- 172.21.15.107:0/1481105852 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f75a801ec70 con 0x56170fff6590 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75c4e7c740 10 client.26503 fetch_fsmap learned FSMap version 43 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75c4e7c740 1 -- 172.21.15.107:0/1481105852 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561710083bb0 con 0x56170fff6590 2024-09-19T04:27:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75a4c00640 1 -- 172.21.15.107:0/1481105852 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ac0035f0 con 0x56170fff6590 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75a4c00640 1 -- 172.21.15.107:0/1481105852 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f75a8025040 con 0x56170fff6590 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75c4e7c740 10 client.26503 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75c4e7c740 10 client.26503 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75c4e7c740 20 client.26503 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x561710084300 0x561710086720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75c4e7c740 4 client.26503 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.356+0000 7f75c4e7c740 1 -- 172.21.15.107:0/1481105852 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56170ff63e60 con 0x561710084300 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.357+0000 7f75bb400640 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x561710084300 0x561710086720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.357+0000 7f75bb400640 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x561710084300 0x561710086720 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=556ec537e0f2086b server_cookie=6991837afa59a716 in_seq=0 out_seq=0 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.357+0000 7f75a4c00640 10 client.26503 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.357+0000 7f75a4c00640 1 -- 172.21.15.107:0/1481105852 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56170ff63e60 con 0x561710084300 2024-09-19T04:27:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.357+0000 7f75a4c00640 10 client.26503 handle_command_reply: tid=0 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.357+0000 7f75c4e7c740 1 client.26503 shutdown 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.357+0000 7f75c4e7c740 1 -- 172.21.15.107:0/1481105852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 msgr2=0x561710004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.357+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710004890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x561710003750 tx=0x7f75a8003f40 comp rx=0 tx=0).stop 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75c4e7c740 1 -- 172.21.15.107:0/1481105852 shutdown_connections 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x561710084300 0x561710086720 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561710000e50 0x561710082460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56170fff6590 0x561710004890 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75a4c00640 0 client.26503 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75c4e7c740 1 --2- 172.21.15.107:0/1481105852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56170fff5c70 0x561710004350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75c4e7c740 1 -- 172.21.15.107:0/1481105852 >> 172.21.15.107:0/1481105852 conn(0x56170fff2200 msgr2=0x56170fff0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75c4e7c740 1 -- 172.21.15.107:0/1481105852 shutdown_connections 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75c4e7c740 1 -- 172.21.15.107:0/1481105852 wait complete. 2024-09-19T04:27:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75c4e7c740 20 client.26503 trim_cache size 0 max 16384 2024-09-19T04:27:41.364 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:27:41.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.358+0000 7f75c2200640 1 -- 172.21.15.107:0/845988175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75a00a3430 msgr2=0x7f75a00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:41.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75a00a3430 0x7f75a00a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f75a80058a0 tx=0x7f75a800b740 comp rx=0 tx=0).stop 2024-09-19T04:27:41.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 -- 172.21.15.107:0/845988175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75bc106f10 msgr2=0x7f75bc19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:41.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75bc106f10 0x7f75bc19ef10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f75b4031a20 tx=0x7f75b4031a50 comp rx=0 tx=0).stop 2024-09-19T04:27:41.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 -- 172.21.15.107:0/845988175 shutdown_connections 2024-09-19T04:27:41.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75a00a3430 0x7f75a00a58f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75bc106f10 0x7f75bc19ef10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75bc102790 0x7f75bc19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 --2- 172.21.15.107:0/845988175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75bc101de0 0x7f75bc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:41.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 -- 172.21.15.107:0/845988175 >> 172.21.15.107:0/845988175 conn(0x7f75bc0fd8c0 msgr2=0x7f75bc0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:41.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 -- 172.21.15.107:0/845988175 shutdown_connections 2024-09-19T04:27:41.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:41.359+0000 7f75c2200640 1 -- 172.21.15.107:0/845988175 wait complete. 2024-09-19T04:27:41.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 235839 2.42 MB/sec execute 239 sec latency 1887.341 ms 2024-09-19T04:27:42.459 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:27:42.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 235839 2.41 MB/sec execute 240 sec latency 2887.400 ms 2024-09-19T04:27:42.830 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:42.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:42 smithi115 ceph-mon[35477]: pgmap v226: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-09-19T04:27:42.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:42 smithi107 ceph-mon[30659]: pgmap v226: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-09-19T04:27:42.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:42 smithi145 ceph-mon[35096]: pgmap v226: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-09-19T04:27:43.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.369+0000 7f6c0aa00640 1 -- 172.21.15.107:0/2978417498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c04112870 msgr2=0x7f6c04114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.369+0000 7f6c0aa00640 1 --2- 172.21.15.107:0/2978417498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c04112870 0x7f6c04114c60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009ca0 tx=0x7f6bfc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.369+0000 7f6c0aa00640 1 -- 172.21.15.107:0/2978417498 shutdown_connections 2024-09-19T04:27:43.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.369+0000 7f6c0aa00640 1 --2- 172.21.15.107:0/2978417498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c04112870 0x7f6c04114c60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.369+0000 7f6c0aa00640 1 --2- 172.21.15.107:0/2978417498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c04077040 0x7f6c04112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.369+0000 7f6c0aa00640 1 --2- 172.21.15.107:0/2978417498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c04076720 0x7f6c04076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.369+0000 7f6c0aa00640 1 -- 172.21.15.107:0/2978417498 >> 172.21.15.107:0/2978417498 conn(0x7f6c040ff8d0 msgr2=0x7f6c04101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.369+0000 7f6c0aa00640 1 -- 172.21.15.107:0/2978417498 shutdown_connections 2024-09-19T04:27:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.369+0000 7f6c0aa00640 1 -- 172.21.15.107:0/2978417498 wait complete. 2024-09-19T04:27:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.370+0000 7f6c0aa00640 1 Processor -- start 2024-09-19T04:27:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.370+0000 7f6c0aa00640 1 -- start start 2024-09-19T04:27:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c0aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c04076720 0x7f6c041a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c0aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c04077040 0x7f6c041a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c0aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c04112870 0x7f6c0419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c0aa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c041183c0 con 0x7f6c04076720 2024-09-19T04:27:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c0aa00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c04118240 con 0x7f6c04112870 2024-09-19T04:27:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c0aa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c04118540 con 0x7f6c04077040 2024-09-19T04:27:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c09600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c04076720 0x7f6c041a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c09600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c04076720 0x7f6c041a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36966/0 (socket says 172.21.15.107:36966) 2024-09-19T04:27:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c09600640 1 -- 172.21.15.107:0/1875901923 learned_addr learned my addr 172.21.15.107:0/1875901923 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c0a000640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c04112870 0x7f6c0419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c08c00640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c04077040 0x7f6c041a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c09600640 1 -- 172.21.15.107:0/1875901923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c04077040 msgr2=0x7f6c041a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c09600640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c04077040 0x7f6c041a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c09600640 1 -- 172.21.15.107:0/1875901923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c04112870 msgr2=0x7f6c0419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c09600640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c04112870 0x7f6c0419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c09600640 1 -- 172.21.15.107:0/1875901923 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0419efa0 con 0x7f6c04076720 2024-09-19T04:27:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.371+0000 7f6c0a000640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c04112870 0x7f6c0419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:43.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.372+0000 7f6c09600640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c04076720 0x7f6c041a4660 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800e9f0 tx=0x7f6bf800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.372+0000 7f6c02000640 1 -- 172.21.15.107:0/1875901923 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800cd80 con 0x7f6c04076720 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.372+0000 7f6c0aa00640 1 -- 172.21.15.107:0/1875901923 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0419f290 con 0x7f6c04076720 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.372+0000 7f6c02000640 1 -- 172.21.15.107:0/1875901923 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf8005500 con 0x7f6c04076720 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.372+0000 7f6c02000640 1 -- 172.21.15.107:0/1875901923 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8010690 con 0x7f6c04076720 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.372+0000 7f6c0aa00640 1 -- 172.21.15.107:0/1875901923 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c041ab410 con 0x7f6c04076720 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.375+0000 7f6c0ec3d740 1 Processor -- start 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.375+0000 7f6c02000640 1 -- 172.21.15.107:0/1875901923 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f6bf80046c0 con 0x7f6c04076720 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.375+0000 7f6c0ec3d740 1 -- start start 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.375+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 0x5630542b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.375+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 0x5630542c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.375+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56305421f3e0 con 0x5630542c4020 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5630541c6d80 con 0x5630542b8e40 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56305421b3a0 con 0x5630542b9760 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 0x5630542b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c08c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c09600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 0x5630542c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 0x5630542b9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39424/0 (socket says 172.21.15.107:39424) 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0a000640 1 -- 172.21.15.107:0/3272736132 learned_addr learned my addr 172.21.15.107:0/3272736132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0a000640 1 -- 172.21.15.107:0/3272736132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 msgr2=0x5630542c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0a000640 1 --2- 172.21.15.107:0/3272736132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 0x5630542c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0a000640 1 -- 172.21.15.107:0/3272736132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 msgr2=0x5630542c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0a000640 1 --2- 172.21.15.107:0/3272736132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c0a000640 1 -- 172.21.15.107:0/3272736132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630541e5ae0 con 0x5630542b8e40 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.376+0000 7f6c08c00640 1 --2- 172.21.15.107:0/3272736132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.377+0000 7f6c02000640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6bec0a33b0 0x7f6bec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.377+0000 7f6c0a000640 1 --2- 172.21.15.107:0/3272736132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 0x5630542b9220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009ca0 tx=0x7f6bfc005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d51e2489076c15b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.377+0000 7f6c02000640 1 -- 172.21.15.107:0/1875901923 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f6bf80d02c0 con 0x7f6c04076720 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.377+0000 7f6c09600640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6bec0a33b0 0x7f6bec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.377+0000 7f6bf6600640 1 -- 172.21.15.107:0/3272736132 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc031db0 con 0x5630542b8e40 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.377+0000 7f6bf6600640 1 -- 172.21.15.107:0/3272736132 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bfc036050 con 0x5630542b8e40 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.377+0000 7f6bf6600640 1 -- 172.21.15.107:0/3272736132 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc004240 con 0x5630542b8e40 2024-09-19T04:27:43.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3272736132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 msgr2=0x5630542b9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c09600640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6bec0a33b0 0x7f6bec0a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5630542b6230 tx=0x7f6bf80d3d70 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/3272736132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 0x5630542b9220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009ca0 tx=0x7f6bfc005f60 comp rx=0 tx=0).stop 2024-09-19T04:27:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3272736132 shutdown_connections 2024-09-19T04:27:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/3272736132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/3272736132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 0x5630542c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/3272736132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 0x5630542b9220 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3272736132 >> 172.21.15.107:0/3272736132 conn(0x5630542b53d0 msgr2=0x5630542b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3272736132 shutdown_connections 2024-09-19T04:27:43.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3272736132 wait complete. 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.378+0000 7f6c0ec3d740 1 -- start start 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0ec3d740 1 Processor -- start 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b8e40 0x5630542c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 0x5630542c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542c4020 0x5630542c1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5630542c6a70 con 0x5630542b8e40 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5630541da0f0 con 0x5630542c4020 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5630542c6bf0 con 0x5630542b9760 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0ec3d740 10 client.0 _ll_register_callbacks cb 0x5630541c1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b8e40 0x5630542c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b8e40 0x5630542c0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36984/0 (socket says 172.21.15.107:36984) 2024-09-19T04:27:43.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.379+0000 7f6c0a000640 1 -- 172.21.15.107:0/884597755 learned_addr learned my addr 172.21.15.107:0/884597755 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.380+0000 7f6c08c00640 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 0x5630542c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.380+0000 7f6c0a000640 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542c4020 0x5630542c1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.380+0000 7f6c08c00640 1 -- 172.21.15.107:0/884597755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542c4020 msgr2=0x5630542c1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.380+0000 7f6c08c00640 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542c4020 0x5630542c1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.380+0000 7f6c08c00640 1 -- 172.21.15.107:0/884597755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b8e40 msgr2=0x5630542c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.380+0000 7f6c08c00640 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b8e40 0x5630542c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.380+0000 7f6c08c00640 1 -- 172.21.15.107:0/884597755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563053f903c0 con 0x5630542b9760 2024-09-19T04:27:43.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.380+0000 7f6c0a000640 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542c4020 0x5630542c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:43.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.380+0000 7f6c08c00640 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 0x5630542c13c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6c0419ff80 tx=0x7f6bf0012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f727d82ad5264d16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.380+0000 7f6beaa00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:27:43.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.381+0000 7f6beaa00640 1 -- 172.21.15.107:0/884597755 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0004320 con 0x5630542b9760 2024-09-19T04:27:43.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.381+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/884597755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bfc036050 con 0x5630542b9760 2024-09-19T04:27:43.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.381+0000 7f6beaa00640 1 -- 172.21.15.107:0/884597755 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f6bf0011940 con 0x5630542b9760 2024-09-19T04:27:43.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.381+0000 7f6beaa00640 1 -- 172.21.15.107:0/884597755 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6bf00044c0 con 0x5630542b9760 2024-09-19T04:27:43.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.381+0000 7f6c0ec3d740 10 client.26515 fetch_fsmap learned FSMap version 43 2024-09-19T04:27:43.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.381+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/884597755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5630542d2b10 con 0x5630542b9760 2024-09-19T04:27:43.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.381+0000 7f6beaa00640 1 -- 172.21.15.107:0/884597755 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bfc036050 con 0x5630542b9760 2024-09-19T04:27:43.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.381+0000 7f6beaa00640 1 -- 172.21.15.107:0/884597755 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f6bf0023020 con 0x5630542b9760 2024-09-19T04:27:43.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.382+0000 7f6c0ec3d740 10 client.26515 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:27:43.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.382+0000 7f6c0ec3d740 10 client.26515 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:27:43.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.382+0000 7f6c0ec3d740 20 client.26515 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:43.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.382+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5630542d3f70 0x5630542d63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.382+0000 7f6c0ec3d740 4 client.26515 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:43.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.382+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/884597755 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56305422c950 con 0x5630542d3f70 2024-09-19T04:27:43.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.382+0000 7f6c0a000640 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5630542d3f70 0x5630542d63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.383+0000 7f6c0a000640 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5630542d3f70 0x5630542d63c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fcab7c95ca569aac server_cookie=2883fbda400807df in_seq=0 out_seq=0 2024-09-19T04:27:43.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.383+0000 7f6beaa00640 10 client.26515 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:27:43.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.383+0000 7f6beaa00640 1 -- 172.21.15.107:0/884597755 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56305422c950 con 0x5630542d3f70 2024-09-19T04:27:43.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.383+0000 7f6beaa00640 10 client.26515 handle_command_reply: tid=0 2024-09-19T04:27:43.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.383+0000 7f6c0ec3d740 1 client.26515 shutdown 2024-09-19T04:27:43.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/884597755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 msgr2=0x5630542c13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 0x5630542c13c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6c0419ff80 tx=0x7f6bf0012b60 comp rx=0 tx=0).stop 2024-09-19T04:27:43.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/884597755 shutdown_connections 2024-09-19T04:27:43.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5630542d3f70 0x5630542d63c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542c4020 0x5630542c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b9760 0x5630542c13c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/884597755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b8e40 0x5630542c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6beaa00640 0 client.26515 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:27:43.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/884597755 >> 172.21.15.107:0/884597755 conn(0x5630542b53d0 msgr2=0x5630542bb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/884597755 shutdown_connections 2024-09-19T04:27:43.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/884597755 wait complete. 2024-09-19T04:27:43.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.384+0000 7f6c0ec3d740 20 client.26515 trim_cache size 0 max 16384 2024-09-19T04:27:43.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.397+0000 7f6c0ec3d740 1 Processor -- start 2024-09-19T04:27:43.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.397+0000 7f6c0ec3d740 1 -- start start 2024-09-19T04:27:43.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b8e40 0x5630542bb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b9760 0x5630542c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542dbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c041b3ca0 con 0x5630542c4020 2024-09-19T04:27:43.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c041b58a0 con 0x5630542b9760 2024-09-19T04:27:43.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c041b43a0 con 0x5630542b8e40 2024-09-19T04:27:43.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542dbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542dbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36992/0 (socket says 172.21.15.107:36992) 2024-09-19T04:27:43.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0a000640 1 -- 172.21.15.107:0/4083980164 learned_addr learned my addr 172.21.15.107:0/4083980164 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:43.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c08c00640 1 --2- 172.21.15.107:0/4083980164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b9760 0x5630542c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0a000640 1 --2- 172.21.15.107:0/4083980164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b8e40 0x5630542bb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0a000640 1 -- 172.21.15.107:0/4083980164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b8e40 msgr2=0x5630542bb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0a000640 1 --2- 172.21.15.107:0/4083980164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b8e40 0x5630542bb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0a000640 1 -- 172.21.15.107:0/4083980164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b9760 msgr2=0x5630542c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0a000640 1 --2- 172.21.15.107:0/4083980164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b9760 0x5630542c2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.398+0000 7f6c0a000640 1 -- 172.21.15.107:0/4083980164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630542b8c70 con 0x5630542c4020 2024-09-19T04:27:43.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.399+0000 7f6c0a000640 1 --2- 172.21.15.107:0/4083980164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542dbcd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009ca0 tx=0x7f6bfc02f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fe506fbead4b0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.399+0000 7f6bf5c00640 1 -- 172.21.15.107:0/4083980164 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc049040 con 0x5630542c4020 2024-09-19T04:27:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.399+0000 7f6bf5c00640 1 -- 172.21.15.107:0/4083980164 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bfc003e50 con 0x5630542c4020 2024-09-19T04:27:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.399+0000 7f6bf5c00640 1 -- 172.21.15.107:0/4083980164 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc004710 con 0x5630542c4020 2024-09-19T04:27:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.400+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/4083980164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 msgr2=0x5630542dbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.400+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/4083980164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542dbcd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009ca0 tx=0x7f6bfc02f390 comp rx=0 tx=0).stop 2024-09-19T04:27:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.400+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/4083980164 shutdown_connections 2024-09-19T04:27:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.400+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/4083980164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542c4020 0x5630542dbcd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.400+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/4083980164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b9760 0x5630542c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.400+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/4083980164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542b8e40 0x5630542bb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.400+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/4083980164 >> 172.21.15.107:0/4083980164 conn(0x5630542b53d0 msgr2=0x5630542c26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.400+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/4083980164 shutdown_connections 2024-09-19T04:27:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.400+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/4083980164 wait complete. 2024-09-19T04:27:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.400+0000 7f6c0ec3d740 1 -- start start 2024-09-19T04:27:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c0ec3d740 1 Processor -- start 2024-09-19T04:27:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 0x5630542c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b9760 0x5630542c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c0ec3d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542c4020 0x5630542c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c041b5140 con 0x5630542b9760 2024-09-19T04:27:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c041b4a70 con 0x5630542b8e40 2024-09-19T04:27:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c0ec3d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c041b1ca0 con 0x5630542c4020 2024-09-19T04:27:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c0ec3d740 10 client.0 _ll_register_callbacks cb 0x7f6bf0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c08c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b9760 0x5630542c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c08c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b9760 0x5630542c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37004/0 (socket says 172.21.15.107:37004) 2024-09-19T04:27:43.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c08c00640 1 -- 172.21.15.107:0/3366523109 learned_addr learned my addr 172.21.15.107:0/3366523109 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:43.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c0a000640 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 0x5630542c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.401+0000 7f6c0a000640 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542c4020 0x5630542c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.402+0000 7f6c08c00640 1 -- 172.21.15.107:0/3366523109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542c4020 msgr2=0x5630542c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.402+0000 7f6c08c00640 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542c4020 0x5630542c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.402+0000 7f6c08c00640 1 -- 172.21.15.107:0/3366523109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 msgr2=0x5630542c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:43.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.402+0000 7f6c08c00640 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 0x5630542c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.402+0000 7f6c08c00640 1 -- 172.21.15.107:0/3366523109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630542b6060 con 0x5630542b9760 2024-09-19T04:27:43.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.402+0000 7f6c08c00640 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b9760 0x5630542c81e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6c0419ff80 tx=0x7f6bf00095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3790b0631a4fd33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:43.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.402+0000 7f6beaa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:27:43.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.402+0000 7f6beaa00640 1 -- 172.21.15.107:0/3366523109 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf002b040 con 0x5630542b9760 2024-09-19T04:27:43.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.402+0000 7f6beaa00640 1 -- 172.21.15.107:0/3366523109 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f6bf00055d0 con 0x5630542b9760 2024-09-19T04:27:43.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.402+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3366523109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bfc003e50 con 0x5630542b9760 2024-09-19T04:27:43.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6beaa00640 1 -- 172.21.15.107:0/3366523109 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6bf0004320 con 0x5630542b9760 2024-09-19T04:27:43.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6c0ec3d740 10 client.16503 fetch_fsmap learned FSMap version 43 2024-09-19T04:27:43.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3366523109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56305433ee80 con 0x5630542b9760 2024-09-19T04:27:43.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6beaa00640 1 -- 172.21.15.107:0/3366523109 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bfc003e50 con 0x5630542b9760 2024-09-19T04:27:43.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6beaa00640 1 -- 172.21.15.107:0/3366523109 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f6bf0037070 con 0x5630542b9760 2024-09-19T04:27:43.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6c0ec3d740 10 client.16503 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:27:43.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6c0ec3d740 10 client.16503 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:27:43.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6c0ec3d740 20 client.16503 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:43.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56305433f6e0 0x563054341b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:43.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6c0ec3d740 4 client.16503 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:27:43.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.403+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3366523109 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563054226e60 con 0x56305433f6e0 2024-09-19T04:27:43.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0a000640 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56305433f6e0 0x563054341b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:43.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0a000640 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56305433f6e0 0x563054341b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=92e4445844f76d4d server_cookie=d41c0eefabc6156 in_seq=0 out_seq=0 2024-09-19T04:27:43.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6beaa00640 10 client.16503 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:27:43.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6beaa00640 1 -- 172.21.15.107:0/3366523109 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563054226e60 con 0x56305433f6e0 2024-09-19T04:27:43.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6beaa00640 10 client.16503 handle_command_reply: tid=0 2024-09-19T04:27:43.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0ec3d740 1 client.16503 shutdown 2024-09-19T04:27:43.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3366523109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b9760 msgr2=0x5630542c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b9760 0x5630542c81e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6c0419ff80 tx=0x7f6bf00095e0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3366523109 shutdown_connections 2024-09-19T04:27:43.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x56305433f6e0 0x563054341b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630542c4020 0x5630542c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630542b9760 0x5630542c81e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0ec3d740 1 --2- 172.21.15.107:0/3366523109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630542b8e40 0x5630542c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.404+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3366523109 >> 172.21.15.107:0/3366523109 conn(0x5630542b53d0 msgr2=0x5630542c1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.405+0000 7f6beaa00640 0 client.16503 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:27:43.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.405+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3366523109 shutdown_connections 2024-09-19T04:27:43.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.405+0000 7f6c0ec3d740 1 -- 172.21.15.107:0/3366523109 wait complete. 2024-09-19T04:27:43.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.405+0000 7f6c0ec3d740 20 client.16503 trim_cache size 0 max 16384 2024-09-19T04:27:43.428 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:27:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.405+0000 7f6c0aa00640 1 -- 172.21.15.107:0/1875901923 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6bec0a33b0 msgr2=0x7f6bec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.405+0000 7f6c0aa00640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6bec0a33b0 0x7f6bec0a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5630542b6230 tx=0x7f6bf80d3d70 comp rx=0 tx=0).stop 2024-09-19T04:27:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.405+0000 7f6c0aa00640 1 -- 172.21.15.107:0/1875901923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c04076720 msgr2=0x7f6c041a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.405+0000 7f6c0aa00640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c04076720 0x7f6c041a4660 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800e9f0 tx=0x7f6bf800edf0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.406+0000 7f6c0aa00640 1 -- 172.21.15.107:0/1875901923 shutdown_connections 2024-09-19T04:27:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.406+0000 7f6c0aa00640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6bec0a33b0 0x7f6bec0a5870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.406+0000 7f6c0aa00640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c04112870 0x7f6c0419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.406+0000 7f6c0aa00640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c04077040 0x7f6c041a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.406+0000 7f6c0aa00640 1 --2- 172.21.15.107:0/1875901923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c04076720 0x7f6c041a4660 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.406+0000 7f6c0aa00640 1 -- 172.21.15.107:0/1875901923 >> 172.21.15.107:0/1875901923 conn(0x7f6c040ff8d0 msgr2=0x7f6c04101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.406+0000 7f6c0aa00640 1 -- 172.21.15.107:0/1875901923 shutdown_connections 2024-09-19T04:27:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:43.406+0000 7f6c0aa00640 1 -- 172.21.15.107:0/1875901923 wait complete. 2024-09-19T04:27:43.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 235839 2.40 MB/sec execute 241 sec latency 3887.508 ms 2024-09-19T04:27:44.137 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:27:44.535 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:44.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 237486 2.43 MB/sec execute 242 sec latency 4157.121 ms 2024-09-19T04:27:44.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:44 smithi115 ceph-mon[35477]: pgmap v227: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 795 KiB/s wr, 85 op/s 2024-09-19T04:27:44.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:44 smithi145 ceph-mon[35096]: pgmap v227: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 795 KiB/s wr, 85 op/s 2024-09-19T04:27:44.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:44 smithi107 ceph-mon[30659]: pgmap v227: 97 pgs: 97 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 795 KiB/s wr, 85 op/s 2024-09-19T04:27:45.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.127+0000 7f7677200640 1 -- 172.21.15.107:0/4259381512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76781047e0 msgr2=0x7f7678108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.127+0000 7f7677200640 1 --2- 172.21.15.107:0/4259381512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76781047e0 0x7f7678108830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7664009900 tx=0x7f766402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.127+0000 7f7677200640 1 -- 172.21.15.107:0/4259381512 shutdown_connections 2024-09-19T04:27:45.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.127+0000 7f7677200640 1 --2- 172.21.15.107:0/4259381512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7678108f60 0x7f7678110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.127+0000 7f7677200640 1 --2- 172.21.15.107:0/4259381512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76781047e0 0x7f7678108830 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.127+0000 7f7677200640 1 --2- 172.21.15.107:0/4259381512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678103e30 0x7f7678104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.127+0000 7f7677200640 1 -- 172.21.15.107:0/4259381512 >> 172.21.15.107:0/4259381512 conn(0x7f76780ff8d0 msgr2=0x7f7678101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.127+0000 7f7677200640 1 -- 172.21.15.107:0/4259381512 shutdown_connections 2024-09-19T04:27:45.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.127+0000 7f7677200640 1 -- 172.21.15.107:0/4259381512 wait complete. 2024-09-19T04:27:45.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.128+0000 7f7677200640 1 Processor -- start 2024-09-19T04:27:45.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.128+0000 7f7677200640 1 -- start start 2024-09-19T04:27:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.128+0000 7f7677200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678103e30 0x7f767819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7677200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76781047e0 0x7f767819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7675e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678103e30 0x7f767819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7675e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678103e30 0x7f767819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37016/0 (socket says 172.21.15.107:37016) 2024-09-19T04:27:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7677200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7678108f60 0x7f767819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7675e00640 1 -- 172.21.15.107:0/1415328997 learned_addr learned my addr 172.21.15.107:0/1415328997 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7678113f80 con 0x7f7678103e30 2024-09-19T04:27:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7678113e00 con 0x7f76781047e0 2024-09-19T04:27:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7678114100 con 0x7f7678108f60 2024-09-19T04:27:45.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7675400640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76781047e0 0x7f767819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7676800640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7678108f60 0x7f767819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7675e00640 1 -- 172.21.15.107:0/1415328997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7678108f60 msgr2=0x7f767819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.129+0000 7f7675e00640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7678108f60 0x7f767819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.130+0000 7f7675e00640 1 -- 172.21.15.107:0/1415328997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76781047e0 msgr2=0x7f767819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.130+0000 7f7675e00640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76781047e0 0x7f767819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.130+0000 7f7675e00640 1 -- 172.21.15.107:0/1415328997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767819f410 con 0x7f7678103e30 2024-09-19T04:27:45.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.130+0000 7f7676800640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7678108f60 0x7f767819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:45.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.130+0000 7f7675e00640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678103e30 0x7f767819a460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f766800e9f0 tx=0x7f766800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.130+0000 7f766ea00640 1 -- 172.21.15.107:0/1415328997 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766800cd80 con 0x7f7678103e30 2024-09-19T04:27:45.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.130+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767819f700 con 0x7f7678103e30 2024-09-19T04:27:45.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.130+0000 7f766ea00640 1 -- 172.21.15.107:0/1415328997 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7668010910 con 0x7f7678103e30 2024-09-19T04:27:45.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.130+0000 7f766ea00640 1 -- 172.21.15.107:0/1415328997 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76680188f0 con 0x7f7678103e30 2024-09-19T04:27:45.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.130+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781a6fe0 con 0x7f7678103e30 2024-09-19T04:27:45.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.134+0000 7f766ea00640 1 -- 172.21.15.107:0/1415328997 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f7668010ac0 con 0x7f7678103e30 2024-09-19T04:27:45.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f767f844740 1 Processor -- start 2024-09-19T04:27:45.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f767f844740 1 -- start start 2024-09-19T04:27:45.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f767f844740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 0x55eeb05eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f767f844740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb05f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f767f844740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 0x55eeb05f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f767f844740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eeb05513e0 con 0x55eeb05f6020 2024-09-19T04:27:45.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f767f844740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55eeb04f8d80 con 0x55eeb05eb760 2024-09-19T04:27:45.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f767f844740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55eeb054d3a0 con 0x55eeb05eae40 2024-09-19T04:27:45.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f7675e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb05f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f7676800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 0x55eeb05eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f7675e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb05f5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39476/0 (socket says 172.21.15.107:39476) 2024-09-19T04:27:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f7676800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 0x55eeb05eb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:60746/0 (socket says 172.21.15.107:60746) 2024-09-19T04:27:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f7675e00640 1 -- 172.21.15.107:0/1640994912 learned_addr learned my addr 172.21.15.107:0/1640994912 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.135+0000 7f7675400640 1 --2- 172.21.15.107:0/1640994912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 0x55eeb05f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.136+0000 7f7675e00640 1 -- 172.21.15.107:0/1640994912 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 msgr2=0x55eeb05eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.136+0000 7f7675e00640 1 --2- 172.21.15.107:0/1640994912 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 0x55eeb05eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.136+0000 7f7675e00640 1 -- 172.21.15.107:0/1640994912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 msgr2=0x55eeb05f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.136+0000 7f7675e00640 1 --2- 172.21.15.107:0/1640994912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 0x55eeb05f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.136+0000 7f7675e00640 1 -- 172.21.15.107:0/1640994912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb0517ae0 con 0x55eeb05eb760 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.136+0000 7f766ea00640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f765c0a3430 0x7f765c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.136+0000 7f7675400640 1 --2- 172.21.15.107:0/1640994912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 0x55eeb05f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.136+0000 7f766ea00640 1 -- 172.21.15.107:0/1415328997 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f76680d0150 con 0x7f7678103e30 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.136+0000 7f7676800640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f765c0a3430 0x7f765c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.136+0000 7f7675e00640 1 --2- 172.21.15.107:0/1640994912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb05f5ae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f76680cd7a0 tx=0x7f76680d8910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbf4f4b64e0b404e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.137+0000 7f7663400640 1 -- 172.21.15.107:0/1640994912 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668098690 con 0x55eeb05eb760 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.137+0000 7f7663400640 1 -- 172.21.15.107:0/1640994912 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f766809caf0 con 0x55eeb05eb760 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.137+0000 7f7663400640 1 -- 172.21.15.107:0/1640994912 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766809bd50 con 0x55eeb05eb760 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.138+0000 7f767f844740 1 -- 172.21.15.107:0/1640994912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 msgr2=0x55eeb05f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.138+0000 7f7676800640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f765c0a3430 0x7f765c0a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7670005fd0 tx=0x7f7670009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.138+0000 7f767f844740 1 --2- 172.21.15.107:0/1640994912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb05f5ae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f76680cd7a0 tx=0x7f76680d8910 comp rx=0 tx=0).stop 2024-09-19T04:27:45.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.138+0000 7f767f844740 1 -- 172.21.15.107:0/1640994912 shutdown_connections 2024-09-19T04:27:45.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.138+0000 7f767f844740 1 --2- 172.21.15.107:0/1640994912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 0x55eeb05f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.138+0000 7f767f844740 1 --2- 172.21.15.107:0/1640994912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb05f5ae0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.138+0000 7f767f844740 1 --2- 172.21.15.107:0/1640994912 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 0x55eeb05eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.138+0000 7f767f844740 1 -- 172.21.15.107:0/1640994912 >> 172.21.15.107:0/1640994912 conn(0x55eeb05e73d0 msgr2=0x55eeb05e77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.138+0000 7f767f844740 1 -- 172.21.15.107:0/1640994912 shutdown_connections 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.138+0000 7f767f844740 1 -- 172.21.15.107:0/1640994912 wait complete. 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.139+0000 7f767f844740 1 -- start start 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.139+0000 7f767f844740 1 Processor -- start 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f767f844740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f767f844740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 0x55eeb05f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f767f844740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 0x55eeb05f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f767f844740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eeb05f8bf0 con 0x55eeb05eb760 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f767f844740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55eeb050c0f0 con 0x55eeb05eae40 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f767f844740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55eeb05f8d70 con 0x55eeb05f6020 2024-09-19T04:27:45.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f767f844740 10 client.0 _ll_register_callbacks cb 0x55eeb04f3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7675e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7675e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39490/0 (socket says 172.21.15.107:39490) 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7675e00640 1 -- 172.21.15.107:0/3535009298 learned_addr learned my addr 172.21.15.107:0/3535009298 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7675400640 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 0x55eeb05f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7676800640 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 0x55eeb05f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7675e00640 1 -- 172.21.15.107:0/3535009298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 msgr2=0x55eeb05f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7675e00640 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 0x55eeb05f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7675e00640 1 -- 172.21.15.107:0/3535009298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 msgr2=0x55eeb05f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7675e00640 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 0x55eeb05f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7675e00640 1 -- 172.21.15.107:0/3535009298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb02c23c0 con 0x55eeb05eae40 2024-09-19T04:27:45.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.140+0000 7f7675400640 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 0x55eeb05f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f7675e00640 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05f2f80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f766809c0f0 tx=0x7f76680cd7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd0f1b420175604 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f7657e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f7657e00640 1 -- 172.21.15.107:0/3535009298 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766809bf30 con 0x55eeb05eae40 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f7657e00640 1 -- 172.21.15.107:0/3535009298 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f766809c850 con 0x55eeb05eae40 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f767f844740 1 -- 172.21.15.107:0/3535009298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f766809caf0 con 0x55eeb05eae40 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f7657e00640 1 -- 172.21.15.107:0/3535009298 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f766809c1c0 con 0x55eeb05eae40 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f767f844740 10 client.26564 fetch_fsmap learned FSMap version 43 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f767f844740 1 -- 172.21.15.107:0/3535009298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55eeb05f8bf0 con 0x55eeb05eae40 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f7657e00640 1 -- 172.21.15.107:0/3535009298 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f766809caf0 con 0x55eeb05eae40 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f7657e00640 1 -- 172.21.15.107:0/3535009298 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f76680a4bd0 con 0x55eeb05eae40 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f767f844740 10 client.26564 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.141+0000 7f767f844740 10 client.26564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:27:45.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.142+0000 7f767f844740 20 client.26564 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:45.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.142+0000 7f767f844740 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55eeb0605e30 0x55eeb0608220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.142+0000 7f767f844740 4 client.26564 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:45.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.142+0000 7f767f844740 1 -- 172.21.15.107:0/3535009298 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeb055e950 con 0x55eeb0605e30 2024-09-19T04:27:45.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.142+0000 7f7675400640 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55eeb0605e30 0x55eeb0608220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.142+0000 7f7657e00640 10 client.26564 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:27:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.142+0000 7f7675400640 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55eeb0605e30 0x55eeb0608220 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4f306680a8bd5b server_cookie=49ee0652ac0b5dc7 in_seq=0 out_seq=0 2024-09-19T04:27:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f7657e00640 1 -- 172.21.15.107:0/3535009298 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eeb055e950 con 0x55eeb0605e30 2024-09-19T04:27:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f7657e00640 10 client.26564 handle_command_reply: tid=0 2024-09-19T04:27:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f767f844740 1 client.26564 shutdown 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f767f844740 1 -- 172.21.15.107:0/3535009298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 msgr2=0x55eeb05f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f767f844740 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05f2f80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f766809c0f0 tx=0x7f76680cd7d0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f767f844740 1 -- 172.21.15.107:0/3535009298 shutdown_connections 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f767f844740 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55eeb0605e30 0x55eeb0608220 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f767f844740 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 0x55eeb05f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f767f844740 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 0x55eeb05f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f767f844740 1 --2- 172.21.15.107:0/3535009298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05f2f80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f7657e00640 0 client.26564 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.143+0000 7f767f844740 1 -- 172.21.15.107:0/3535009298 >> 172.21.15.107:0/3535009298 conn(0x55eeb05e73d0 msgr2=0x55eeb05ed0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.144+0000 7f767f844740 1 -- 172.21.15.107:0/3535009298 shutdown_connections 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.144+0000 7f767f844740 1 -- 172.21.15.107:0/3535009298 wait complete. 2024-09-19T04:27:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.144+0000 7f767f844740 20 client.26564 trim_cache size 0 max 16384 2024-09-19T04:27:45.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.162+0000 7f767f844740 1 Processor -- start 2024-09-19T04:27:45.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.162+0000 7f767f844740 1 -- start start 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.162+0000 7f767f844740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 0x55eeb060d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.162+0000 7f767f844740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb060ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.162+0000 7f767f844740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 0x55eeb05f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f767f844740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76781af8b0 con 0x55eeb05f6020 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f767f844740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76781b14b0 con 0x55eeb05eb760 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f767f844740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f76781affb0 con 0x55eeb05eae40 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb060ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 0x55eeb060d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb060ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39498/0 (socket says 172.21.15.107:39498) 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 0x55eeb060d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:60768/0 (socket says 172.21.15.107:60768) 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675400640 1 -- 172.21.15.107:0/3243697519 learned_addr learned my addr 172.21.15.107:0/3243697519 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:45.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7676800640 1 --2- 172.21.15.107:0/3243697519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 0x55eeb05f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675400640 1 -- 172.21.15.107:0/3243697519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 msgr2=0x55eeb060d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675400640 1 --2- 172.21.15.107:0/3243697519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 0x55eeb060d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675400640 1 -- 172.21.15.107:0/3243697519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 msgr2=0x55eeb05f01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675400640 1 --2- 172.21.15.107:0/3243697519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 0x55eeb05f01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675400640 1 -- 172.21.15.107:0/3243697519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb05f58f0 con 0x55eeb05eb760 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.163+0000 7f7675400640 1 --2- 172.21.15.107:0/3243697519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb060ddf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f766402a910 tx=0x7f76640058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4331ffc70258100c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.164+0000 7f7662a00640 1 -- 172.21.15.107:0/3243697519 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664049070 con 0x55eeb05eb760 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.164+0000 7f7662a00640 1 -- 172.21.15.107:0/3243697519 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7664005020 con 0x55eeb05eb760 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.164+0000 7f7662a00640 1 -- 172.21.15.107:0/3243697519 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664035040 con 0x55eeb05eb760 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.164+0000 7f767f844740 1 -- 172.21.15.107:0/3243697519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 msgr2=0x55eeb060ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.164+0000 7f767f844740 1 --2- 172.21.15.107:0/3243697519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb060ddf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f766402a910 tx=0x7f76640058d0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.164+0000 7f767f844740 1 -- 172.21.15.107:0/3243697519 shutdown_connections 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.165+0000 7f767f844740 1 --2- 172.21.15.107:0/3243697519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05f6020 0x55eeb05f01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.165+0000 7f767f844740 1 --2- 172.21.15.107:0/3243697519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eb760 0x55eeb060ddf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.165+0000 7f767f844740 1 --2- 172.21.15.107:0/3243697519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05eae40 0x55eeb060d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.165+0000 7f767f844740 1 -- 172.21.15.107:0/3243697519 >> 172.21.15.107:0/3243697519 conn(0x55eeb05e73d0 msgr2=0x55eeb05e6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.165+0000 7f767f844740 1 -- 172.21.15.107:0/3243697519 shutdown_connections 2024-09-19T04:27:45.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.165+0000 7f767f844740 1 -- 172.21.15.107:0/3243697519 wait complete. 2024-09-19T04:27:45.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.165+0000 7f767f844740 1 -- start start 2024-09-19T04:27:45.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.165+0000 7f767f844740 1 Processor -- start 2024-09-19T04:27:45.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f767f844740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f767f844740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 0x55eeb05faa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f767f844740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 0x55eeb0678090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f767f844740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76781b0d50 con 0x55eeb05eb760 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f767f844740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76781b0680 con 0x55eeb05eae40 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f767f844740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f76781ad890 con 0x55eeb05f6020 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f767f844740 10 client.0 _ll_register_callbacks cb 0x7f76700099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f7675e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f7675e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05fa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39514/0 (socket says 172.21.15.107:39514) 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f7675e00640 1 -- 172.21.15.107:0/1986000663 learned_addr learned my addr 172.21.15.107:0/1986000663 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f7676800640 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 0x55eeb0678090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.166+0000 7f7675400640 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 0x55eeb05faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7675e00640 1 -- 172.21.15.107:0/1986000663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 msgr2=0x55eeb0678090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7675e00640 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 0x55eeb0678090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7675e00640 1 -- 172.21.15.107:0/1986000663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 msgr2=0x55eeb05faa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7675e00640 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 0x55eeb05faa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7675e00640 1 -- 172.21.15.107:0/1986000663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb06786a0 con 0x55eeb05eae40 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7676800640 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 0x55eeb0678090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7675400640 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 0x55eeb05faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7675e00640 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05fa4e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f76680a4fd0 tx=0x7f76680d8910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ce65e945cff0c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7657e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7657e00640 1 -- 172.21.15.107:0/1986000663 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766809bf30 con 0x55eeb05eae40 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f7657e00640 1 -- 172.21.15.107:0/1986000663 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f766809b840 con 0x55eeb05eae40 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.167+0000 7f767f844740 1 -- 172.21.15.107:0/1986000663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7664005020 con 0x55eeb05eae40 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.168+0000 7f7657e00640 1 -- 172.21.15.107:0/1986000663 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7668098690 con 0x55eeb05eae40 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.168+0000 7f767f844740 10 client.26570 fetch_fsmap learned FSMap version 43 2024-09-19T04:27:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.168+0000 7f767f844740 1 -- 172.21.15.107:0/1986000663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55eeb0679960 con 0x55eeb05eae40 2024-09-19T04:27:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.168+0000 7f7657e00640 1 -- 172.21.15.107:0/1986000663 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7664005020 con 0x55eeb05eae40 2024-09-19T04:27:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.169+0000 7f7657e00640 1 -- 172.21.15.107:0/1986000663 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f7668098140 con 0x55eeb05eae40 2024-09-19T04:27:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.169+0000 7f767f844740 10 client.26570 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:27:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.169+0000 7f767f844740 10 client.26570 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:27:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.169+0000 7f767f844740 20 client.26570 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.169+0000 7f767f844740 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55eeb0679e80 0x55eeb067c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.169+0000 7f767f844740 4 client.26570 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.169+0000 7f767f844740 1 -- 172.21.15.107:0/1986000663 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eeb0558e60 con 0x55eeb0679e80 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.169+0000 7f7675400640 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55eeb0679e80 0x55eeb067c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f7675400640 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55eeb0679e80 0x55eeb067c270 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c4d88295b19045e server_cookie=85483e7dfededeae in_seq=0 out_seq=0 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f7657e00640 10 client.26570 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f7657e00640 1 -- 172.21.15.107:0/1986000663 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eeb0558e60 con 0x55eeb0679e80 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f7657e00640 10 client.26570 handle_command_reply: tid=0 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 client.26570 shutdown 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 -- 172.21.15.107:0/1986000663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 msgr2=0x55eeb05fa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05fa4e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f76680a4fd0 tx=0x7f76680d8910 comp rx=0 tx=0).stop 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 -- 172.21.15.107:0/1986000663 shutdown_connections 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55eeb0679e80 0x55eeb067c270 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55eeb05f6020 0x55eeb0678090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeb05eb760 0x55eeb05faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 --2- 172.21.15.107:0/1986000663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eeb05eae40 0x55eeb05fa4e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 -- 172.21.15.107:0/1986000663 >> 172.21.15.107:0/1986000663 conn(0x55eeb05e73d0 msgr2=0x55eeb05f47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f7657e00640 0 client.26570 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:27:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 -- 172.21.15.107:0/1986000663 shutdown_connections 2024-09-19T04:27:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 1 -- 172.21.15.107:0/1986000663 wait complete. 2024-09-19T04:27:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.170+0000 7f767f844740 20 client.26570 trim_cache size 0 max 16384 2024-09-19T04:27:45.175 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:27:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.171+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f765c0a3430 msgr2=0x7f765c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.171+0000 7f7677200640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f765c0a3430 0x7f765c0a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7670005fd0 tx=0x7f7670009500 comp rx=0 tx=0).stop 2024-09-19T04:27:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.171+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678103e30 msgr2=0x7f767819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.171+0000 7f7677200640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678103e30 0x7f767819a460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f766800e9f0 tx=0x7f766800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.171+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 shutdown_connections 2024-09-19T04:27:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.171+0000 7f7677200640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f765c0a3430 0x7f765c0a58f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.171+0000 7f7677200640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7678108f60 0x7f767819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.171+0000 7f7677200640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76781047e0 0x7f767819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.171+0000 7f7677200640 1 --2- 172.21.15.107:0/1415328997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678103e30 0x7f767819a460 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.171+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 >> 172.21.15.107:0/1415328997 conn(0x7f76780ff8d0 msgr2=0x7f7678100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.172+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 shutdown_connections 2024-09-19T04:27:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:45.172+0000 7f7677200640 1 -- 172.21.15.107:0/1415328997 wait complete. 2024-09-19T04:27:45.684 INFO:tasks.workunit.client.0.smithi107.stdout: 1 238487 2.44 MB/sec execute 243 sec latency 45.398 ms 2024-09-19T04:27:46.031 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:27:46.438 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:46.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 238487 2.43 MB/sec execute 244 sec latency 1045.439 ms 2024-09-19T04:27:46.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:46 smithi107 ceph-mon[30659]: pgmap v228: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 989 KiB/s wr, 120 op/s 2024-09-19T04:27:46.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:46 smithi115 ceph-mon[35477]: pgmap v228: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 989 KiB/s wr, 120 op/s 2024-09-19T04:27:46.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:46 smithi145 ceph-mon[35096]: pgmap v228: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 989 KiB/s wr, 120 op/s 2024-09-19T04:27:47.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.015+0000 7f40ee400640 1 -- 172.21.15.107:0/333279655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40e810e3c0 msgr2=0x7f40e8110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:47.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.015+0000 7f40ee400640 1 --2- 172.21.15.107:0/333279655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40e810e3c0 0x7f40e8110810 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009ca0 tx=0x7f40e002a950 comp rx=0 tx=0).stop 2024-09-19T04:27:47.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.015+0000 7f40ee400640 1 -- 172.21.15.107:0/333279655 shutdown_connections 2024-09-19T04:27:47.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.015+0000 7f40ee400640 1 --2- 172.21.15.107:0/333279655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40e810e3c0 0x7f40e8110810 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.015+0000 7f40ee400640 1 --2- 172.21.15.107:0/333279655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40e8074f40 0x7f40e810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.015+0000 7f40ee400640 1 --2- 172.21.15.107:0/333279655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e8074620 0x7f40e8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.015+0000 7f40ee400640 1 -- 172.21.15.107:0/333279655 >> 172.21.15.107:0/333279655 conn(0x7f40e80fd8c0 msgr2=0x7f40e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:47.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.015+0000 7f40ee400640 1 -- 172.21.15.107:0/333279655 shutdown_connections 2024-09-19T04:27:47.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.015+0000 7f40ee400640 1 -- 172.21.15.107:0/333279655 wait complete. 2024-09-19T04:27:47.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.016+0000 7f40ee400640 1 Processor -- start 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.016+0000 7f40ee400640 1 -- start start 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40ee400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40e8074620 0x7f40e819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40ee400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40e8074f40 0x7f40e819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40ee400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e810e3c0 0x7f40e819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40ee400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81141a0 con 0x7f40e8074f40 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40ee400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8114020 con 0x7f40e810e3c0 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40ee400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8114320 con 0x7f40e8074620 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40e8074f40 0x7f40e819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40e7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e810e3c0 0x7f40e819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40e8074f40 0x7f40e819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37076/0 (socket says 172.21.15.107:37076) 2024-09-19T04:27:47.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40e6a00640 1 -- 172.21.15.107:0/3406153606 learned_addr learned my addr 172.21.15.107:0/3406153606 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40e7400640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40e8074620 0x7f40e819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40e7e00640 1 -- 172.21.15.107:0/3406153606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40e8074620 msgr2=0x7f40e819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40e7e00640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40e8074620 0x7f40e819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40e7e00640 1 -- 172.21.15.107:0/3406153606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40e8074f40 msgr2=0x7f40e819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40e7e00640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40e8074f40 0x7f40e819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.017+0000 7f40e7e00640 1 -- 172.21.15.107:0/3406153606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e819f610 con 0x7f40e810e3c0 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.018+0000 7f40e7e00640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e810e3c0 0x7f40e819ef30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005e30 tx=0x7f40e0003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.018+0000 7f40dfe00640 1 -- 172.21.15.107:0/3406153606 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0034040 con 0x7f40e810e3c0 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.018+0000 7f40dfe00640 1 -- 172.21.15.107:0/3406153606 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e002f940 con 0x7f40e810e3c0 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.018+0000 7f40ee400640 1 -- 172.21.15.107:0/3406153606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e819f8a0 con 0x7f40e810e3c0 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.018+0000 7f40dfe00640 1 -- 172.21.15.107:0/3406153606 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00045e0 con 0x7f40e810e3c0 2024-09-19T04:27:47.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.018+0000 7f40ee400640 1 -- 172.21.15.107:0/3406153606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e81a71e0 con 0x7f40e810e3c0 2024-09-19T04:27:47.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.021+0000 7f40f0e5d740 1 Processor -- start 2024-09-19T04:27:47.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40dfe00640 1 -- 172.21.15.107:0/3406153606 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f40e0037020 con 0x7f40e810e3c0 2024-09-19T04:27:47.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40f0e5d740 1 -- start start 2024-09-19T04:27:47.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b7c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b86910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b89240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c8d6ae23e0 con 0x55c8d6b7bc70 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c8d6a89d80 con 0x55c8d6b86e50 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c8d6ade3a0 con 0x55c8d6b7c590 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b7c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b7c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37088/0 (socket says 172.21.15.107:37088) 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40e6a00640 1 -- 172.21.15.107:0/3121071643 learned_addr learned my addr 172.21.15.107:0/3121071643 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.022+0000 7f40e7400640 1 --2- 172.21.15.107:0/3121071643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b86910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.023+0000 7f40e6a00640 1 --2- 172.21.15.107:0/3121071643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b89240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.023+0000 7f40e7400640 1 -- 172.21.15.107:0/3121071643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 msgr2=0x55c8d6b89240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.023+0000 7f40e7400640 1 --2- 172.21.15.107:0/3121071643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b89240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.023+0000 7f40e7400640 1 -- 172.21.15.107:0/3121071643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 msgr2=0x55c8d6b7c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.023+0000 7f40e7400640 1 --2- 172.21.15.107:0/3121071643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b7c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.023+0000 7f40e7400640 1 -- 172.21.15.107:0/3121071643 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d6aa8ae0 con 0x55c8d6b7c590 2024-09-19T04:27:47.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.023+0000 7f40e6a00640 1 --2- 172.21.15.107:0/3121071643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b7c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:47.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.023+0000 7f40e7400640 1 --2- 172.21.15.107:0/3121071643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b86910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009c20 tx=0x7f40d8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afc36bea4eb8f2c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:47.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.023+0000 7f40dfe00640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40cc0a3430 0x7f40cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.023+0000 7f40dcc00640 1 -- 172.21.15.107:0/3121071643 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8004050 con 0x55c8d6b7c590 2024-09-19T04:27:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40dcc00640 1 -- 172.21.15.107:0/3121071643 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40d8005500 con 0x55c8d6b7c590 2024-09-19T04:27:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40dcc00640 1 -- 172.21.15.107:0/3121071643 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d80035f0 con 0x55c8d6b7c590 2024-09-19T04:27:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40dfe00640 1 -- 172.21.15.107:0/3406153606 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f40e00f0890 con 0x7f40e810e3c0 2024-09-19T04:27:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40e6a00640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40cc0a3430 0x7f40cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3121071643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 msgr2=0x55c8d6b86910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3121071643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b86910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009c20 tx=0x7f40d8011810 comp rx=0 tx=0).stop 2024-09-19T04:27:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3121071643 shutdown_connections 2024-09-19T04:27:47.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3121071643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b89240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3121071643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b86910 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3121071643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b7c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3121071643 >> 172.21.15.107:0/3121071643 conn(0x55c8d6b78200 msgr2=0x55c8d6b785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:47.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40e6a00640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40cc0a3430 0x7f40cc0a58f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009620 tx=0x7f40d4009550 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:47.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.024+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3121071643 shutdown_connections 2024-09-19T04:27:47.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.025+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3121071643 wait complete. 2024-09-19T04:27:47.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.025+0000 7f40f0e5d740 1 -- start start 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.025+0000 7f40f0e5d740 1 Processor -- start 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b83ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b84720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c8d6b89a20 con 0x55c8d6b7bc70 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c8d6a9d0f0 con 0x55c8d6b86e50 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c8d6b89ba0 con 0x55c8d6b7c590 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40f0e5d740 10 client.0 _ll_register_callbacks cb 0x55c8d6a84bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b83ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b83ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37100/0 (socket says 172.21.15.107:37100) 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b84720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e7400640 1 -- 172.21.15.107:0/3886873761 learned_addr learned my addr 172.21.15.107:0/3886873761 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e7400640 1 -- 172.21.15.107:0/3886873761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 msgr2=0x55c8d6b841e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e7400640 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b841e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e7400640 1 -- 172.21.15.107:0/3886873761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 msgr2=0x55c8d6b84720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e7400640 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b84720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e7400640 1 -- 172.21.15.107:0/3886873761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d68533c0 con 0x55c8d6b7bc70 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.026+0000 7f40e7e00640 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b84720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.027+0000 7f40e7400640 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b83ca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f40d8000c00 tx=0x7f40d8004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55ac6f8c0126aba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.027+0000 7f40d1600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.027+0000 7f40d1600640 1 -- 172.21.15.107:0/3886873761 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8029070 con 0x55c8d6b7bc70 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.027+0000 7f40d1600640 1 -- 172.21.15.107:0/3886873761 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f40d8005250 con 0x55c8d6b7bc70 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.027+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3886873761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40d8005500 con 0x55c8d6b7bc70 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.027+0000 7f40d1600640 1 -- 172.21.15.107:0/3886873761 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f40d8035050 con 0x55c8d6b7bc70 2024-09-19T04:27:47.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.027+0000 7f40f0e5d740 10 client.16533 fetch_fsmap learned FSMap version 43 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.027+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3886873761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c8d6b95a20 con 0x55c8d6b7bc70 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.028+0000 7f40d1600640 1 -- 172.21.15.107:0/3886873761 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40d8005500 con 0x55c8d6b7bc70 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.028+0000 7f40d1600640 1 -- 172.21.15.107:0/3886873761 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f40d8023430 con 0x55c8d6b7bc70 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.028+0000 7f40f0e5d740 10 client.16533 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.028+0000 7f40f0e5d740 10 client.16533 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.028+0000 7f40f0e5d740 20 client.16533 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.028+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55c8d6b96d70 0x55c8d6b99160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.028+0000 7f40f0e5d740 4 client.16533 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.028+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3886873761 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8d6aef950 con 0x55c8d6b96d70 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.028+0000 7f40e6a00640 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55c8d6b96d70 0x55c8d6b99160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.029+0000 7f40e6a00640 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55c8d6b96d70 0x55c8d6b99160 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bcb5f7bfaa76230 server_cookie=fcbad4d428ee184d in_seq=0 out_seq=0 2024-09-19T04:27:47.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.029+0000 7f40d1600640 10 client.16533 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:27:47.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.029+0000 7f40d1600640 1 -- 172.21.15.107:0/3886873761 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8d6aef950 con 0x55c8d6b96d70 2024-09-19T04:27:47.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.029+0000 7f40d1600640 10 client.16533 handle_command_reply: tid=0 2024-09-19T04:27:47.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 client.16533 shutdown 2024-09-19T04:27:47.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3886873761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 msgr2=0x55c8d6b83ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:47.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b83ca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f40d8000c00 tx=0x7f40d8004e50 comp rx=0 tx=0).stop 2024-09-19T04:27:47.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3886873761 shutdown_connections 2024-09-19T04:27:47.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55c8d6b96d70 0x55c8d6b99160 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b84720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b841e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3886873761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b83ca0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40d1600640 0 client.16533 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:27:47.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3886873761 >> 172.21.15.107:0/3886873761 conn(0x55c8d6b78200 msgr2=0x55c8d6b7df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:47.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3886873761 shutdown_connections 2024-09-19T04:27:47.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3886873761 wait complete. 2024-09-19T04:27:47.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.030+0000 7f40f0e5d740 20 client.16533 trim_cache size 0 max 16384 2024-09-19T04:27:47.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.044+0000 7f40f0e5d740 1 Processor -- start 2024-09-19T04:27:47.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.044+0000 7f40f0e5d740 1 -- start start 2024-09-19T04:27:47.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.044+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b7e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.044+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b9e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.044+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b9ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.044+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81af9d0 con 0x55c8d6b7bc70 2024-09-19T04:27:47.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.044+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81b15d0 con 0x55c8d6b86e50 2024-09-19T04:27:47.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.044+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81b00d0 con 0x55c8d6b7c590 2024-09-19T04:27:47.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.044+0000 7f40e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b7e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40e7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b9ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40e7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b9ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39568/0 (socket says 172.21.15.107:39568) 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40e7e00640 1 -- 172.21.15.107:0/2529189732 learned_addr learned my addr 172.21.15.107:0/2529189732 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40e6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b9e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40e7e00640 1 -- 172.21.15.107:0/2529189732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 msgr2=0x55c8d6b9e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40e7e00640 1 --2- 172.21.15.107:0/2529189732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b9e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40e7e00640 1 -- 172.21.15.107:0/2529189732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 msgr2=0x55c8d6b7e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40e7e00640 1 --2- 172.21.15.107:0/2529189732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b7e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40e7e00640 1 -- 172.21.15.107:0/2529189732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d6b78030 con 0x55c8d6b86e50 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40e7e00640 1 --2- 172.21.15.107:0/2529189732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b9ee80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40e00b9610 tx=0x7f40e00bc800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c7798180d9cca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40d3e00640 1 -- 172.21.15.107:0/2529189732 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00bcc40 con 0x55c8d6b86e50 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40d3e00640 1 -- 172.21.15.107:0/2529189732 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e00c0ff0 con 0x55c8d6b86e50 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.045+0000 7f40d3e00640 1 -- 172.21.15.107:0/2529189732 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00c5420 con 0x55c8d6b86e50 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.046+0000 7f40f0e5d740 1 -- 172.21.15.107:0/2529189732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 msgr2=0x55c8d6b9ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.046+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/2529189732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b9ee80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40e00b9610 tx=0x7f40e00bc800 comp rx=0 tx=0).stop 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.046+0000 7f40f0e5d740 1 -- 172.21.15.107:0/2529189732 shutdown_connections 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.046+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/2529189732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b86e50 0x55c8d6b9ee80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.046+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/2529189732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b7c590 0x55c8d6b9e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.046+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/2529189732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b7e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.046+0000 7f40f0e5d740 1 -- 172.21.15.107:0/2529189732 >> 172.21.15.107:0/2529189732 conn(0x55c8d6b78200 msgr2=0x55c8d6b85d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.046+0000 7f40f0e5d740 1 -- 172.21.15.107:0/2529189732 shutdown_connections 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.046+0000 7f40f0e5d740 1 -- 172.21.15.107:0/2529189732 wait complete. 2024-09-19T04:27:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.047+0000 7f40f0e5d740 1 -- start start 2024-09-19T04:27:47.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.047+0000 7f40f0e5d740 1 Processor -- start 2024-09-19T04:27:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.047+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b8f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.047+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b7c590 0x55c8d6b8f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.047+0000 7f40f0e5d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b86e50 0x55c8d6b80450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.047+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81b0e70 con 0x55c8d6b7bc70 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.047+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81b07a0 con 0x55c8d6b7c590 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.047+0000 7f40f0e5d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81ad9d0 con 0x55c8d6b86e50 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.047+0000 7f40f0e5d740 10 client.0 _ll_register_callbacks cb 0x7f40d8009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b86e50 0x55c8d6b80450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b7c590 0x55c8d6b8f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b8f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b7c590 0x55c8d6b8f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39582/0 (socket says 172.21.15.107:39582) 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b86e50 0x55c8d6b80450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:60820/0 (socket says 172.21.15.107:60820) 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e6a00640 1 -- 172.21.15.107:0/3831489907 learned_addr learned my addr 172.21.15.107:0/3831489907 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e6a00640 1 -- 172.21.15.107:0/3831489907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b86e50 msgr2=0x55c8d6b80450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e6a00640 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b86e50 0x55c8d6b80450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e6a00640 1 -- 172.21.15.107:0/3831489907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 msgr2=0x55c8d6b8f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e6a00640 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b8f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e6a00640 1 -- 172.21.15.107:0/3831489907 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d6b73770 con 0x55c8d6b7c590 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.048+0000 7f40e7400640 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b8f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.049+0000 7f40e6a00640 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b7c590 0x55c8d6b8f570 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009bd0 tx=0x7f40d4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58d8c4341b10b4c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.049+0000 7f40d1600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.049+0000 7f40d1600640 1 -- 172.21.15.107:0/3831489907 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4032070 con 0x55c8d6b7c590 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.049+0000 7f40d1600640 1 -- 172.21.15.107:0/3831489907 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f40d402d630 con 0x55c8d6b7c590 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.049+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3831489907 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e00c0ff0 con 0x55c8d6b7c590 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.049+0000 7f40d1600640 1 -- 172.21.15.107:0/3831489907 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f40d403b050 con 0x55c8d6b7c590 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.049+0000 7f40f0e5d740 10 client.26588 fetch_fsmap learned FSMap version 43 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.049+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3831489907 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c8d6c01180 con 0x55c8d6b7c590 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.049+0000 7f40d1600640 1 -- 172.21.15.107:0/3831489907 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e00c0ff0 con 0x55c8d6b7c590 2024-09-19T04:27:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.050+0000 7f40d1600640 1 -- 172.21.15.107:0/3831489907 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f40d4035020 con 0x55c8d6b7c590 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.050+0000 7f40f0e5d740 10 client.26588 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.050+0000 7f40f0e5d740 10 client.26588 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.050+0000 7f40f0e5d740 20 client.26588 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.050+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55c8d6c01930 0x55c8d6c03d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.050+0000 7f40f0e5d740 4 client.26588 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.050+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3831489907 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8d6ae9e60 con 0x55c8d6c01930 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.050+0000 7f40e7400640 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55c8d6c01930 0x55c8d6c03d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.050+0000 7f40e7400640 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55c8d6c01930 0x55c8d6c03d50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c431732c843e6a19 server_cookie=e82daccb7ca3d59e in_seq=0 out_seq=0 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.050+0000 7f40d1600640 10 client.26588 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40d1600640 1 -- 172.21.15.107:0/3831489907 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8d6ae9e60 con 0x55c8d6c01930 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40d1600640 10 client.26588 handle_command_reply: tid=0 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 client.26588 shutdown 2024-09-19T04:27:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3831489907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b7c590 msgr2=0x55c8d6b8f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b7c590 0x55c8d6b8f570 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009bd0 tx=0x7f40d4004730 comp rx=0 tx=0).stop 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3831489907 shutdown_connections 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55c8d6c01930 0x55c8d6c03d50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8d6b86e50 0x55c8d6b80450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8d6b7c590 0x55c8d6b8f570 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 --2- 172.21.15.107:0/3831489907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c8d6b7bc70 0x55c8d6b8f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40d1600640 0 client.26588 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3831489907 >> 172.21.15.107:0/3831489907 conn(0x55c8d6b78200 msgr2=0x55c8d6b82440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3831489907 shutdown_connections 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 1 -- 172.21.15.107:0/3831489907 wait complete. 2024-09-19T04:27:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.051+0000 7f40f0e5d740 20 client.26588 trim_cache size 0 max 16384 2024-09-19T04:27:47.058 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:27:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 -- 172.21.15.107:0/3406153606 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40cc0a3430 msgr2=0x7f40cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40cc0a3430 0x7f40cc0a58f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009620 tx=0x7f40d4009550 comp rx=0 tx=0).stop 2024-09-19T04:27:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 -- 172.21.15.107:0/3406153606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e810e3c0 msgr2=0x7f40e819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e810e3c0 0x7f40e819ef30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005e30 tx=0x7f40e0003b80 comp rx=0 tx=0).stop 2024-09-19T04:27:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 -- 172.21.15.107:0/3406153606 shutdown_connections 2024-09-19T04:27:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40cc0a3430 0x7f40cc0a58f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e810e3c0 0x7f40e819ef30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40e8074f40 0x7f40e819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 --2- 172.21.15.107:0/3406153606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40e8074620 0x7f40e819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 -- 172.21.15.107:0/3406153606 >> 172.21.15.107:0/3406153606 conn(0x7f40e80fd8c0 msgr2=0x7f40e80fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 -- 172.21.15.107:0/3406153606 shutdown_connections 2024-09-19T04:27:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:47.052+0000 7f40ee400640 1 -- 172.21.15.107:0/3406153606 wait complete. 2024-09-19T04:27:47.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 238487 2.42 MB/sec execute 245 sec latency 2045.487 ms 2024-09-19T04:27:47.901 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:27:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:48.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 238487 2.41 MB/sec execute 246 sec latency 3045.537 ms 2024-09-19T04:27:48.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.825+0000 7fd584400640 1 -- 172.21.15.107:0/489782646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c106f10 msgr2=0x7fd57c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:48.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.825+0000 7fd584400640 1 --2- 172.21.15.107:0/489782646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c106f10 0x7fd57c10ea90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd574009ca0 tx=0x7fd57402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.825+0000 7fd584400640 1 -- 172.21.15.107:0/489782646 shutdown_connections 2024-09-19T04:27:48.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.825+0000 7fd584400640 1 --2- 172.21.15.107:0/489782646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c106f10 0x7fd57c10ea90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.825+0000 7fd584400640 1 --2- 172.21.15.107:0/489782646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd57c102790 0x7fd57c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.825+0000 7fd584400640 1 --2- 172.21.15.107:0/489782646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd57c101de0 0x7fd57c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.825+0000 7fd584400640 1 -- 172.21.15.107:0/489782646 >> 172.21.15.107:0/489782646 conn(0x7fd57c0fd8c0 msgr2=0x7fd57c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:48.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.826+0000 7fd584400640 1 -- 172.21.15.107:0/489782646 shutdown_connections 2024-09-19T04:27:48.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.826+0000 7fd584400640 1 -- 172.21.15.107:0/489782646 wait complete. 2024-09-19T04:27:48.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.826+0000 7fd584400640 1 Processor -- start 2024-09-19T04:27:48.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd584400640 1 -- start start 2024-09-19T04:27:48.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd584400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd57c101de0 0x7fd57c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd584400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd57c102790 0x7fd57c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd584400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c106f10 0x7fd57c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd584400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c111fc0 con 0x7fd57c106f10 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd584400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c111e40 con 0x7fd57c101de0 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd584400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c112140 con 0x7fd57c102790 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd581a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd57c101de0 0x7fd57c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd582400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c106f10 0x7fd57c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd581000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd57c102790 0x7fd57c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd581a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd57c101de0 0x7fd57c19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39604/0 (socket says 172.21.15.107:39604) 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd581000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd57c102790 0x7fd57c19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:60848/0 (socket says 172.21.15.107:60848) 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.827+0000 7fd581a00640 1 -- 172.21.15.107:0/3800079560 learned_addr learned my addr 172.21.15.107:0/3800079560 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd581a00640 1 -- 172.21.15.107:0/3800079560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd57c102790 msgr2=0x7fd57c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:48.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd581a00640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd57c102790 0x7fd57c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd581a00640 1 -- 172.21.15.107:0/3800079560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c106f10 msgr2=0x7fd57c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd581a00640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c106f10 0x7fd57c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd581a00640 1 -- 172.21.15.107:0/3800079560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c19f610 con 0x7fd57c101de0 2024-09-19T04:27:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd582400640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c106f10 0x7fd57c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd581a00640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd57c101de0 0x7fd57c19a660 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd57000e9f0 tx=0x7fd57000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd57a000640 1 -- 172.21.15.107:0/3800079560 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57000cd80 con 0x7fd57c101de0 2024-09-19T04:27:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd584400640 1 -- 172.21.15.107:0/3800079560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c19f900 con 0x7fd57c101de0 2024-09-19T04:27:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd57a000640 1 -- 172.21.15.107:0/3800079560 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd570005500 con 0x7fd57c101de0 2024-09-19T04:27:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.828+0000 7fd57a000640 1 -- 172.21.15.107:0/3800079560 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570010690 con 0x7fd57c101de0 2024-09-19T04:27:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.829+0000 7fd584400640 1 -- 172.21.15.107:0/3800079560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c1a71e0 con 0x7fd57c101de0 2024-09-19T04:27:48.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.833+0000 7fd586e50740 1 Processor -- start 2024-09-19T04:27:48.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.833+0000 7fd586e50740 1 -- start start 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.833+0000 7fd57a000640 1 -- 172.21.15.107:0/3800079560 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fd5700046c0 con 0x7fd57c101de0 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.833+0000 7fd586e50740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 0x55ef8874d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.833+0000 7fd586e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef88757910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.833+0000 7fd586e50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef8875a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.833+0000 7fd586e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ef886b33e0 con 0x55ef8874d590 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.833+0000 7fd586e50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ef8865ad80 con 0x55ef88757e50 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.833+0000 7fd586e50740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ef886af3a0 con 0x55ef8874cc70 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.834+0000 7fd581a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef88757910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.834+0000 7fd581000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef8875a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.834+0000 7fd581000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef8875a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39620/0 (socket says 172.21.15.107:39620) 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.834+0000 7fd581000640 1 -- 172.21.15.107:0/738255010 learned_addr learned my addr 172.21.15.107:0/738255010 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:48.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.834+0000 7fd582400640 1 --2- 172.21.15.107:0/738255010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 0x55ef8874d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.834+0000 7fd581000640 1 -- 172.21.15.107:0/738255010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 msgr2=0x55ef8874d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.834+0000 7fd581000640 1 --2- 172.21.15.107:0/738255010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 0x55ef8874d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.834+0000 7fd581000640 1 -- 172.21.15.107:0/738255010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 msgr2=0x55ef88757910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.834+0000 7fd581000640 1 --2- 172.21.15.107:0/738255010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef88757910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.834+0000 7fd581000640 1 -- 172.21.15.107:0/738255010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef88679ae0 con 0x55ef88757e50 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.835+0000 7fd581000640 1 --2- 172.21.15.107:0/738255010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef8875a240 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd568009c10 tx=0x7fd568011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568d85a4f04d7013 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.835+0000 7fd56ea00640 1 -- 172.21.15.107:0/738255010 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568004230 con 0x55ef88757e50 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.835+0000 7fd56ea00640 1 -- 172.21.15.107:0/738255010 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5680043d0 con 0x55ef88757e50 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.835+0000 7fd56ea00640 1 -- 172.21.15.107:0/738255010 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568004e60 con 0x55ef88757e50 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.835+0000 7fd57a000640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5640a3430 0x7fd5640a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.835+0000 7fd57a000640 1 -- 172.21.15.107:0/3800079560 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7fd5700d00e0 con 0x7fd57c101de0 2024-09-19T04:27:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.835+0000 7fd582400640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5640a3430 0x7fd5640a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.835+0000 7fd586e50740 1 -- 172.21.15.107:0/738255010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 msgr2=0x55ef8875a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.835+0000 7fd586e50740 1 --2- 172.21.15.107:0/738255010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef8875a240 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd568009c10 tx=0x7fd568011810 comp rx=0 tx=0).stop 2024-09-19T04:27:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.836+0000 7fd586e50740 1 -- 172.21.15.107:0/738255010 shutdown_connections 2024-09-19T04:27:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.836+0000 7fd586e50740 1 --2- 172.21.15.107:0/738255010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef8875a240 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.836+0000 7fd586e50740 1 --2- 172.21.15.107:0/738255010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef88757910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.836+0000 7fd586e50740 1 --2- 172.21.15.107:0/738255010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 0x55ef8874d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.836+0000 7fd586e50740 1 -- 172.21.15.107:0/738255010 >> 172.21.15.107:0/738255010 conn(0x55ef88749200 msgr2=0x55ef887495e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.836+0000 7fd586e50740 1 -- 172.21.15.107:0/738255010 shutdown_connections 2024-09-19T04:27:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.836+0000 7fd582400640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5640a3430 0x7fd5640a58f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd574006010 tx=0x7fd574005d80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.836+0000 7fd586e50740 1 -- 172.21.15.107:0/738255010 wait complete. 2024-09-19T04:27:48.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.836+0000 7fd586e50740 1 -- start start 2024-09-19T04:27:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.837+0000 7fd586e50740 1 Processor -- start 2024-09-19T04:27:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.837+0000 7fd586e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874cc70 0x55ef88754cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.837+0000 7fd586e50740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874d590 0x55ef887551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.837+0000 7fd586e50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef88755730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.837+0000 7fd586e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ef8875aa20 con 0x55ef8874cc70 2024-09-19T04:27:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.837+0000 7fd586e50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ef8866e0f0 con 0x55ef88757e50 2024-09-19T04:27:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.837+0000 7fd586e50740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ef8875aba0 con 0x55ef8874d590 2024-09-19T04:27:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.837+0000 7fd586e50740 10 client.0 _ll_register_callbacks cb 0x55ef88655bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.837+0000 7fd581a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874cc70 0x55ef88754cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.837+0000 7fd581a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874cc70 0x55ef88754cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37134/0 (socket says 172.21.15.107:37134) 2024-09-19T04:27:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd581a00640 1 -- 172.21.15.107:0/4016392051 learned_addr learned my addr 172.21.15.107:0/4016392051 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd581000640 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874d590 0x55ef887551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd582400640 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef88755730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd581a00640 1 -- 172.21.15.107:0/4016392051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874d590 msgr2=0x55ef887551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd581a00640 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874d590 0x55ef887551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd581a00640 1 -- 172.21.15.107:0/4016392051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 msgr2=0x55ef88755730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd581a00640 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef88755730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd581a00640 1 -- 172.21.15.107:0/4016392051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef884243c0 con 0x55ef8874cc70 2024-09-19T04:27:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd581a00640 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874cc70 0x55ef88754cb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd57000e9c0 tx=0x7fd570004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b8f73923fd0c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd563400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd563400640 1 -- 172.21.15.107:0/4016392051 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57009bec0 con 0x55ef8874cc70 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd563400640 1 -- 172.21.15.107:0/4016392051 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7fd5700d2110 con 0x55ef8874cc70 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.838+0000 7fd586e50740 1 -- 172.21.15.107:0/4016392051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5680043d0 con 0x55ef8874cc70 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd563400640 1 -- 172.21.15.107:0/4016392051 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd57009bce0 con 0x55ef8874cc70 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd586e50740 10 client.16551 fetch_fsmap learned FSMap version 44 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd586e50740 1 -- 172.21.15.107:0/4016392051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ef88766ac0 con 0x55ef8874cc70 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd563400640 1 -- 172.21.15.107:0/4016392051 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5680043d0 con 0x55ef8874cc70 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd563400640 1 -- 172.21.15.107:0/4016392051 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fd5700b5100 con 0x55ef8874cc70 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd586e50740 10 client.16551 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd586e50740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd586e50740 20 client.16551 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd586e50740 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ef88767f80 0x55ef8876a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd586e50740 4 client.16551 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.839+0000 7fd586e50740 1 -- 172.21.15.107:0/4016392051 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef886c0950 con 0x55ef88767f80 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.840+0000 7fd581000640 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ef88767f80 0x55ef8876a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.840+0000 7fd581000640 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ef88767f80 0x55ef8876a3d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d42706db4be9287 server_cookie=6fa25c1b5a126d34 in_seq=0 out_seq=0 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.840+0000 7fd563400640 10 client.16551 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.841+0000 7fd563400640 1 -- 172.21.15.107:0/4016392051 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef886c0950 con 0x55ef88767f80 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.841+0000 7fd563400640 10 client.16551 handle_command_reply: tid=0 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.841+0000 7fd586e50740 1 client.16551 shutdown 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.841+0000 7fd586e50740 1 -- 172.21.15.107:0/4016392051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874cc70 msgr2=0x55ef88754cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.841+0000 7fd586e50740 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874cc70 0x55ef88754cb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd57000e9c0 tx=0x7fd570004ae0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.842+0000 7fd586e50740 1 -- 172.21.15.107:0/4016392051 shutdown_connections 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.842+0000 7fd586e50740 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ef88767f80 0x55ef8876a3d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.842+0000 7fd586e50740 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef88755730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.842+0000 7fd586e50740 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874d590 0x55ef887551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.842+0000 7fd586e50740 1 --2- 172.21.15.107:0/4016392051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874cc70 0x55ef88754cb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.842+0000 7fd586e50740 1 -- 172.21.15.107:0/4016392051 >> 172.21.15.107:0/4016392051 conn(0x55ef88749200 msgr2=0x55ef8874ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.842+0000 7fd563400640 0 client.16551 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.842+0000 7fd586e50740 1 -- 172.21.15.107:0/4016392051 shutdown_connections 2024-09-19T04:27:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.842+0000 7fd586e50740 1 -- 172.21.15.107:0/4016392051 wait complete. 2024-09-19T04:27:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.842+0000 7fd586e50740 20 client.16551 trim_cache size 0 max 16384 2024-09-19T04:27:48.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.855+0000 7fd586e50740 1 Processor -- start 2024-09-19T04:27:48.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.855+0000 7fd586e50740 1 -- start start 2024-09-19T04:27:48.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd586e50740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 0x55ef8874f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd586e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef88756c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd586e50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef8876fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd586e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c1afa70 con 0x55ef8874d590 2024-09-19T04:27:48.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd586e50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c1b1610 con 0x55ef88757e50 2024-09-19T04:27:48.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd586e50740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c1b0170 con 0x55ef8874cc70 2024-09-19T04:27:48.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd581000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef88756c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd581000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef88756c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43594/0 (socket says 172.21.15.107:43594) 2024-09-19T04:27:48.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd581000640 1 -- 172.21.15.107:0/957276887 learned_addr learned my addr 172.21.15.107:0/957276887 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:48.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd582400640 1 --2- 172.21.15.107:0/957276887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef8876fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd581a00640 1 --2- 172.21.15.107:0/957276887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 0x55ef8874f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd581000640 1 -- 172.21.15.107:0/957276887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 msgr2=0x55ef8874f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:48.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd581000640 1 --2- 172.21.15.107:0/957276887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 0x55ef8874f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd581000640 1 -- 172.21.15.107:0/957276887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 msgr2=0x55ef8876fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd581000640 1 --2- 172.21.15.107:0/957276887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef8876fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd581000640 1 -- 172.21.15.107:0/957276887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef8874caa0 con 0x55ef8874d590 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.856+0000 7fd581a00640 1 --2- 172.21.15.107:0/957276887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 0x55ef8874f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.857+0000 7fd581000640 1 --2- 172.21.15.107:0/957276887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef88756c20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd568004990 tx=0x7fd568004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb7e2b15c291c5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.857+0000 7fd56e000640 1 -- 172.21.15.107:0/957276887 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56801e040 con 0x55ef8874d590 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.857+0000 7fd56e000640 1 -- 172.21.15.107:0/957276887 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd56801f040 con 0x55ef8874d590 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.857+0000 7fd56e000640 1 -- 172.21.15.107:0/957276887 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568026710 con 0x55ef8874d590 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.857+0000 7fd586e50740 1 -- 172.21.15.107:0/957276887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 msgr2=0x55ef88756c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.857+0000 7fd586e50740 1 --2- 172.21.15.107:0/957276887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef88756c20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd568004990 tx=0x7fd568004920 comp rx=0 tx=0).stop 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.858+0000 7fd586e50740 1 -- 172.21.15.107:0/957276887 shutdown_connections 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.858+0000 7fd586e50740 1 --2- 172.21.15.107:0/957276887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef88757e50 0x55ef8876fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.858+0000 7fd586e50740 1 --2- 172.21.15.107:0/957276887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef88756c20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.858+0000 7fd586e50740 1 --2- 172.21.15.107:0/957276887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef8874cc70 0x55ef8874f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.858+0000 7fd586e50740 1 -- 172.21.15.107:0/957276887 >> 172.21.15.107:0/957276887 conn(0x55ef88749200 msgr2=0x55ef88756510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.858+0000 7fd586e50740 1 -- 172.21.15.107:0/957276887 shutdown_connections 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.858+0000 7fd586e50740 1 -- 172.21.15.107:0/957276887 wait complete. 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.858+0000 7fd586e50740 1 -- start start 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.858+0000 7fd586e50740 1 Processor -- start 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd586e50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef8874cc70 0x55ef88760390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd586e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef8875b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd586e50740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef88757e50 0x55ef8875bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd586e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c1b0f10 con 0x55ef8874d590 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd586e50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c1b0840 con 0x55ef8874cc70 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd586e50740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c1ada70 con 0x55ef88757e50 2024-09-19T04:27:48.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd586e50740 10 client.0 _ll_register_callbacks cb 0x7fd5700d3880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:48.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef8875b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef8874cc70 0x55ef88760390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef8874cc70 0x55ef88760390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57644/0 (socket says 172.21.15.107:57644) 2024-09-19T04:27:48.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581a00640 1 -- 172.21.15.107:0/2261340886 learned_addr learned my addr 172.21.15.107:0/2261340886 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:48.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef8875b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43602/0 (socket says 172.21.15.107:43602) 2024-09-19T04:27:48.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd582400640 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef88757e50 0x55ef8875bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581000640 1 -- 172.21.15.107:0/2261340886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef88757e50 msgr2=0x55ef8875bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:48.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581000640 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef88757e50 0x55ef8875bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581000640 1 -- 172.21.15.107:0/2261340886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef8874cc70 msgr2=0x55ef88760390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:48.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581000640 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef8874cc70 0x55ef88760390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581000640 1 -- 172.21.15.107:0/2261340886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef88744770 con 0x55ef8874d590 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd581a00640 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef8874cc70 0x55ef88760390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.859+0000 7fd582400640 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef88757e50 0x55ef8875bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.860+0000 7fd581000640 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef8875b710 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd5680048f0 tx=0x7fd568009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a21b8f62f6c91b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.860+0000 7fd563400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.860+0000 7fd563400640 1 -- 172.21.15.107:0/2261340886 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568038070 con 0x55ef8874d590 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.860+0000 7fd586e50740 1 -- 172.21.15.107:0/2261340886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd56801f040 con 0x55ef8874d590 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.860+0000 7fd563400640 1 -- 172.21.15.107:0/2261340886 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7fd568004240 con 0x55ef8874d590 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.860+0000 7fd563400640 1 -- 172.21.15.107:0/2261340886 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd568043050 con 0x55ef8874d590 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.860+0000 7fd586e50740 10 client.16557 fetch_fsmap learned FSMap version 44 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.860+0000 7fd586e50740 1 -- 172.21.15.107:0/2261340886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ef887db090 con 0x55ef8874d590 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.861+0000 7fd563400640 1 -- 172.21.15.107:0/2261340886 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd56801f040 con 0x55ef8874d590 2024-09-19T04:27:48.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.861+0000 7fd563400640 1 -- 172.21.15.107:0/2261340886 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fd56802e020 con 0x55ef8874d590 2024-09-19T04:27:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.861+0000 7fd586e50740 10 client.16557 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:27:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.861+0000 7fd586e50740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.861+0000 7fd586e50740 20 client.16557 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.861+0000 7fd586e50740 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ef887db560 0x55ef887dd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.861+0000 7fd586e50740 4 client.16557 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:27:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.861+0000 7fd586e50740 1 -- 172.21.15.107:0/2261340886 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef886bae60 con 0x55ef887db560 2024-09-19T04:27:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.861+0000 7fd581a00640 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ef887db560 0x55ef887dd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd581a00640 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ef887db560 0x55ef887dd950 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1567216ff96980 server_cookie=64503361097b2c2e in_seq=0 out_seq=0 2024-09-19T04:27:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd563400640 10 client.16557 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd563400640 1 -- 172.21.15.107:0/2261340886 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef886bae60 con 0x55ef887db560 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd563400640 10 client.16557 handle_command_reply: tid=0 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd586e50740 1 client.16557 shutdown 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd586e50740 1 -- 172.21.15.107:0/2261340886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 msgr2=0x55ef8875b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd586e50740 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef8875b710 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd5680048f0 tx=0x7fd568009b40 comp rx=0 tx=0).stop 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd586e50740 1 -- 172.21.15.107:0/2261340886 shutdown_connections 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd586e50740 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ef887db560 0x55ef887dd950 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd586e50740 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef88757e50 0x55ef8875bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd586e50740 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef8874d590 0x55ef8875b710 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd586e50740 1 --2- 172.21.15.107:0/2261340886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef8874cc70 0x55ef88760390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd563400640 0 client.16557 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd586e50740 1 -- 172.21.15.107:0/2261340886 >> 172.21.15.107:0/2261340886 conn(0x55ef88749200 msgr2=0x55ef88747b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.862+0000 7fd586e50740 1 -- 172.21.15.107:0/2261340886 shutdown_connections 2024-09-19T04:27:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd586e50740 1 -- 172.21.15.107:0/2261340886 wait complete. 2024-09-19T04:27:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd586e50740 20 client.16557 trim_cache size 0 max 16384 2024-09-19T04:27:48.873 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:27:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd584400640 1 -- 172.21.15.107:0/3800079560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5640a3430 msgr2=0x7fd5640a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd584400640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5640a3430 0x7fd5640a58f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd574006010 tx=0x7fd574005d80 comp rx=0 tx=0).stop 2024-09-19T04:27:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd584400640 1 -- 172.21.15.107:0/3800079560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd57c101de0 msgr2=0x7fd57c19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd584400640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd57c101de0 0x7fd57c19a660 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd57000e9f0 tx=0x7fd57000edf0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd584400640 1 -- 172.21.15.107:0/3800079560 shutdown_connections 2024-09-19T04:27:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd584400640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5640a3430 0x7fd5640a58f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd584400640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c106f10 0x7fd57c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd584400640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd57c102790 0x7fd57c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd584400640 1 --2- 172.21.15.107:0/3800079560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd57c101de0 0x7fd57c19a660 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.863+0000 7fd584400640 1 -- 172.21.15.107:0/3800079560 >> 172.21.15.107:0/3800079560 conn(0x7fd57c0fd8c0 msgr2=0x7fd57c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.864+0000 7fd584400640 1 -- 172.21.15.107:0/3800079560 shutdown_connections 2024-09-19T04:27:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:48.864+0000 7fd584400640 1 -- 172.21.15.107:0/3800079560 wait complete. 2024-09-19T04:27:48.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:48 smithi115 ceph-mon[35477]: pgmap v229: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 805 KiB/s wr, 88 op/s 2024-09-19T04:27:48.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:48 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:48.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:48 smithi145 ceph-mon[35096]: pgmap v229: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 805 KiB/s wr, 88 op/s 2024-09-19T04:27:48.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:48 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:48.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:48 smithi107 ceph-mon[30659]: pgmap v229: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 805 KiB/s wr, 88 op/s 2024-09-19T04:27:48.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:48 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:49.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 239305 2.42 MB/sec execute 247 sec latency 3578.979 ms 2024-09-19T04:27:50.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 239388 2.41 MB/sec execute 248 sec latency 941.571 ms 2024-09-19T04:27:50.839 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:27:50.839 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:27:50.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:50 smithi115 ceph-mon[35477]: pgmap v230: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 805 KiB/s wr, 88 op/s 2024-09-19T04:27:50.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:50 smithi145 ceph-mon[35096]: pgmap v230: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 805 KiB/s wr, 88 op/s 2024-09-19T04:27:50.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:50 smithi107 ceph-mon[30659]: pgmap v230: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 805 KiB/s wr, 88 op/s 2024-09-19T04:27:51.217 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:51.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 239388 2.40 MB/sec execute 249 sec latency 1941.733 ms 2024-09-19T04:27:51.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.832+0000 7fea31a00640 1 -- 172.21.15.107:0/2974053648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea2c102c50 msgr2=0x7fea2c103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:51.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.832+0000 7fea31a00640 1 --2- 172.21.15.107:0/2974053648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea2c102c50 0x7fea2c103030 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009ca0 tx=0x7fea1c02a950 comp rx=0 tx=0).stop 2024-09-19T04:27:51.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.832+0000 7fea31a00640 1 -- 172.21.15.107:0/2974053648 shutdown_connections 2024-09-19T04:27:51.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.832+0000 7fea31a00640 1 --2- 172.21.15.107:0/2974053648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea2c10bff0 0x7fea2c10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.832+0000 7fea31a00640 1 --2- 172.21.15.107:0/2974053648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea2c103570 0x7fea2c10ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.832+0000 7fea31a00640 1 --2- 172.21.15.107:0/2974053648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea2c102c50 0x7fea2c103030 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.832+0000 7fea31a00640 1 -- 172.21.15.107:0/2974053648 >> 172.21.15.107:0/2974053648 conn(0x7fea2c0fc610 msgr2=0x7fea2c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:51.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.833+0000 7fea31a00640 1 -- 172.21.15.107:0/2974053648 shutdown_connections 2024-09-19T04:27:51.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.833+0000 7fea31a00640 1 -- 172.21.15.107:0/2974053648 wait complete. 2024-09-19T04:27:51.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.833+0000 7fea31a00640 1 Processor -- start 2024-09-19T04:27:51.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea31a00640 1 -- start start 2024-09-19T04:27:51.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea31a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea2c102c50 0x7fea2c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea31a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea2c103570 0x7fea2c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea31a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea2c10bff0 0x7fea2c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea31a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c111e10 con 0x7fea2c103570 2024-09-19T04:27:51.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea31a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c111c90 con 0x7fea2c10bff0 2024-09-19T04:27:51.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea31a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c111f90 con 0x7fea2c102c50 2024-09-19T04:27:51.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea23400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea2c103570 0x7fea2c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea31000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea2c10bff0 0x7fea2c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea31000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea2c10bff0 0x7fea2c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57668/0 (socket says 172.21.15.107:57668) 2024-09-19T04:27:51.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea23400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea2c103570 0x7fea2c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43622/0 (socket says 172.21.15.107:43622) 2024-09-19T04:27:51.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea31000640 1 -- 172.21.15.107:0/1932590647 learned_addr learned my addr 172.21.15.107:0/1932590647 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:51.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.834+0000 7fea2be00640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea2c102c50 0x7fea2c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea31000640 1 -- 172.21.15.107:0/1932590647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea2c102c50 msgr2=0x7fea2c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:51.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea31000640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea2c102c50 0x7fea2c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea31000640 1 -- 172.21.15.107:0/1932590647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea2c103570 msgr2=0x7fea2c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:51.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea31000640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea2c103570 0x7fea2c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea31000640 1 -- 172.21.15.107:0/1932590647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c19f450 con 0x7fea2c10bff0 2024-09-19T04:27:51.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea31000640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea2c10bff0 0x7fea2c19ed70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea2400e9b0 tx=0x7fea2400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:51.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea29600640 1 -- 172.21.15.107:0/1932590647 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2400cd80 con 0x7fea2c10bff0 2024-09-19T04:27:51.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea31a00640 1 -- 172.21.15.107:0/1932590647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c19f740 con 0x7fea2c10bff0 2024-09-19T04:27:51.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea29600640 1 -- 172.21.15.107:0/1932590647 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea240035f0 con 0x7fea2c10bff0 2024-09-19T04:27:51.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea29600640 1 -- 172.21.15.107:0/1932590647 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24010670 con 0x7fea2c10bff0 2024-09-19T04:27:51.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.835+0000 7fea31a00640 1 -- 172.21.15.107:0/1932590647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c1a7020 con 0x7fea2c10bff0 2024-09-19T04:27:51.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.838+0000 7fea29600640 1 -- 172.21.15.107:0/1932590647 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fea240037a0 con 0x7fea2c10bff0 2024-09-19T04:27:51.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.839+0000 7fea35a56740 1 Processor -- start 2024-09-19T04:27:51.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.839+0000 7fea35a56740 1 -- start start 2024-09-19T04:27:51.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.839+0000 7fea35a56740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.839+0000 7fea35a56740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.839+0000 7fea35a56740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.839+0000 7fea35a56740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e98290f290 con 0x55e9829a8cb0 2024-09-19T04:27:51.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.839+0000 7fea35a56740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e9828b6b40 con 0x55e9829b3e90 2024-09-19T04:27:51.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.839+0000 7fea35a56740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e98290b250 con 0x55e9829a95d0 2024-09-19T04:27:51.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.839+0000 7fea2be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea2be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59760/0 (socket says 172.21.15.107:59760) 2024-09-19T04:27:51.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea2be00640 1 -- 172.21.15.107:0/4204434357 learned_addr learned my addr 172.21.15.107:0/4204434357 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:51.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.839+0000 7fea23400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea23400640 1 --2- 172.21.15.107:0/4204434357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea2be00640 1 -- 172.21.15.107:0/4204434357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 msgr2=0x55e9829b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:51.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea2be00640 1 --2- 172.21.15.107:0/4204434357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea2be00640 1 -- 172.21.15.107:0/4204434357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 msgr2=0x55e9829a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea2be00640 1 --2- 172.21.15.107:0/4204434357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea2be00640 1 -- 172.21.15.107:0/4204434357 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9826806a0 con 0x55e9829a95d0 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea29600640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea100a33b0 0x7fea100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea23400640 1 --2- 172.21.15.107:0/4204434357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea29600640 1 -- 172.21.15.107:0/1932590647 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7fea240cffe0 con 0x7fea2c10bff0 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.840+0000 7fea2be00640 1 --2- 172.21.15.107:0/4204434357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829b3950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fea1c02ae70 tx=0x7fea1c02f450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba39b1215a763fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.841+0000 7fea23400640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea100a33b0 0x7fea100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.841+0000 7fea21600640 1 -- 172.21.15.107:0/4204434357 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c048070 con 0x55e9829a95d0 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.841+0000 7fea21600640 1 -- 172.21.15.107:0/4204434357 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c003e50 con 0x55e9829a95d0 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.841+0000 7fea21600640 1 -- 172.21.15.107:0/4204434357 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c034040 con 0x55e9829a95d0 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 -- 172.21.15.107:0/4204434357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 msgr2=0x55e9829b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:51.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 --2- 172.21.15.107:0/4204434357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829b3950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fea1c02ae70 tx=0x7fea1c02f450 comp rx=0 tx=0).stop 2024-09-19T04:27:51.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea23400640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea100a33b0 0x7fea100a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55e9829a65c0 tx=0x7fea1400b650 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:51.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 -- 172.21.15.107:0/4204434357 shutdown_connections 2024-09-19T04:27:51.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 --2- 172.21.15.107:0/4204434357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 --2- 172.21.15.107:0/4204434357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829b3950 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 --2- 172.21.15.107:0/4204434357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 -- 172.21.15.107:0/4204434357 >> 172.21.15.107:0/4204434357 conn(0x55e9829a5240 msgr2=0x55e9829a5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:51.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 -- 172.21.15.107:0/4204434357 shutdown_connections 2024-09-19T04:27:51.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 -- 172.21.15.107:0/4204434357 wait complete. 2024-09-19T04:27:51.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 -- start start 2024-09-19T04:27:51.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.842+0000 7fea35a56740 1 Processor -- start 2024-09-19T04:27:51.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea35a56740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea35a56740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a95d0 0x55e9829b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea35a56740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea35a56740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e9829b6b00 con 0x55e9829a95d0 2024-09-19T04:27:51.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea35a56740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e9828c9fa0 con 0x55e9829b3e90 2024-09-19T04:27:51.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea35a56740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e9829b6c80 con 0x55e9829a8cb0 2024-09-19T04:27:51.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea35a56740 10 client.0 _ll_register_callbacks cb 0x55e98261dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:51.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea2be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea2be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59762/0 (socket says 172.21.15.107:59762) 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea31000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea23400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a95d0 0x55e9829b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.843+0000 7fea2be00640 1 -- 172.21.15.107:0/3558796617 learned_addr learned my addr 172.21.15.107:0/3558796617 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea2be00640 1 -- 172.21.15.107:0/3558796617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 msgr2=0x55e9829b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea2be00640 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea2be00640 1 -- 172.21.15.107:0/3558796617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a95d0 msgr2=0x55e9829b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea2be00640 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a95d0 0x55e9829b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea2be00640 1 -- 172.21.15.107:0/3558796617 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9829b1720 con 0x55e9829a8cb0 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea31000640 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea2be00640 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829b3700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea1c02af40 tx=0x7fea1c02b0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f5bc10ae451502 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea1a000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea1a000640 1 -- 172.21.15.107:0/3558796617 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c048030 con 0x55e9829a8cb0 2024-09-19T04:27:51.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea35a56740 1 -- 172.21.15.107:0/3558796617 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1c003e50 con 0x55e9829a8cb0 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea1a000640 1 -- 172.21.15.107:0/3558796617 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7fea1c040050 con 0x55e9829a8cb0 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea1a000640 1 -- 172.21.15.107:0/3558796617 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fea1c034040 con 0x55e9829a8cb0 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.844+0000 7fea35a56740 10 client.26587 fetch_fsmap learned FSMap version 44 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.845+0000 7fea35a56740 1 -- 172.21.15.107:0/3558796617 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e9829b6b00 con 0x55e9829a8cb0 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.845+0000 7fea1a000640 1 -- 172.21.15.107:0/3558796617 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c003e50 con 0x55e9829a8cb0 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.845+0000 7fea1a000640 1 -- 172.21.15.107:0/3558796617 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fea1c055030 con 0x55e9829a8cb0 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.845+0000 7fea35a56740 10 client.26587 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.845+0000 7fea35a56740 10 client.26587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.845+0000 7fea35a56740 20 client.26587 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.845+0000 7fea35a56740 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e9829c3af0 0x55e9829c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.845+0000 7fea35a56740 4 client.26587 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.845+0000 7fea35a56740 1 -- 172.21.15.107:0/3558796617 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e98291c800 con 0x55e9829c3af0 2024-09-19T04:27:51.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.846+0000 7fea23400640 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e9829c3af0 0x55e9829c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.846+0000 7fea23400640 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e9829c3af0 0x55e9829c5ee0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e96747b3518ed2b server_cookie=426f01ff821d9d19 in_seq=0 out_seq=0 2024-09-19T04:27:51.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.846+0000 7fea1a000640 10 client.26587 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea1a000640 1 -- 172.21.15.107:0/3558796617 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e98291c800 con 0x55e9829c3af0 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea1a000640 10 client.26587 handle_command_reply: tid=0 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 client.26587 shutdown 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 -- 172.21.15.107:0/3558796617 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 msgr2=0x55e9829b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829b3700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea1c02af40 tx=0x7fea1c02b0d0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 -- 172.21.15.107:0/3558796617 shutdown_connections 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e9829c3af0 0x55e9829c5ee0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a95d0 0x55e9829b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 --2- 172.21.15.107:0/3558796617 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829b3700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 -- 172.21.15.107:0/3558796617 >> 172.21.15.107:0/3558796617 conn(0x55e9829a5240 msgr2=0x55e9829aaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:51.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea1a000640 0 client.26587 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:51.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 -- 172.21.15.107:0/3558796617 shutdown_connections 2024-09-19T04:27:51.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 1 -- 172.21.15.107:0/3558796617 wait complete. 2024-09-19T04:27:51.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.847+0000 7fea35a56740 20 client.26587 trim_cache size 0 max 16384 2024-09-19T04:27:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.866+0000 7fea35a56740 1 Processor -- start 2024-09-19T04:27:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.866+0000 7fea35a56740 1 -- start start 2024-09-19T04:27:51.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea35a56740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea35a56740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea35a56740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea35a56740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1af890 con 0x55e9829a8cb0 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea35a56740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b1430 con 0x55e9829b3e90 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea35a56740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1aff90 con 0x55e9829a95d0 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea2be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea2be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43646/0 (socket says 172.21.15.107:43646) 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea2be00640 1 -- 172.21.15.107:0/2942131394 learned_addr learned my addr 172.21.15.107:0/2942131394 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea23400640 1 --2- 172.21.15.107:0/2942131394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.867+0000 7fea31000640 1 --2- 172.21.15.107:0/2942131394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.868+0000 7fea23400640 1 -- 172.21.15.107:0/2942131394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 msgr2=0x55e9829cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.868+0000 7fea23400640 1 --2- 172.21.15.107:0/2942131394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.868+0000 7fea23400640 1 -- 172.21.15.107:0/2942131394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 msgr2=0x55e9829b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.868+0000 7fea23400640 1 --2- 172.21.15.107:0/2942131394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.868+0000 7fea23400640 1 -- 172.21.15.107:0/2942131394 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9829b65c0 con 0x55e9829a95d0 2024-09-19T04:27:51.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.868+0000 7fea2be00640 1 --2- 172.21.15.107:0/2942131394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:51.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.868+0000 7fea23400640 1 --2- 172.21.15.107:0/2942131394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829cb6e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea1400b6c0 tx=0x7fea140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44d2dcf14c047ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.868+0000 7fea20c00640 1 -- 172.21.15.107:0/2942131394 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1400ee20 con 0x55e9829a95d0 2024-09-19T04:27:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.868+0000 7fea20c00640 1 -- 172.21.15.107:0/2942131394 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea14005020 con 0x55e9829a95d0 2024-09-19T04:27:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.868+0000 7fea20c00640 1 -- 172.21.15.107:0/2942131394 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea140228c0 con 0x55e9829a95d0 2024-09-19T04:27:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.869+0000 7fea35a56740 1 -- 172.21.15.107:0/2942131394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 msgr2=0x55e9829cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.869+0000 7fea35a56740 1 --2- 172.21.15.107:0/2942131394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829cb6e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea1400b6c0 tx=0x7fea140046c0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.869+0000 7fea35a56740 1 -- 172.21.15.107:0/2942131394 shutdown_connections 2024-09-19T04:27:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.869+0000 7fea35a56740 1 --2- 172.21.15.107:0/2942131394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829b3e90 0x55e9829cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.869+0000 7fea35a56740 1 --2- 172.21.15.107:0/2942131394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a95d0 0x55e9829cb6e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.869+0000 7fea35a56740 1 --2- 172.21.15.107:0/2942131394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829a8cb0 0x55e9829b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.869+0000 7fea35a56740 1 -- 172.21.15.107:0/2942131394 >> 172.21.15.107:0/2942131394 conn(0x55e9829a5240 msgr2=0x55e9829b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.869+0000 7fea35a56740 1 -- 172.21.15.107:0/2942131394 shutdown_connections 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.869+0000 7fea35a56740 1 -- 172.21.15.107:0/2942131394 wait complete. 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.870+0000 7fea35a56740 1 -- start start 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.870+0000 7fea35a56740 1 Processor -- start 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.870+0000 7fea35a56740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829bc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.870+0000 7fea35a56740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829b3e90 0x55e9829b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.870+0000 7fea35a56740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829ba030 0x55e9829ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.870+0000 7fea35a56740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b0d30 con 0x55e9829b3e90 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.870+0000 7fea35a56740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b0660 con 0x55e9829ba030 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.870+0000 7fea35a56740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1ad890 con 0x55e9829a8cb0 2024-09-19T04:27:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.870+0000 7fea35a56740 10 client.0 _ll_register_callbacks cb 0x7fea1c046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea2be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829bc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea2be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829bc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59776/0 (socket says 172.21.15.107:59776) 2024-09-19T04:27:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea2be00640 1 -- 172.21.15.107:0/1747637431 learned_addr learned my addr 172.21.15.107:0/1747637431 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea23400640 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829b3e90 0x55e9829b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea31000640 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829ba030 0x55e9829ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea31000640 1 -- 172.21.15.107:0/1747637431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 msgr2=0x55e9829bc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea31000640 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829bc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea31000640 1 -- 172.21.15.107:0/1747637431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829b3e90 msgr2=0x55e9829b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea31000640 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829b3e90 0x55e9829b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea31000640 1 -- 172.21.15.107:0/1747637431 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e982a2d890 con 0x55e9829ba030 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea31000640 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829ba030 0x55e9829ba450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea240cd6f0 tx=0x7fea2409cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ce97757b3baf1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.871+0000 7fea1a000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea1a000640 1 -- 172.21.15.107:0/1747637431 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea240bc2b0 con 0x55e9829ba030 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea1a000640 1 -- 172.21.15.107:0/1747637431 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7fea2409b6d0 con 0x55e9829ba030 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea35a56740 1 -- 172.21.15.107:0/1747637431 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea14005020 con 0x55e9829ba030 2024-09-19T04:27:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea1a000640 1 -- 172.21.15.107:0/1747637431 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fea240a4ef0 con 0x55e9829ba030 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea35a56740 10 client.26624 fetch_fsmap learned FSMap version 44 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea35a56740 1 -- 172.21.15.107:0/1747637431 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e982a2ebd0 con 0x55e9829ba030 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea1a000640 1 -- 172.21.15.107:0/1747637431 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea14005020 con 0x55e9829ba030 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea1a000640 1 -- 172.21.15.107:0/1747637431 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fea24004000 con 0x55e9829ba030 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea35a56740 10 client.26624 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea35a56740 10 client.26624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea35a56740 20 client.26624 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea35a56740 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e982a2f2f0 0x55e982a31710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea35a56740 4 client.26624 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.872+0000 7fea35a56740 1 -- 172.21.15.107:0/1747637431 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e9829a5df0 con 0x55e982a2f2f0 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.873+0000 7fea2be00640 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e982a2f2f0 0x55e982a31710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.873+0000 7fea2be00640 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e982a2f2f0 0x55e982a31710 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9852731fb8c10f0a server_cookie=f6ee1ccdb9958cf9 in_seq=0 out_seq=0 2024-09-19T04:27:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.873+0000 7fea1a000640 10 client.26624 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.874+0000 7fea1a000640 1 -- 172.21.15.107:0/1747637431 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e9829a5df0 con 0x55e982a2f2f0 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.874+0000 7fea1a000640 10 client.26624 handle_command_reply: tid=0 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.874+0000 7fea35a56740 1 client.26624 shutdown 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 1 -- 172.21.15.107:0/1747637431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829ba030 msgr2=0x55e9829ba450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829ba030 0x55e9829ba450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea240cd6f0 tx=0x7fea2409cac0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 1 -- 172.21.15.107:0/1747637431 shutdown_connections 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e982a2f2f0 0x55e982a31710 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9829ba030 0x55e9829ba450 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9829b3e90 0x55e9829b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 1 --2- 172.21.15.107:0/1747637431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9829a8cb0 0x55e9829bc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 1 -- 172.21.15.107:0/1747637431 >> 172.21.15.107:0/1747637431 conn(0x55e9829a5240 msgr2=0x55e9829aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea1a000640 0 client.26624 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 1 -- 172.21.15.107:0/1747637431 shutdown_connections 2024-09-19T04:27:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 1 -- 172.21.15.107:0/1747637431 wait complete. 2024-09-19T04:27:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.875+0000 7fea35a56740 20 client.26624 trim_cache size 0 max 16384 2024-09-19T04:27:51.880 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:27:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:27:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "096b3454-b551-43c5-8782-7d87d0895050", 2024-09-19T04:27:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:27:51.880 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:27:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 -- 172.21.15.107:0/1932590647 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea100a33b0 msgr2=0x7fea100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea100a33b0 0x7fea100a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55e9829a65c0 tx=0x7fea1400b650 comp rx=0 tx=0).stop 2024-09-19T04:27:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 -- 172.21.15.107:0/1932590647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea2c10bff0 msgr2=0x7fea2c19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea2c10bff0 0x7fea2c19ed70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea2400e9b0 tx=0x7fea2400ee70 comp rx=0 tx=0).stop 2024-09-19T04:27:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 -- 172.21.15.107:0/1932590647 shutdown_connections 2024-09-19T04:27:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea100a33b0 0x7fea100a5870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea2c10bff0 0x7fea2c19ed70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea2c103570 0x7fea2c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 --2- 172.21.15.107:0/1932590647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea2c102c50 0x7fea2c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 -- 172.21.15.107:0/1932590647 >> 172.21.15.107:0/1932590647 conn(0x7fea2c0fc610 msgr2=0x7fea2c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 -- 172.21.15.107:0/1932590647 shutdown_connections 2024-09-19T04:27:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:51.876+0000 7fea31a00640 1 -- 172.21.15.107:0/1932590647 wait complete. 2024-09-19T04:27:52.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 241203 2.44 MB/sec execute 250 sec latency 2157.096 ms 2024-09-19T04:27:52.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:27:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: pgmap v231: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-19T04:27:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:52.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:52.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.948 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:52 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: pgmap v231: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-19T04:27:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:52.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:52 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: pgmap v231: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-19T04:27:52.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:52.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:52.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:52.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:52.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:52.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:52.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:52.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:52 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:53.245 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:27:53.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 241961 2.44 MB/sec execute 251 sec latency 220.602 ms 2024-09-19T04:27:53.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.802+0000 7efd37e00640 1 -- 172.21.15.107:0/2172782504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd38074510 msgr2=0x7efd380748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:53.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.802+0000 7efd37e00640 1 --2- 172.21.15.107:0/2172782504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd38074510 0x7efd380748f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efd28009860 tx=0x7efd2802a3d0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.802+0000 7efd37e00640 1 -- 172.21.15.107:0/2172782504 shutdown_connections 2024-09-19T04:27:53.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.802+0000 7efd37e00640 1 --2- 172.21.15.107:0/2172782504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd3810e280 0x7efd38110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.802+0000 7efd37e00640 1 --2- 172.21.15.107:0/2172782504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd38074e30 0x7efd3810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.802+0000 7efd37e00640 1 --2- 172.21.15.107:0/2172782504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd38074510 0x7efd380748f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.802+0000 7efd37e00640 1 -- 172.21.15.107:0/2172782504 >> 172.21.15.107:0/2172782504 conn(0x7efd380fd740 msgr2=0x7efd380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:53.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.802+0000 7efd37e00640 1 -- 172.21.15.107:0/2172782504 shutdown_connections 2024-09-19T04:27:53.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.802+0000 7efd37e00640 1 -- 172.21.15.107:0/2172782504 wait complete. 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.803+0000 7efd37e00640 1 Processor -- start 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd37e00640 1 -- start start 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd37e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd38074510 0x7efd3819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd37e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd38074e30 0x7efd3819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd37e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3810e280 0x7efd3819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd37e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd38113f70 con 0x7efd38074e30 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd37e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd38113df0 con 0x7efd3810e280 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd37e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efd381140f0 con 0x7efd38074510 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd36000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd38074e30 0x7efd3819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd36000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd38074e30 0x7efd3819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43662/0 (socket says 172.21.15.107:43662) 2024-09-19T04:27:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd36000640 1 -- 172.21.15.107:0/893732370 learned_addr learned my addr 172.21.15.107:0/893732370 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.804+0000 7efd36a00640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd38074510 0x7efd3819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd37400640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3810e280 0x7efd3819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd36000640 1 -- 172.21.15.107:0/893732370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd38074510 msgr2=0x7efd3819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd36000640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd38074510 0x7efd3819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd36000640 1 -- 172.21.15.107:0/893732370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3810e280 msgr2=0x7efd3819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd36000640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3810e280 0x7efd3819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd36000640 1 -- 172.21.15.107:0/893732370 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3819f420 con 0x7efd38074e30 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd36000640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd38074e30 0x7efd3819a9b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efd2400e9f0 tx=0x7efd2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd2f400640 1 -- 172.21.15.107:0/893732370 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2400cd80 con 0x7efd38074e30 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd2f400640 1 -- 172.21.15.107:0/893732370 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd24010910 con 0x7efd38074e30 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd37e00640 1 -- 172.21.15.107:0/893732370 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3819f710 con 0x7efd38074e30 2024-09-19T04:27:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.805+0000 7efd2f400640 1 -- 172.21.15.107:0/893732370 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24018990 con 0x7efd38074e30 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.806+0000 7efd37e00640 1 -- 172.21.15.107:0/893732370 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd381a6ff0 con 0x7efd38074e30 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.808+0000 7efd405d2740 1 Processor -- start 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd405d2740 1 -- start start 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd2f400640 1 -- 172.21.15.107:0/893732370 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7efd24010ac0 con 0x7efd38074e30 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd405d2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 0x559da9da9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd405d2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9db39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd405d2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 0x559da9db6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd405d2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559da9d0f3e0 con 0x559da9da9670 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd405d2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559da9cb6d80 con 0x559da9db3f30 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd405d2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559da9d0b3a0 con 0x559da9da8d50 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd36a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9db39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd37400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 0x559da9da9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd36a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9db39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43666/0 (socket says 172.21.15.107:43666) 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd36a00640 1 -- 172.21.15.107:0/3117733804 learned_addr learned my addr 172.21.15.107:0/3117733804 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.809+0000 7efd36000640 1 --2- 172.21.15.107:0/3117733804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 0x559da9db6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.810+0000 7efd37400640 1 -- 172.21.15.107:0/3117733804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 msgr2=0x559da9db6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.810+0000 7efd37400640 1 --2- 172.21.15.107:0/3117733804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 0x559da9db6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.810+0000 7efd37400640 1 -- 172.21.15.107:0/3117733804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 msgr2=0x559da9db39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.810+0000 7efd37400640 1 --2- 172.21.15.107:0/3117733804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9db39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.810+0000 7efd37400640 1 -- 172.21.15.107:0/3117733804 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da9cd5ae0 con 0x559da9da8d50 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.810+0000 7efd36000640 1 --2- 172.21.15.107:0/3117733804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 0x559da9db6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.810+0000 7efd37400640 1 --2- 172.21.15.107:0/3117733804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 0x559da9da9130 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd30005fd0 tx=0x7efd300095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91de94f19091037a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.810+0000 7efd23e00640 1 -- 172.21.15.107:0/3117733804 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30003b70 con 0x559da9da8d50 2024-09-19T04:27:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.810+0000 7efd23e00640 1 -- 172.21.15.107:0/3117733804 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd30003d10 con 0x559da9da8d50 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.810+0000 7efd23e00640 1 -- 172.21.15.107:0/3117733804 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30004560 con 0x559da9da8d50 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd2f400640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd1c0a33b0 0x7efd1c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd405d2740 1 -- 172.21.15.107:0/3117733804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 msgr2=0x559da9da9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd405d2740 1 --2- 172.21.15.107:0/3117733804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 0x559da9da9130 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd30005fd0 tx=0x7efd300095c0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd405d2740 1 -- 172.21.15.107:0/3117733804 shutdown_connections 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd405d2740 1 --2- 172.21.15.107:0/3117733804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 0x559da9db6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd405d2740 1 --2- 172.21.15.107:0/3117733804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9db39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd405d2740 1 --2- 172.21.15.107:0/3117733804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 0x559da9da9130 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd405d2740 1 -- 172.21.15.107:0/3117733804 >> 172.21.15.107:0/3117733804 conn(0x559da9da52e0 msgr2=0x559da9da56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd2f400640 1 -- 172.21.15.107:0/893732370 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7efd240d0070 con 0x7efd38074e30 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd405d2740 1 -- 172.21.15.107:0/3117733804 shutdown_connections 2024-09-19T04:27:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd36a00640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd1c0a33b0 0x7efd1c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.811+0000 7efd405d2740 1 -- 172.21.15.107:0/3117733804 wait complete. 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.812+0000 7efd405d2740 1 -- start start 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.812+0000 7efd36a00640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd1c0a33b0 0x7efd1c0a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd280095f0 tx=0x7efd28000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.812+0000 7efd405d2740 1 Processor -- start 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.812+0000 7efd405d2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da8d50 0x559da9db0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.812+0000 7efd405d2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da9670 0x559da9db1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.812+0000 7efd405d2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 0x559da9db1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.812+0000 7efd405d2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559da9db6ba0 con 0x559da9da8d50 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.812+0000 7efd405d2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559da9cca0f0 con 0x559da9da9670 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.812+0000 7efd405d2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559da9db6d20 con 0x559da9db3f30 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd405d2740 10 client.0 _ll_register_callbacks cb 0x559da9cbb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd36000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da9670 0x559da9db1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd36a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da8d50 0x559da9db0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd36000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da9670 0x559da9db1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:57744/0 (socket says 172.21.15.107:57744) 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd36000640 1 -- 172.21.15.107:0/334334739 learned_addr learned my addr 172.21.15.107:0/334334739 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd37400640 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 0x559da9db1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd36000640 1 -- 172.21.15.107:0/334334739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 msgr2=0x559da9db1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd36000640 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 0x559da9db1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd36000640 1 -- 172.21.15.107:0/334334739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da8d50 msgr2=0x559da9db0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd36000640 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da8d50 0x559da9db0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd36000640 1 -- 172.21.15.107:0/334334739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da9a803c0 con 0x559da9da9670 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd37400640 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 0x559da9db1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd36000640 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da9670 0x559da9db1410 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efd2400e9c0 tx=0x7efd2409c280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=918a3958cd4f7f8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd20c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:27:53.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.813+0000 7efd20c00640 1 -- 172.21.15.107:0/334334739 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd240cdc00 con 0x559da9da9670 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd20c00640 1 -- 172.21.15.107:0/334334739 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7efd240bc0e0 con 0x559da9da9670 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd405d2740 1 -- 172.21.15.107:0/334334739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd30003d10 con 0x559da9da9670 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd20c00640 1 -- 172.21.15.107:0/334334739 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efd240a06c0 con 0x559da9da9670 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd405d2740 10 client.26636 fetch_fsmap learned FSMap version 45 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd405d2740 1 -- 172.21.15.107:0/334334739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559da9dc2950 con 0x559da9da9670 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd20c00640 1 -- 172.21.15.107:0/334334739 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd30003d10 con 0x559da9da9670 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd20c00640 1 -- 172.21.15.107:0/334334739 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7efd24098110 con 0x559da9da9670 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd405d2740 10 client.26636 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd405d2740 10 client.26636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd405d2740 20 client.26636 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd405d2740 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x559da9dc3e10 0x559da9dc6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd405d2740 4 client.26636 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.814+0000 7efd405d2740 1 -- 172.21.15.107:0/334334739 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559da9d1c950 con 0x559da9dc3e10 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.815+0000 7efd36a00640 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x559da9dc3e10 0x559da9dc6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.815+0000 7efd36a00640 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x559da9dc3e10 0x559da9dc6220 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7458b46b062b2c5 server_cookie=b0f9f4c21082c8cd in_seq=0 out_seq=0 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.815+0000 7efd20c00640 10 client.26636 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd20c00640 1 -- 172.21.15.107:0/334334739 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559da9d1c950 con 0x559da9dc3e10 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd20c00640 10 client.26636 handle_command_reply: tid=0 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd405d2740 1 client.26636 shutdown 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd405d2740 1 -- 172.21.15.107:0/334334739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da9670 msgr2=0x559da9db1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd405d2740 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da9670 0x559da9db1410 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efd2400e9c0 tx=0x7efd2409c280 comp rx=0 tx=0).stop 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd405d2740 1 -- 172.21.15.107:0/334334739 shutdown_connections 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd405d2740 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x559da9dc3e10 0x559da9dc6220 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd405d2740 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 0x559da9db1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd405d2740 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da9670 0x559da9db1410 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd405d2740 1 --2- 172.21.15.107:0/334334739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da8d50 0x559da9db0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd20c00640 0 client.26636 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:53.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.816+0000 7efd405d2740 1 -- 172.21.15.107:0/334334739 >> 172.21.15.107:0/334334739 conn(0x559da9da52e0 msgr2=0x559da9dab060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:53.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.817+0000 7efd405d2740 1 -- 172.21.15.107:0/334334739 shutdown_connections 2024-09-19T04:27:53.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.817+0000 7efd405d2740 1 -- 172.21.15.107:0/334334739 wait complete. 2024-09-19T04:27:53.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.817+0000 7efd405d2740 20 client.26636 trim_cache size 0 max 16384 2024-09-19T04:27:53.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.834+0000 7efd405d2740 1 Processor -- start 2024-09-19T04:27:53.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.834+0000 7efd405d2740 1 -- start start 2024-09-19T04:27:53.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd405d2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da8d50 0x559da9dab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd405d2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dcbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd405d2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 0x559da9dcbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd405d2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd381af880 con 0x559da9da9670 2024-09-19T04:27:53.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd405d2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd381b13f0 con 0x559da9da8d50 2024-09-19T04:27:53.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd405d2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efd381aff80 con 0x559da9db3f30 2024-09-19T04:27:53.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd36000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dcbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd36000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dcbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43684/0 (socket says 172.21.15.107:43684) 2024-09-19T04:27:53.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd36000640 1 -- 172.21.15.107:0/1346545520 learned_addr learned my addr 172.21.15.107:0/1346545520 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:53.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd36a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da8d50 0x559da9dab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.835+0000 7efd37400640 1 --2- 172.21.15.107:0/1346545520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 0x559da9dcbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.836+0000 7efd36000640 1 -- 172.21.15.107:0/1346545520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 msgr2=0x559da9dcbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:53.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.836+0000 7efd36000640 1 --2- 172.21.15.107:0/1346545520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 0x559da9dcbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.836+0000 7efd36000640 1 -- 172.21.15.107:0/1346545520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da8d50 msgr2=0x559da9dab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:53.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.836+0000 7efd36000640 1 --2- 172.21.15.107:0/1346545520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da8d50 0x559da9dab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.836+0000 7efd36000640 1 -- 172.21.15.107:0/1346545520 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da9da6390 con 0x559da9da9670 2024-09-19T04:27:53.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.836+0000 7efd36a00640 1 --2- 172.21.15.107:0/1346545520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da8d50 0x559da9dab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:53.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.836+0000 7efd36000640 1 --2- 172.21.15.107:0/1346545520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dcbaa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efd2409a670 tx=0x7efd2409c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ede9d0a16b78eeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:53.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.836+0000 7efd23400640 1 -- 172.21.15.107:0/1346545520 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24098110 con 0x559da9da9670 2024-09-19T04:27:53.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.836+0000 7efd23400640 1 -- 172.21.15.107:0/1346545520 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd240982b0 con 0x559da9da9670 2024-09-19T04:27:53.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.836+0000 7efd23400640 1 -- 172.21.15.107:0/1346545520 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24018d30 con 0x559da9da9670 2024-09-19T04:27:53.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 -- 172.21.15.107:0/1346545520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 msgr2=0x559da9dcbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:53.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 --2- 172.21.15.107:0/1346545520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dcbaa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efd2409a670 tx=0x7efd2409c540 comp rx=0 tx=0).stop 2024-09-19T04:27:53.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 -- 172.21.15.107:0/1346545520 shutdown_connections 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 --2- 172.21.15.107:0/1346545520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9db3f30 0x559da9dcbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 --2- 172.21.15.107:0/1346545520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dcbaa0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 --2- 172.21.15.107:0/1346545520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9da8d50 0x559da9dab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 -- 172.21.15.107:0/1346545520 >> 172.21.15.107:0/1346545520 conn(0x559da9da52e0 msgr2=0x559da9db2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 -- 172.21.15.107:0/1346545520 shutdown_connections 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 -- 172.21.15.107:0/1346545520 wait complete. 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 -- start start 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.837+0000 7efd405d2740 1 Processor -- start 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd405d2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 0x559da9dbc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd405d2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dadd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd405d2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 0x559da9dae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd405d2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd381b0d20 con 0x559da9da9670 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd405d2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd381b0650 con 0x559da9db3f30 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd405d2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efd381ad880 con 0x559da9da8d50 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd405d2740 10 client.0 _ll_register_callbacks cb 0x7efd24098730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd36000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dadd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd37400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 0x559da9dae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd36000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dadd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43688/0 (socket says 172.21.15.107:43688) 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd36000640 1 -- 172.21.15.107:0/304580066 learned_addr learned my addr 172.21.15.107:0/304580066 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.838+0000 7efd36a00640 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 0x559da9dbc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd37400640 1 -- 172.21.15.107:0/304580066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 msgr2=0x559da9dbc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd37400640 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 0x559da9dbc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd37400640 1 -- 172.21.15.107:0/304580066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 msgr2=0x559da9dadd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:27:53.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd37400640 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dadd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd37400640 1 -- 172.21.15.107:0/304580066 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da9daebc0 con 0x559da9db3f30 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd36000640 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dadd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd37400640 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 0x559da9dae390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efd3000ecb0 tx=0x7efd30003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee98de9e321efa8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd20c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd20c00640 1 -- 172.21.15.107:0/304580066 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30005250 con 0x559da9db3f30 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd20c00640 1 -- 172.21.15.107:0/304580066 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7efd300053f0 con 0x559da9db3f30 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd405d2740 1 -- 172.21.15.107:0/304580066 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd240982b0 con 0x559da9db3f30 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd20c00640 1 -- 172.21.15.107:0/304580066 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efd30034050 con 0x559da9db3f30 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd405d2740 10 client.26642 fetch_fsmap learned FSMap version 45 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd405d2740 1 -- 172.21.15.107:0/304580066 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559da9e373c0 con 0x559da9db3f30 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.839+0000 7efd20c00640 1 -- 172.21.15.107:0/304580066 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd240982b0 con 0x559da9db3f30 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.840+0000 7efd20c00640 1 -- 172.21.15.107:0/304580066 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7efd3000b040 con 0x559da9db3f30 2024-09-19T04:27:53.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.840+0000 7efd405d2740 10 client.26642 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.840+0000 7efd405d2740 10 client.26642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.840+0000 7efd405d2740 20 client.26642 populate_metadata read hostname 'smithi107' 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.840+0000 7efd405d2740 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x559da9e37810 0x559da9e39c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.840+0000 7efd405d2740 4 client.26642 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.840+0000 7efd405d2740 1 -- 172.21.15.107:0/304580066 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559da9d16e60 con 0x559da9e37810 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.840+0000 7efd36a00640 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x559da9e37810 0x559da9e39c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.840+0000 7efd36a00640 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x559da9e37810 0x559da9e39c00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ce226e07379c43 server_cookie=7398efc02ce64c51 in_seq=0 out_seq=0 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.840+0000 7efd20c00640 10 client.26642 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd20c00640 1 -- 172.21.15.107:0/304580066 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559da9d16e60 con 0x559da9e37810 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd20c00640 10 client.26642 handle_command_reply: tid=0 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 client.26642 shutdown 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 -- 172.21.15.107:0/304580066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 msgr2=0x559da9dae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:53.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 0x559da9dae390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efd3000ecb0 tx=0x7efd30003c10 comp rx=0 tx=0).stop 2024-09-19T04:27:53.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 -- 172.21.15.107:0/304580066 shutdown_connections 2024-09-19T04:27:53.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x559da9e37810 0x559da9e39c00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559da9db3f30 0x559da9dae390 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559da9da9670 0x559da9dadd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd20c00640 0 client.26642 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:27:53.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 --2- 172.21.15.107:0/304580066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559da9da8d50 0x559da9dbc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 -- 172.21.15.107:0/304580066 >> 172.21.15.107:0/304580066 conn(0x559da9da52e0 msgr2=0x559da9da3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:53.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 -- 172.21.15.107:0/304580066 shutdown_connections 2024-09-19T04:27:53.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 1 -- 172.21.15.107:0/304580066 wait complete. 2024-09-19T04:27:53.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.841+0000 7efd405d2740 20 client.26642 trim_cache size 0 max 16384 2024-09-19T04:27:53.849 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:27:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:27:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:27:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "096b3454-b551-43c5-8782-7d87d0895050": { 2024-09-19T04:27:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:27:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "096b3454-b551-43c5-8782-7d87d0895050", 2024-09-19T04:27:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:27:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:27:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:27:53.849 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:27:53.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 -- 172.21.15.107:0/893732370 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd1c0a33b0 msgr2=0x7efd1c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:53.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd1c0a33b0 0x7efd1c0a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd280095f0 tx=0x7efd28000bc0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 -- 172.21.15.107:0/893732370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd38074e30 msgr2=0x7efd3819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:27:53.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd38074e30 0x7efd3819a9b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efd2400e9f0 tx=0x7efd2400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 -- 172.21.15.107:0/893732370 shutdown_connections 2024-09-19T04:27:53.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efd1c0a33b0 0x7efd1c0a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3810e280 0x7efd3819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd38074e30 0x7efd3819a9b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 --2- 172.21.15.107:0/893732370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd38074510 0x7efd3819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:27:53.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 -- 172.21.15.107:0/893732370 >> 172.21.15.107:0/893732370 conn(0x7efd380fd740 msgr2=0x7efd380fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:27:53.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 -- 172.21.15.107:0/893732370 shutdown_connections 2024-09-19T04:27:53.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:27:53.842+0000 7efd37e00640 1 -- 172.21.15.107:0/893732370 wait complete. 2024-09-19T04:27:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:53 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:27:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:53 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:53.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:53 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:27:53.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:53 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:53.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:53 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:53.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:53 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:27:53.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:53 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:53.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:53 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:27:53.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:53 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:53.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:53 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:53 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:27:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:53 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:54.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:53 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:27:54.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:53 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:27:54.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:53 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:27:54.685 INFO:tasks.workunit.client.0.smithi107.stdout: 1 242690 2.45 MB/sec execute 252 sec latency 752.994 ms 2024-09-19T04:27:54.833 INFO:tasks.cephfs.filesystem:scrub status for tag:096b3454-b551-43c5-8782-7d87d0895050 - {'path': '/', 'tag': '096b3454-b551-43c5-8782-7d87d0895050', 'options': 'recursive,force'} 2024-09-19T04:27:54.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:54 smithi145 ceph-mon[35096]: pgmap v232: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 580 KiB/s wr, 69 op/s 2024-09-19T04:27:54.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:54 smithi107 ceph-mon[30659]: pgmap v232: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 580 KiB/s wr, 69 op/s 2024-09-19T04:27:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:54 smithi115 ceph-mon[35477]: pgmap v232: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 580 KiB/s wr, 69 op/s 2024-09-19T04:27:55.686 INFO:tasks.workunit.client.0.smithi107.stdout: 1 242931 2.44 MB/sec execute 253 sec latency 675.909 ms 2024-09-19T04:27:56.686 INFO:tasks.workunit.client.0.smithi107.stdout: 1 242931 2.43 MB/sec execute 254 sec latency 1676.001 ms 2024-09-19T04:27:56.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:56 smithi145 ceph-mon[35096]: pgmap v233: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 894 KiB/s wr, 117 op/s 2024-09-19T04:27:56.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:56 smithi107 ceph-mon[30659]: pgmap v233: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 894 KiB/s wr, 117 op/s 2024-09-19T04:27:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:56 smithi115 ceph-mon[35477]: pgmap v233: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 894 KiB/s wr, 117 op/s 2024-09-19T04:27:57.686 INFO:tasks.workunit.client.0.smithi107.stdout: 1 242931 2.42 MB/sec execute 255 sec latency 2676.133 ms 2024-09-19T04:27:58.686 INFO:tasks.workunit.client.0.smithi107.stdout: 1 242931 2.41 MB/sec execute 256 sec latency 3676.269 ms 2024-09-19T04:27:58.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:27:58 smithi145 ceph-mon[35096]: pgmap v234: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 700 KiB/s wr, 82 op/s 2024-09-19T04:27:58.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:27:58 smithi107 ceph-mon[30659]: pgmap v234: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 700 KiB/s wr, 82 op/s 2024-09-19T04:27:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:27:58 smithi115 ceph-mon[35477]: pgmap v234: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 700 KiB/s wr, 82 op/s 2024-09-19T04:27:59.686 INFO:tasks.workunit.client.0.smithi107.stdout: 1 244588 2.44 MB/sec execute 257 sec latency 3944.558 ms 2024-09-19T04:28:00.686 INFO:tasks.workunit.client.0.smithi107.stdout: 1 245730 2.45 MB/sec execute 258 sec latency 15.477 ms 2024-09-19T04:28:00.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:00 smithi107 ceph-mon[30659]: pgmap v235: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 700 KiB/s wr, 82 op/s 2024-09-19T04:28:00.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:00 smithi145 ceph-mon[35096]: pgmap v235: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 700 KiB/s wr, 82 op/s 2024-09-19T04:28:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:00 smithi115 ceph-mon[35477]: pgmap v235: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 700 KiB/s wr, 82 op/s 2024-09-19T04:28:01.687 INFO:tasks.workunit.client.0.smithi107.stdout: 1 246478 2.46 MB/sec execute 259 sec latency 283.875 ms 2024-09-19T04:28:02.687 INFO:tasks.workunit.client.0.smithi107.stdout: 1 246478 2.45 MB/sec execute 260 sec latency 1284.327 ms 2024-09-19T04:28:02.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:02 smithi115 ceph-mon[35477]: pgmap v236: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 901 KiB/s wr, 118 op/s 2024-09-19T04:28:02.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:02 smithi115 ceph-mon[35477]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:28:02.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:02 smithi145 ceph-mon[35096]: pgmap v236: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 901 KiB/s wr, 118 op/s 2024-09-19T04:28:02.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:02 smithi145 ceph-mon[35096]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:28:02.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:02 smithi107 ceph-mon[30659]: pgmap v236: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 901 KiB/s wr, 118 op/s 2024-09-19T04:28:02.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:02 smithi107 ceph-mon[30659]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:28:03.687 INFO:tasks.workunit.client.0.smithi107.stdout: 1 246478 2.44 MB/sec execute 261 sec latency 2284.445 ms 2024-09-19T04:28:04.687 INFO:tasks.workunit.client.0.smithi107.stdout: 1 247382 2.44 MB/sec execute 262 sec latency 2817.792 ms 2024-09-19T04:28:04.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:04 smithi145 ceph-mon[35096]: pgmap v238: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 619 KiB/s wr, 101 op/s 2024-09-19T04:28:04.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:04 smithi107 ceph-mon[30659]: pgmap v238: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 619 KiB/s wr, 101 op/s 2024-09-19T04:28:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:04 smithi115 ceph-mon[35477]: pgmap v238: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 619 KiB/s wr, 101 op/s 2024-09-19T04:28:05.687 INFO:tasks.workunit.client.0.smithi107.stdout: 1 248878 2.47 MB/sec execute 263 sec latency 11.556 ms 2024-09-19T04:28:06.687 INFO:tasks.workunit.client.0.smithi107.stdout: 1 250021 2.48 MB/sec execute 264 sec latency 287.782 ms 2024-09-19T04:28:06.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:06 smithi107 ceph-mon[30659]: pgmap v239: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-09-19T04:28:06.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:06 smithi107 ceph-mon[30659]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:28:06.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:06 smithi145 ceph-mon[35096]: pgmap v239: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-09-19T04:28:06.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:06 smithi145 ceph-mon[35096]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:28:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:06 smithi115 ceph-mon[35477]: pgmap v239: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-09-19T04:28:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:06 smithi115 ceph-mon[35477]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:28:07.687 INFO:tasks.workunit.client.0.smithi107.stdout: 1 250021 2.47 MB/sec execute 265 sec latency 1287.942 ms 2024-09-19T04:28:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:07 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:28:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:07 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:28:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:07 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:28:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:07 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:28:07.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:07 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:28:08.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:07 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:28:08.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:07 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:28:08.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:07 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:28:08.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:07 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:28:08.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:07 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:28:08.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:07 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:28:08.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:07 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:28:08.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:07 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:28:08.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:07 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:28:08.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:07 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:28:08.687 INFO:tasks.workunit.client.0.smithi107.stdout: 1 250021 2.46 MB/sec execute 266 sec latency 2288.104 ms 2024-09-19T04:28:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:08 smithi115 ceph-mon[35477]: pgmap v241: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-09-19T04:28:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:08 smithi145 ceph-mon[35096]: pgmap v241: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-09-19T04:28:09.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:08 smithi107 ceph-mon[30659]: pgmap v241: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-09-19T04:28:09.688 INFO:tasks.workunit.client.0.smithi107.stdout: 1 250930 2.46 MB/sec execute 267 sec latency 2819.280 ms 2024-09-19T04:28:10.687 INFO:tasks.workunit.client.0.smithi107.stdout: 1 252416 2.50 MB/sec execute 268 sec latency 9.786 ms 2024-09-19T04:28:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:10 smithi115 ceph-mon[35477]: pgmap v242: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-19T04:28:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:10 smithi145 ceph-mon[35096]: pgmap v242: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-19T04:28:11.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:10 smithi107 ceph-mon[30659]: pgmap v242: 97 pgs: 97 active+clean; 173 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-19T04:28:11.688 INFO:tasks.workunit.client.0.smithi107.stdout: 1 252602 2.49 MB/sec execute 269 sec latency 846.595 ms 2024-09-19T04:28:12.688 INFO:tasks.workunit.client.0.smithi107.stdout: 1 252602 2.48 MB/sec execute 270 sec latency 1846.733 ms 2024-09-19T04:28:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:12 smithi115 ceph-mon[35477]: pgmap v243: 97 pgs: 97 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-09-19T04:28:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:12 smithi145 ceph-mon[35096]: pgmap v243: 97 pgs: 97 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-09-19T04:28:13.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:12 smithi107 ceph-mon[30659]: pgmap v243: 97 pgs: 97 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-09-19T04:28:13.688 INFO:tasks.workunit.client.0.smithi107.stdout: 1 252602 2.47 MB/sec execute 271 sec latency 2846.846 ms 2024-09-19T04:28:14.688 INFO:tasks.workunit.client.0.smithi107.stdout: 1 252670 2.46 MB/sec execute 272 sec latency 3385.503 ms 2024-09-19T04:28:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:14 smithi115 ceph-mon[35477]: pgmap v244: 97 pgs: 97 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-09-19T04:28:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:14 smithi145 ceph-mon[35096]: pgmap v244: 97 pgs: 97 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-09-19T04:28:15.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:14 smithi107 ceph-mon[30659]: pgmap v244: 97 pgs: 97 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-09-19T04:28:15.688 INFO:tasks.workunit.client.0.smithi107.stdout: 1 252670 2.45 MB/sec execute 273 sec latency 1407.045 ms 2024-09-19T04:28:16.688 INFO:tasks.workunit.client.0.smithi107.stdout: 1 252670 2.44 MB/sec execute 274 sec latency 2407.174 ms 2024-09-19T04:28:17.012 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:16 smithi107 ceph-mon[30659]: pgmap v245: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-19T04:28:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:16 smithi115 ceph-mon[35477]: pgmap v245: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-19T04:28:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:16 smithi145 ceph-mon[35096]: pgmap v245: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-19T04:28:17.688 INFO:tasks.workunit.client.0.smithi107.stdout: 1 252670 2.43 MB/sec execute 275 sec latency 3407.291 ms 2024-09-19T04:28:18.688 INFO:tasks.workunit.client.0.smithi107.stdout: 1 252670 2.43 MB/sec execute 276 sec latency 4407.415 ms 2024-09-19T04:28:19.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:18 smithi145 ceph-mon[35096]: pgmap v246: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-09-19T04:28:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:18 smithi115 ceph-mon[35477]: pgmap v246: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-09-19T04:28:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:18 smithi107 ceph-mon[30659]: pgmap v246: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-09-19T04:28:19.689 INFO:tasks.workunit.client.0.smithi107.stdout: 1 253489 2.43 MB/sec execute 277 sec latency 4938.588 ms 2024-09-19T04:28:20.689 INFO:tasks.workunit.client.0.smithi107.stdout: 1 253572 2.43 MB/sec execute 278 sec latency 940.055 ms 2024-09-19T04:28:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:20 smithi115 ceph-mon[35477]: pgmap v247: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 971 KiB/s wr, 87 op/s 2024-09-19T04:28:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:20 smithi145 ceph-mon[35096]: pgmap v247: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 971 KiB/s wr, 87 op/s 2024-09-19T04:28:21.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:20 smithi107 ceph-mon[30659]: pgmap v247: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 971 KiB/s wr, 87 op/s 2024-09-19T04:28:21.689 INFO:tasks.workunit.client.0.smithi107.stdout: 1 253572 2.42 MB/sec execute 279 sec latency 1940.194 ms 2024-09-19T04:28:22.689 INFO:tasks.workunit.client.0.smithi107.stdout: 1 253572 2.41 MB/sec execute 280 sec latency 2940.329 ms 2024-09-19T04:28:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:22 smithi115 ceph-mon[35477]: pgmap v248: 97 pgs: 97 active+clean; 181 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-09-19T04:28:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:22 smithi145 ceph-mon[35096]: pgmap v248: 97 pgs: 97 active+clean; 181 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-09-19T04:28:23.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:22 smithi107 ceph-mon[30659]: pgmap v248: 97 pgs: 97 active+clean; 181 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-09-19T04:28:23.689 INFO:tasks.workunit.client.0.smithi107.stdout: 1 253572 2.40 MB/sec execute 281 sec latency 3940.465 ms 2024-09-19T04:28:24.689 INFO:tasks.workunit.client.0.smithi107.stdout: 1 255234 2.42 MB/sec execute 282 sec latency 4207.722 ms 2024-09-19T04:28:24.835 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:28:25.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:24 smithi115 ceph-mon[35477]: pgmap v249: 97 pgs: 97 active+clean; 181 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 953 KiB/s wr, 59 op/s 2024-09-19T04:28:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:24 smithi145 ceph-mon[35096]: pgmap v249: 97 pgs: 97 active+clean; 181 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 953 KiB/s wr, 59 op/s 2024-09-19T04:28:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:24 smithi107 ceph-mon[30659]: pgmap v249: 97 pgs: 97 active+clean; 181 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 953 KiB/s wr, 59 op/s 2024-09-19T04:28:25.252 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:25.689 INFO:tasks.workunit.client.0.smithi107.stdout: 1 256143 2.43 MB/sec execute 283 sec latency 124.255 ms 2024-09-19T04:28:25.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.806+0000 7f0630600640 1 -- 172.21.15.107:0/3895824381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06281049a0 msgr2=0x7f06281089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.806+0000 7f0630600640 1 --2- 172.21.15.107:0/3895824381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06281049a0 0x7f06281089f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0614009900 tx=0x7f061402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.806+0000 7f0630600640 1 -- 172.21.15.107:0/3895824381 shutdown_connections 2024-09-19T04:28:25.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.806+0000 7f0630600640 1 --2- 172.21.15.107:0/3895824381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0628109120 0x7f0628110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.806+0000 7f0630600640 1 --2- 172.21.15.107:0/3895824381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06281049a0 0x7f06281089f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.806+0000 7f0630600640 1 --2- 172.21.15.107:0/3895824381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628103ff0 0x7f06281043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.806+0000 7f0630600640 1 -- 172.21.15.107:0/3895824381 >> 172.21.15.107:0/3895824381 conn(0x7f06280ffa90 msgr2=0x7f0628101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.806+0000 7f0630600640 1 -- 172.21.15.107:0/3895824381 shutdown_connections 2024-09-19T04:28:25.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.806+0000 7f0630600640 1 -- 172.21.15.107:0/3895824381 wait complete. 2024-09-19T04:28:25.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.807+0000 7f0630600640 1 Processor -- start 2024-09-19T04:28:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.808+0000 7f0630600640 1 -- start start 2024-09-19T04:28:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.808+0000 7f0630600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628103ff0 0x7f062819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.808+0000 7f0630600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06281049a0 0x7f062819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.808+0000 7f0630600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628109120 0x7f062819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.808+0000 7f0630600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0628114120 con 0x7f0628103ff0 2024-09-19T04:28:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.808+0000 7f0630600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0628113fa0 con 0x7f06281049a0 2024-09-19T04:28:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.808+0000 7f0630600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f06281142a0 con 0x7f0628109120 2024-09-19T04:28:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.808+0000 7f062dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628103ff0 0x7f062819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.808+0000 7f062dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628103ff0 0x7f062819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46238/0 (socket says 172.21.15.107:46238) 2024-09-19T04:28:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.808+0000 7f062dc00640 1 -- 172.21.15.107:0/2846902328 learned_addr learned my addr 172.21.15.107:0/2846902328 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f062d200640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06281049a0 0x7f062819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f062e600640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628109120 0x7f062819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f062dc00640 1 -- 172.21.15.107:0/2846902328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628109120 msgr2=0x7f062819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f062dc00640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628109120 0x7f062819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f062dc00640 1 -- 172.21.15.107:0/2846902328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06281049a0 msgr2=0x7f062819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f062dc00640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06281049a0 0x7f062819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f062dc00640 1 -- 172.21.15.107:0/2846902328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062819f5d0 con 0x7f0628103ff0 2024-09-19T04:28:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f062dc00640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628103ff0 0x7f062819a620 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f061c00ea60 tx=0x7f061c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f0626000640 1 -- 172.21.15.107:0/2846902328 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c00cd20 con 0x7f0628103ff0 2024-09-19T04:28:25.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f0630600640 1 -- 172.21.15.107:0/2846902328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062819f8c0 con 0x7f0628103ff0 2024-09-19T04:28:25.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.809+0000 7f0626000640 1 -- 172.21.15.107:0/2846902328 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c0035f0 con 0x7f0628103ff0 2024-09-19T04:28:25.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.810+0000 7f0626000640 1 -- 172.21.15.107:0/2846902328 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c010690 con 0x7f0628103ff0 2024-09-19T04:28:25.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.810+0000 7f0630600640 1 -- 172.21.15.107:0/2846902328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06281a71a0 con 0x7f0628103ff0 2024-09-19T04:28:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.812+0000 7f0633025740 1 Processor -- start 2024-09-19T04:28:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.813+0000 7f0626000640 1 -- 172.21.15.107:0/2846902328 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f061c0037a0 con 0x7f0628103ff0 2024-09-19T04:28:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.813+0000 7f0633025740 1 -- start start 2024-09-19T04:28:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.813+0000 7f0633025740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d6bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.813+0000 7f0633025740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6c4b0 0x562333d76830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.813+0000 7f0633025740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d76d70 0x562333d79160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.813+0000 7f0633025740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562333cd23e0 con 0x562333d76d70 2024-09-19T04:28:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.813+0000 7f0633025740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562333c79d80 con 0x562333d6bb90 2024-09-19T04:28:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.813+0000 7f0633025740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562333cce3a0 con 0x562333d6c4b0 2024-09-19T04:28:25.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.813+0000 7f062e600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d6bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.813+0000 7f062e600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d6bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:50874/0 (socket says 172.21.15.107:50874) 2024-09-19T04:28:25.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.814+0000 7f062e600640 1 -- 172.21.15.107:0/2812677882 learned_addr learned my addr 172.21.15.107:0/2812677882 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:25.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.814+0000 7f062d200640 1 --2- 172.21.15.107:0/2812677882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d76d70 0x562333d79160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.814+0000 7f062dc00640 1 --2- 172.21.15.107:0/2812677882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6c4b0 0x562333d76830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.814+0000 7f062e600640 1 -- 172.21.15.107:0/2812677882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6c4b0 msgr2=0x562333d76830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.814+0000 7f062e600640 1 --2- 172.21.15.107:0/2812677882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6c4b0 0x562333d76830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.814+0000 7f062e600640 1 -- 172.21.15.107:0/2812677882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d76d70 msgr2=0x562333d79160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.814+0000 7f062e600640 1 --2- 172.21.15.107:0/2812677882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d76d70 0x562333d79160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.814+0000 7f062e600640 1 -- 172.21.15.107:0/2812677882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562333c98ae0 con 0x562333d6bb90 2024-09-19T04:28:25.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.814+0000 7f062e600640 1 --2- 172.21.15.107:0/2812677882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d6bf70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0620009f60 tx=0x7f0620009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=567fa5265269cff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.814+0000 7f0626000640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06100a33b0 0x7f06100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.815+0000 7f061aa00640 1 -- 172.21.15.107:0/2812677882 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062001e070 con 0x562333d6bb90 2024-09-19T04:28:25.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.815+0000 7f0626000640 1 -- 172.21.15.107:0/2846902328 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f061c0cffe0 con 0x7f0628103ff0 2024-09-19T04:28:25.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.815+0000 7f061aa00640 1 -- 172.21.15.107:0/2812677882 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0620003c30 con 0x562333d6bb90 2024-09-19T04:28:25.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.815+0000 7f061aa00640 1 -- 172.21.15.107:0/2812677882 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062000fa20 con 0x562333d6bb90 2024-09-19T04:28:25.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.815+0000 7f062dc00640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06100a33b0 0x7f06100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f0633025740 1 -- 172.21.15.107:0/2812677882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 msgr2=0x562333d6bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f0633025740 1 --2- 172.21.15.107:0/2812677882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d6bf70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0620009f60 tx=0x7f0620009f90 comp rx=0 tx=0).stop 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f062dc00640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06100a33b0 0x7f06100a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f061c0cd800 tx=0x7f061c0d3b30 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f0633025740 1 -- 172.21.15.107:0/2812677882 shutdown_connections 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f0633025740 1 --2- 172.21.15.107:0/2812677882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d76d70 0x562333d79160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f0633025740 1 --2- 172.21.15.107:0/2812677882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6c4b0 0x562333d76830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f0633025740 1 --2- 172.21.15.107:0/2812677882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d6bf70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f0633025740 1 -- 172.21.15.107:0/2812677882 >> 172.21.15.107:0/2812677882 conn(0x562333d68120 msgr2=0x562333d68500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f0633025740 1 -- 172.21.15.107:0/2812677882 shutdown_connections 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f0633025740 1 -- 172.21.15.107:0/2812677882 wait complete. 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.816+0000 7f0633025740 1 -- start start 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f0633025740 1 Processor -- start 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f0633025740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d73cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f0633025740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 0x562333d74230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f0633025740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d76d70 0x562333d74770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f0633025740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562333d799e0 con 0x562333d6c4b0 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f0633025740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562333c8d0f0 con 0x562333d76d70 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f0633025740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562333d79b60 con 0x562333d6bb90 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f0633025740 10 client.0 _ll_register_callbacks cb 0x562333c7e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f062d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 0x562333d74230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f062d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 0x562333d74230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46246/0 (socket says 172.21.15.107:46246) 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f062d200640 1 -- 172.21.15.107:0/4134708791 learned_addr learned my addr 172.21.15.107:0/4134708791 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.817+0000 7f062e600640 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d73cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f062e600640 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d76d70 0x562333d74770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f062d200640 1 -- 172.21.15.107:0/4134708791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 msgr2=0x562333d73cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f062d200640 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d73cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f062d200640 1 -- 172.21.15.107:0/4134708791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d76d70 msgr2=0x562333d74770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f062d200640 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d76d70 0x562333d74770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f062d200640 1 -- 172.21.15.107:0/4134708791 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562333a433c0 con 0x562333d6c4b0 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f062e600640 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d73cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f062d200640 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 0x562333d74230 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f061402a910 tx=0x7f06140317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d406fbb0dcd3be24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f060f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f060f400640 1 -- 172.21.15.107:0/4134708791 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614035040 con 0x562333d6c4b0 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f060f400640 1 -- 172.21.15.107:0/4134708791 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f0614004240 con 0x562333d6c4b0 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.818+0000 7f0633025740 1 -- 172.21.15.107:0/4134708791 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0620003c30 con 0x562333d6c4b0 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f060f400640 1 -- 172.21.15.107:0/4134708791 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0614004440 con 0x562333d6c4b0 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f0633025740 10 client.16605 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f0633025740 1 -- 172.21.15.107:0/4134708791 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562333d85710 con 0x562333d6c4b0 2024-09-19T04:28:25.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f060f400640 1 -- 172.21.15.107:0/4134708791 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0620003c30 con 0x562333d6c4b0 2024-09-19T04:28:25.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f060f400640 1 -- 172.21.15.107:0/4134708791 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f061404d070 con 0x562333d6c4b0 2024-09-19T04:28:25.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f0633025740 10 client.16605 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:25.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f0633025740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:25.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f0633025740 20 client.16605 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:25.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f0633025740 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x562333d86c20 0x562333d89040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f0633025740 4 client.16605 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:25.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.819+0000 7f0633025740 1 -- 172.21.15.107:0/4134708791 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562333cdf950 con 0x562333d86c20 2024-09-19T04:28:25.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.820+0000 7f062e600640 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x562333d86c20 0x562333d89040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.820+0000 7f062e600640 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x562333d86c20 0x562333d89040 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b7baace9a33a7c server_cookie=a91bfe6c354b89df in_seq=0 out_seq=0 2024-09-19T04:28:25.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.820+0000 7f060f400640 10 client.16605 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:25.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.821+0000 7f060f400640 1 -- 172.21.15.107:0/4134708791 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562333cdf950 con 0x562333d86c20 2024-09-19T04:28:25.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.821+0000 7f060f400640 10 client.16605 handle_command_reply: tid=0 2024-09-19T04:28:25.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.821+0000 7f0633025740 1 client.16605 shutdown 2024-09-19T04:28:25.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.821+0000 7f0633025740 1 -- 172.21.15.107:0/4134708791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 msgr2=0x562333d74230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.821+0000 7f0633025740 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 0x562333d74230 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f061402a910 tx=0x7f06140317e0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.822+0000 7f0633025740 1 -- 172.21.15.107:0/4134708791 shutdown_connections 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.822+0000 7f0633025740 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x562333d86c20 0x562333d89040 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.822+0000 7f0633025740 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d76d70 0x562333d74770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.822+0000 7f0633025740 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 0x562333d74230 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.822+0000 7f0633025740 1 --2- 172.21.15.107:0/4134708791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d73cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.822+0000 7f0633025740 1 -- 172.21.15.107:0/4134708791 >> 172.21.15.107:0/4134708791 conn(0x562333d68120 msgr2=0x562333d6de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.822+0000 7f060f400640 0 client.16605 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.822+0000 7f0633025740 1 -- 172.21.15.107:0/4134708791 shutdown_connections 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.822+0000 7f0633025740 1 -- 172.21.15.107:0/4134708791 wait complete. 2024-09-19T04:28:25.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.822+0000 7f0633025740 20 client.16605 trim_cache size 0 max 16384 2024-09-19T04:28:25.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.839+0000 7f0633025740 1 Processor -- start 2024-09-19T04:28:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f0633025740 1 -- start start 2024-09-19T04:28:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f0633025740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f0633025740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 0x562333d8e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f0633025740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d76d70 0x562333d8ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f0633025740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06281afa70 con 0x562333d6c4b0 2024-09-19T04:28:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f0633025740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f06281b15e0 con 0x562333d6bb90 2024-09-19T04:28:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f0633025740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f06281b0170 con 0x562333d76d70 2024-09-19T04:28:25.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f062e600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f062d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 0x562333d8e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f062e600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d745c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:50886/0 (socket says 172.21.15.107:50886) 2024-09-19T04:28:25.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f062e600640 1 -- 172.21.15.107:0/1766825579 learned_addr learned my addr 172.21.15.107:0/1766825579 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:25.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f062e600640 1 --2- 172.21.15.107:0/1766825579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d76d70 0x562333d8ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f062e600640 1 -- 172.21.15.107:0/1766825579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d76d70 msgr2=0x562333d8ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f062e600640 1 --2- 172.21.15.107:0/1766825579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d76d70 0x562333d8ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f062e600640 1 -- 172.21.15.107:0/1766825579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 msgr2=0x562333d8e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f062e600640 1 --2- 172.21.15.107:0/1766825579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 0x562333d8e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.840+0000 7f062e600640 1 -- 172.21.15.107:0/1766825579 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562333d896c0 con 0x562333d6bb90 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.841+0000 7f062e600640 1 --2- 172.21.15.107:0/1766825579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d745c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0620009f00 tx=0x7f0620009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14024531f210eca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.841+0000 7f061a000640 1 -- 172.21.15.107:0/1766825579 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620036070 con 0x562333d6bb90 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.841+0000 7f061a000640 1 -- 172.21.15.107:0/1766825579 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f062001f040 con 0x562333d6bb90 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.841+0000 7f061a000640 1 -- 172.21.15.107:0/1766825579 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062001e040 con 0x562333d6bb90 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 -- 172.21.15.107:0/1766825579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 msgr2=0x562333d745c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 --2- 172.21.15.107:0/1766825579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d745c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0620009f00 tx=0x7f0620009f90 comp rx=0 tx=0).stop 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 -- 172.21.15.107:0/1766825579 shutdown_connections 2024-09-19T04:28:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 --2- 172.21.15.107:0/1766825579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d76d70 0x562333d8ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 --2- 172.21.15.107:0/1766825579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d6c4b0 0x562333d8e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 --2- 172.21.15.107:0/1766825579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6bb90 0x562333d745c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 -- 172.21.15.107:0/1766825579 >> 172.21.15.107:0/1766825579 conn(0x562333d68120 msgr2=0x562333d72de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 -- 172.21.15.107:0/1766825579 shutdown_connections 2024-09-19T04:28:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 -- 172.21.15.107:0/1766825579 wait complete. 2024-09-19T04:28:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 -- start start 2024-09-19T04:28:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.842+0000 7f0633025740 1 Processor -- start 2024-09-19T04:28:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f0633025740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d7f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f0633025740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6c4b0 0x562333d7a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f0633025740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d76d70 0x562333d7ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f0633025740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06281b0f10 con 0x562333d76d70 2024-09-19T04:28:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f0633025740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f06281b0840 con 0x562333d6c4b0 2024-09-19T04:28:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f0633025740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f06281ada70 con 0x562333d6bb90 2024-09-19T04:28:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f0633025740 10 client.0 _ll_register_callbacks cb 0x7f0614005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f062e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d76d70 0x562333d7ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f062e600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d7f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f062e600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d7f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35202/0 (socket says 172.21.15.107:35202) 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f062e600640 1 -- 172.21.15.107:0/653364648 learned_addr learned my addr 172.21.15.107:0/653364648 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.843+0000 7f062d200640 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6c4b0 0x562333d7a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f062e600640 1 -- 172.21.15.107:0/653364648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6c4b0 msgr2=0x562333d7a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f062e600640 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6c4b0 0x562333d7a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f062e600640 1 -- 172.21.15.107:0/653364648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d76d70 msgr2=0x562333d7ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f062e600640 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d76d70 0x562333d7ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f062e600640 1 -- 172.21.15.107:0/653364648 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562333d7b310 con 0x562333d6bb90 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f062e600640 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d7f3b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f062001c040 tx=0x7f0620005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=def278b085dceb58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f060f400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f060f400640 1 -- 172.21.15.107:0/653364648 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620044060 con 0x562333d6bb90 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f060f400640 1 -- 172.21.15.107:0/653364648 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f0620004630 con 0x562333d6bb90 2024-09-19T04:28:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f0633025740 1 -- 172.21.15.107:0/653364648 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f062001f040 con 0x562333d6bb90 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f060f400640 1 -- 172.21.15.107:0/653364648 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0620036040 con 0x562333d6bb90 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f0633025740 10 client.26629 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.844+0000 7f0633025740 1 -- 172.21.15.107:0/653364648 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562333df1790 con 0x562333d6bb90 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.845+0000 7f060f400640 1 -- 172.21.15.107:0/653364648 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f062001f040 con 0x562333d6bb90 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.845+0000 7f060f400640 1 -- 172.21.15.107:0/653364648 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f062003c070 con 0x562333d6bb90 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.845+0000 7f0633025740 10 client.26629 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.845+0000 7f0633025740 10 client.26629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.845+0000 7f0633025740 20 client.26629 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.845+0000 7f0633025740 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x562333df1b60 0x562333df3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.845+0000 7f0633025740 4 client.26629 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.845+0000 7f0633025740 1 -- 172.21.15.107:0/653364648 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562333cd9e60 con 0x562333df1b60 2024-09-19T04:28:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.845+0000 7f062d200640 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x562333df1b60 0x562333df3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f062d200640 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x562333df1b60 0x562333df3f50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128d9e62d4fcc64d server_cookie=498596249c736610 in_seq=0 out_seq=0 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f060f400640 10 client.26629 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f060f400640 1 -- 172.21.15.107:0/653364648 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562333cd9e60 con 0x562333df1b60 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f060f400640 10 client.26629 handle_command_reply: tid=0 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 client.26629 shutdown 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 -- 172.21.15.107:0/653364648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 msgr2=0x562333d7f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d7f3b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f062001c040 tx=0x7f0620005fd0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 -- 172.21.15.107:0/653364648 shutdown_connections 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x562333df1b60 0x562333df3f50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562333d76d70 0x562333d7ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562333d6c4b0 0x562333d7a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f060f400640 0 client.26629 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 --2- 172.21.15.107:0/653364648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562333d6bb90 0x562333d7f3b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 -- 172.21.15.107:0/653364648 >> 172.21.15.107:0/653364648 conn(0x562333d68120 msgr2=0x562333d6d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 -- 172.21.15.107:0/653364648 shutdown_connections 2024-09-19T04:28:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.846+0000 7f0633025740 1 -- 172.21.15.107:0/653364648 wait complete. 2024-09-19T04:28:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0633025740 20 client.26629 trim_cache size 0 max 16384 2024-09-19T04:28:25.854 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:28:25.854 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:28:25.854 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:28:25.855 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:28:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 -- 172.21.15.107:0/2846902328 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06100a33b0 msgr2=0x7f06100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06100a33b0 0x7f06100a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f061c0cd800 tx=0x7f061c0d3b30 comp rx=0 tx=0).stop 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 -- 172.21.15.107:0/2846902328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628103ff0 msgr2=0x7f062819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628103ff0 0x7f062819a620 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f061c00ea60 tx=0x7f061c00ee60 comp rx=0 tx=0).stop 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 -- 172.21.15.107:0/2846902328 shutdown_connections 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06100a33b0 0x7f06100a5870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628109120 0x7f062819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06281049a0 0x7f062819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 --2- 172.21.15.107:0/2846902328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628103ff0 0x7f062819a620 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 -- 172.21.15.107:0/2846902328 >> 172.21.15.107:0/2846902328 conn(0x7f06280ffa90 msgr2=0x7f0628100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 -- 172.21.15.107:0/2846902328 shutdown_connections 2024-09-19T04:28:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:25.847+0000 7f0630600640 1 -- 172.21.15.107:0/2846902328 wait complete. 2024-09-19T04:28:26.689 INFO:tasks.workunit.client.0.smithi107.stdout: 1 256143 2.42 MB/sec execute 284 sec latency 1124.325 ms 2024-09-19T04:28:26.813 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:28:26.813 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:28:27.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:26 smithi107 ceph-mon[30659]: pgmap v250: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-09-19T04:28:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:26 smithi115 ceph-mon[35477]: pgmap v250: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-09-19T04:28:27.217 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:26 smithi145 ceph-mon[35096]: pgmap v250: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-09-19T04:28:27.689 INFO:tasks.workunit.client.0.smithi107.stdout: 1 256143 2.42 MB/sec execute 285 sec latency 2124.413 ms 2024-09-19T04:28:27.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.863+0000 7f44e3e00640 1 -- 172.21.15.107:0/242465873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e41047e0 msgr2=0x7f44e4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:27.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.863+0000 7f44e3e00640 1 --2- 172.21.15.107:0/242465873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e41047e0 0x7f44e4108830 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009900 tx=0x7f44cc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:28:27.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.863+0000 7f44e3e00640 1 -- 172.21.15.107:0/242465873 shutdown_connections 2024-09-19T04:28:27.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.863+0000 7f44e3e00640 1 --2- 172.21.15.107:0/242465873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44e4108f60 0x7f44e4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:27.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.863+0000 7f44e3e00640 1 --2- 172.21.15.107:0/242465873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e41047e0 0x7f44e4108830 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:27.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.863+0000 7f44e3e00640 1 --2- 172.21.15.107:0/242465873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e4103e30 0x7f44e4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:27.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.863+0000 7f44e3e00640 1 -- 172.21.15.107:0/242465873 >> 172.21.15.107:0/242465873 conn(0x7f44e40ff910 msgr2=0x7f44e4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:27.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.863+0000 7f44e3e00640 1 -- 172.21.15.107:0/242465873 shutdown_connections 2024-09-19T04:28:27.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.863+0000 7f44e3e00640 1 -- 172.21.15.107:0/242465873 wait complete. 2024-09-19T04:28:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.864+0000 7f44e3e00640 1 Processor -- start 2024-09-19T04:28:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.864+0000 7f44e3e00640 1 -- start start 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e4103e30 0x7f44e419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44e41047e0 0x7f44e419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e4108f60 0x7f44e419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4113fc0 con 0x7f44e4108f60 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e3e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4113e40 con 0x7f44e4103e30 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4114140 con 0x7f44e41047e0 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e4103e30 0x7f44e419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e4108f60 0x7f44e419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e4103e30 0x7f44e419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:50916/0 (socket says 172.21.15.107:50916) 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e2a00640 1 -- 172.21.15.107:0/716958220 learned_addr learned my addr 172.21.15.107:0/716958220 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44dbe00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44e41047e0 0x7f44e419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e2a00640 1 -- 172.21.15.107:0/716958220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44e41047e0 msgr2=0x7f44e419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:27.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e2a00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44e41047e0 0x7f44e419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:27.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e2a00640 1 -- 172.21.15.107:0/716958220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e4108f60 msgr2=0x7f44e419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:27.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e2a00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e4108f60 0x7f44e419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:27.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e2a00640 1 -- 172.21.15.107:0/716958220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e419f430 con 0x7f44e4103e30 2024-09-19T04:28:27.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.865+0000 7f44e3400640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e4108f60 0x7f44e419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:27.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.866+0000 7f44e2a00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e4103e30 0x7f44e419a480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f44d400ea60 tx=0x7f44d400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:27.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.866+0000 7f44e8a00640 1 -- 172.21.15.107:0/716958220 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d400cde0 con 0x7f44e4103e30 2024-09-19T04:28:27.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.866+0000 7f44e8a00640 1 -- 172.21.15.107:0/716958220 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d40035f0 con 0x7f44e4103e30 2024-09-19T04:28:27.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.866+0000 7f44e8a00640 1 -- 172.21.15.107:0/716958220 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4010670 con 0x7f44e4103e30 2024-09-19T04:28:27.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.866+0000 7f44e3e00640 1 -- 172.21.15.107:0/716958220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e419f720 con 0x7f44e4103e30 2024-09-19T04:28:27.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.866+0000 7f44e3e00640 1 -- 172.21.15.107:0/716958220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e41a7000 con 0x7f44e4103e30 2024-09-19T04:28:27.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.867+0000 7f44e3e00640 1 -- 172.21.15.107:0/716958220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e4105310 con 0x7f44e4103e30 2024-09-19T04:28:27.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.869+0000 7f44e8a00640 1 -- 172.21.15.107:0/716958220 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f44d40037a0 con 0x7f44e4103e30 2024-09-19T04:28:27.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.871+0000 7f44e8a00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f44c80a33b0 0x7f44c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:27.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.871+0000 7f44e8a00640 1 -- 172.21.15.107:0/716958220 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f44d40d0100 con 0x7f44e4103e30 2024-09-19T04:28:27.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.871+0000 7f44dbe00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f44c80a33b0 0x7f44c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:27.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.871+0000 7f44dbe00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f44c80a33b0 0x7f44c80a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44e40ffd30 tx=0x7f44cc005a20 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:27.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:27.875+0000 7f44e8a00640 1 -- 172.21.15.107:0/716958220 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f44d409caa0 con 0x7f44e4103e30 2024-09-19T04:28:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.086+0000 7f44e3e00640 1 -- 172.21.15.107:0/716958220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44e4104210 con 0x7f44e4103e30 2024-09-19T04:28:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.087+0000 7f44e8a00640 1 -- 172.21.15.107:0/716958220 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11996 (secure 0 0 0) 0x7f44d409c3a0 con 0x7f44e4103e30 2024-09-19T04:28:28.089 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:28:28.089 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":45,"btime":"2024-09-19T04:27:52:688570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:27:51.707236+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:28:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 45 2024-09-19T04:28:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.089+0000 7f44e3e00640 1 -- 172.21.15.107:0/716958220 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f44c80a33b0 msgr2=0x7f44c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.089+0000 7f44e3e00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f44c80a33b0 0x7f44c80a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44e40ffd30 tx=0x7f44cc005a20 comp rx=0 tx=0).stop 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.089+0000 7f44e3e00640 1 -- 172.21.15.107:0/716958220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e4103e30 msgr2=0x7f44e419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.089+0000 7f44e3e00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e4103e30 0x7f44e419a480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f44d400ea60 tx=0x7f44d400ef20 comp rx=0 tx=0).stop 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.090+0000 7f44e3e00640 1 -- 172.21.15.107:0/716958220 shutdown_connections 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.090+0000 7f44e3e00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f44c80a33b0 0x7f44c80a5870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.090+0000 7f44e3e00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e4108f60 0x7f44e419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.090+0000 7f44e3e00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44e41047e0 0x7f44e419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.090+0000 7f44e3e00640 1 --2- 172.21.15.107:0/716958220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e4103e30 0x7f44e419a480 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.090+0000 7f44e3e00640 1 -- 172.21.15.107:0/716958220 >> 172.21.15.107:0/716958220 conn(0x7f44e40ff910 msgr2=0x7f44e4100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.090+0000 7f44e3e00640 1 -- 172.21.15.107:0/716958220 shutdown_connections 2024-09-19T04:28:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:28.090+0000 7f44e3e00640 1 -- 172.21.15.107:0/716958220 wait complete. 2024-09-19T04:28:28.690 INFO:tasks.workunit.client.0.smithi107.stdout: 1 256143 2.41 MB/sec execute 286 sec latency 3124.477 ms 2024-09-19T04:28:28.860 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:28:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:28 smithi115 ceph-mon[35477]: pgmap v251: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 555 KiB/s wr, 63 op/s 2024-09-19T04:28:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:28 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/716958220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:28:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:28 smithi145 ceph-mon[35096]: pgmap v251: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 555 KiB/s wr, 63 op/s 2024-09-19T04:28:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:28 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/716958220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:28:29.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:28 smithi107 ceph-mon[30659]: pgmap v251: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 555 KiB/s wr, 63 op/s 2024-09-19T04:28:29.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:28 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/716958220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:28:29.280 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:29.690 INFO:tasks.workunit.client.0.smithi107.stdout: 1 256211 2.40 MB/sec execute 287 sec latency 3657.702 ms 2024-09-19T04:28:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.889+0000 7f9824000640 1 -- 172.21.15.107:0/4127627748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f981c101420 msgr2=0x7f981c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.889+0000 7f9824000640 1 --2- 172.21.15.107:0/4127627748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f981c101420 0x7f981c10d950 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9814009ca0 tx=0x7f981402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.889+0000 7f9824000640 1 -- 172.21.15.107:0/4127627748 shutdown_connections 2024-09-19T04:28:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.889+0000 7f9824000640 1 --2- 172.21.15.107:0/4127627748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f981c101420 0x7f981c10d950 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.889+0000 7f9824000640 1 --2- 172.21.15.107:0/4127627748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f981c100a80 0x7f981c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.889+0000 7f9824000640 1 --2- 172.21.15.107:0/4127627748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981c106a80 0x7f981c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.889+0000 7f9824000640 1 -- 172.21.15.107:0/4127627748 >> 172.21.15.107:0/4127627748 conn(0x7f981c0fc7a0 msgr2=0x7f981c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.889+0000 7f9824000640 1 -- 172.21.15.107:0/4127627748 shutdown_connections 2024-09-19T04:28:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.889+0000 7f9824000640 1 -- 172.21.15.107:0/4127627748 wait complete. 2024-09-19T04:28:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.890+0000 7f9824000640 1 Processor -- start 2024-09-19T04:28:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.890+0000 7f9824000640 1 -- start start 2024-09-19T04:28:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9824000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981c100a80 0x7f981c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9824000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f981c101420 0x7f981c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9824000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f981c106a80 0x7f981c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9824000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f981c110ed0 con 0x7f981c106a80 2024-09-19T04:28:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9824000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f981c110d50 con 0x7f981c101420 2024-09-19T04:28:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9824000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f981c111050 con 0x7f981c100a80 2024-09-19T04:28:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9821600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981c100a80 0x7f981c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9820c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f981c101420 0x7f981c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9822000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f981c106a80 0x7f981c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9820c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f981c101420 0x7f981c19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36264/0 (socket says 172.21.15.107:36264) 2024-09-19T04:28:29.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9820c00640 1 -- 172.21.15.107:0/3799911499 learned_addr learned my addr 172.21.15.107:0/3799911499 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:29.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.891+0000 7f9821600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981c100a80 0x7f981c19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33858/0 (socket says 172.21.15.107:33858) 2024-09-19T04:28:29.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.892+0000 7f9821600640 1 -- 172.21.15.107:0/3799911499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f981c101420 msgr2=0x7f981c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.892+0000 7f9821600640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f981c101420 0x7f981c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.892+0000 7f9821600640 1 -- 172.21.15.107:0/3799911499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f981c106a80 msgr2=0x7f981c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.892+0000 7f9821600640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f981c106a80 0x7f981c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.892+0000 7f9821600640 1 -- 172.21.15.107:0/3799911499 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981c19f520 con 0x7f981c100a80 2024-09-19T04:28:29.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.892+0000 7f9821600640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981c100a80 0x7f981c19a570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f981000c970 tx=0x7f981000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.893+0000 7f981a000640 1 -- 172.21.15.107:0/3799911499 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810007bb0 con 0x7f981c100a80 2024-09-19T04:28:29.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.893+0000 7f981a000640 1 -- 172.21.15.107:0/3799911499 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9810007d50 con 0x7f981c100a80 2024-09-19T04:28:29.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.893+0000 7f981a000640 1 -- 172.21.15.107:0/3799911499 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98100187c0 con 0x7f981c100a80 2024-09-19T04:28:29.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.893+0000 7f9824000640 1 -- 172.21.15.107:0/3799911499 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981c19f810 con 0x7f981c100a80 2024-09-19T04:28:29.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.893+0000 7f9824000640 1 -- 172.21.15.107:0/3799911499 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981c1a70f0 con 0x7f981c100a80 2024-09-19T04:28:29.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.897+0000 7f981a000640 1 -- 172.21.15.107:0/3799911499 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f981000f450 con 0x7f981c100a80 2024-09-19T04:28:29.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.897+0000 7f9826a2f740 1 Processor -- start 2024-09-19T04:28:29.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.897+0000 7f9826a2f740 1 -- start start 2024-09-19T04:28:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2acc70 0x55b6cb2ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2ad590 0x55b6cb2b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2ba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9826a2f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b6cb2133e0 con 0x55b6cb2acc70 2024-09-19T04:28:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9826a2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b6cb1bad80 con 0x55b6cb2ad590 2024-09-19T04:28:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9826a2f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b6cb20f3a0 con 0x55b6cb2b7e50 2024-09-19T04:28:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9821600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2ad590 0x55b6cb2b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9822000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2acc70 0x55b6cb2ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9821600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2ad590 0x55b6cb2b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36280/0 (socket says 172.21.15.107:36280) 2024-09-19T04:28:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9821600640 1 -- 172.21.15.107:0/571120935 learned_addr learned my addr 172.21.15.107:0/571120935 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.898+0000 7f9820c00640 1 --2- 172.21.15.107:0/571120935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2ba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.899+0000 7f981a000640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98040a3430 0x7f98040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.899+0000 7f9821600640 1 -- 172.21.15.107:0/571120935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2b7e50 msgr2=0x55b6cb2ba240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.899+0000 7f9821600640 1 --2- 172.21.15.107:0/571120935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2ba240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.899+0000 7f9821600640 1 -- 172.21.15.107:0/571120935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2acc70 msgr2=0x55b6cb2ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.899+0000 7f9821600640 1 --2- 172.21.15.107:0/571120935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2acc70 0x55b6cb2ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.899+0000 7f9821600640 1 -- 172.21.15.107:0/571120935 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6cb1d9ae0 con 0x55b6cb2ad590 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.899+0000 7f9822000640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98040a3430 0x7f98040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.899+0000 7f981a000640 1 -- 172.21.15.107:0/3799911499 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f98100cfc10 con 0x7f981c100a80 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.899+0000 7f9821600640 1 --2- 172.21.15.107:0/571120935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2ad590 0x55b6cb2b7910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f98100cd260 tx=0x7f981006dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23e5ebd388240dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.899+0000 7f980e600640 1 -- 172.21.15.107:0/571120935 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98100d3050 con 0x55b6cb2ad590 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f980e600640 1 -- 172.21.15.107:0/571120935 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981006d110 con 0x55b6cb2ad590 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f980e600640 1 -- 172.21.15.107:0/571120935 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981009c5b0 con 0x55b6cb2ad590 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f9826a2f740 1 -- 172.21.15.107:0/571120935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2ad590 msgr2=0x55b6cb2b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f9826a2f740 1 --2- 172.21.15.107:0/571120935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2ad590 0x55b6cb2b7910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f98100cd260 tx=0x7f981006dc30 comp rx=0 tx=0).stop 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f9822000640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98040a3430 0x7f98040a58f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9814006010 tx=0x7f98140095c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f9826a2f740 1 -- 172.21.15.107:0/571120935 shutdown_connections 2024-09-19T04:28:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f9826a2f740 1 --2- 172.21.15.107:0/571120935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2ba240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f9826a2f740 1 --2- 172.21.15.107:0/571120935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2ad590 0x55b6cb2b7910 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f9826a2f740 1 --2- 172.21.15.107:0/571120935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2acc70 0x55b6cb2ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f9826a2f740 1 -- 172.21.15.107:0/571120935 >> 172.21.15.107:0/571120935 conn(0x55b6cb2a9200 msgr2=0x55b6cb2a95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.900+0000 7f9826a2f740 1 -- 172.21.15.107:0/571120935 shutdown_connections 2024-09-19T04:28:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.901+0000 7f9826a2f740 1 -- 172.21.15.107:0/571120935 wait complete. 2024-09-19T04:28:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.901+0000 7f9826a2f740 1 -- start start 2024-09-19T04:28:29.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.901+0000 7f9826a2f740 1 Processor -- start 2024-09-19T04:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2acc70 0x55b6cb2b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 0x55b6cb2b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9826a2f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b6cb2baa20 con 0x55b6cb2b7e50 2024-09-19T04:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9826a2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b6cb1ce0f0 con 0x55b6cb2acc70 2024-09-19T04:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9826a2f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b6cb2baba0 con 0x55b6cb2ad590 2024-09-19T04:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9826a2f740 10 client.0 _ll_register_callbacks cb 0x55b6cb1b5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9821600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2acc70 0x55b6cb2b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9821600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2acc70 0x55b6cb2b4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36286/0 (socket says 172.21.15.107:36286) 2024-09-19T04:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9821600640 1 -- 172.21.15.107:0/1508973613 learned_addr learned my addr 172.21.15.107:0/1508973613 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9820c00640 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 0x55b6cb2b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9822000640 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9821600640 1 -- 172.21.15.107:0/1508973613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 msgr2=0x55b6cb2b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9821600640 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 0x55b6cb2b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9821600640 1 -- 172.21.15.107:0/1508973613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2b7e50 msgr2=0x55b6cb2b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9821600640 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.902+0000 7f9821600640 1 -- 172.21.15.107:0/1508973613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6caf843c0 con 0x55b6cb2acc70 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.903+0000 7f9822000640 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.903+0000 7f9821600640 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2acc70 0x55b6cb2b4cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9810098310 tx=0x7f98100cd2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3dfabe853a324f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.903+0000 7f9802a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:28:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.903+0000 7f9802a00640 1 -- 172.21.15.107:0/1508973613 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981009b9f0 con 0x55b6cb2acc70 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.903+0000 7f9802a00640 1 -- 172.21.15.107:0/1508973613 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f98100d3170 con 0x55b6cb2acc70 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.903+0000 7f9826a2f740 1 -- 172.21.15.107:0/1508973613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6cb220950 con 0x55b6cb2acc70 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.903+0000 7f9802a00640 1 -- 172.21.15.107:0/1508973613 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f981009b810 con 0x55b6cb2acc70 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.903+0000 7f9826a2f740 10 client.26678 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.903+0000 7f9826a2f740 1 -- 172.21.15.107:0/1508973613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b6cb2c69e0 con 0x55b6cb2acc70 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.903+0000 7f9802a00640 1 -- 172.21.15.107:0/1508973613 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b6cb220950 con 0x55b6cb2acc70 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.904+0000 7f9802a00640 1 -- 172.21.15.107:0/1508973613 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f98100d8070 con 0x55b6cb2acc70 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.904+0000 7f9826a2f740 10 client.26678 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.904+0000 7f9826a2f740 10 client.26678 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.904+0000 7f9826a2f740 20 client.26678 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.904+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b6cb2c7d30 0x55b6cb2ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.904+0000 7f9826a2f740 4 client.26678 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.904+0000 7f9826a2f740 1 -- 172.21.15.107:0/1508973613 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6cb21ae60 con 0x55b6cb2c7d30 2024-09-19T04:28:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.904+0000 7f9820c00640 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b6cb2c7d30 0x55b6cb2ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.905+0000 7f9820c00640 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b6cb2c7d30 0x55b6cb2ca120 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4dfaa04f4a7c5c11 server_cookie=df2be691f9b0d19c in_seq=0 out_seq=0 2024-09-19T04:28:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.905+0000 7f9802a00640 10 client.26678 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:28:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.905+0000 7f9802a00640 1 -- 172.21.15.107:0/1508973613 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6cb21ae60 con 0x55b6cb2c7d30 2024-09-19T04:28:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9802a00640 10 client.26678 handle_command_reply: tid=0 2024-09-19T04:28:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 client.26678 shutdown 2024-09-19T04:28:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 -- 172.21.15.107:0/1508973613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2acc70 msgr2=0x55b6cb2b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2acc70 0x55b6cb2b4cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9810098310 tx=0x7f98100cd2c0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 -- 172.21.15.107:0/1508973613 shutdown_connections 2024-09-19T04:28:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b6cb2c7d30 0x55b6cb2ca120 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 0x55b6cb2b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1508973613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2acc70 0x55b6cb2b4cb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 -- 172.21.15.107:0/1508973613 >> 172.21.15.107:0/1508973613 conn(0x55b6cb2a9200 msgr2=0x55b6cb2aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9802a00640 0 client.26678 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:28:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 -- 172.21.15.107:0/1508973613 shutdown_connections 2024-09-19T04:28:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 1 -- 172.21.15.107:0/1508973613 wait complete. 2024-09-19T04:28:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.906+0000 7f9826a2f740 20 client.26678 trim_cache size 0 max 16384 2024-09-19T04:28:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.919+0000 7f9826a2f740 1 Processor -- start 2024-09-19T04:28:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9826a2f740 1 -- start start 2024-09-19T04:28:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2acc70 0x55b6cb2b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9826a2f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1afa10 con 0x55b6cb2acc70 2024-09-19T04:28:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9826a2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1b1580 con 0x55b6cb2b7e50 2024-09-19T04:28:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9826a2f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1b0110 con 0x55b6cb2ad590 2024-09-19T04:28:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9820c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9820c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33868/0 (socket says 172.21.15.107:33868) 2024-09-19T04:28:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9821600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2acc70 0x55b6cb2b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9820c00640 1 -- 172.21.15.107:0/3861599490 learned_addr learned my addr 172.21.15.107:0/3861599490 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.920+0000 7f9822000640 1 --2- 172.21.15.107:0/3861599490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.921+0000 7f9820c00640 1 -- 172.21.15.107:0/3861599490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2b7e50 msgr2=0x55b6cb2cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.921+0000 7f9820c00640 1 --2- 172.21.15.107:0/3861599490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.921+0000 7f9820c00640 1 -- 172.21.15.107:0/3861599490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2acc70 msgr2=0x55b6cb2b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.921+0000 7f9820c00640 1 --2- 172.21.15.107:0/3861599490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2acc70 0x55b6cb2b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.921+0000 7f9820c00640 1 -- 172.21.15.107:0/3861599490 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6cb2a9030 con 0x55b6cb2ad590 2024-09-19T04:28:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.921+0000 7f9820c00640 1 --2- 172.21.15.107:0/3861599490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cd140 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9808005fd0 tx=0x7f980800bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154ea6c568139d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.921+0000 7f980dc00640 1 -- 172.21.15.107:0/3861599490 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808003e80 con 0x55b6cb2ad590 2024-09-19T04:28:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.921+0000 7f980dc00640 1 -- 172.21.15.107:0/3861599490 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9808005500 con 0x55b6cb2ad590 2024-09-19T04:28:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.921+0000 7f980dc00640 1 -- 172.21.15.107:0/3861599490 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98080116a0 con 0x55b6cb2ad590 2024-09-19T04:28:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.922+0000 7f9826a2f740 1 -- 172.21.15.107:0/3861599490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 msgr2=0x55b6cb2cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.922+0000 7f9826a2f740 1 --2- 172.21.15.107:0/3861599490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cd140 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9808005fd0 tx=0x7f980800bec0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.922+0000 7f9826a2f740 1 -- 172.21.15.107:0/3861599490 shutdown_connections 2024-09-19T04:28:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.922+0000 7f9826a2f740 1 --2- 172.21.15.107:0/3861599490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.922+0000 7f9826a2f740 1 --2- 172.21.15.107:0/3861599490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cd140 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.922+0000 7f9826a2f740 1 --2- 172.21.15.107:0/3861599490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2acc70 0x55b6cb2b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.922+0000 7f9826a2f740 1 -- 172.21.15.107:0/3861599490 >> 172.21.15.107:0/3861599490 conn(0x55b6cb2a9200 msgr2=0x55b6cb2b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.922+0000 7f9826a2f740 1 -- 172.21.15.107:0/3861599490 shutdown_connections 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.922+0000 7f9826a2f740 1 -- 172.21.15.107:0/3861599490 wait complete. 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.922+0000 7f9826a2f740 1 -- start start 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9826a2f740 1 Processor -- start 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2acc70 0x55b6cb2cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9826a2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9826a2f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1b0eb0 con 0x55b6cb2ad590 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9826a2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1b07e0 con 0x55b6cb2b7e50 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9826a2f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1ada10 con 0x55b6cb2acc70 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9826a2f740 10 client.0 _ll_register_callbacks cb 0x7f9814005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9820c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9820c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33208/0 (socket says 172.21.15.107:33208) 2024-09-19T04:28:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9820c00640 1 -- 172.21.15.107:0/1903389347 learned_addr learned my addr 172.21.15.107:0/1903389347 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9821600640 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2acc70 0x55b6cb2cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.923+0000 7f9822000640 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.924+0000 7f9821600640 1 -- 172.21.15.107:0/1903389347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2b7e50 msgr2=0x55b6cb2cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.924+0000 7f9821600640 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.924+0000 7f9821600640 1 -- 172.21.15.107:0/1903389347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2ad590 msgr2=0x55b6cb2cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.924+0000 7f9821600640 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.924+0000 7f9821600640 1 -- 172.21.15.107:0/1903389347 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6cb2a4770 con 0x55b6cb2acc70 2024-09-19T04:28:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.924+0000 7f9820c00640 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.924+0000 7f9821600640 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2acc70 0x55b6cb2cda80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9810005940 tx=0x7f98100cd420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a564feca7db794e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.924+0000 7f9802a00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:28:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.925+0000 7f9802a00640 1 -- 172.21.15.107:0/1903389347 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981009b9f0 con 0x55b6cb2acc70 2024-09-19T04:28:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.925+0000 7f9802a00640 1 -- 172.21.15.107:0/1903389347 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f981009a1b0 con 0x55b6cb2acc70 2024-09-19T04:28:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.925+0000 7f9826a2f740 1 -- 172.21.15.107:0/1903389347 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9808005500 con 0x55b6cb2acc70 2024-09-19T04:28:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.925+0000 7f9802a00640 1 -- 172.21.15.107:0/1903389347 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f98100a13f0 con 0x55b6cb2acc70 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.925+0000 7f9826a2f740 10 client.26650 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.925+0000 7f9826a2f740 1 -- 172.21.15.107:0/1903389347 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b6cb332680 con 0x55b6cb2acc70 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.925+0000 7f9802a00640 1 -- 172.21.15.107:0/1903389347 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9808005500 con 0x55b6cb2acc70 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.925+0000 7f9802a00640 1 -- 172.21.15.107:0/1903389347 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f98100d8030 con 0x55b6cb2acc70 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.926+0000 7f9826a2f740 10 client.26650 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.926+0000 7f9826a2f740 10 client.26650 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.926+0000 7f9826a2f740 20 client.26650 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.926+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b6cb332c60 0x55b6cb335080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.926+0000 7f9826a2f740 4 client.26650 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.926+0000 7f9826a2f740 1 -- 172.21.15.107:0/1903389347 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6cb197010 con 0x55b6cb332c60 2024-09-19T04:28:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.926+0000 7f9820c00640 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b6cb332c60 0x55b6cb335080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9820c00640 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b6cb332c60 0x55b6cb335080 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4c53f40b0655268 server_cookie=89b6150dfb6a98d9 in_seq=0 out_seq=0 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9802a00640 10 client.26650 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9802a00640 1 -- 172.21.15.107:0/1903389347 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6cb197010 con 0x55b6cb332c60 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9802a00640 10 client.26650 handle_command_reply: tid=0 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 client.26650 shutdown 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 -- 172.21.15.107:0/1903389347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2acc70 msgr2=0x55b6cb2cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2acc70 0x55b6cb2cda80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9810005940 tx=0x7f98100cd420 comp rx=0 tx=0).stop 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 -- 172.21.15.107:0/1903389347 shutdown_connections 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b6cb332c60 0x55b6cb335080 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6cb2b7e50 0x55b6cb2cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9802a00640 0 client.26650 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6cb2ad590 0x55b6cb2cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 --2- 172.21.15.107:0/1903389347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b6cb2acc70 0x55b6cb2cda80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 -- 172.21.15.107:0/1903389347 >> 172.21.15.107:0/1903389347 conn(0x55b6cb2a9200 msgr2=0x55b6cb2b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 -- 172.21.15.107:0/1903389347 shutdown_connections 2024-09-19T04:28:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.927+0000 7f9826a2f740 1 -- 172.21.15.107:0/1903389347 wait complete. 2024-09-19T04:28:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9826a2f740 20 client.26650 trim_cache size 0 max 16384 2024-09-19T04:28:29.933 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:28:29.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9824000640 1 -- 172.21.15.107:0/3799911499 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98040a3430 msgr2=0x7f98040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9824000640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98040a3430 0x7f98040a58f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9814006010 tx=0x7f98140095c0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9824000640 1 -- 172.21.15.107:0/3799911499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981c100a80 msgr2=0x7f981c19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:29.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9824000640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981c100a80 0x7f981c19a570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f981000c970 tx=0x7f981000ce30 comp rx=0 tx=0).stop 2024-09-19T04:28:29.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9824000640 1 -- 172.21.15.107:0/3799911499 shutdown_connections 2024-09-19T04:28:29.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9824000640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f98040a3430 0x7f98040a58f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9824000640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f981c106a80 0x7f981c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9824000640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f981c101420 0x7f981c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9824000640 1 --2- 172.21.15.107:0/3799911499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981c100a80 0x7f981c19a570 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:29.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.928+0000 7f9824000640 1 -- 172.21.15.107:0/3799911499 >> 172.21.15.107:0/3799911499 conn(0x7f981c0fc7a0 msgr2=0x7f981c06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:29.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.929+0000 7f9824000640 1 -- 172.21.15.107:0/3799911499 shutdown_connections 2024-09-19T04:28:29.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:29.929+0000 7f9824000640 1 -- 172.21.15.107:0/3799911499 wait complete. 2024-09-19T04:28:30.690 INFO:tasks.workunit.client.0.smithi107.stdout: 1 256211 2.39 MB/sec execute 288 sec latency 1417.036 ms 2024-09-19T04:28:30.727 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:28:30.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:30 smithi107 ceph-mon[30659]: pgmap v252: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 555 KiB/s wr, 63 op/s 2024-09-19T04:28:31.096 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:30 smithi115 ceph-mon[35477]: pgmap v252: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 555 KiB/s wr, 63 op/s 2024-09-19T04:28:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:30 smithi145 ceph-mon[35096]: pgmap v252: 97 pgs: 97 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 555 KiB/s wr, 63 op/s 2024-09-19T04:28:31.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.628+0000 7f526f200640 1 -- 172.21.15.107:0/3852743117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f527010bfc0 msgr2=0x7f527010e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.628+0000 7f526f200640 1 --2- 172.21.15.107:0/3852743117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f527010bfc0 0x7f527010e460 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5268009900 tx=0x7f526802a480 comp rx=0 tx=0).stop 2024-09-19T04:28:31.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.628+0000 7f526f200640 1 -- 172.21.15.107:0/3852743117 shutdown_connections 2024-09-19T04:28:31.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.628+0000 7f526f200640 1 --2- 172.21.15.107:0/3852743117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f527010bfc0 0x7f527010e460 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.628+0000 7f526f200640 1 --2- 172.21.15.107:0/3852743117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52701034b0 0x7f527010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.628+0000 7f526f200640 1 --2- 172.21.15.107:0/3852743117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5270102b90 0x7f5270102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.628+0000 7f526f200640 1 -- 172.21.15.107:0/3852743117 >> 172.21.15.107:0/3852743117 conn(0x7f52700fc610 msgr2=0x7f52700fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.628+0000 7f526f200640 1 -- 172.21.15.107:0/3852743117 shutdown_connections 2024-09-19T04:28:31.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.628+0000 7f526f200640 1 -- 172.21.15.107:0/3852743117 wait complete. 2024-09-19T04:28:31.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.629+0000 7f526f200640 1 Processor -- start 2024-09-19T04:28:31.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.629+0000 7f526f200640 1 -- start start 2024-09-19T04:28:31.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5270102b90 0x7f527019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52701034b0 0x7f527019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526f200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f527010bfc0 0x7f527019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5270111dc0 con 0x7f5270102b90 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526f200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5270111c40 con 0x7f527010bfc0 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5270111f40 con 0x7f52701034b0 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5270102b90 0x7f527019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5270102b90 0x7f527019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33220/0 (socket says 172.21.15.107:33220) 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526de00640 1 -- 172.21.15.107:0/4222477574 learned_addr learned my addr 172.21.15.107:0/4222477574 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526d400640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52701034b0 0x7f527019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.630+0000 7f526e800640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f527010bfc0 0x7f527019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f526de00640 1 -- 172.21.15.107:0/4222477574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52701034b0 msgr2=0x7f527019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f526de00640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52701034b0 0x7f527019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f526de00640 1 -- 172.21.15.107:0/4222477574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f527010bfc0 msgr2=0x7f527019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f526de00640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f527010bfc0 0x7f527019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f526de00640 1 -- 172.21.15.107:0/4222477574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527019f510 con 0x7f5270102b90 2024-09-19T04:28:31.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f526d400640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52701034b0 0x7f527019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:31.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f526de00640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5270102b90 0x7f527019a560 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f526000eb90 tx=0x7f526000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f5266a00640 1 -- 172.21.15.107:0/4222477574 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260004d20 con 0x7f5270102b90 2024-09-19T04:28:31.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f5266a00640 1 -- 172.21.15.107:0/4222477574 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f526000ee90 con 0x7f5270102b90 2024-09-19T04:28:31.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f5266a00640 1 -- 172.21.15.107:0/4222477574 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260010830 con 0x7f5270102b90 2024-09-19T04:28:31.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f526f200640 1 -- 172.21.15.107:0/4222477574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527019f800 con 0x7f5270102b90 2024-09-19T04:28:31.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.631+0000 7f526f200640 1 -- 172.21.15.107:0/4222477574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52701a70e0 con 0x7f5270102b90 2024-09-19T04:28:31.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.634+0000 7f527764d740 1 Processor -- start 2024-09-19T04:28:31.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.634+0000 7f5266a00640 1 -- 172.21.15.107:0/4222477574 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f5260005020 con 0x7f5270102b90 2024-09-19T04:28:31.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f527764d740 1 -- start start 2024-09-19T04:28:31.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f527764d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fae40 0x559f890fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f527764d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fb760 0x559f89105ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f527764d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 0x559f89108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f527764d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559f890613e0 con 0x559f890fb760 2024-09-19T04:28:31.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f527764d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559f89008d80 con 0x559f890fae40 2024-09-19T04:28:31.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f527764d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559f8905d3a0 con 0x559f89106020 2024-09-19T04:28:31.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f526e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fae40 0x559f890fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f526e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fae40 0x559f890fb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36330/0 (socket says 172.21.15.107:36330) 2024-09-19T04:28:31.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f526e800640 1 -- 172.21.15.107:0/3612971417 learned_addr learned my addr 172.21.15.107:0/3612971417 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f526de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fb760 0x559f89105ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.635+0000 7f526d400640 1 --2- 172.21.15.107:0/3612971417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 0x559f89108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f526e800640 1 -- 172.21.15.107:0/3612971417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 msgr2=0x559f89108410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f526e800640 1 --2- 172.21.15.107:0/3612971417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 0x559f89108410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f526e800640 1 -- 172.21.15.107:0/3612971417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fb760 msgr2=0x559f89105ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f526e800640 1 --2- 172.21.15.107:0/3612971417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fb760 0x559f89105ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f526e800640 1 -- 172.21.15.107:0/3612971417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f89027ae0 con 0x559f890fae40 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f526d400640 1 --2- 172.21.15.107:0/3612971417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 0x559f89108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f526e800640 1 --2- 172.21.15.107:0/3612971417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fae40 0x559f890fb220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5268033000 tx=0x7f5268005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e4b87db685e0c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f5266a00640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f52540a3430 0x7f52540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f5266a00640 1 -- 172.21.15.107:0/4222477574 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f52600d0300 con 0x7f5270102b90 2024-09-19T04:28:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f526de00640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f52540a3430 0x7f52540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f525b000640 1 -- 172.21.15.107:0/3612971417 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268040070 con 0x559f890fae40 2024-09-19T04:28:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f525b000640 1 -- 172.21.15.107:0/3612971417 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52680317b0 con 0x559f890fae40 2024-09-19T04:28:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.636+0000 7f525b000640 1 -- 172.21.15.107:0/3612971417 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268035040 con 0x559f890fae40 2024-09-19T04:28:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.637+0000 7f527764d740 1 -- 172.21.15.107:0/3612971417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fae40 msgr2=0x559f890fb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.637+0000 7f526de00640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f52540a3430 0x7f52540a58f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x559f890f8230 tx=0x7f52600d3db0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.637+0000 7f527764d740 1 --2- 172.21.15.107:0/3612971417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fae40 0x559f890fb220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5268033000 tx=0x7f5268005d50 comp rx=0 tx=0).stop 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.637+0000 7f527764d740 1 -- 172.21.15.107:0/3612971417 shutdown_connections 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.637+0000 7f527764d740 1 --2- 172.21.15.107:0/3612971417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 0x559f89108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.637+0000 7f527764d740 1 --2- 172.21.15.107:0/3612971417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fb760 0x559f89105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.637+0000 7f527764d740 1 --2- 172.21.15.107:0/3612971417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fae40 0x559f890fb220 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.637+0000 7f527764d740 1 -- 172.21.15.107:0/3612971417 >> 172.21.15.107:0/3612971417 conn(0x559f890f73d0 msgr2=0x559f890f77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.637+0000 7f527764d740 1 -- 172.21.15.107:0/3612971417 shutdown_connections 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.638+0000 7f527764d740 1 -- 172.21.15.107:0/3612971417 wait complete. 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.638+0000 7f527764d740 1 -- start start 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.638+0000 7f527764d740 1 Processor -- start 2024-09-19T04:28:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f527764d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 0x559f89102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f527764d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 0x559f891033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f527764d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 0x559f89103930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f527764d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559f89108bf0 con 0x559f890fae40 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f527764d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559f8901c0f0 con 0x559f890fb760 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f527764d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559f89108d70 con 0x559f89106020 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f527764d740 10 client.0 _ll_register_callbacks cb 0x559f89003bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f526e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 0x559f89102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f526d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 0x559f891033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f526e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 0x559f89102eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33236/0 (socket says 172.21.15.107:33236) 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f526e800640 1 -- 172.21.15.107:0/3342487189 learned_addr learned my addr 172.21.15.107:0/3342487189 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f526e800640 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 0x559f89103930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f526e800640 1 -- 172.21.15.107:0/3342487189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 msgr2=0x559f89103930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f526e800640 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 0x559f89103930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f526e800640 1 -- 172.21.15.107:0/3342487189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 msgr2=0x559f891033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f526e800640 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 0x559f891033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.639+0000 7f526e800640 1 -- 172.21.15.107:0/3342487189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f88dd23c0 con 0x559f890fae40 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.640+0000 7f526d400640 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 0x559f891033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.640+0000 7f526e800640 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 0x559f89102eb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5268009850 tx=0x7f5268005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deaa6c1292946b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.640+0000 7f524f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.640+0000 7f524f400640 1 -- 172.21.15.107:0/3342487189 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268048070 con 0x559f890fae40 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.640+0000 7f524f400640 1 -- 172.21.15.107:0/3342487189 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f5268004020 con 0x559f890fae40 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.640+0000 7f527764d740 1 -- 172.21.15.107:0/3342487189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52680317b0 con 0x559f890fae40 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.640+0000 7f524f400640 1 -- 172.21.15.107:0/3342487189 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5268040070 con 0x559f890fae40 2024-09-19T04:28:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.640+0000 7f527764d740 10 client.16647 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.640+0000 7f527764d740 1 -- 172.21.15.107:0/3342487189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559f89114bc0 con 0x559f890fae40 2024-09-19T04:28:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.641+0000 7f524f400640 1 -- 172.21.15.107:0/3342487189 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52680317b0 con 0x559f890fae40 2024-09-19T04:28:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.641+0000 7f524f400640 1 -- 172.21.15.107:0/3342487189 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f5268054030 con 0x559f890fae40 2024-09-19T04:28:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.641+0000 7f527764d740 10 client.16647 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.641+0000 7f527764d740 10 client.16647 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:28:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.641+0000 7f527764d740 20 client.16647 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.641+0000 7f527764d740 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559f89115f10 0x559f89118300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.641+0000 7f527764d740 4 client.16647 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.641+0000 7f527764d740 1 -- 172.21.15.107:0/3342487189 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f8906e950 con 0x559f89115f10 2024-09-19T04:28:31.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.641+0000 7f526d400640 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559f89115f10 0x559f89118300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.642+0000 7f526d400640 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559f89115f10 0x559f89118300 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88df8ba5f516a7ed server_cookie=af964b9aaa24ab75 in_seq=0 out_seq=0 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.642+0000 7f524f400640 10 client.16647 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.643+0000 7f524f400640 1 -- 172.21.15.107:0/3342487189 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f8906e950 con 0x559f89115f10 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.643+0000 7f524f400640 10 client.16647 handle_command_reply: tid=0 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.643+0000 7f527764d740 1 client.16647 shutdown 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 1 -- 172.21.15.107:0/3342487189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 msgr2=0x559f89102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 0x559f89102eb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5268009850 tx=0x7f5268005460 comp rx=0 tx=0).stop 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 1 -- 172.21.15.107:0/3342487189 shutdown_connections 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559f89115f10 0x559f89118300 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f89106020 0x559f89103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 0x559f891033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 1 --2- 172.21.15.107:0/3342487189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 0x559f89102eb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 1 -- 172.21.15.107:0/3342487189 >> 172.21.15.107:0/3342487189 conn(0x559f890f73d0 msgr2=0x559f890fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f524f400640 0 client.16647 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:28:31.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 1 -- 172.21.15.107:0/3342487189 shutdown_connections 2024-09-19T04:28:31.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 1 -- 172.21.15.107:0/3342487189 wait complete. 2024-09-19T04:28:31.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.644+0000 7f527764d740 20 client.16647 trim_cache size 0 max 16384 2024-09-19T04:28:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f527764d740 1 Processor -- start 2024-09-19T04:28:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f527764d740 1 -- start start 2024-09-19T04:28:31.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f527764d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fae40 0x559f89104e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f527764d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 0x559f8911d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f527764d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f89106020 0x559f8911de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f527764d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52701af790 con 0x559f89106020 2024-09-19T04:28:31.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f527764d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f52701b1300 con 0x559f890fb760 2024-09-19T04:28:31.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f527764d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52701afe90 con 0x559f890fae40 2024-09-19T04:28:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f526e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f89106020 0x559f8911de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f526e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f89106020 0x559f8911de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33238/0 (socket says 172.21.15.107:33238) 2024-09-19T04:28:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f526e800640 1 -- 172.21.15.107:0/2288052296 learned_addr learned my addr 172.21.15.107:0/2288052296 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f526d400640 1 --2- 172.21.15.107:0/2288052296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 0x559f8911d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.658+0000 7f526e800640 1 --2- 172.21.15.107:0/2288052296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fae40 0x559f89104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.659+0000 7f526d400640 1 -- 172.21.15.107:0/2288052296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fae40 msgr2=0x559f89104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.659+0000 7f526d400640 1 --2- 172.21.15.107:0/2288052296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fae40 0x559f89104e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.659+0000 7f526d400640 1 -- 172.21.15.107:0/2288052296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f89106020 msgr2=0x559f8911de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.659+0000 7f526d400640 1 --2- 172.21.15.107:0/2288052296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f89106020 0x559f8911de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.659+0000 7f526d400640 1 -- 172.21.15.107:0/2288052296 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f89105820 con 0x559f890fb760 2024-09-19T04:28:31.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.659+0000 7f526e800640 1 --2- 172.21.15.107:0/2288052296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f89106020 0x559f8911de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:31.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.659+0000 7f526d400640 1 --2- 172.21.15.107:0/2288052296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 0x559f8911d900 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x559f891188a0 tx=0x7f525c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7984ae630598d682 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.659+0000 7f525a600640 1 -- 172.21.15.107:0/2288052296 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c00f8f0 con 0x559f890fb760 2024-09-19T04:28:31.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.659+0000 7f525a600640 1 -- 172.21.15.107:0/2288052296 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f525c005020 con 0x559f890fb760 2024-09-19T04:28:31.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.659+0000 7f525a600640 1 -- 172.21.15.107:0/2288052296 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c004b70 con 0x559f890fb760 2024-09-19T04:28:31.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.660+0000 7f527764d740 1 -- 172.21.15.107:0/2288052296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 msgr2=0x559f8911d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.660+0000 7f527764d740 1 --2- 172.21.15.107:0/2288052296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 0x559f8911d900 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x559f891188a0 tx=0x7f525c005f90 comp rx=0 tx=0).stop 2024-09-19T04:28:31.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.660+0000 7f527764d740 1 -- 172.21.15.107:0/2288052296 shutdown_connections 2024-09-19T04:28:31.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.660+0000 7f527764d740 1 --2- 172.21.15.107:0/2288052296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f89106020 0x559f8911de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.660+0000 7f527764d740 1 --2- 172.21.15.107:0/2288052296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f890fb760 0x559f8911d900 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.660+0000 7f527764d740 1 --2- 172.21.15.107:0/2288052296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fae40 0x559f89104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.660+0000 7f527764d740 1 -- 172.21.15.107:0/2288052296 >> 172.21.15.107:0/2288052296 conn(0x559f890f73d0 msgr2=0x559f89104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.660+0000 7f527764d740 1 -- 172.21.15.107:0/2288052296 shutdown_connections 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.660+0000 7f527764d740 1 -- 172.21.15.107:0/2288052296 wait complete. 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.661+0000 7f527764d740 1 -- start start 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.661+0000 7f527764d740 1 Processor -- start 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.661+0000 7f527764d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 0x559f8910e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.661+0000 7f527764d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fb760 0x559f890ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.661+0000 7f527764d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f89106020 0x559f890ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.661+0000 7f527764d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52701b0c30 con 0x559f890fae40 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.661+0000 7f527764d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f52701b0560 con 0x559f89106020 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.661+0000 7f527764d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52701ad790 con 0x559f890fb760 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.661+0000 7f527764d740 10 client.0 _ll_register_callbacks cb 0x7f52600d3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fb760 0x559f890ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 0x559f8910e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fb760 0x559f890ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33930/0 (socket says 172.21.15.107:33930) 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526d400640 1 -- 172.21.15.107:0/1732922441 learned_addr learned my addr 172.21.15.107:0/1732922441 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526d400640 1 -- 172.21.15.107:0/1732922441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f89106020 msgr2=0x559f890ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526e800640 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f89106020 0x559f890ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526d400640 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f89106020 0x559f890ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526d400640 1 -- 172.21.15.107:0/1732922441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 msgr2=0x559f8910e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526d400640 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 0x559f8910e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526d400640 1 -- 172.21.15.107:0/1732922441 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f89100370 con 0x559f890fb760 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526e800640 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f89106020 0x559f890ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:28:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f526d400640 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fb760 0x559f890ff8f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f525c0122d0 tx=0x7f525c012300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7273659022809c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.662+0000 7f524f400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f524f400640 1 -- 172.21.15.107:0/1732922441 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c0051d0 con 0x559f890fb760 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f524f400640 1 -- 172.21.15.107:0/1732922441 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f525c005370 con 0x559f890fb760 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f527764d740 1 -- 172.21.15.107:0/1732922441 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f525c005020 con 0x559f890fb760 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f524f400640 1 -- 172.21.15.107:0/1732922441 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f525c005570 con 0x559f890fb760 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f527764d740 10 client.26668 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f527764d740 1 -- 172.21.15.107:0/1732922441 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559f891805e0 con 0x559f890fb760 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f524f400640 1 -- 172.21.15.107:0/1732922441 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f525c005020 con 0x559f890fb760 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f524f400640 1 -- 172.21.15.107:0/1732922441 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f525c039020 con 0x559f890fb760 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f527764d740 10 client.26668 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f527764d740 10 client.26668 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f527764d740 20 client.26668 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.663+0000 7f527764d740 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559f89180d20 0x559f89183140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.664+0000 7f527764d740 4 client.26668 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:28:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.664+0000 7f527764d740 1 -- 172.21.15.107:0/1732922441 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f89068e60 con 0x559f89180d20 2024-09-19T04:28:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.664+0000 7f526e800640 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559f89180d20 0x559f89183140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.664+0000 7f524f400640 10 client.26668 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:28:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.664+0000 7f526e800640 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559f89180d20 0x559f89183140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=debea6d5f27f30d4 server_cookie=542ec60b44de7ec6 in_seq=0 out_seq=0 2024-09-19T04:28:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f524f400640 1 -- 172.21.15.107:0/1732922441 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f89068e60 con 0x559f89180d20 2024-09-19T04:28:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f524f400640 10 client.26668 handle_command_reply: tid=0 2024-09-19T04:28:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 client.26668 shutdown 2024-09-19T04:28:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 -- 172.21.15.107:0/1732922441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fb760 msgr2=0x559f890ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fb760 0x559f890ff8f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f525c0122d0 tx=0x7f525c012300 comp rx=0 tx=0).stop 2024-09-19T04:28:31.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 -- 172.21.15.107:0/1732922441 shutdown_connections 2024-09-19T04:28:31.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x559f89180d20 0x559f89183140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f89106020 0x559f890ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f890fb760 0x559f890ff8f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 --2- 172.21.15.107:0/1732922441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f890fae40 0x559f8910e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f524f400640 0 client.26668 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:28:31.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 -- 172.21.15.107:0/1732922441 >> 172.21.15.107:0/1732922441 conn(0x559f890f73d0 msgr2=0x559f89104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 -- 172.21.15.107:0/1732922441 shutdown_connections 2024-09-19T04:28:31.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 1 -- 172.21.15.107:0/1732922441 wait complete. 2024-09-19T04:28:31.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.665+0000 7f527764d740 20 client.26668 trim_cache size 0 max 16384 2024-09-19T04:28:31.672 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:28:31.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 -- 172.21.15.107:0/4222477574 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f52540a3430 msgr2=0x7f52540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f52540a3430 0x7f52540a58f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x559f890f8230 tx=0x7f52600d3db0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 -- 172.21.15.107:0/4222477574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5270102b90 msgr2=0x7f527019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:31.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5270102b90 0x7f527019a560 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f526000eb90 tx=0x7f526000c6a0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 -- 172.21.15.107:0/4222477574 shutdown_connections 2024-09-19T04:28:31.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f52540a3430 0x7f52540a58f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f527010bfc0 0x7f527019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52701034b0 0x7f527019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 --2- 172.21.15.107:0/4222477574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5270102b90 0x7f527019a560 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:31.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 -- 172.21.15.107:0/4222477574 >> 172.21.15.107:0/4222477574 conn(0x7f52700fc610 msgr2=0x7f52700fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:31.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 -- 172.21.15.107:0/4222477574 shutdown_connections 2024-09-19T04:28:31.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:31.666+0000 7f526f200640 1 -- 172.21.15.107:0/4222477574 wait complete. 2024-09-19T04:28:31.690 INFO:tasks.workunit.client.0.smithi107.stdout: 1 256211 2.38 MB/sec execute 289 sec latency 2417.089 ms 2024-09-19T04:28:32.587 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:28:32.690 INFO:tasks.workunit.client.0.smithi107.stdout: 1 256211 2.37 MB/sec execute 290 sec latency 3417.144 ms 2024-09-19T04:28:32.957 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:32.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:32 smithi107 ceph-mon[30659]: pgmap v253: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 876 KiB/s wr, 87 op/s 2024-09-19T04:28:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:32 smithi115 ceph-mon[35477]: pgmap v253: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 876 KiB/s wr, 87 op/s 2024-09-19T04:28:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:32 smithi145 ceph-mon[35096]: pgmap v253: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 876 KiB/s wr, 87 op/s 2024-09-19T04:28:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.529+0000 7feab3400640 1 -- 172.21.15.107:0/3977423839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaac1049a0 msgr2=0x7feaac1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.529+0000 7feab3400640 1 --2- 172.21.15.107:0/3977423839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaac1049a0 0x7feaac1089f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fea98009900 tx=0x7fea9802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.529+0000 7feab3400640 1 -- 172.21.15.107:0/3977423839 shutdown_connections 2024-09-19T04:28:33.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.529+0000 7feab3400640 1 --2- 172.21.15.107:0/3977423839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac109120 0x7feaac110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.529+0000 7feab3400640 1 --2- 172.21.15.107:0/3977423839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaac1049a0 0x7feaac1089f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.529+0000 7feab3400640 1 --2- 172.21.15.107:0/3977423839 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feaac103ff0 0x7feaac1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.529+0000 7feab3400640 1 -- 172.21.15.107:0/3977423839 >> 172.21.15.107:0/3977423839 conn(0x7feaac0ffa90 msgr2=0x7feaac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:33.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.530+0000 7feab3400640 1 -- 172.21.15.107:0/3977423839 shutdown_connections 2024-09-19T04:28:33.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.530+0000 7feab3400640 1 -- 172.21.15.107:0/3977423839 wait complete. 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.530+0000 7feab3400640 1 Processor -- start 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab3400640 1 -- start start 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaac103ff0 0x7feaac19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac1049a0 0x7feaac19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feaac109120 0x7feaac19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab3400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feaac114140 con 0x7feaac103ff0 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab3400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feaac113fc0 con 0x7feaac1049a0 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab3400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1142c0 con 0x7feaac109120 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaac103ff0 0x7feaac19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaac103ff0 0x7feaac19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33260/0 (socket says 172.21.15.107:33260) 2024-09-19T04:28:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab0a00640 1 -- 172.21.15.107:0/610134063 learned_addr learned my addr 172.21.15.107:0/610134063 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feaabe00640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac1049a0 0x7feaac19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.531+0000 7feab1400640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feaac109120 0x7feaac19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feab0a00640 1 -- 172.21.15.107:0/610134063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feaac109120 msgr2=0x7feaac19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feab0a00640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feaac109120 0x7feaac19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feab0a00640 1 -- 172.21.15.107:0/610134063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac1049a0 msgr2=0x7feaac19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feab0a00640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac1049a0 0x7feaac19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feab0a00640 1 -- 172.21.15.107:0/610134063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac19f5d0 con 0x7feaac103ff0 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feab1400640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feaac109120 0x7feaac19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feab0a00640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaac103ff0 0x7feaac19a620 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feaa000d770 tx=0x7feaa000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feaa9600640 1 -- 172.21.15.107:0/610134063 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0017070 con 0x7feaac103ff0 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feab3400640 1 -- 172.21.15.107:0/610134063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaac19f8c0 con 0x7feaac103ff0 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feaa9600640 1 -- 172.21.15.107:0/610134063 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa0005500 con 0x7feaac103ff0 2024-09-19T04:28:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feaa9600640 1 -- 172.21.15.107:0/610134063 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0004e00 con 0x7feaac103ff0 2024-09-19T04:28:33.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.532+0000 7feab3400640 1 -- 172.21.15.107:0/610134063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaac1a71a0 con 0x7feaac103ff0 2024-09-19T04:28:33.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.535+0000 7feab5fde740 1 Processor -- start 2024-09-19T04:28:33.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.535+0000 7feaa9600640 1 -- 172.21.15.107:0/610134063 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7feaa0004fa0 con 0x7feaac103ff0 2024-09-19T04:28:33.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.535+0000 7feab5fde740 1 -- start start 2024-09-19T04:28:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feab5fde740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce32846050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feab5fde740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32846590 0x55ce32850910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feab5fde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32853240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feab5fde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ce327ac3e0 con 0x55ce32850e50 2024-09-19T04:28:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feab5fde740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ce32753d80 con 0x55ce32845c70 2024-09-19T04:28:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feab5fde740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ce327a83a0 con 0x55ce32846590 2024-09-19T04:28:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feaabe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32853240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feab1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce32846050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feaabe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32853240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33270/0 (socket says 172.21.15.107:33270) 2024-09-19T04:28:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feab1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce32846050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36378/0 (socket says 172.21.15.107:36378) 2024-09-19T04:28:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feaabe00640 1 -- 172.21.15.107:0/3046441716 learned_addr learned my addr 172.21.15.107:0/3046441716 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.536+0000 7feab0a00640 1 --2- 172.21.15.107:0/3046441716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32846590 0x55ce32850910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.537+0000 7feaabe00640 1 -- 172.21.15.107:0/3046441716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32846590 msgr2=0x55ce32850910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:33.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.537+0000 7feaabe00640 1 --2- 172.21.15.107:0/3046441716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32846590 0x55ce32850910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.537+0000 7feaabe00640 1 -- 172.21.15.107:0/3046441716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 msgr2=0x55ce32846050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:33.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.537+0000 7feaabe00640 1 --2- 172.21.15.107:0/3046441716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce32846050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.537+0000 7feaabe00640 1 -- 172.21.15.107:0/3046441716 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce32772ae0 con 0x55ce32850e50 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.537+0000 7feab1400640 1 --2- 172.21.15.107:0/3046441716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce32846050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.537+0000 7feaa9600640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea940a33b0 0x7fea940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.537+0000 7feaa9600640 1 -- 172.21.15.107:0/610134063 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7feaa00cf480 con 0x7feaac103ff0 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.537+0000 7feaabe00640 1 --2- 172.21.15.107:0/3046441716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32853240 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fea98009900 tx=0x7fea98005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9b62f7b493fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.537+0000 7feab1400640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea940a33b0 0x7fea940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.538+0000 7fea9dc00640 1 -- 172.21.15.107:0/3046441716 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98040070 con 0x55ce32850e50 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.538+0000 7fea9dc00640 1 -- 172.21.15.107:0/3046441716 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea980319f0 con 0x55ce32850e50 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.538+0000 7fea9dc00640 1 -- 172.21.15.107:0/3046441716 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98035040 con 0x55ce32850e50 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.538+0000 7feab5fde740 1 -- 172.21.15.107:0/3046441716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 msgr2=0x55ce32853240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.538+0000 7feab1400640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea940a33b0 0x7fea940a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feaa4005fd0 tx=0x7feaa4009780 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.538+0000 7feab5fde740 1 --2- 172.21.15.107:0/3046441716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32853240 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fea98009900 tx=0x7fea98005e80 comp rx=0 tx=0).stop 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.539+0000 7feab5fde740 1 -- 172.21.15.107:0/3046441716 shutdown_connections 2024-09-19T04:28:33.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.539+0000 7feab5fde740 1 --2- 172.21.15.107:0/3046441716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32853240 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.539+0000 7feab5fde740 1 --2- 172.21.15.107:0/3046441716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32846590 0x55ce32850910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.539+0000 7feab5fde740 1 --2- 172.21.15.107:0/3046441716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce32846050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.539+0000 7feab5fde740 1 -- 172.21.15.107:0/3046441716 >> 172.21.15.107:0/3046441716 conn(0x55ce32842200 msgr2=0x55ce328425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:33.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.539+0000 7feab5fde740 1 -- 172.21.15.107:0/3046441716 shutdown_connections 2024-09-19T04:28:33.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.539+0000 7feab5fde740 1 -- 172.21.15.107:0/3046441716 wait complete. 2024-09-19T04:28:33.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.539+0000 7feab5fde740 1 -- start start 2024-09-19T04:28:33.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.539+0000 7feab5fde740 1 Processor -- start 2024-09-19T04:28:33.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab5fde740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce3284dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab5fde740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32846590 0x55ce3284e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab5fde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce3284e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab5fde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ce32853a20 con 0x55ce32850e50 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab5fde740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ce327670f0 con 0x55ce32845c70 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab5fde740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ce32853ba0 con 0x55ce32846590 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab5fde740 10 client.0 _ll_register_callbacks cb 0x55ce3274ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab0a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce3284dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce3284e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab0a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce3284dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36390/0 (socket says 172.21.15.107:36390) 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab0a00640 1 -- 172.21.15.107:0/1836870049 learned_addr learned my addr 172.21.15.107:0/1836870049 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feaabe00640 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32846590 0x55ce3284e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab0a00640 1 -- 172.21.15.107:0/1836870049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32846590 msgr2=0x55ce3284e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.540+0000 7feab0a00640 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32846590 0x55ce3284e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7feab0a00640 1 -- 172.21.15.107:0/1836870049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 msgr2=0x55ce3284e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7feab0a00640 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce3284e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7feab0a00640 1 -- 172.21.15.107:0/1836870049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce3251d3c0 con 0x55ce32845c70 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7feab0a00640 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce3284dc70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009610 tx=0x7feaa0097a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ced94668fc546c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7fea92000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7fea92000640 1 -- 172.21.15.107:0/1836870049 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0020040 con 0x55ce32845c70 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7feab5fde740 1 -- 172.21.15.107:0/1836870049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea980319f0 con 0x55ce32845c70 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7fea92000640 1 -- 172.21.15.107:0/1836870049 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7feaa0097d90 con 0x55ce32845c70 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7fea92000640 1 -- 172.21.15.107:0/1836870049 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7feaa00056b0 con 0x55ce32845c70 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7feab5fde740 10 client.26711 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.541+0000 7feab5fde740 1 -- 172.21.15.107:0/1836870049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ce3285f9e0 con 0x55ce32845c70 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.542+0000 7fea92000640 1 -- 172.21.15.107:0/1836870049 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea980319f0 con 0x55ce32845c70 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.542+0000 7fea92000640 1 -- 172.21.15.107:0/1836870049 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7feaa0023060 con 0x55ce32845c70 2024-09-19T04:28:33.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.542+0000 7feab5fde740 10 client.26711 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:33.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.542+0000 7feab5fde740 10 client.26711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:28:33.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.542+0000 7feab5fde740 20 client.26711 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:33.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.542+0000 7feab5fde740 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55ce32860df0 0x55ce328631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.542+0000 7feab5fde740 4 client.26711 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:33.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.542+0000 7feab5fde740 1 -- 172.21.15.107:0/1836870049 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce327b9950 con 0x55ce32860df0 2024-09-19T04:28:33.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.542+0000 7feaabe00640 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55ce32860df0 0x55ce328631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.543+0000 7feaabe00640 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55ce32860df0 0x55ce328631e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb2929bb4a4421ef server_cookie=2783edea6ee2711e in_seq=0 out_seq=0 2024-09-19T04:28:33.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.543+0000 7fea92000640 10 client.26711 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:28:33.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7fea92000640 1 -- 172.21.15.107:0/1836870049 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce327b9950 con 0x55ce32860df0 2024-09-19T04:28:33.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7fea92000640 10 client.26711 handle_command_reply: tid=0 2024-09-19T04:28:33.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 client.26711 shutdown 2024-09-19T04:28:33.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 -- 172.21.15.107:0/1836870049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 msgr2=0x55ce3284dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:33.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce3284dc70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009610 tx=0x7feaa0097a80 comp rx=0 tx=0).stop 2024-09-19T04:28:33.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 -- 172.21.15.107:0/1836870049 shutdown_connections 2024-09-19T04:28:33.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55ce32860df0 0x55ce328631e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce3284e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32846590 0x55ce3284e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 --2- 172.21.15.107:0/1836870049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32845c70 0x55ce3284dc70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7fea92000640 0 client.26711 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:28:33.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 -- 172.21.15.107:0/1836870049 >> 172.21.15.107:0/1836870049 conn(0x55ce32842200 msgr2=0x55ce32847ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:33.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 -- 172.21.15.107:0/1836870049 shutdown_connections 2024-09-19T04:28:33.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 1 -- 172.21.15.107:0/1836870049 wait complete. 2024-09-19T04:28:33.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.544+0000 7feab5fde740 20 client.26711 trim_cache size 0 max 16384 2024-09-19T04:28:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feab5fde740 1 Processor -- start 2024-09-19T04:28:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feab5fde740 1 -- start start 2024-09-19T04:28:33.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feab5fde740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 0x55ce328486d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feab5fde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32846590 0x55ce32868a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feab5fde740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32850e50 0x55ce32868f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feab5fde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1af9a0 con 0x55ce32846590 2024-09-19T04:28:33.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feab5fde740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1b1510 con 0x55ce32850e50 2024-09-19T04:28:33.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feab5fde740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1b00a0 con 0x55ce32845c70 2024-09-19T04:28:33.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feaabe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32846590 0x55ce32868a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feaabe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32846590 0x55ce32868a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33284/0 (socket says 172.21.15.107:33284) 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.558+0000 7feaabe00640 1 -- 172.21.15.107:0/1353873129 learned_addr learned my addr 172.21.15.107:0/1353873129 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7feab0a00640 1 --2- 172.21.15.107:0/1353873129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 0x55ce328486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7feab1400640 1 --2- 172.21.15.107:0/1353873129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32850e50 0x55ce32868f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7feaabe00640 1 -- 172.21.15.107:0/1353873129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 msgr2=0x55ce328486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7feaabe00640 1 --2- 172.21.15.107:0/1353873129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 0x55ce328486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7feaabe00640 1 -- 172.21.15.107:0/1353873129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32850e50 msgr2=0x55ce32868f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7feaabe00640 1 --2- 172.21.15.107:0/1353873129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32850e50 0x55ce32868f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7feaabe00640 1 -- 172.21.15.107:0/1353873129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce32842030 con 0x55ce32846590 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7feab1400640 1 --2- 172.21.15.107:0/1353873129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32850e50 0x55ce32868f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7feaabe00640 1 --2- 172.21.15.107:0/1353873129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32846590 0x55ce32868a10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fea98031550 tx=0x7fea980314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb2004e819cf356 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7fea9d200640 1 -- 172.21.15.107:0/1353873129 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98040040 con 0x55ce32846590 2024-09-19T04:28:33.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7fea9d200640 1 -- 172.21.15.107:0/1353873129 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea980035f0 con 0x55ce32846590 2024-09-19T04:28:33.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.559+0000 7fea9d200640 1 -- 172.21.15.107:0/1353873129 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98035040 con 0x55ce32846590 2024-09-19T04:28:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.560+0000 7feab5fde740 1 -- 172.21.15.107:0/1353873129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32846590 msgr2=0x55ce32868a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.560+0000 7feab5fde740 1 --2- 172.21.15.107:0/1353873129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32846590 0x55ce32868a10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fea98031550 tx=0x7fea980314e0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.560+0000 7feab5fde740 1 -- 172.21.15.107:0/1353873129 shutdown_connections 2024-09-19T04:28:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.560+0000 7feab5fde740 1 --2- 172.21.15.107:0/1353873129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32850e50 0x55ce32868f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.560+0000 7feab5fde740 1 --2- 172.21.15.107:0/1353873129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32846590 0x55ce32868a10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.560+0000 7feab5fde740 1 --2- 172.21.15.107:0/1353873129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 0x55ce328486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.560+0000 7feab5fde740 1 -- 172.21.15.107:0/1353873129 >> 172.21.15.107:0/1353873129 conn(0x55ce32842200 msgr2=0x55ce3284cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.560+0000 7feab5fde740 1 -- 172.21.15.107:0/1353873129 shutdown_connections 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.560+0000 7feab5fde740 1 -- 172.21.15.107:0/1353873129 wait complete. 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.561+0000 7feab5fde740 1 -- start start 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.561+0000 7feab5fde740 1 Processor -- start 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.561+0000 7feab5fde740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 0x55ce32859460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.561+0000 7feab5fde740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32846590 0x55ce32856f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.561+0000 7feab5fde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32857470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.561+0000 7feab5fde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1b0e40 con 0x55ce32850e50 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.561+0000 7feab5fde740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1b0770 con 0x55ce32846590 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.561+0000 7feab5fde740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1ad9a0 con 0x55ce32845c70 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab5fde740 10 client.0 _ll_register_callbacks cb 0x7feaa000d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab0a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 0x55ce32859460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab0a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 0x55ce32859460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33996/0 (socket says 172.21.15.107:33996) 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab0a00640 1 -- 172.21.15.107:0/2728954087 learned_addr learned my addr 172.21.15.107:0/2728954087 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feaabe00640 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32846590 0x55ce32856f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab1400640 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32857470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab0a00640 1 -- 172.21.15.107:0/2728954087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32846590 msgr2=0x55ce32856f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab0a00640 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32846590 0x55ce32856f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab0a00640 1 -- 172.21.15.107:0/2728954087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 msgr2=0x55ce32857470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab0a00640 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32857470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab0a00640 1 -- 172.21.15.107:0/2728954087 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce32857ca0 con 0x55ce32845c70 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab1400640 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32857470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7feab0a00640 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 0x55ce32859460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feaa009b420 tx=0x7feaa0099c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90a0e0bea184d276 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.562+0000 7fea92000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7fea92000640 1 -- 172.21.15.107:0/2728954087 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0014070 con 0x55ce32845c70 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7fea92000640 1 -- 172.21.15.107:0/2728954087 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7feaa00cf900 con 0x55ce32845c70 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7feab5fde740 1 -- 172.21.15.107:0/2728954087 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea980035f0 con 0x55ce32845c70 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7fea92000640 1 -- 172.21.15.107:0/2728954087 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7feaa0020040 con 0x55ce32845c70 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7feab5fde740 10 client.26686 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7feab5fde740 1 -- 172.21.15.107:0/2728954087 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ce328cb9e0 con 0x55ce32845c70 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7fea92000640 1 -- 172.21.15.107:0/2728954087 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea980035f0 con 0x55ce32845c70 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7fea92000640 1 -- 172.21.15.107:0/2728954087 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7feaa009ab70 con 0x55ce32845c70 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7feab5fde740 10 client.26686 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7feab5fde740 10 client.26686 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7feab5fde740 20 client.26686 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7feab5fde740 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55ce328cbfc0 0x55ce328ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:33.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7feab5fde740 4 client.26686 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:28:33.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.563+0000 7feab5fde740 1 -- 172.21.15.107:0/2728954087 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce327b3e60 con 0x55ce328cbfc0 2024-09-19T04:28:33.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.564+0000 7feaabe00640 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55ce328cbfc0 0x55ce328ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.564+0000 7feaabe00640 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55ce328cbfc0 0x55ce328ce3e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75142c852a9eadc8 server_cookie=4d3319a176e462f8 in_seq=0 out_seq=0 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.564+0000 7fea92000640 10 client.26686 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.564+0000 7fea92000640 1 -- 172.21.15.107:0/2728954087 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce327b3e60 con 0x55ce328cbfc0 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.564+0000 7fea92000640 10 client.26686 handle_command_reply: tid=0 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.564+0000 7feab5fde740 1 client.26686 shutdown 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 1 -- 172.21.15.107:0/2728954087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 msgr2=0x55ce32859460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 0x55ce32859460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feaa009b420 tx=0x7feaa0099c30 comp rx=0 tx=0).stop 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 1 -- 172.21.15.107:0/2728954087 shutdown_connections 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55ce328cbfc0 0x55ce328ce3e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce32850e50 0x55ce32857470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce32846590 0x55ce32856f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 1 --2- 172.21.15.107:0/2728954087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ce32845c70 0x55ce32859460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7fea92000640 0 client.26686 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:28:33.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 1 -- 172.21.15.107:0/2728954087 >> 172.21.15.107:0/2728954087 conn(0x55ce32842200 msgr2=0x55ce32840b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:33.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 1 -- 172.21.15.107:0/2728954087 shutdown_connections 2024-09-19T04:28:33.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 1 -- 172.21.15.107:0/2728954087 wait complete. 2024-09-19T04:28:33.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab5fde740 20 client.26686 trim_cache size 0 max 16384 2024-09-19T04:28:33.571 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:28:33.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab3400640 1 -- 172.21.15.107:0/610134063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea940a33b0 msgr2=0x7fea940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:33.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab3400640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea940a33b0 0x7fea940a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feaa4005fd0 tx=0x7feaa4009780 comp rx=0 tx=0).stop 2024-09-19T04:28:33.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab3400640 1 -- 172.21.15.107:0/610134063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaac103ff0 msgr2=0x7feaac19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:33.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.565+0000 7feab3400640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaac103ff0 0x7feaac19a620 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feaa000d770 tx=0x7feaa000dc30 comp rx=0 tx=0).stop 2024-09-19T04:28:33.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.566+0000 7feab3400640 1 -- 172.21.15.107:0/610134063 shutdown_connections 2024-09-19T04:28:33.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.566+0000 7feab3400640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fea940a33b0 0x7fea940a5870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.566+0000 7feab3400640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feaac109120 0x7feaac19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.566+0000 7feab3400640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac1049a0 0x7feaac19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.566+0000 7feab3400640 1 --2- 172.21.15.107:0/610134063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaac103ff0 0x7feaac19a620 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:33.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.566+0000 7feab3400640 1 -- 172.21.15.107:0/610134063 >> 172.21.15.107:0/610134063 conn(0x7feaac0ffa90 msgr2=0x7feaac100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:33.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.566+0000 7feab3400640 1 -- 172.21.15.107:0/610134063 shutdown_connections 2024-09-19T04:28:33.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:33.566+0000 7feab3400640 1 -- 172.21.15.107:0/610134063 wait complete. 2024-09-19T04:28:33.690 INFO:tasks.workunit.client.0.smithi107.stdout: 1 256211 2.37 MB/sec execute 291 sec latency 4417.237 ms 2024-09-19T04:28:34.433 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:28:34.690 INFO:tasks.workunit.client.0.smithi107.stdout: 1 257070 2.37 MB/sec execute 292 sec latency 4946.798 ms 2024-09-19T04:28:34.834 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:35.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:34 smithi107 ceph-mon[30659]: pgmap v254: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 534 KiB/s wr, 61 op/s 2024-09-19T04:28:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:34 smithi115 ceph-mon[35477]: pgmap v254: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 534 KiB/s wr, 61 op/s 2024-09-19T04:28:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:34 smithi145 ceph-mon[35096]: pgmap v254: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 534 KiB/s wr, 61 op/s 2024-09-19T04:28:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.458+0000 7f9ef9000640 1 -- 172.21.15.107:0/989322522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef4103e30 msgr2=0x7f9ef4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.458+0000 7f9ef9000640 1 --2- 172.21.15.107:0/989322522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef4103e30 0x7f9ef4104210 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009900 tx=0x7f9ee402a410 comp rx=0 tx=0).stop 2024-09-19T04:28:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.458+0000 7f9ef9000640 1 -- 172.21.15.107:0/989322522 shutdown_connections 2024-09-19T04:28:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.458+0000 7f9ef9000640 1 --2- 172.21.15.107:0/989322522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ef4108f60 0x7f9ef4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.458+0000 7f9ef9000640 1 --2- 172.21.15.107:0/989322522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ef41047e0 0x7f9ef4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.458+0000 7f9ef9000640 1 --2- 172.21.15.107:0/989322522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef4103e30 0x7f9ef4104210 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.458+0000 7f9ef9000640 1 -- 172.21.15.107:0/989322522 >> 172.21.15.107:0/989322522 conn(0x7f9ef40ff8d0 msgr2=0x7f9ef4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.458+0000 7f9ef9000640 1 -- 172.21.15.107:0/989322522 shutdown_connections 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.458+0000 7f9ef9000640 1 -- 172.21.15.107:0/989322522 wait complete. 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.459+0000 7f9ef9000640 1 Processor -- start 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.459+0000 7f9ef9000640 1 -- start start 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.459+0000 7f9ef9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef4103e30 0x7f9ef419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.459+0000 7f9ef9000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ef41047e0 0x7f9ef419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.459+0000 7f9ef9000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ef4108f60 0x7f9ef419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.459+0000 7f9ef9000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4114010 con 0x7f9ef4103e30 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.459+0000 7f9ef9000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4113e90 con 0x7f9ef4108f60 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.459+0000 7f9ef9000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4114190 con 0x7f9ef41047e0 2024-09-19T04:28:35.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.459+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef4103e30 0x7f9ef419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef4103e30 0x7f9ef419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33324/0 (socket says 172.21.15.107:33324) 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9ef3400640 1 -- 172.21.15.107:0/2555859779 learned_addr learned my addr 172.21.15.107:0/2555859779 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ef41047e0 0x7f9ef419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9ef3e00640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ef4108f60 0x7f9ef419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9eeaa00640 1 -- 172.21.15.107:0/2555859779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ef4108f60 msgr2=0x7f9ef419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ef4108f60 0x7f9ef419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9eeaa00640 1 -- 172.21.15.107:0/2555859779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef4103e30 msgr2=0x7f9ef419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef4103e30 0x7f9ef419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9eeaa00640 1 -- 172.21.15.107:0/2555859779 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef419f400 con 0x7f9ef41047e0 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9ef3e00640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ef4108f60 0x7f9ef419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9ef3400640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef4103e30 0x7f9ef419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.460+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ef41047e0 0x7f9ef419a990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00e9f0 tx=0x7f9edc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:35.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.461+0000 7f9ef0c00640 1 -- 172.21.15.107:0/2555859779 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00ccc0 con 0x7f9ef41047e0 2024-09-19T04:28:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.461+0000 7f9ef9000640 1 -- 172.21.15.107:0/2555859779 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef419f6f0 con 0x7f9ef41047e0 2024-09-19T04:28:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.461+0000 7f9ef0c00640 1 -- 172.21.15.107:0/2555859779 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9edc010910 con 0x7f9ef41047e0 2024-09-19T04:28:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.461+0000 7f9ef0c00640 1 -- 172.21.15.107:0/2555859779 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc018960 con 0x7f9ef41047e0 2024-09-19T04:28:35.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.461+0000 7f9ef9000640 1 -- 172.21.15.107:0/2555859779 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef41a6fd0 con 0x7f9ef41047e0 2024-09-19T04:28:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.464+0000 7f9efd087740 1 Processor -- start 2024-09-19T04:28:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.464+0000 7f9ef0c00640 1 -- 172.21.15.107:0/2555859779 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f9edc010ac0 con 0x7f9ef41047e0 2024-09-19T04:28:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.464+0000 7f9efd087740 1 -- start start 2024-09-19T04:28:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.464+0000 7f9efd087740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5be40 0x5644c5a5c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.464+0000 7f9efd087740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5c760 0x5644c5a66ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9efd087740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a67020 0x5644c5a69410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9efd087740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5644c59c23e0 con 0x5644c5a5c760 2024-09-19T04:28:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9efd087740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5644c5969d80 con 0x5644c5a5be40 2024-09-19T04:28:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9efd087740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5644c59be3a0 con 0x5644c5a67020 2024-09-19T04:28:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9ef3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5be40 0x5644c5a5c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5c760 0x5644c5a66ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5c760 0x5644c5a66ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33330/0 (socket says 172.21.15.107:33330) 2024-09-19T04:28:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9ef3400640 1 -- 172.21.15.107:0/2872086564 learned_addr learned my addr 172.21.15.107:0/2872086564 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/2872086564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a67020 0x5644c5a69410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9ef3400640 1 -- 172.21.15.107:0/2872086564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a67020 msgr2=0x5644c5a69410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9ef3400640 1 --2- 172.21.15.107:0/2872086564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a67020 0x5644c5a69410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9ef3400640 1 -- 172.21.15.107:0/2872086564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5be40 msgr2=0x5644c5a5c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9ef3400640 1 --2- 172.21.15.107:0/2872086564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5be40 0x5644c5a5c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.465+0000 7f9ef3400640 1 -- 172.21.15.107:0/2872086564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c5988ae0 con 0x5644c5a5c760 2024-09-19T04:28:35.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.466+0000 7f9ef0c00640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9ed80a3430 0x7f9ed80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.466+0000 7f9ef3400640 1 --2- 172.21.15.107:0/2872086564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5c760 0x5644c5a66ae0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005c40 tx=0x7f9ee4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cff29aaf41d624 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.466+0000 7f9ef0c00640 1 -- 172.21.15.107:0/2555859779 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f9edc0cfff0 con 0x7f9ef41047e0 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.466+0000 7f9ef3e00640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9ed80a3430 0x7f9ed80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.466+0000 7f9ee8c00640 1 -- 172.21.15.107:0/2872086564 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4040070 con 0x5644c5a5c760 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.466+0000 7f9ee8c00640 1 -- 172.21.15.107:0/2872086564 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee40047a0 con 0x5644c5a5c760 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.466+0000 7f9ee8c00640 1 -- 172.21.15.107:0/2872086564 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4035040 con 0x5644c5a5c760 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.467+0000 7f9efd087740 1 -- 172.21.15.107:0/2872086564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5c760 msgr2=0x5644c5a66ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.467+0000 7f9ef3e00640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9ed80a3430 0x7f9ed80a58f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009fd0 tx=0x7f9eec009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.467+0000 7f9efd087740 1 --2- 172.21.15.107:0/2872086564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5c760 0x5644c5a66ae0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005c40 tx=0x7f9ee4005dd0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.467+0000 7f9efd087740 1 -- 172.21.15.107:0/2872086564 shutdown_connections 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.467+0000 7f9efd087740 1 --2- 172.21.15.107:0/2872086564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a67020 0x5644c5a69410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.467+0000 7f9efd087740 1 --2- 172.21.15.107:0/2872086564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5c760 0x5644c5a66ae0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.467+0000 7f9efd087740 1 --2- 172.21.15.107:0/2872086564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5be40 0x5644c5a5c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.467+0000 7f9efd087740 1 -- 172.21.15.107:0/2872086564 >> 172.21.15.107:0/2872086564 conn(0x5644c5a583d0 msgr2=0x5644c5a587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.467+0000 7f9efd087740 1 -- 172.21.15.107:0/2872086564 shutdown_connections 2024-09-19T04:28:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.467+0000 7f9efd087740 1 -- 172.21.15.107:0/2872086564 wait complete. 2024-09-19T04:28:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.468+0000 7f9efd087740 1 -- start start 2024-09-19T04:28:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.468+0000 7f9efd087740 1 Processor -- start 2024-09-19T04:28:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.468+0000 7f9efd087740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 0x5644c5a63f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.468+0000 7f9efd087740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.468+0000 7f9efd087740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 0x5644c5a649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.468+0000 7f9efd087740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5644c5a69bf0 con 0x5644c5a67020 2024-09-19T04:28:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.468+0000 7f9efd087740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5644c597d0f0 con 0x5644c5a5c760 2024-09-19T04:28:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.468+0000 7f9efd087740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5644c5a69d70 con 0x5644c5a5be40 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.468+0000 7f9efd087740 10 client.0 _ll_register_callbacks cb 0x5644c5964bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9eeaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 0x5644c5a63f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 0x5644c5a63f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:34054/0 (socket says 172.21.15.107:34054) 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9ef3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 0x5644c5a649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9ef3400640 1 -- 172.21.15.107:0/2639614080 learned_addr learned my addr 172.21.15.107:0/2639614080 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9eeaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a644a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36426/0 (socket says 172.21.15.107:36426) 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9eeaa00640 1 -- 172.21.15.107:0/2639614080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 msgr2=0x5644c5a63f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 0x5644c5a63f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9eeaa00640 1 -- 172.21.15.107:0/2639614080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 msgr2=0x5644c5a649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 0x5644c5a649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9eeaa00640 1 -- 172.21.15.107:0/2639614080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c57333c0 con 0x5644c5a5c760 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9ef3400640 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 0x5644c5a63f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a644a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0cd760 tx=0x7f9edc0d3730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daea02ba870daa73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9ee1600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.469+0000 7f9ee1600640 1 -- 172.21.15.107:0/2639614080 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc09bee0 con 0x5644c5a5c760 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9ee1600640 1 -- 172.21.15.107:0/2639614080 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f9edc0d2110 con 0x5644c5a5c760 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9efd087740 1 -- 172.21.15.107:0/2639614080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee40047a0 con 0x5644c5a5c760 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9ee1600640 1 -- 172.21.15.107:0/2639614080 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9edc09bd00 con 0x5644c5a5c760 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9efd087740 10 client.26729 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9efd087740 1 -- 172.21.15.107:0/2639614080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5644c5a75c80 con 0x5644c5a5c760 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9ee1600640 1 -- 172.21.15.107:0/2639614080 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee40047a0 con 0x5644c5a5c760 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9ee1600640 1 -- 172.21.15.107:0/2639614080 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f9edc0a0170 con 0x5644c5a5c760 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9efd087740 10 client.26729 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9efd087740 10 client.26729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:28:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9efd087740 20 client.26729 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9efd087740 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5644c5a77140 0x5644c5a79550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9efd087740 4 client.26729 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.470+0000 7f9efd087740 1 -- 172.21.15.107:0/2639614080 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644c59cf950 con 0x5644c5a77140 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.471+0000 7f9ef3400640 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5644c5a77140 0x5644c5a79550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.471+0000 7f9ef3400640 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5644c5a77140 0x5644c5a79550 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34e7e573f5736919 server_cookie=57aa5598810b281a in_seq=0 out_seq=0 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.471+0000 7f9ee1600640 10 client.26729 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9ee1600640 1 -- 172.21.15.107:0/2639614080 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644c59cf950 con 0x5644c5a77140 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9ee1600640 10 client.26729 handle_command_reply: tid=0 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 client.26729 shutdown 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 -- 172.21.15.107:0/2639614080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 msgr2=0x5644c5a644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a644a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0cd760 tx=0x7f9edc0d3730 comp rx=0 tx=0).stop 2024-09-19T04:28:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 -- 172.21.15.107:0/2639614080 shutdown_connections 2024-09-19T04:28:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5644c5a77140 0x5644c5a79550 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 0x5644c5a649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a644a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9ee1600640 0 client.26729 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:28:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 --2- 172.21.15.107:0/2639614080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 0x5644c5a63f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 -- 172.21.15.107:0/2639614080 >> 172.21.15.107:0/2639614080 conn(0x5644c5a583d0 msgr2=0x5644c5a5e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 -- 172.21.15.107:0/2639614080 shutdown_connections 2024-09-19T04:28:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.472+0000 7f9efd087740 1 -- 172.21.15.107:0/2639614080 wait complete. 2024-09-19T04:28:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.473+0000 7f9efd087740 20 client.26729 trim_cache size 0 max 16384 2024-09-19T04:28:35.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.486+0000 7f9efd087740 1 Processor -- start 2024-09-19T04:28:35.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.486+0000 7f9efd087740 1 -- start start 2024-09-19T04:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.486+0000 7f9efd087740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5be40 0x5644c5a65ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.486+0000 7f9efd087740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a7ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.486+0000 7f9efd087740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a67020 0x5644c5a7f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.486+0000 7f9efd087740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41af850 con 0x5644c5a5be40 2024-09-19T04:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.486+0000 7f9efd087740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41b13c0 con 0x5644c5a5c760 2024-09-19T04:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.486+0000 7f9efd087740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41aff50 con 0x5644c5a67020 2024-09-19T04:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5be40 0x5644c5a65ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5be40 0x5644c5a65ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33350/0 (socket says 172.21.15.107:33350) 2024-09-19T04:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ef3400640 1 -- 172.21.15.107:0/3936107728 learned_addr learned my addr 172.21.15.107:0/3936107728 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/3936107728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a7ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ef3e00640 1 --2- 172.21.15.107:0/3936107728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a67020 0x5644c5a7f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ef3400640 1 -- 172.21.15.107:0/3936107728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a67020 msgr2=0x5644c5a7f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ef3400640 1 --2- 172.21.15.107:0/3936107728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a67020 0x5644c5a7f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ef3400640 1 -- 172.21.15.107:0/3936107728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 msgr2=0x5644c5a7ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ef3400640 1 --2- 172.21.15.107:0/3936107728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a7ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ef3400640 1 -- 172.21.15.107:0/3936107728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c5a58200 con 0x5644c5a5be40 2024-09-19T04:28:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/3936107728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a7ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ef3400640 1 --2- 172.21.15.107:0/3936107728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5be40 0x5644c5a65ed0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4031520 tx=0x7f9ee4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ba4041f4b8b418 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.487+0000 7f9ee3e00640 1 -- 172.21.15.107:0/3936107728 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4058070 con 0x5644c5a5be40 2024-09-19T04:28:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.488+0000 7f9ee3e00640 1 -- 172.21.15.107:0/3936107728 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee4003cd0 con 0x5644c5a5be40 2024-09-19T04:28:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.488+0000 7f9ee3e00640 1 -- 172.21.15.107:0/3936107728 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4040040 con 0x5644c5a5be40 2024-09-19T04:28:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.488+0000 7f9efd087740 1 -- 172.21.15.107:0/3936107728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5be40 msgr2=0x5644c5a65ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.488+0000 7f9efd087740 1 --2- 172.21.15.107:0/3936107728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5be40 0x5644c5a65ed0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4031520 tx=0x7f9ee4009260 comp rx=0 tx=0).stop 2024-09-19T04:28:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.488+0000 7f9efd087740 1 -- 172.21.15.107:0/3936107728 shutdown_connections 2024-09-19T04:28:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.488+0000 7f9efd087740 1 --2- 172.21.15.107:0/3936107728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a67020 0x5644c5a7f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.488+0000 7f9efd087740 1 --2- 172.21.15.107:0/3936107728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a7ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.488+0000 7f9efd087740 1 --2- 172.21.15.107:0/3936107728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a5be40 0x5644c5a65ed0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.488+0000 7f9efd087740 1 -- 172.21.15.107:0/3936107728 >> 172.21.15.107:0/3936107728 conn(0x5644c5a583d0 msgr2=0x5644c5a657c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.488+0000 7f9efd087740 1 -- 172.21.15.107:0/3936107728 shutdown_connections 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.489+0000 7f9efd087740 1 -- 172.21.15.107:0/3936107728 wait complete. 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.489+0000 7f9efd087740 1 -- start start 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.489+0000 7f9efd087740 1 Processor -- start 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.489+0000 7f9efd087740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 0x5644c5a6f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.489+0000 7f9efd087740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a6a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.489+0000 7f9efd087740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 0x5644c5a6adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.489+0000 7f9efd087740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41b0cf0 con 0x5644c5a67020 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.489+0000 7f9efd087740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41b0620 con 0x5644c5a5c760 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.489+0000 7f9efd087740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41ad850 con 0x5644c5a5be40 2024-09-19T04:28:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9efd087740 10 client.0 _ll_register_callbacks cb 0x7f9edc0d3830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9eeaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a6a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 0x5644c5a6f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9eeaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a6a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36442/0 (socket says 172.21.15.107:36442) 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9eeaa00640 1 -- 172.21.15.107:0/4267863743 learned_addr learned my addr 172.21.15.107:0/4267863743 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9ef3e00640 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 0x5644c5a6adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9eeaa00640 1 -- 172.21.15.107:0/4267863743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 msgr2=0x5644c5a6f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 0x5644c5a6f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9eeaa00640 1 -- 172.21.15.107:0/4267863743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 msgr2=0x5644c5a6adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 0x5644c5a6adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9eeaa00640 1 -- 172.21.15.107:0/4267863743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c5a57c80 con 0x5644c5a5c760 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9ef3e00640 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 0x5644c5a6adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9eeaa00640 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a6a8b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00c6a0 tx=0x7f9edc098c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=647c8ff149ab9dfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.490+0000 7f9ee1600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:28:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9ee1600640 1 -- 172.21.15.107:0/4267863743 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc09c3a0 con 0x5644c5a5c760 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9ee1600640 1 -- 172.21.15.107:0/4267863743 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f9edc0bc0b0 con 0x5644c5a5c760 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9efd087740 1 -- 172.21.15.107:0/4267863743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee4003cd0 con 0x5644c5a5c760 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9ee1600640 1 -- 172.21.15.107:0/4267863743 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9edc0a06a0 con 0x5644c5a5c760 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9efd087740 10 client.26735 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9efd087740 1 -- 172.21.15.107:0/4267863743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5644c5ae1a30 con 0x5644c5a5c760 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9ee1600640 1 -- 172.21.15.107:0/4267863743 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee4003cd0 con 0x5644c5a5c760 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9ee1600640 1 -- 172.21.15.107:0/4267863743 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f9edc018b00 con 0x5644c5a5c760 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9efd087740 10 client.26735 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9efd087740 10 client.26735 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.491+0000 7f9efd087740 20 client.26735 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.492+0000 7f9efd087740 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5644c5ae1f90 0x5644c5ae43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.492+0000 7f9efd087740 4 client.26735 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:28:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.492+0000 7f9efd087740 1 -- 172.21.15.107:0/4267863743 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644c59c9e60 con 0x5644c5ae1f90 2024-09-19T04:28:35.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.492+0000 7f9ef3400640 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5644c5ae1f90 0x5644c5ae43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.492+0000 7f9ef3400640 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5644c5ae1f90 0x5644c5ae43b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=443d35330bebc656 server_cookie=a4956a0d8878466a in_seq=0 out_seq=0 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.492+0000 7f9ee1600640 10 client.26735 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9ee1600640 1 -- 172.21.15.107:0/4267863743 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644c59c9e60 con 0x5644c5ae1f90 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9ee1600640 10 client.26735 handle_command_reply: tid=0 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 client.26735 shutdown 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 -- 172.21.15.107:0/4267863743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 msgr2=0x5644c5a6a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a6a8b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00c6a0 tx=0x7f9edc098c40 comp rx=0 tx=0).stop 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 -- 172.21.15.107:0/4267863743 shutdown_connections 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5644c5ae1f90 0x5644c5ae43b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644c5a67020 0x5644c5a6adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644c5a5c760 0x5644c5a6a8b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9ee1600640 0 client.26735 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:28:35.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 --2- 172.21.15.107:0/4267863743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644c5a5be40 0x5644c5a6f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 -- 172.21.15.107:0/4267863743 >> 172.21.15.107:0/4267863743 conn(0x5644c5a583d0 msgr2=0x5644c5a657c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:35.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 -- 172.21.15.107:0/4267863743 shutdown_connections 2024-09-19T04:28:35.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 1 -- 172.21.15.107:0/4267863743 wait complete. 2024-09-19T04:28:35.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.493+0000 7f9efd087740 20 client.26735 trim_cache size 0 max 16384 2024-09-19T04:28:35.499 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:28:35.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 -- 172.21.15.107:0/2555859779 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9ed80a3430 msgr2=0x7f9ed80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:35.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9ed80a3430 0x7f9ed80a58f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009fd0 tx=0x7f9eec009500 comp rx=0 tx=0).stop 2024-09-19T04:28:35.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 -- 172.21.15.107:0/2555859779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ef41047e0 msgr2=0x7f9ef419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:35.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ef41047e0 0x7f9ef419a990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00e9f0 tx=0x7f9edc00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 -- 172.21.15.107:0/2555859779 shutdown_connections 2024-09-19T04:28:35.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9ed80a3430 0x7f9ed80a58f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ef4108f60 0x7f9ef419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ef41047e0 0x7f9ef419a990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 --2- 172.21.15.107:0/2555859779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef4103e30 0x7f9ef419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:35.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 -- 172.21.15.107:0/2555859779 >> 172.21.15.107:0/2555859779 conn(0x7f9ef40ff8d0 msgr2=0x7f9ef4100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:35.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 -- 172.21.15.107:0/2555859779 shutdown_connections 2024-09-19T04:28:35.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:35.494+0000 7f9ef9000640 1 -- 172.21.15.107:0/2555859779 wait complete. 2024-09-19T04:28:35.690 INFO:tasks.workunit.client.0.smithi107.stdout: 1 257113 2.37 MB/sec execute 293 sec latency 968.655 ms 2024-09-19T04:28:36.510 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:28:36.691 INFO:tasks.workunit.client.0.smithi107.stdout: 1 257113 2.36 MB/sec execute 294 sec latency 1969.063 ms 2024-09-19T04:28:36.885 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:37.170 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:36 smithi107 ceph-mon[30659]: pgmap v255: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 739 KiB/s wr, 90 op/s 2024-09-19T04:28:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:36 smithi115 ceph-mon[35477]: pgmap v255: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 739 KiB/s wr, 90 op/s 2024-09-19T04:28:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:36 smithi145 ceph-mon[35096]: pgmap v255: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 739 KiB/s wr, 90 op/s 2024-09-19T04:28:37.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.614+0000 7f9daac00640 1 -- 172.21.15.107:0/1176400056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9da4102780 msgr2=0x7f9da41067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:37.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.614+0000 7f9daac00640 1 --2- 172.21.15.107:0/1176400056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9da4102780 0x7f9da41067d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009900 tx=0x7f9d9002a410 comp rx=0 tx=0).stop 2024-09-19T04:28:37.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.615+0000 7f9daac00640 1 -- 172.21.15.107:0/1176400056 shutdown_connections 2024-09-19T04:28:37.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.615+0000 7f9daac00640 1 --2- 172.21.15.107:0/1176400056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da4106f00 0x7f9da410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.615+0000 7f9daac00640 1 --2- 172.21.15.107:0/1176400056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9da4102780 0x7f9da41067d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.615+0000 7f9daac00640 1 --2- 172.21.15.107:0/1176400056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4101dd0 0x7f9da41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.615+0000 7f9daac00640 1 -- 172.21.15.107:0/1176400056 >> 172.21.15.107:0/1176400056 conn(0x7f9da40fd890 msgr2=0x7f9da40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.615+0000 7f9daac00640 1 -- 172.21.15.107:0/1176400056 shutdown_connections 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.615+0000 7f9daac00640 1 -- 172.21.15.107:0/1176400056 wait complete. 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.616+0000 7f9daac00640 1 Processor -- start 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.616+0000 7f9daac00640 1 -- start start 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.616+0000 7f9daac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da4101dd0 0x7f9da419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.616+0000 7f9daac00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9da4102780 0x7f9da419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.616+0000 7f9daac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4106f00 0x7f9da419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.616+0000 7f9daac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4111f70 con 0x7f9da4106f00 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.616+0000 7f9daac00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4111df0 con 0x7f9da4102780 2024-09-19T04:28:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.616+0000 7f9daac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41120f0 con 0x7f9da4101dd0 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.616+0000 7f9da3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9da4102780 0x7f9da419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.617+0000 7f9da3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9da4102780 0x7f9da419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36462/0 (socket says 172.21.15.107:36462) 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.617+0000 7f9da3400640 1 -- 172.21.15.107:0/3936175365 learned_addr learned my addr 172.21.15.107:0/3936175365 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.617+0000 7f9da8c00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4106f00 0x7f9da419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.617+0000 7f9da8c00640 1 -- 172.21.15.107:0/3936175365 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da4101dd0 msgr2=0x7f9da419a690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.617+0000 7f9da8c00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da4101dd0 0x7f9da419a690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.617+0000 7f9da8c00640 1 -- 172.21.15.107:0/3936175365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9da4102780 msgr2=0x7f9da419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.617+0000 7f9da8c00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9da4102780 0x7f9da419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.617+0000 7f9da8c00640 1 -- 172.21.15.107:0/3936175365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da419f640 con 0x7f9da4106f00 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.617+0000 7f9da3400640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9da4102780 0x7f9da419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.617+0000 7f9da8c00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4106f00 0x7f9da419ef60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009fd0 tx=0x7f9d9c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.618+0000 7f9da0c00640 1 -- 172.21.15.107:0/3936175365 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00ce20 con 0x7f9da4106f00 2024-09-19T04:28:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.618+0000 7f9da0c00640 1 -- 172.21.15.107:0/3936175365 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c005020 con 0x7f9da4106f00 2024-09-19T04:28:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.618+0000 7f9daac00640 1 -- 172.21.15.107:0/3936175365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da419f930 con 0x7f9da4106f00 2024-09-19T04:28:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.618+0000 7f9da0c00640 1 -- 172.21.15.107:0/3936175365 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c010720 con 0x7f9da4106f00 2024-09-19T04:28:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.618+0000 7f9daac00640 1 -- 172.21.15.107:0/3936175365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da41a7210 con 0x7f9da4106f00 2024-09-19T04:28:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.622+0000 7f9dad649740 1 Processor -- start 2024-09-19T04:28:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.622+0000 7f9da0c00640 1 -- 172.21.15.107:0/3936175365 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f9d9c004a40 con 0x7f9da4106f00 2024-09-19T04:28:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.623+0000 7f9dad649740 1 -- start start 2024-09-19T04:28:37.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.623+0000 7f9dad649740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edbc70 0x55a753edc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.623+0000 7f9dad649740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.623+0000 7f9dad649740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753ee6e50 0x55a753ee9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.623+0000 7f9dad649740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a753e423e0 con 0x55a753ee6e50 2024-09-19T04:28:37.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.623+0000 7f9dad649740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a753de9d80 con 0x55a753edc590 2024-09-19T04:28:37.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.623+0000 7f9dad649740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a753e3e3a0 con 0x55a753edbc70 2024-09-19T04:28:37.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.623+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.623+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36466/0 (socket says 172.21.15.107:36466) 2024-09-19T04:28:37.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.623+0000 7f9da3e00640 1 -- 172.21.15.107:0/3317486865 learned_addr learned my addr 172.21.15.107:0/3317486865 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da3400640 1 --2- 172.21.15.107:0/3317486865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753ee6e50 0x55a753ee9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da3e00640 1 -- 172.21.15.107:0/3317486865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edbc70 msgr2=0x55a753edc050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da3e00640 1 --2- 172.21.15.107:0/3317486865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edbc70 0x55a753edc050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da3e00640 1 -- 172.21.15.107:0/3317486865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753ee6e50 msgr2=0x55a753ee9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da3e00640 1 --2- 172.21.15.107:0/3317486865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753ee6e50 0x55a753ee9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da3e00640 1 -- 172.21.15.107:0/3317486865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a753e08ae0 con 0x55a753edc590 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da3400640 1 --2- 172.21.15.107:0/3317486865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753ee6e50 0x55a753ee9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da0c00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9d880a33b0 0x7f9d880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da0c00640 1 -- 172.21.15.107:0/3936175365 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f9d9c0d0320 con 0x7f9da4106f00 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da3e00640 1 --2- 172.21.15.107:0/3317486865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee6910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009bd0 tx=0x7f9d9400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67737119fa40b1e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:37.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.624+0000 7f9da3400640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9d880a33b0 0x7f9d880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.625+0000 7f9d99200640 1 -- 172.21.15.107:0/3317486865 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94005020 con 0x55a753edc590 2024-09-19T04:28:37.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.625+0000 7f9d99200640 1 -- 172.21.15.107:0/3317486865 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d940047a0 con 0x55a753edc590 2024-09-19T04:28:37.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.625+0000 7f9d99200640 1 -- 172.21.15.107:0/3317486865 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94010690 con 0x55a753edc590 2024-09-19T04:28:37.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.626+0000 7f9da3400640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9d880a33b0 0x7f9d880a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55a753ed9580 tx=0x7f9d900046c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:37.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.626+0000 7f9dad649740 1 -- 172.21.15.107:0/3317486865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 msgr2=0x55a753ee6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:37.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.626+0000 7f9dad649740 1 --2- 172.21.15.107:0/3317486865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee6910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009bd0 tx=0x7f9d9400e720 comp rx=0 tx=0).stop 2024-09-19T04:28:37.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.626+0000 7f9dad649740 1 -- 172.21.15.107:0/3317486865 shutdown_connections 2024-09-19T04:28:37.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.626+0000 7f9dad649740 1 --2- 172.21.15.107:0/3317486865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753ee6e50 0x55a753ee9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.626+0000 7f9dad649740 1 --2- 172.21.15.107:0/3317486865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee6910 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.626+0000 7f9dad649740 1 --2- 172.21.15.107:0/3317486865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edbc70 0x55a753edc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.626+0000 7f9dad649740 1 -- 172.21.15.107:0/3317486865 >> 172.21.15.107:0/3317486865 conn(0x55a753ed8200 msgr2=0x55a753ed85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.626+0000 7f9dad649740 1 -- 172.21.15.107:0/3317486865 shutdown_connections 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.626+0000 7f9dad649740 1 -- 172.21.15.107:0/3317486865 wait complete. 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.627+0000 7f9dad649740 1 -- start start 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.627+0000 7f9dad649740 1 Processor -- start 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.627+0000 7f9dad649740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edbc70 0x55a753ee3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.627+0000 7f9dad649740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edc590 0x55a753ee4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.627+0000 7f9dad649740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 0x55a753ee4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.627+0000 7f9dad649740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a753ee9a20 con 0x55a753edc590 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.627+0000 7f9dad649740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a753dfd0f0 con 0x55a753ee6e50 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.627+0000 7f9dad649740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a753ee9ba0 con 0x55a753edbc70 2024-09-19T04:28:37.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.627+0000 7f9dad649740 10 client.0 _ll_register_callbacks cb 0x55a753de4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:37.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edbc70 0x55a753ee3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edc590 0x55a753ee4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da8c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 0x55a753ee4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edc590 0x55a753ee4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33386/0 (socket says 172.21.15.107:33386) 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da3400640 1 -- 172.21.15.107:0/369444219 learned_addr learned my addr 172.21.15.107:0/369444219 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da8c00640 1 -- 172.21.15.107:0/369444219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edbc70 msgr2=0x55a753ee3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da8c00640 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edbc70 0x55a753ee3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da8c00640 1 -- 172.21.15.107:0/369444219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edc590 msgr2=0x55a753ee4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da8c00640 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edc590 0x55a753ee4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da8c00640 1 -- 172.21.15.107:0/369444219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a753bb33c0 con 0x55a753ee6e50 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9da8c00640 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 0x55a753ee4770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0cda30 tx=0x7f9d9c09c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16c94bd397fd7ea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.628+0000 7f9d8d600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.629+0000 7f9d8d600640 1 -- 172.21.15.107:0/369444219 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0b4020 con 0x55a753ee6e50 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.629+0000 7f9d8d600640 1 -- 172.21.15.107:0/369444219 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f9d9c0a4da0 con 0x55a753ee6e50 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.629+0000 7f9dad649740 1 -- 172.21.15.107:0/369444219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d940047a0 con 0x55a753ee6e50 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.629+0000 7f9d8d600640 1 -- 172.21.15.107:0/369444219 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9d9c0a4fa0 con 0x55a753ee6e50 2024-09-19T04:28:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.629+0000 7f9dad649740 10 client.26747 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:37.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.629+0000 7f9dad649740 1 -- 172.21.15.107:0/369444219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a753ef5b30 con 0x55a753ee6e50 2024-09-19T04:28:37.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.629+0000 7f9d8d600640 1 -- 172.21.15.107:0/369444219 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d940047a0 con 0x55a753ee6e50 2024-09-19T04:28:37.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.630+0000 7f9d8d600640 1 -- 172.21.15.107:0/369444219 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f9d9c0672c0 con 0x55a753ee6e50 2024-09-19T04:28:37.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.630+0000 7f9dad649740 10 client.26747 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:37.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.630+0000 7f9dad649740 10 client.26747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:37.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.631+0000 7f9dad649740 20 client.26747 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:37.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.631+0000 7f9dad649740 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a753ef6fe0 0x55a753ef9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.631+0000 7f9dad649740 4 client.26747 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:37.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.631+0000 7f9dad649740 1 -- 172.21.15.107:0/369444219 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a753e4f950 con 0x55a753ef6fe0 2024-09-19T04:28:37.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.664+0000 7f9da3e00640 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a753ef6fe0 0x55a753ef9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.691 INFO:tasks.workunit.client.0.smithi107.stdout: 1 257113 2.35 MB/sec execute 295 sec latency 2969.104 ms 2024-09-19T04:28:37.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.759+0000 7f9da3e00640 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a753ef6fe0 0x55a753ef9460 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612b61d549b0b9 server_cookie=d031557ded059677 in_seq=0 out_seq=0 2024-09-19T04:28:37.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.759+0000 7f9d8d600640 10 client.26747 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9d8d600640 1 -- 172.21.15.107:0/369444219 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a753e4f950 con 0x55a753ef6fe0 2024-09-19T04:28:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9d8d600640 10 client.26747 handle_command_reply: tid=0 2024-09-19T04:28:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9dad649740 1 client.26747 shutdown 2024-09-19T04:28:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9dad649740 1 -- 172.21.15.107:0/369444219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 msgr2=0x55a753ee4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9dad649740 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 0x55a753ee4770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0cda30 tx=0x7f9d9c09c680 comp rx=0 tx=0).stop 2024-09-19T04:28:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9dad649740 1 -- 172.21.15.107:0/369444219 shutdown_connections 2024-09-19T04:28:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9dad649740 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a753ef6fe0 0x55a753ef9460 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9dad649740 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 0x55a753ee4770 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9dad649740 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edc590 0x55a753ee4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9dad649740 1 --2- 172.21.15.107:0/369444219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edbc70 0x55a753ee3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9dad649740 1 -- 172.21.15.107:0/369444219 >> 172.21.15.107:0/369444219 conn(0x55a753ed8200 msgr2=0x55a753eddee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9d8d600640 0 client.26747 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.771+0000 7f9dad649740 1 -- 172.21.15.107:0/369444219 shutdown_connections 2024-09-19T04:28:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.772+0000 7f9dad649740 1 -- 172.21.15.107:0/369444219 wait complete. 2024-09-19T04:28:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.772+0000 7f9dad649740 20 client.26747 trim_cache size 0 max 16384 2024-09-19T04:28:37.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.786+0000 7f9dad649740 1 Processor -- start 2024-09-19T04:28:37.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.786+0000 7f9dad649740 1 -- start start 2024-09-19T04:28:37.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.786+0000 7f9dad649740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753ede6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.786+0000 7f9dad649740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.786+0000 7f9dad649740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753ee6e50 0x55a753efed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9dad649740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8400e6e0 con 0x55a753edbc70 2024-09-19T04:28:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9dad649740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84010250 con 0x55a753edc590 2024-09-19T04:28:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9dad649740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8400ede0 con 0x55a753ee6e50 2024-09-19T04:28:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753ede6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:36486/0 (socket says 172.21.15.107:36486) 2024-09-19T04:28:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da3400640 1 -- 172.21.15.107:0/708969219 learned_addr learned my addr 172.21.15.107:0/708969219 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da8c00640 1 --2- 172.21.15.107:0/708969219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753ee6e50 0x55a753efed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da3400640 1 -- 172.21.15.107:0/708969219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753ee6e50 msgr2=0x55a753efed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da3400640 1 --2- 172.21.15.107:0/708969219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753ee6e50 0x55a753efed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da3400640 1 -- 172.21.15.107:0/708969219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 msgr2=0x55a753ede6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da3400640 1 --2- 172.21.15.107:0/708969219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753ede6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da3400640 1 -- 172.21.15.107:0/708969219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a753ee2ca0 con 0x55a753edc590 2024-09-19T04:28:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.787+0000 7f9da3e00640 1 --2- 172.21.15.107:0/708969219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753ede6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.794+0000 7f9da3400640 1 --2- 172.21.15.107:0/708969219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee25c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9d90000c00 tx=0x7f9d9003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b189cd91d609e581 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.795+0000 7f9d8fe00640 1 -- 172.21.15.107:0/708969219 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90035040 con 0x55a753edc590 2024-09-19T04:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.795+0000 7f9d8fe00640 1 -- 172.21.15.107:0/708969219 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9004f070 con 0x55a753edc590 2024-09-19T04:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.795+0000 7f9d8fe00640 1 -- 172.21.15.107:0/708969219 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d900547d0 con 0x55a753edc590 2024-09-19T04:28:37.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.796+0000 7f9dad649740 1 -- 172.21.15.107:0/708969219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 msgr2=0x55a753ee25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.796+0000 7f9dad649740 1 --2- 172.21.15.107:0/708969219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee25c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9d90000c00 tx=0x7f9d9003f000 comp rx=0 tx=0).stop 2024-09-19T04:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.796+0000 7f9dad649740 1 -- 172.21.15.107:0/708969219 shutdown_connections 2024-09-19T04:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.796+0000 7f9dad649740 1 --2- 172.21.15.107:0/708969219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753ee6e50 0x55a753efed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.796+0000 7f9dad649740 1 --2- 172.21.15.107:0/708969219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753edc590 0x55a753ee25c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.796+0000 7f9dad649740 1 --2- 172.21.15.107:0/708969219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753ede6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.796+0000 7f9dad649740 1 -- 172.21.15.107:0/708969219 >> 172.21.15.107:0/708969219 conn(0x55a753ed8200 msgr2=0x55a753ee5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.796+0000 7f9dad649740 1 -- 172.21.15.107:0/708969219 shutdown_connections 2024-09-19T04:28:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.796+0000 7f9dad649740 1 -- 172.21.15.107:0/708969219 wait complete. 2024-09-19T04:28:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.796+0000 7f9dad649740 1 -- start start 2024-09-19T04:28:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.797+0000 7f9dad649740 1 Processor -- start 2024-09-19T04:28:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.797+0000 7f9dad649740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753eeacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.797+0000 7f9dad649740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edc590 0x55a753eeb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.797+0000 7f9dad649740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 0x55a753eeb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.797+0000 7f9dad649740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8400fb80 con 0x55a753edbc70 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.797+0000 7f9dad649740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8400f4b0 con 0x55a753ee6e50 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.797+0000 7f9dad649740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8400c6e0 con 0x55a753edc590 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.797+0000 7f9dad649740 10 client.0 _ll_register_callbacks cb 0x7f9d9c098860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.797+0000 7f9da3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edc590 0x55a753eeb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.797+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753eeacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.798+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753eeacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33400/0 (socket says 172.21.15.107:33400) 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.798+0000 7f9da3e00640 1 -- 172.21.15.107:0/1321183786 learned_addr learned my addr 172.21.15.107:0/1321183786 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.798+0000 7f9da8c00640 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 0x55a753eeb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.798+0000 7f9da3400640 1 -- 172.21.15.107:0/1321183786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 msgr2=0x55a753eeb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.798+0000 7f9da3400640 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 0x55a753eeb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.798+0000 7f9da3400640 1 -- 172.21.15.107:0/1321183786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 msgr2=0x55a753eeacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.798+0000 7f9da3400640 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753eeacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.798+0000 7f9da3400640 1 -- 172.21.15.107:0/1321183786 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a753f60e00 con 0x55a753edc590 2024-09-19T04:28:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.798+0000 7f9da3e00640 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753eeacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.806+0000 7f9da3400640 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edc590 0x55a753eeb210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55a753edca20 tx=0x7f9d90031ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8855b00f461af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.806+0000 7f9d8d600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:28:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.807+0000 7f9d8d600640 1 -- 172.21.15.107:0/1321183786 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d900319c0 con 0x55a753edc590 2024-09-19T04:28:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.807+0000 7f9d8d600640 1 -- 172.21.15.107:0/1321183786 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f9d90031b60 con 0x55a753edc590 2024-09-19T04:28:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.807+0000 7f9dad649740 1 -- 172.21.15.107:0/1321183786 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d9004f070 con 0x55a753edc590 2024-09-19T04:28:37.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.807+0000 7f9d8d600640 1 -- 172.21.15.107:0/1321183786 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9d9006a050 con 0x55a753edc590 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.807+0000 7f9dad649740 10 client.26716 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.807+0000 7f9dad649740 1 -- 172.21.15.107:0/1321183786 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a753f620e0 con 0x55a753edc590 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.807+0000 7f9d8d600640 1 -- 172.21.15.107:0/1321183786 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9004f070 con 0x55a753edc590 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.808+0000 7f9d8d600640 1 -- 172.21.15.107:0/1321183786 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f9d9003d470 con 0x55a753edc590 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.808+0000 7f9dad649740 10 client.26716 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.808+0000 7f9dad649740 10 client.26716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.808+0000 7f9dad649740 20 client.26716 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.808+0000 7f9dad649740 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a753f62830 0x55a753f64c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.808+0000 7f9dad649740 4 client.26716 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.808+0000 7f9dad649740 1 -- 172.21.15.107:0/1321183786 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a753e49e60 con 0x55a753f62830 2024-09-19T04:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.808+0000 7f9da3e00640 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a753f62830 0x55a753f64c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:37.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9da3e00640 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a753f62830 0x55a753f64c50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e11b5c2592138c61 server_cookie=fd3a3fd28183003b in_seq=0 out_seq=0 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9d8d600640 10 client.26716 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9d8d600640 1 -- 172.21.15.107:0/1321183786 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a753e49e60 con 0x55a753f62830 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9d8d600640 10 client.26716 handle_command_reply: tid=0 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 client.26716 shutdown 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 -- 172.21.15.107:0/1321183786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edc590 msgr2=0x55a753eeb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edc590 0x55a753eeb210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55a753edca20 tx=0x7f9d90031ed0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 -- 172.21.15.107:0/1321183786 shutdown_connections 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55a753f62830 0x55a753f64c50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a753ee6e50 0x55a753eeb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a753edc590 0x55a753eeb210 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 --2- 172.21.15.107:0/1321183786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a753edbc70 0x55a753eeacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 -- 172.21.15.107:0/1321183786 >> 172.21.15.107:0/1321183786 conn(0x55a753ed8200 msgr2=0x55a753ed6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:37.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9d8d600640 0 client.26716 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:37.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 -- 172.21.15.107:0/1321183786 shutdown_connections 2024-09-19T04:28:37.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.809+0000 7f9dad649740 1 -- 172.21.15.107:0/1321183786 wait complete. 2024-09-19T04:28:37.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.810+0000 7f9dad649740 20 client.26716 trim_cache size 0 max 16384 2024-09-19T04:28:37.813 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:28:37.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.810+0000 7f9daac00640 1 -- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9d880a33b0 msgr2=0x7f9d880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:37.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.810+0000 7f9daac00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9d880a33b0 0x7f9d880a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55a753ed9580 tx=0x7f9d900046c0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.810+0000 7f9daac00640 1 -- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4106f00 msgr2=0x7f9da419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:37.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.810+0000 7f9daac00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4106f00 0x7f9da419ef60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009fd0 tx=0x7f9d9c00ef10 comp rx=0 tx=0).stop 2024-09-19T04:28:37.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.810+0000 7f9daac00640 1 -- 172.21.15.107:0/3936175365 shutdown_connections 2024-09-19T04:28:37.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.810+0000 7f9daac00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f9d880a33b0 0x7f9d880a5870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.810+0000 7f9daac00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4106f00 0x7f9da419ef60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.811+0000 7f9daac00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9da4102780 0x7f9da419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.811+0000 7f9daac00640 1 --2- 172.21.15.107:0/3936175365 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da4101dd0 0x7f9da419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.811+0000 7f9daac00640 1 -- 172.21.15.107:0/3936175365 >> 172.21.15.107:0/3936175365 conn(0x7f9da40fd890 msgr2=0x7f9da40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.811+0000 7f9daac00640 1 -- 172.21.15.107:0/3936175365 shutdown_connections 2024-09-19T04:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:37.811+0000 7f9daac00640 1 -- 172.21.15.107:0/3936175365 wait complete. 2024-09-19T04:28:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:37 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:28:38.203 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:37 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:28:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:37 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:28:38.691 INFO:tasks.workunit.client.0.smithi107.stdout: 1 257113 2.34 MB/sec execute 296 sec latency 3969.188 ms 2024-09-19T04:28:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:38 smithi107 ceph-mon[30659]: pgmap v256: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 527 KiB/s wr, 52 op/s 2024-09-19T04:28:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:38 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:28:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:38 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:28:38.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:38 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:28:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:38 smithi115 ceph-mon[35477]: pgmap v256: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 527 KiB/s wr, 52 op/s 2024-09-19T04:28:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:38 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:28:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:38 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:28:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:38 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:28:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:38 smithi145 ceph-mon[35096]: pgmap v256: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 527 KiB/s wr, 52 op/s 2024-09-19T04:28:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:38 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:28:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:38 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:28:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:38 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:28:39.691 INFO:tasks.workunit.client.0.smithi107.stdout: 1 258195 2.35 MB/sec execute 297 sec latency 4500.048 ms 2024-09-19T04:28:39.767 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:28:39.767 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:28:40.185 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:40.691 INFO:tasks.workunit.client.0.smithi107.stdout: 1 259500 2.37 MB/sec execute 298 sec latency 40.200 ms 2024-09-19T04:28:40.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.856+0000 7f8b02e00640 1 -- 172.21.15.107:0/2916894187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8afc1049c0 msgr2=0x7f8afc108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:40.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.856+0000 7f8b02e00640 1 --2- 172.21.15.107:0/2916894187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8afc1049c0 0x7f8afc108a10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009900 tx=0x7f8ae802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.856+0000 7f8b02e00640 1 -- 172.21.15.107:0/2916894187 shutdown_connections 2024-09-19T04:28:40.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.856+0000 7f8b02e00640 1 --2- 172.21.15.107:0/2916894187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8afc109140 0x7f8afc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.856+0000 7f8b02e00640 1 --2- 172.21.15.107:0/2916894187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8afc1049c0 0x7f8afc108a10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.856+0000 7f8b02e00640 1 --2- 172.21.15.107:0/2916894187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8afc104010 0x7f8afc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.856+0000 7f8b02e00640 1 -- 172.21.15.107:0/2916894187 >> 172.21.15.107:0/2916894187 conn(0x7f8afc0ffa50 msgr2=0x7f8afc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:40.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.856+0000 7f8b02e00640 1 -- 172.21.15.107:0/2916894187 shutdown_connections 2024-09-19T04:28:40.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:40 smithi107 ceph-mon[30659]: pgmap v257: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 527 KiB/s wr, 52 op/s 2024-09-19T04:28:40.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.857+0000 7f8b02e00640 1 -- 172.21.15.107:0/2916894187 wait complete. 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8b02e00640 1 Processor -- start 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8b02e00640 1 -- start start 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8b02e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8afc104010 0x7f8afc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8b02e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8afc1049c0 0x7f8afc19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8b02e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8afc109140 0x7f8afc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8b02e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc114160 con 0x7f8afc104010 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8b02e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc113fe0 con 0x7f8afc1049c0 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8b02e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc1142e0 con 0x7f8afc109140 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8afb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8afc1049c0 0x7f8afc19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8b00e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8afc109140 0x7f8afc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8afb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8afc1049c0 0x7f8afc19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:40620/0 (socket says 172.21.15.107:40620) 2024-09-19T04:28:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8afb400640 1 -- 172.21.15.107:0/1645124632 learned_addr learned my addr 172.21.15.107:0/1645124632 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.858+0000 7f8afbe00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8afc104010 0x7f8afc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8b00e00640 1 -- 172.21.15.107:0/1645124632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8afc1049c0 msgr2=0x7f8afc19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8b00e00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8afc1049c0 0x7f8afc19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8b00e00640 1 -- 172.21.15.107:0/1645124632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8afc104010 msgr2=0x7f8afc19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8b00e00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8afc104010 0x7f8afc19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8b00e00640 1 -- 172.21.15.107:0/1645124632 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc19f6b0 con 0x7f8afc109140 2024-09-19T04:28:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8b00e00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8afc109140 0x7f8afc19efd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009fd0 tx=0x7f8af400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8af8c00640 1 -- 172.21.15.107:0/1645124632 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af400ce20 con 0x7f8afc109140 2024-09-19T04:28:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8af8c00640 1 -- 172.21.15.107:0/1645124632 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af4005020 con 0x7f8afc109140 2024-09-19T04:28:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8b02e00640 1 -- 172.21.15.107:0/1645124632 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8afc19f9a0 con 0x7f8afc109140 2024-09-19T04:28:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8af8c00640 1 -- 172.21.15.107:0/1645124632 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4010720 con 0x7f8afc109140 2024-09-19T04:28:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.859+0000 7f8b02e00640 1 -- 172.21.15.107:0/1645124632 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8afc1a7280 con 0x7f8afc109140 2024-09-19T04:28:40.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.864+0000 7f8b059ba740 1 Processor -- start 2024-09-19T04:28:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.864+0000 7f8af8c00640 1 -- 172.21.15.107:0/1645124632 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f8af4005540 con 0x7f8afc109140 2024-09-19T04:28:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.864+0000 7f8b059ba740 1 -- start start 2024-09-19T04:28:40.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.864+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017ddac0 0x560c017ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.864+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017de3e0 0x560c017e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.864+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.864+0000 7f8b059ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560c01744290 con 0x560c017ddac0 2024-09-19T04:28:40.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.864+0000 7f8b059ba740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560c016ebb40 con 0x560c017e8ca0 2024-09-19T04:28:40.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.864+0000 7f8b059ba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560c01740250 con 0x560c017de3e0 2024-09-19T04:28:40.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017de3e0 0x560c017e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017de3e0 0x560c017e8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53480/0 (socket says 172.21.15.107:53480) 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afbe00640 1 -- 172.21.15.107:0/588098703 learned_addr learned my addr 172.21.15.107:0/588098703 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afb400640 1 --2- 172.21.15.107:0/588098703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017ddac0 0x560c017ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afbe00640 1 -- 172.21.15.107:0/588098703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 msgr2=0x560c017eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afb400640 1 --2- 172.21.15.107:0/588098703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afbe00640 1 --2- 172.21.15.107:0/588098703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afbe00640 1 -- 172.21.15.107:0/588098703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017ddac0 msgr2=0x560c017ddea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afbe00640 1 --2- 172.21.15.107:0/588098703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017ddac0 0x560c017ddea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afbe00640 1 -- 172.21.15.107:0/588098703 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c014b56a0 con 0x560c017de3e0 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afb400640 1 --2- 172.21.15.107:0/588098703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.865+0000 7f8afbe00640 1 --2- 172.21.15.107:0/588098703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017de3e0 0x560c017e8760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009cf0 tx=0x7f8aec011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd822adf0393e4c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.866+0000 7f8af1600640 1 -- 172.21.15.107:0/588098703 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec004230 con 0x560c017de3e0 2024-09-19T04:28:40.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.866+0000 7f8af1600640 1 -- 172.21.15.107:0/588098703 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aec0043d0 con 0x560c017de3e0 2024-09-19T04:28:40.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.866+0000 7f8af1600640 1 -- 172.21.15.107:0/588098703 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec019960 con 0x560c017de3e0 2024-09-19T04:28:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.866+0000 7f8af8c00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ae00a33b0 0x7f8ae00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.866+0000 7f8af8c00640 1 -- 172.21.15.107:0/1645124632 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f8af40d02f0 con 0x7f8afc109140 2024-09-19T04:28:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.866+0000 7f8b059ba740 1 -- 172.21.15.107:0/588098703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017de3e0 msgr2=0x560c017e8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.866+0000 7f8b059ba740 1 --2- 172.21.15.107:0/588098703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017de3e0 0x560c017e8760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009cf0 tx=0x7f8aec011810 comp rx=0 tx=0).stop 2024-09-19T04:28:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.866+0000 7f8afb400640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ae00a33b0 0x7f8ae00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.867+0000 7f8b059ba740 1 -- 172.21.15.107:0/588098703 shutdown_connections 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.867+0000 7f8b059ba740 1 --2- 172.21.15.107:0/588098703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.867+0000 7f8b059ba740 1 --2- 172.21.15.107:0/588098703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017de3e0 0x560c017e8760 secure :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009cf0 tx=0x7f8aec011810 comp rx=0 tx=0).stop 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.867+0000 7f8b059ba740 1 --2- 172.21.15.107:0/588098703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017ddac0 0x560c017ddea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.867+0000 7f8b059ba740 1 -- 172.21.15.107:0/588098703 >> 172.21.15.107:0/588098703 conn(0x560c017da050 msgr2=0x560c017da430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.867+0000 7f8b059ba740 1 -- 172.21.15.107:0/588098703 shutdown_connections 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.867+0000 7f8b059ba740 1 -- 172.21.15.107:0/588098703 wait complete. 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.867+0000 7f8afb400640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ae00a33b0 0x7f8ae00a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8031550 tx=0x7f8ae8005a20 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.867+0000 7f8b059ba740 1 -- start start 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8b059ba740 1 Processor -- start 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e6540 0x560c017f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8b059ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560c017eb910 con 0x560c017e6540 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8b059ba740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560c016fefa0 con 0x560c017e8ca0 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8b059ba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560c017eba90 con 0x560c017ddac0 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8b059ba740 10 client.0 _ll_register_callbacks cb 0x560c01452ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8afbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8afb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8afbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017e5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53484/0 (socket says 172.21.15.107:53484) 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8afbe00640 1 -- 172.21.15.107:0/3069106800 learned_addr learned my addr 172.21.15.107:0/3069106800 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8afb400640 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017e6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:40636/0 (socket says 172.21.15.107:40636) 2024-09-19T04:28:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.868+0000 7f8b00e00640 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e6540 0x560c017f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.869+0000 7f8afbe00640 1 -- 172.21.15.107:0/3069106800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 msgr2=0x560c017e6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.869+0000 7f8afbe00640 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017e6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.869+0000 7f8afbe00640 1 -- 172.21.15.107:0/3069106800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e6540 msgr2=0x560c017f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.869+0000 7f8afbe00640 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e6540 0x560c017f4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.869+0000 7f8afbe00640 1 -- 172.21.15.107:0/3069106800 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c017f52f0 con 0x560c017ddac0 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.869+0000 7f8afb400640 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.869+0000 7f8afbe00640 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017e5ac0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8aec01a000 tx=0x7f8aec003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=848f24da913eca2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.869+0000 7f8ae6000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.869+0000 7f8ae6000640 1 -- 172.21.15.107:0/3069106800 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec027070 con 0x560c017ddac0 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.870+0000 7f8ae6000640 1 -- 172.21.15.107:0/3069106800 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f8aec005240 con 0x560c017ddac0 2024-09-19T04:28:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.870+0000 7f8b059ba740 1 -- 172.21.15.107:0/3069106800 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aec0043d0 con 0x560c017ddac0 2024-09-19T04:28:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.870+0000 7f8ae6000640 1 -- 172.21.15.107:0/3069106800 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8aec033050 con 0x560c017ddac0 2024-09-19T04:28:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.870+0000 7f8b059ba740 10 client.26728 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.870+0000 7f8b059ba740 1 -- 172.21.15.107:0/3069106800 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560c017f7dc0 con 0x560c017ddac0 2024-09-19T04:28:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.870+0000 7f8ae6000640 1 -- 172.21.15.107:0/3069106800 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aec0043d0 con 0x560c017ddac0 2024-09-19T04:28:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.871+0000 7f8ae6000640 1 -- 172.21.15.107:0/3069106800 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f8aec004ba0 con 0x560c017ddac0 2024-09-19T04:28:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.871+0000 7f8b059ba740 10 client.26728 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.871+0000 7f8b059ba740 10 client.26728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.871+0000 7f8b059ba740 20 client.26728 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.871+0000 7f8b059ba740 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560c017f9170 0x560c017fb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.871+0000 7f8b059ba740 4 client.26728 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.871+0000 7f8b059ba740 1 -- 172.21.15.107:0/3069106800 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c01751800 con 0x560c017f9170 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.871+0000 7f8afb400640 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560c017f9170 0x560c017fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.871+0000 7f8afb400640 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560c017f9170 0x560c017fb560 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8123075dc0e74988 server_cookie=6c044d32ecc120c9 in_seq=0 out_seq=0 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.871+0000 7f8ae6000640 10 client.26728 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.872+0000 7f8ae6000640 1 -- 172.21.15.107:0/3069106800 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c01751800 con 0x560c017f9170 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.872+0000 7f8ae6000640 10 client.26728 handle_command_reply: tid=0 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.872+0000 7f8b059ba740 1 client.26728 shutdown 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.872+0000 7f8b059ba740 1 -- 172.21.15.107:0/3069106800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 msgr2=0x560c017e5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.872+0000 7f8b059ba740 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017e5ac0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8aec01a000 tx=0x7f8aec003820 comp rx=0 tx=0).stop 2024-09-19T04:28:40.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.872+0000 7f8b059ba740 1 -- 172.21.15.107:0/3069106800 shutdown_connections 2024-09-19T04:28:40.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.872+0000 7f8b059ba740 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560c017f9170 0x560c017fb560 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.872+0000 7f8b059ba740 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e6540 0x560c017f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.873+0000 7f8b059ba740 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.873+0000 7f8b059ba740 1 --2- 172.21.15.107:0/3069106800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017e5ac0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.873+0000 7f8b059ba740 1 -- 172.21.15.107:0/3069106800 >> 172.21.15.107:0/3069106800 conn(0x560c017da050 msgr2=0x560c017dfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:40.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.873+0000 7f8ae6000640 0 client.26728 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:40.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.873+0000 7f8b059ba740 1 -- 172.21.15.107:0/3069106800 shutdown_connections 2024-09-19T04:28:40.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.873+0000 7f8b059ba740 1 -- 172.21.15.107:0/3069106800 wait complete. 2024-09-19T04:28:40.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.873+0000 7f8b059ba740 20 client.26728 trim_cache size 0 max 16384 2024-09-19T04:28:40.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.894+0000 7f8b059ba740 1 Processor -- start 2024-09-19T04:28:40.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.894+0000 7f8b059ba740 1 -- start start 2024-09-19T04:28:40.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.894+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.894+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e6540 0x560c017f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.894+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e8ca0 0x560c017e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.894+0000 7f8b059ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc1afae0 con 0x560c017e8ca0 2024-09-19T04:28:40.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.894+0000 7f8b059ba740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc1b1650 con 0x560c017e6540 2024-09-19T04:28:40.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.894+0000 7f8b059ba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc1b01e0 con 0x560c017ddac0 2024-09-19T04:28:40.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e6540 0x560c017f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017d99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53498/0 (socket says 172.21.15.107:53498) 2024-09-19T04:28:40.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afbe00640 1 -- 172.21.15.107:0/377377626 learned_addr learned my addr 172.21.15.107:0/377377626 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:40.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e6540 0x560c017f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:40652/0 (socket says 172.21.15.107:40652) 2024-09-19T04:28:40.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8b00e00640 1 --2- 172.21.15.107:0/377377626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e8ca0 0x560c017e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afbe00640 1 -- 172.21.15.107:0/377377626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e6540 msgr2=0x560c017f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:40.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afbe00640 1 --2- 172.21.15.107:0/377377626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e6540 0x560c017f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afbe00640 1 -- 172.21.15.107:0/377377626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e8ca0 msgr2=0x560c017e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:40.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afbe00640 1 --2- 172.21.15.107:0/377377626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e8ca0 0x560c017e7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afbe00640 1 -- 172.21.15.107:0/377377626 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c017e4050 con 0x560c017ddac0 2024-09-19T04:28:40.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8b00e00640 1 --2- 172.21.15.107:0/377377626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e8ca0 0x560c017e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:40.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afb400640 1 --2- 172.21.15.107:0/377377626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e6540 0x560c017f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:40.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.895+0000 7f8afbe00640 1 --2- 172.21.15.107:0/377377626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017d99b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8aec011f50 tx=0x7f8aec003910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6e5c173706bd547 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:40.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.896+0000 7f8af0c00640 1 -- 172.21.15.107:0/377377626 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec03c070 con 0x560c017ddac0 2024-09-19T04:28:40.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.896+0000 7f8af0c00640 1 -- 172.21.15.107:0/377377626 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aec011d10 con 0x560c017ddac0 2024-09-19T04:28:40.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.896+0000 7f8af0c00640 1 -- 172.21.15.107:0/377377626 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec027050 con 0x560c017ddac0 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.897+0000 7f8b059ba740 1 -- 172.21.15.107:0/377377626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 msgr2=0x560c017d99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.897+0000 7f8b059ba740 1 --2- 172.21.15.107:0/377377626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017d99b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8aec011f50 tx=0x7f8aec003910 comp rx=0 tx=0).stop 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.897+0000 7f8b059ba740 1 -- 172.21.15.107:0/377377626 shutdown_connections 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.897+0000 7f8b059ba740 1 --2- 172.21.15.107:0/377377626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017e8ca0 0x560c017e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.897+0000 7f8b059ba740 1 --2- 172.21.15.107:0/377377626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e6540 0x560c017f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.897+0000 7f8b059ba740 1 --2- 172.21.15.107:0/377377626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017ddac0 0x560c017d99b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.897+0000 7f8b059ba740 1 -- 172.21.15.107:0/377377626 >> 172.21.15.107:0/377377626 conn(0x560c017da050 msgr2=0x560c017e7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.897+0000 7f8b059ba740 1 -- 172.21.15.107:0/377377626 shutdown_connections 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.897+0000 7f8b059ba740 1 -- 172.21.15.107:0/377377626 wait complete. 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.898+0000 7f8b059ba740 1 -- start start 2024-09-19T04:28:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.898+0000 7f8b059ba740 1 Processor -- start 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.898+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017ddac0 0x560c017f0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.898+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017e6540 0x560c017f1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8b059ba740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017e2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8b059ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc1b0f80 con 0x560c017ddac0 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8b059ba740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc1b08b0 con 0x560c017e8ca0 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8b059ba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc1adae0 con 0x560c017e6540 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8b059ba740 10 client.0 _ll_register_callbacks cb 0x7f8aec009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8afb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017e6540 0x560c017f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8afb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017e6540 0x560c017f1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53508/0 (socket says 172.21.15.107:53508) 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8afb400640 1 -- 172.21.15.107:0/2793524446 learned_addr learned my addr 172.21.15.107:0/2793524446 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8b00e00640 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8afbe00640 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017ddac0 0x560c017f0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8afb400640 1 -- 172.21.15.107:0/2793524446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 msgr2=0x560c017e2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:40.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8afb400640 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017e2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8afb400640 1 -- 172.21.15.107:0/2793524446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017ddac0 msgr2=0x560c017f0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:40.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8afb400640 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017ddac0 0x560c017f0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.899+0000 7f8afb400640 1 -- 172.21.15.107:0/2793524446 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c017e2a00 con 0x560c017e6540 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.900+0000 7f8afb400640 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017e6540 0x560c017f1400 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8054fd0 tx=0x7f8ae8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9519e8235b71272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.900+0000 7f8ae6000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.900+0000 7f8ae6000640 1 -- 172.21.15.107:0/2793524446 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8035040 con 0x560c017e6540 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.900+0000 7f8b059ba740 1 -- 172.21.15.107:0/2793524446 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aec011d10 con 0x560c017e6540 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.900+0000 7f8ae6000640 1 -- 172.21.15.107:0/2793524446 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f8ae8005450 con 0x560c017e6540 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8ae6000640 1 -- 172.21.15.107:0/2793524446 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8ae8005650 con 0x560c017e6540 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8b059ba740 10 client.26734 fetch_fsmap learned FSMap version 45 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8b059ba740 1 -- 172.21.15.107:0/2793524446 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560c01863740 con 0x560c017e6540 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8ae6000640 1 -- 172.21.15.107:0/2793524446 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aec011d10 con 0x560c017e6540 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8ae6000640 1 -- 172.21.15.107:0/2793524446 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f8ae8064020 con 0x560c017e6540 2024-09-19T04:28:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8b059ba740 10 client.26734 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T04:28:40.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8b059ba740 10 client.26734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:40.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8b059ba740 20 client.26734 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:40.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8b059ba740 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560c01863e50 0x560c01866270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:40.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8b059ba740 4 client.26734 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:28:40.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.901+0000 7f8b059ba740 1 -- 172.21.15.107:0/2793524446 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560c0174bd10 con 0x560c01863e50 2024-09-19T04:28:40.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.902+0000 7f8afbe00640 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560c01863e50 0x560c01866270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:40.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.902+0000 7f8afbe00640 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560c01863e50 0x560c01866270 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4525ab9ec3b704d7 server_cookie=86621e410dda572d in_seq=0 out_seq=0 2024-09-19T04:28:40.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.902+0000 7f8ae6000640 10 client.26734 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:40.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.903+0000 7f8ae6000640 1 -- 172.21.15.107:0/2793524446 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560c0174bd10 con 0x560c01863e50 2024-09-19T04:28:40.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.903+0000 7f8ae6000640 10 client.26734 handle_command_reply: tid=0 2024-09-19T04:28:40.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.903+0000 7f8b059ba740 1 client.26734 shutdown 2024-09-19T04:28:40.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 1 -- 172.21.15.107:0/2793524446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017e6540 msgr2=0x560c017f1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:40.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017e6540 0x560c017f1400 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8054fd0 tx=0x7f8ae8009260 comp rx=0 tx=0).stop 2024-09-19T04:28:40.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 1 -- 172.21.15.107:0/2793524446 shutdown_connections 2024-09-19T04:28:40.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560c01863e50 0x560c01866270 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560c017e8ca0 0x560c017e2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560c017e6540 0x560c017f1400 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 1 --2- 172.21.15.107:0/2793524446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c017ddac0 0x560c017f0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8ae6000640 0 client.26734 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:40.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 1 -- 172.21.15.107:0/2793524446 >> 172.21.15.107:0/2793524446 conn(0x560c017da050 msgr2=0x560c017e7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:40.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 1 -- 172.21.15.107:0/2793524446 shutdown_connections 2024-09-19T04:28:40.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 1 -- 172.21.15.107:0/2793524446 wait complete. 2024-09-19T04:28:40.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.904+0000 7f8b059ba740 20 client.26734 trim_cache size 0 max 16384 2024-09-19T04:28:40.908 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:28:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:28:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "6127dce7-e347-49b8-9a9b-eb5f3a7c8260", 2024-09-19T04:28:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:28:40.908 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 -- 172.21.15.107:0/1645124632 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ae00a33b0 msgr2=0x7f8ae00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ae00a33b0 0x7f8ae00a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8031550 tx=0x7f8ae8005a20 comp rx=0 tx=0).stop 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 -- 172.21.15.107:0/1645124632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8afc109140 msgr2=0x7f8afc19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8afc109140 0x7f8afc19efd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009fd0 tx=0x7f8af400ef10 comp rx=0 tx=0).stop 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 -- 172.21.15.107:0/1645124632 shutdown_connections 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ae00a33b0 0x7f8ae00a5870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8afc109140 0x7f8afc19efd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8afc1049c0 0x7f8afc19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 --2- 172.21.15.107:0/1645124632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8afc104010 0x7f8afc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 -- 172.21.15.107:0/1645124632 >> 172.21.15.107:0/1645124632 conn(0x7f8afc0ffa50 msgr2=0x7f8afc100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:40.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.905+0000 7f8b02e00640 1 -- 172.21.15.107:0/1645124632 shutdown_connections 2024-09-19T04:28:40.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:40.906+0000 7f8b02e00640 1 -- 172.21.15.107:0/1645124632 wait complete. 2024-09-19T04:28:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:40 smithi115 ceph-mon[35477]: pgmap v257: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 527 KiB/s wr, 52 op/s 2024-09-19T04:28:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:40 smithi145 ceph-mon[35096]: pgmap v257: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 527 KiB/s wr, 52 op/s 2024-09-19T04:28:41.691 INFO:tasks.workunit.client.0.smithi107.stdout: 1 260637 2.38 MB/sec execute 299 sec latency 50.991 ms 2024-09-19T04:28:41.824 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:28:42.196 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:28:42.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:28:42.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:28:42.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:41 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:28:42.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:41 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:28:42.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:28:42.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:41 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 260661 2.37 MB/sec execute 300 sec latency 1006.048 ms 2024-09-19T04:28:42.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.858+0000 7f047ec00640 1 -- 172.21.15.107:0/4093261112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0478103ff0 msgr2=0x7f04781043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:42.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.858+0000 7f047ec00640 1 --2- 172.21.15.107:0/4093261112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0478103ff0 0x7f04781043d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0468009900 tx=0x7f046802a400 comp rx=0 tx=0).stop 2024-09-19T04:28:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.858+0000 7f047ec00640 1 -- 172.21.15.107:0/4093261112 shutdown_connections 2024-09-19T04:28:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.858+0000 7f047ec00640 1 --2- 172.21.15.107:0/4093261112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0478109120 0x7f0478110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.858+0000 7f047ec00640 1 --2- 172.21.15.107:0/4093261112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04781049a0 0x7f04781089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.858+0000 7f047ec00640 1 --2- 172.21.15.107:0/4093261112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0478103ff0 0x7f04781043d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.858+0000 7f047ec00640 1 -- 172.21.15.107:0/4093261112 >> 172.21.15.107:0/4093261112 conn(0x7f04780ffa90 msgr2=0x7f0478101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.858+0000 7f047ec00640 1 -- 172.21.15.107:0/4093261112 shutdown_connections 2024-09-19T04:28:42.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.858+0000 7f047ec00640 1 -- 172.21.15.107:0/4093261112 wait complete. 2024-09-19T04:28:42.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:42 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:28:42.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:42 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:42.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:42 smithi107 ceph-mon[30659]: pgmap v258: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1014 KiB/s wr, 89 op/s 2024-09-19T04:28:42.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:42 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:28:42.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.859+0000 7f047ec00640 1 Processor -- start 2024-09-19T04:28:42.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.859+0000 7f047ec00640 1 -- start start 2024-09-19T04:28:42.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.859+0000 7f047ec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0478103ff0 0x7f047819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.859+0000 7f047ec00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04781049a0 0x7f047819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.859+0000 7f047ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0478109120 0x7f047819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.859+0000 7f047ec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0478114170 con 0x7f0478103ff0 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.859+0000 7f047ec00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0478113ff0 con 0x7f04781049a0 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.859+0000 7f047ec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04781142f0 con 0x7f0478109120 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f0477e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0478103ff0 0x7f047819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f0477e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0478103ff0 0x7f047819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51758/0 (socket says 172.21.15.107:51758) 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f0477e00640 1 -- 172.21.15.107:0/2205637853 learned_addr learned my addr 172.21.15.107:0/2205637853 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f0477400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04781049a0 0x7f047819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f047cc00640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0478109120 0x7f047819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f0477400640 1 -- 172.21.15.107:0/2205637853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0478109120 msgr2=0x7f047819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f0477400640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0478109120 0x7f047819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f0477400640 1 -- 172.21.15.107:0/2205637853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0478103ff0 msgr2=0x7f047819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:42.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f0477400640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0478103ff0 0x7f047819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f0477400640 1 -- 172.21.15.107:0/2205637853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047819f640 con 0x7f04781049a0 2024-09-19T04:28:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.860+0000 7f0477400640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04781049a0 0x7f047819abd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f046400e9f0 tx=0x7f046400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.861+0000 7f0474c00640 1 -- 172.21.15.107:0/2205637853 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046400ccc0 con 0x7f04781049a0 2024-09-19T04:28:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.861+0000 7f0474c00640 1 -- 172.21.15.107:0/2205637853 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0464010910 con 0x7f04781049a0 2024-09-19T04:28:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.861+0000 7f0474c00640 1 -- 172.21.15.107:0/2205637853 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464018800 con 0x7f04781049a0 2024-09-19T04:28:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.861+0000 7f047ec00640 1 -- 172.21.15.107:0/2205637853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047819f930 con 0x7f04781049a0 2024-09-19T04:28:42.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.861+0000 7f047ec00640 1 -- 172.21.15.107:0/2205637853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04781a7210 con 0x7f04781049a0 2024-09-19T04:28:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.864+0000 7f048165e740 1 Processor -- start 2024-09-19T04:28:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.864+0000 7f048165e740 1 -- start start 2024-09-19T04:28:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.864+0000 7f0474c00640 1 -- 172.21.15.107:0/2205637853 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f0464010ac0 con 0x7f04781049a0 2024-09-19T04:28:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.864+0000 7f048165e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124eb90 0x56326124ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.864+0000 7f048165e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x563261259830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.864+0000 7f048165e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 0x56326125c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f048165e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5632611b53e0 con 0x563261259d70 2024-09-19T04:28:42.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f048165e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56326115cd80 con 0x56326124eb90 2024-09-19T04:28:42.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f048165e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5632611b13a0 con 0x56326124f4b0 2024-09-19T04:28:42.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f0477400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 0x56326125c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f047cc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124eb90 0x56326124ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f0477e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x563261259830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f047cc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124eb90 0x56326124ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:40682/0 (socket says 172.21.15.107:40682) 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f047cc00640 1 -- 172.21.15.107:0/885545242 learned_addr learned my addr 172.21.15.107:0/885545242 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f047cc00640 1 -- 172.21.15.107:0/885545242 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 msgr2=0x563261259830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f047cc00640 1 --2- 172.21.15.107:0/885545242 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x563261259830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f047cc00640 1 -- 172.21.15.107:0/885545242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 msgr2=0x56326125c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f047cc00640 1 --2- 172.21.15.107:0/885545242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 0x56326125c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.865+0000 7f047cc00640 1 -- 172.21.15.107:0/885545242 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56326117bae0 con 0x56326124eb90 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.866+0000 7f047cc00640 1 --2- 172.21.15.107:0/885545242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124eb90 0x56326124ef70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0470009f60 tx=0x7f0470009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26c4ab0e5862471c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.866+0000 7f046d200640 1 -- 172.21.15.107:0/885545242 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047001e070 con 0x56326124eb90 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.866+0000 7f046d200640 1 -- 172.21.15.107:0/885545242 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0470003c30 con 0x56326124eb90 2024-09-19T04:28:42.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.866+0000 7f046d200640 1 -- 172.21.15.107:0/885545242 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047000fa20 con 0x56326124eb90 2024-09-19T04:28:42.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.866+0000 7f0474c00640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f045c0a33b0 0x7f045c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.866+0000 7f0474c00640 1 -- 172.21.15.107:0/2205637853 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f04640d0060 con 0x7f04781049a0 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.866+0000 7f0477e00640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f045c0a33b0 0x7f045c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.867+0000 7f048165e740 1 -- 172.21.15.107:0/885545242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124eb90 msgr2=0x56326124ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.867+0000 7f048165e740 1 --2- 172.21.15.107:0/885545242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124eb90 0x56326124ef70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0470009f60 tx=0x7f0470009f90 comp rx=0 tx=0).stop 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.867+0000 7f048165e740 1 -- 172.21.15.107:0/885545242 shutdown_connections 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.867+0000 7f048165e740 1 --2- 172.21.15.107:0/885545242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 0x56326125c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.867+0000 7f048165e740 1 --2- 172.21.15.107:0/885545242 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x563261259830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.867+0000 7f048165e740 1 --2- 172.21.15.107:0/885545242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124eb90 0x56326124ef70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.867+0000 7f048165e740 1 -- 172.21.15.107:0/885545242 >> 172.21.15.107:0/885545242 conn(0x56326124b120 msgr2=0x56326124b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.867+0000 7f048165e740 1 -- 172.21.15.107:0/885545242 shutdown_connections 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.867+0000 7f048165e740 1 -- 172.21.15.107:0/885545242 wait complete. 2024-09-19T04:28:42.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.867+0000 7f0477e00640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f045c0a33b0 0x7f045c0a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f04680096c0 tx=0x7f04680046c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.868+0000 7f048165e740 1 -- start start 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.868+0000 7f048165e740 1 Processor -- start 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.868+0000 7f048165e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 0x563261256ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.868+0000 7f048165e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124f4b0 0x5632612571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.868+0000 7f048165e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563261259d70 0x563261257720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.868+0000 7f048165e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56326125c9e0 con 0x56326124eb90 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.868+0000 7f048165e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5632611700f0 con 0x56326124f4b0 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.868+0000 7f048165e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56326125cb60 con 0x563261259d70 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.868+0000 7f048165e740 10 client.0 _ll_register_callbacks cb 0x5632611619e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0477400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124f4b0 0x5632612571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0477400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124f4b0 0x5632612571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:40688/0 (socket says 172.21.15.107:40688) 2024-09-19T04:28:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0477400640 1 -- 172.21.15.107:0/213117941 learned_addr learned my addr 172.21.15.107:0/213117941 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0477e00640 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 0x563261256ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f047cc00640 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563261259d70 0x563261257720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0477400640 1 -- 172.21.15.107:0/213117941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563261259d70 msgr2=0x563261257720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0477400640 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563261259d70 0x563261257720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0477400640 1 -- 172.21.15.107:0/213117941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 msgr2=0x563261256ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0477400640 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 0x563261256ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0477400640 1 -- 172.21.15.107:0/213117941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563260f263c0 con 0x56326124f4b0 2024-09-19T04:28:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0477400640 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124f4b0 0x5632612571e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f046409c000 tx=0x7f0464098660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb78cd786213d201 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:42.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0461600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:28:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0461600640 1 -- 172.21.15.107:0/213117941 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046409bc60 con 0x56326124f4b0 2024-09-19T04:28:42.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f0461600640 1 -- 172.21.15.107:0/213117941 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f04640cd9f0 con 0x56326124f4b0 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.869+0000 7f048165e740 1 -- 172.21.15.107:0/213117941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0470003c30 con 0x56326124f4b0 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f0461600640 1 -- 172.21.15.107:0/213117941 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f04640cdbf0 con 0x56326124f4b0 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f048165e740 10 client.26780 fetch_fsmap learned FSMap version 46 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f048165e740 1 -- 172.21.15.107:0/213117941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563261268710 con 0x56326124f4b0 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f0461600640 1 -- 172.21.15.107:0/213117941 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0470003c30 con 0x56326124f4b0 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f0461600640 1 -- 172.21.15.107:0/213117941 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f04640bc110 con 0x56326124f4b0 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f048165e740 10 client.26780 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f048165e740 10 client.26780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f048165e740 20 client.26780 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f048165e740 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563261269b20 0x56326126bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f048165e740 4 client.26780 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.870+0000 7f048165e740 1 -- 172.21.15.107:0/213117941 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632611c2950 con 0x563261269b20 2024-09-19T04:28:42.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.871+0000 7f0477e00640 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563261269b20 0x56326126bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.871+0000 7f0461600640 10 client.26780 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.871+0000 7f0477e00640 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563261269b20 0x56326126bf50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19d6580ba703f3a0 server_cookie=8e039a7ee51c02ed in_seq=0 out_seq=0 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f0461600640 1 -- 172.21.15.107:0/213117941 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632611c2950 con 0x563261269b20 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f0461600640 10 client.26780 handle_command_reply: tid=0 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f048165e740 1 client.26780 shutdown 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f048165e740 1 -- 172.21.15.107:0/213117941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124f4b0 msgr2=0x5632612571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f048165e740 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124f4b0 0x5632612571e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f046409c000 tx=0x7f0464098660 comp rx=0 tx=0).stop 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f048165e740 1 -- 172.21.15.107:0/213117941 shutdown_connections 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f048165e740 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x563261269b20 0x56326126bf50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f048165e740 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563261259d70 0x563261257720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f048165e740 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326124f4b0 0x5632612571e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f048165e740 1 --2- 172.21.15.107:0/213117941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 0x563261256ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f0461600640 0 client.26780 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.872+0000 7f048165e740 1 -- 172.21.15.107:0/213117941 >> 172.21.15.107:0/213117941 conn(0x56326124b120 msgr2=0x563261250e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.873+0000 7f048165e740 1 -- 172.21.15.107:0/213117941 shutdown_connections 2024-09-19T04:28:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.873+0000 7f048165e740 1 -- 172.21.15.107:0/213117941 wait complete. 2024-09-19T04:28:42.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.873+0000 7f048165e740 20 client.26780 trim_cache size 0 max 16384 2024-09-19T04:28:42.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.891+0000 7f048165e740 1 Processor -- start 2024-09-19T04:28:42.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.891+0000 7f048165e740 1 -- start start 2024-09-19T04:28:42.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.891+0000 7f048165e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 0x563261258c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.891+0000 7f048165e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x563261271820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.891+0000 7f048165e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563261259d70 0x563261271d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.891+0000 7f048165e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04781afa50 con 0x56326124eb90 2024-09-19T04:28:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.891+0000 7f048165e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b15c0 con 0x563261259d70 2024-09-19T04:28:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.891+0000 7f048165e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b0150 con 0x56326124f4b0 2024-09-19T04:28:42.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.892+0000 7f0477e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 0x563261258c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.892+0000 7f0477e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 0x563261258c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51784/0 (socket says 172.21.15.107:51784) 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.892+0000 7f0477e00640 1 -- 172.21.15.107:0/1448261225 learned_addr learned my addr 172.21.15.107:0/1448261225 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.892+0000 7f0477400640 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x563261271820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.892+0000 7f0477e00640 1 -- 172.21.15.107:0/1448261225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 msgr2=0x563261271820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.892+0000 7f0477e00640 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x563261271820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.892+0000 7f0477e00640 1 -- 172.21.15.107:0/1448261225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563261259d70 msgr2=0x563261271d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.892+0000 7f047cc00640 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563261259d70 0x563261271d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.893+0000 7f0477e00640 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563261259d70 0x563261271d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.893+0000 7f0477e00640 1 -- 172.21.15.107:0/1448261225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563261259610 con 0x56326124eb90 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.893+0000 7f0477400640 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x563261271820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:28:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.893+0000 7f047cc00640 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563261259d70 0x563261271d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:28:42.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.893+0000 7f0477e00640 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 0x563261258c10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0468004770 tx=0x7f046803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a62b43942c2f8d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:42.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.894+0000 7f0463e00640 1 -- 172.21.15.107:0/1448261225 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468035040 con 0x56326124eb90 2024-09-19T04:28:42.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.894+0000 7f0463e00640 1 -- 172.21.15.107:0/1448261225 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0468059070 con 0x56326124eb90 2024-09-19T04:28:42.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.894+0000 7f0463e00640 1 -- 172.21.15.107:0/1448261225 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468054930 con 0x56326124eb90 2024-09-19T04:28:42.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.895+0000 7f048165e740 1 -- 172.21.15.107:0/1448261225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 msgr2=0x563261258c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:42.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.895+0000 7f048165e740 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 0x563261258c10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0468004770 tx=0x7f046803f000 comp rx=0 tx=0).stop 2024-09-19T04:28:42.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.895+0000 7f048165e740 1 -- 172.21.15.107:0/1448261225 shutdown_connections 2024-09-19T04:28:42.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.895+0000 7f048165e740 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563261259d70 0x563261271d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.895+0000 7f048165e740 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x563261271820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.895+0000 7f048165e740 1 --2- 172.21.15.107:0/1448261225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56326124eb90 0x563261258c10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.895+0000 7f048165e740 1 -- 172.21.15.107:0/1448261225 >> 172.21.15.107:0/1448261225 conn(0x56326124b120 msgr2=0x563261258500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:42.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.895+0000 7f048165e740 1 -- 172.21.15.107:0/1448261225 shutdown_connections 2024-09-19T04:28:42.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.895+0000 7f048165e740 1 -- 172.21.15.107:0/1448261225 wait complete. 2024-09-19T04:28:42.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.896+0000 7f048165e740 1 -- start start 2024-09-19T04:28:42.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.896+0000 7f048165e740 1 Processor -- start 2024-09-19T04:28:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.896+0000 7f048165e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x56326125edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.896+0000 7f048165e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 0x56326125f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.896+0000 7f048165e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326125f830 0x5632612d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.896+0000 7f048165e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b0ef0 con 0x563261259d70 2024-09-19T04:28:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.896+0000 7f048165e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b0820 con 0x56326125f830 2024-09-19T04:28:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.896+0000 7f048165e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04781ada50 con 0x56326124f4b0 2024-09-19T04:28:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.896+0000 7f048165e740 10 client.0 _ll_register_callbacks cb 0x7f0464009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 0x56326125f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x56326125edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x56326125edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53560/0 (socket says 172.21.15.107:53560) 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477e00640 1 -- 172.21.15.107:0/3364552085 learned_addr learned my addr 172.21.15.107:0/3364552085 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 0x56326125f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51786/0 (socket says 172.21.15.107:51786) 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f047cc00640 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326125f830 0x5632612d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477e00640 1 -- 172.21.15.107:0/3364552085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326125f830 msgr2=0x5632612d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477e00640 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326125f830 0x5632612d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477e00640 1 -- 172.21.15.107:0/3364552085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 msgr2=0x56326125f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477e00640 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 0x56326125f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477e00640 1 -- 172.21.15.107:0/3364552085 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632612d4d40 con 0x56326124f4b0 2024-09-19T04:28:42.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.897+0000 7f0477400640 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 0x56326125f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:28:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.898+0000 7f0477e00640 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x56326125edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0468000c00 tx=0x7f0468031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4b7187f8e79ada0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:28:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.898+0000 7f0461600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:28:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.898+0000 7f0461600640 1 -- 172.21.15.107:0/3364552085 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680549c0 con 0x56326124f4b0 2024-09-19T04:28:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.898+0000 7f048165e740 1 -- 172.21.15.107:0/3364552085 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0468059070 con 0x56326124f4b0 2024-09-19T04:28:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.898+0000 7f0461600640 1 -- 172.21.15.107:0/3364552085 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f0468054b60 con 0x56326124f4b0 2024-09-19T04:28:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.898+0000 7f0461600640 1 -- 172.21.15.107:0/3364552085 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f04680035f0 con 0x56326124f4b0 2024-09-19T04:28:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.899+0000 7f048165e740 10 client.26752 fetch_fsmap learned FSMap version 46 2024-09-19T04:28:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.899+0000 7f048165e740 1 -- 172.21.15.107:0/3364552085 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5632612d5f10 con 0x56326124f4b0 2024-09-19T04:28:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.900+0000 7f0461600640 1 -- 172.21.15.107:0/3364552085 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0468059070 con 0x56326124f4b0 2024-09-19T04:28:42.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.900+0000 7f0461600640 1 -- 172.21.15.107:0/3364552085 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f046806e020 con 0x56326124f4b0 2024-09-19T04:28:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.900+0000 7f048165e740 10 client.26752 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T04:28:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.900+0000 7f048165e740 10 client.26752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:28:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.900+0000 7f048165e740 20 client.26752 populate_metadata read hostname 'smithi107' 2024-09-19T04:28:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.900+0000 7f048165e740 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5632612d6580 0x5632612d8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:28:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.900+0000 7f048165e740 4 client.26752 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:28:42.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.900+0000 7f048165e740 1 -- 172.21.15.107:0/3364552085 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5632611bce60 con 0x5632612d6580 2024-09-19T04:28:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.900+0000 7f0477400640 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5632612d6580 0x5632612d8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:28:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.901+0000 7f0461600640 10 client.26752 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.902+0000 7f0477400640 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5632612d6580 0x5632612d8970 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6a5d39007359659 server_cookie=9b75a368a2a2625c in_seq=0 out_seq=0 2024-09-19T04:28:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.902+0000 7f0461600640 1 -- 172.21.15.107:0/3364552085 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5632611bce60 con 0x5632612d6580 2024-09-19T04:28:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.902+0000 7f0461600640 10 client.26752 handle_command_reply: tid=0 2024-09-19T04:28:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.902+0000 7f048165e740 1 client.26752 shutdown 2024-09-19T04:28:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.902+0000 7f048165e740 1 -- 172.21.15.107:0/3364552085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 msgr2=0x56326125edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:42.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.902+0000 7f048165e740 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x56326125edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0468000c00 tx=0x7f0468031630 comp rx=0 tx=0).stop 2024-09-19T04:28:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.903+0000 7f048165e740 1 -- 172.21.15.107:0/3364552085 shutdown_connections 2024-09-19T04:28:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.903+0000 7f048165e740 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5632612d6580 0x5632612d8970 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.903+0000 7f048165e740 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56326125f830 0x5632612d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.903+0000 7f048165e740 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563261259d70 0x56326125f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.903+0000 7f048165e740 1 --2- 172.21.15.107:0/3364552085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56326124f4b0 0x56326125edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.903+0000 7f0461600640 0 client.26752 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:28:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.903+0000 7f048165e740 1 -- 172.21.15.107:0/3364552085 >> 172.21.15.107:0/3364552085 conn(0x56326124b120 msgr2=0x563261258500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.903+0000 7f048165e740 1 -- 172.21.15.107:0/3364552085 shutdown_connections 2024-09-19T04:28:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.903+0000 7f048165e740 1 -- 172.21.15.107:0/3364552085 wait complete. 2024-09-19T04:28:42.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.904+0000 7f048165e740 20 client.26752 trim_cache size 0 max 16384 2024-09-19T04:28:42.907 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:28:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:28:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:28:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "6127dce7-e347-49b8-9a9b-eb5f3a7c8260": { 2024-09-19T04:28:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:28:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "6127dce7-e347-49b8-9a9b-eb5f3a7c8260", 2024-09-19T04:28:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:28:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:28:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:28:42.908 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:28:42.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 -- 172.21.15.107:0/2205637853 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f045c0a33b0 msgr2=0x7f045c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:42.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f045c0a33b0 0x7f045c0a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f04680096c0 tx=0x7f04680046c0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 -- 172.21.15.107:0/2205637853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04781049a0 msgr2=0x7f047819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:28:42.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04781049a0 0x7f047819abd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f046400e9f0 tx=0x7f046400edf0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 -- 172.21.15.107:0/2205637853 shutdown_connections 2024-09-19T04:28:42.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f045c0a33b0 0x7f045c0a5870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0478109120 0x7f047819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04781049a0 0x7f047819abd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 --2- 172.21.15.107:0/2205637853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0478103ff0 0x7f047819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:28:42.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 -- 172.21.15.107:0/2205637853 >> 172.21.15.107:0/2205637853 conn(0x7f04780ffa90 msgr2=0x7f0478100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:28:42.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.905+0000 7f047ec00640 1 -- 172.21.15.107:0/2205637853 shutdown_connections 2024-09-19T04:28:42.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:28:42.906+0000 7f047ec00640 1 -- 172.21.15.107:0/2205637853 wait complete. 2024-09-19T04:28:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:42 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:28:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:42 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:43.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:42 smithi115 ceph-mon[35477]: pgmap v258: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1014 KiB/s wr, 89 op/s 2024-09-19T04:28:43.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:42 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:28:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:42 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:28:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:42 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:28:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:42 smithi145 ceph-mon[35096]: pgmap v258: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1014 KiB/s wr, 89 op/s 2024-09-19T04:28:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:42 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:28:43.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 260661 2.37 MB/sec execute 301 sec latency 2006.107 ms 2024-09-19T04:28:43.755 INFO:tasks.cephfs.filesystem:scrub status for tag:6127dce7-e347-49b8-9a9b-eb5f3a7c8260 - {'path': '/', 'tag': '6127dce7-e347-49b8-9a9b-eb5f3a7c8260', 'options': 'recursive,force'} 2024-09-19T04:28:44.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 261737 2.38 MB/sec execute 302 sec latency 2527.553 ms 2024-09-19T04:28:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:44 smithi115 ceph-mon[35477]: pgmap v259: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 693 KiB/s wr, 65 op/s 2024-09-19T04:28:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:44 smithi145 ceph-mon[35096]: pgmap v259: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 693 KiB/s wr, 65 op/s 2024-09-19T04:28:45.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:44 smithi107 ceph-mon[30659]: pgmap v259: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 693 KiB/s wr, 65 op/s 2024-09-19T04:28:45.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 263108 2.40 MB/sec execute 303 sec latency 17.374 ms 2024-09-19T04:28:46.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 264204 2.41 MB/sec execute 304 sec latency 165.437 ms 2024-09-19T04:28:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:46 smithi115 ceph-mon[35477]: pgmap v260: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-19T04:28:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:46 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:28:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:46 smithi145 ceph-mon[35096]: pgmap v260: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-19T04:28:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:46 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:28:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:46 smithi107 ceph-mon[30659]: pgmap v260: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-19T04:28:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:46 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:28:47.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 264204 2.40 MB/sec execute 305 sec latency 1165.543 ms 2024-09-19T04:28:47.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:47 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:28:47.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:47 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:28:47.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:47 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:28:47.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:47 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:28:48.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:47 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:28:48.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:47 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:28:48.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:47 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:28:48.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:47 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:28:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:47 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:28:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:47 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:28:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:47 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:28:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:47 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:28:48.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 264204 2.39 MB/sec execute 306 sec latency 2165.631 ms 2024-09-19T04:28:49.125 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:48 smithi145 ceph-mon[35096]: pgmap v261: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-19T04:28:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:48 smithi115 ceph-mon[35477]: pgmap v261: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-19T04:28:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:48 smithi107 ceph-mon[30659]: pgmap v261: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-19T04:28:49.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 265883 2.41 MB/sec execute 307 sec latency 2419.862 ms 2024-09-19T04:28:50.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 267168 2.43 MB/sec execute 308 sec latency 17.428 ms 2024-09-19T04:28:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:50 smithi115 ceph-mon[35477]: pgmap v262: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-19T04:28:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:50 smithi145 ceph-mon[35096]: pgmap v262: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-19T04:28:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:50 smithi107 ceph-mon[30659]: pgmap v262: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-19T04:28:51.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 267752 2.43 MB/sec execute 309 sec latency 454.936 ms 2024-09-19T04:28:52.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 267752 2.42 MB/sec execute 310 sec latency 1455.016 ms 2024-09-19T04:28:52.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:52 smithi107 ceph-mon[30659]: pgmap v263: 97 pgs: 97 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-09-19T04:28:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:52 smithi115 ceph-mon[35477]: pgmap v263: 97 pgs: 97 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-09-19T04:28:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:52 smithi145 ceph-mon[35096]: pgmap v263: 97 pgs: 97 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-09-19T04:28:53.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 267752 2.41 MB/sec execute 311 sec latency 2455.101 ms 2024-09-19T04:28:54.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 268681 2.42 MB/sec execute 312 sec latency 2972.459 ms 2024-09-19T04:28:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:54 smithi115 ceph-mon[35477]: pgmap v264: 97 pgs: 97 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 954 KiB/s wr, 107 op/s 2024-09-19T04:28:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:54 smithi145 ceph-mon[35096]: pgmap v264: 97 pgs: 97 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 954 KiB/s wr, 107 op/s 2024-09-19T04:28:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:54 smithi107 ceph-mon[30659]: pgmap v264: 97 pgs: 97 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 954 KiB/s wr, 107 op/s 2024-09-19T04:28:55.701 INFO:tasks.workunit.client.0.smithi107.stdout: 1 270166 2.44 MB/sec execute 313 sec latency 13.159 ms 2024-09-19T04:28:56.702 INFO:tasks.workunit.client.0.smithi107.stdout: 1 271294 2.45 MB/sec execute 314 sec latency 112.406 ms 2024-09-19T04:28:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:56 smithi115 ceph-mon[35477]: pgmap v265: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-09-19T04:28:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:56 smithi145 ceph-mon[35096]: pgmap v265: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-09-19T04:28:57.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:56 smithi107 ceph-mon[30659]: pgmap v265: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-09-19T04:28:57.702 INFO:tasks.workunit.client.0.smithi107.stdout: 1 271294 2.44 MB/sec execute 315 sec latency 1112.522 ms 2024-09-19T04:28:58.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 271294 2.44 MB/sec execute 316 sec latency 2112.653 ms 2024-09-19T04:28:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:28:58 smithi115 ceph-mon[35477]: pgmap v266: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 681 KiB/s wr, 98 op/s 2024-09-19T04:28:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:28:58 smithi145 ceph-mon[35096]: pgmap v266: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 681 KiB/s wr, 98 op/s 2024-09-19T04:28:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:28:58 smithi107 ceph-mon[30659]: pgmap v266: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 681 KiB/s wr, 98 op/s 2024-09-19T04:28:59.702 INFO:tasks.workunit.client.0.smithi107.stdout: 1 272362 2.45 MB/sec execute 317 sec latency 2633.979 ms 2024-09-19T04:29:00.702 INFO:tasks.workunit.client.0.smithi107.stdout: 1 273723 2.46 MB/sec execute 318 sec latency 13.941 ms 2024-09-19T04:29:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:00 smithi115 ceph-mon[35477]: pgmap v267: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 681 KiB/s wr, 98 op/s 2024-09-19T04:29:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:00 smithi145 ceph-mon[35096]: pgmap v267: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 681 KiB/s wr, 98 op/s 2024-09-19T04:29:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:00 smithi107 ceph-mon[30659]: pgmap v267: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 681 KiB/s wr, 98 op/s 2024-09-19T04:29:01.702 INFO:tasks.workunit.client.0.smithi107.stdout: 1 274657 2.47 MB/sec execute 319 sec latency 24.448 ms 2024-09-19T04:29:02.702 INFO:tasks.workunit.client.0.smithi107.stdout: 1 274843 2.47 MB/sec execute 320 sec latency 747.593 ms 2024-09-19T04:29:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:02 smithi115 ceph-mon[35477]: pgmap v268: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-19T04:29:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:02 smithi115 ceph-mon[35477]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:29:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:02 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:29:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:02 smithi145 ceph-mon[35096]: pgmap v268: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-19T04:29:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:02 smithi145 ceph-mon[35096]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:29:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:02 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:29:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:02 smithi107 ceph-mon[30659]: pgmap v268: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-19T04:29:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:02 smithi107 ceph-mon[30659]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:29:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:02 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:29:03.702 INFO:tasks.workunit.client.0.smithi107.stdout: 1 274843 2.46 MB/sec execute 321 sec latency 1747.760 ms 2024-09-19T04:29:04.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 275735 2.46 MB/sec execute 322 sec latency 2264.021 ms 2024-09-19T04:29:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:04 smithi115 ceph-mon[35477]: pgmap v270: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-09-19T04:29:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:04 smithi145 ceph-mon[35096]: pgmap v270: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-09-19T04:29:05.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:04 smithi107 ceph-mon[30659]: pgmap v270: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-09-19T04:29:05.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 277231 2.49 MB/sec execute 323 sec latency 10.154 ms 2024-09-19T04:29:06.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 278385 2.49 MB/sec execute 324 sec latency 293.263 ms 2024-09-19T04:29:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:06 smithi115 ceph-mon[35477]: pgmap v271: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-19T04:29:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:06 smithi115 ceph-mon[35477]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:29:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:06 smithi145 ceph-mon[35096]: pgmap v271: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-19T04:29:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:06 smithi145 ceph-mon[35096]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:29:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:06 smithi107 ceph-mon[30659]: pgmap v271: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-19T04:29:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:06 smithi107 ceph-mon[30659]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:29:07.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 278385 2.49 MB/sec execute 325 sec latency 1293.355 ms 2024-09-19T04:29:08.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 278385 2.48 MB/sec execute 326 sec latency 2293.458 ms 2024-09-19T04:29:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:08 smithi115 ceph-mon[35477]: pgmap v273: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-09-19T04:29:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:08 smithi145 ceph-mon[35096]: pgmap v273: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-09-19T04:29:09.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:08 smithi107 ceph-mon[30659]: pgmap v273: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-09-19T04:29:09.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 280069 2.50 MB/sec execute 327 sec latency 2545.679 ms 2024-09-19T04:29:10.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 281292 2.51 MB/sec execute 328 sec latency 11.101 ms 2024-09-19T04:29:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:10 smithi115 ceph-mon[35477]: pgmap v274: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 736 KiB/s wr, 126 op/s 2024-09-19T04:29:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:10 smithi145 ceph-mon[35096]: pgmap v274: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 736 KiB/s wr, 126 op/s 2024-09-19T04:29:11.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:10 smithi107 ceph-mon[30659]: pgmap v274: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 736 KiB/s wr, 126 op/s 2024-09-19T04:29:11.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 281934 2.51 MB/sec execute 329 sec latency 605.570 ms 2024-09-19T04:29:12.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 281934 2.51 MB/sec execute 330 sec latency 1605.675 ms 2024-09-19T04:29:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:12 smithi115 ceph-mon[35477]: pgmap v275: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-09-19T04:29:13.200 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:12 smithi107 ceph-mon[30659]: pgmap v275: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-09-19T04:29:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:12 smithi145 ceph-mon[35096]: pgmap v275: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-09-19T04:29:13.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 281934 2.50 MB/sec execute 331 sec latency 2605.725 ms 2024-09-19T04:29:13.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:29:14.138 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:14.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 282992 2.51 MB/sec execute 332 sec latency 3126.606 ms 2024-09-19T04:29:14.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.889+0000 7fbe0f200640 1 -- 172.21.15.107:0/268126090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe10102360 msgr2=0x7fbe1010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:14.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.889+0000 7fbe0f200640 1 --2- 172.21.15.107:0/268126090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe10102360 0x7fbe1010e890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009c10 tx=0x7fbe0802a6f0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.889+0000 7fbe0f200640 1 -- 172.21.15.107:0/268126090 shutdown_connections 2024-09-19T04:29:14.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.889+0000 7fbe0f200640 1 --2- 172.21.15.107:0/268126090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe10102360 0x7fbe1010e890 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.889+0000 7fbe0f200640 1 --2- 172.21.15.107:0/268126090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe101019c0 0x7fbe10101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.889+0000 7fbe0f200640 1 --2- 172.21.15.107:0/268126090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe101079c0 0x7fbe10107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.889+0000 7fbe0f200640 1 -- 172.21.15.107:0/268126090 >> 172.21.15.107:0/268126090 conn(0x7fbe100fd740 msgr2=0x7fbe100ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:14.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.889+0000 7fbe0f200640 1 -- 172.21.15.107:0/268126090 shutdown_connections 2024-09-19T04:29:14.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.889+0000 7fbe0f200640 1 -- 172.21.15.107:0/268126090 wait complete. 2024-09-19T04:29:14.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.891+0000 7fbe0f200640 1 Processor -- start 2024-09-19T04:29:14.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.891+0000 7fbe0f200640 1 -- start start 2024-09-19T04:29:14.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.891+0000 7fbe0f200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe101019c0 0x7fbe101982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.891+0000 7fbe0f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe10102360 0x7fbe101987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.891+0000 7fbe0f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe101079c0 0x7fbe1019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.891+0000 7fbe0f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10111dc0 con 0x7fbe101079c0 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.891+0000 7fbe0f200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10111c40 con 0x7fbe101019c0 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.891+0000 7fbe0f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10111f40 con 0x7fbe10102360 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe101019c0 0x7fbe101982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe101019c0 0x7fbe101982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39008/0 (socket says 172.21.15.107:39008) 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0de00640 1 -- 172.21.15.107:0/1534719407 learned_addr learned my addr 172.21.15.107:0/1534719407 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0d400640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe10102360 0x7fbe101987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0e800640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe101079c0 0x7fbe1019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0de00640 1 -- 172.21.15.107:0/1534719407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe10102360 msgr2=0x7fbe101987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0de00640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe10102360 0x7fbe101987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0de00640 1 -- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe101079c0 msgr2=0x7fbe1019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:14.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0de00640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe101079c0 0x7fbe1019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0de00640 1 -- 172.21.15.107:0/1534719407 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1019d250 con 0x7fbe101019c0 2024-09-19T04:29:14.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0e800640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe101079c0 0x7fbe1019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:14.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.892+0000 7fbe0de00640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe101019c0 0x7fbe101982a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000ea60 tx=0x7fbe0000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:14.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.893+0000 7fbe06a00640 1 -- 172.21.15.107:0/1534719407 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000cde0 con 0x7fbe101019c0 2024-09-19T04:29:14.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.893+0000 7fbe06a00640 1 -- 172.21.15.107:0/1534719407 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe000035f0 con 0x7fbe101019c0 2024-09-19T04:29:14.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.893+0000 7fbe06a00640 1 -- 172.21.15.107:0/1534719407 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00010670 con 0x7fbe101019c0 2024-09-19T04:29:14.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.893+0000 7fbe0f200640 1 -- 172.21.15.107:0/1534719407 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1019d540 con 0x7fbe101019c0 2024-09-19T04:29:14.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.893+0000 7fbe0f200640 1 -- 172.21.15.107:0/1534719407 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe101a4e20 con 0x7fbe101019c0 2024-09-19T04:29:14.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.896+0000 7fbe06a00640 1 -- 172.21.15.107:0/1534719407 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fbe000037a0 con 0x7fbe101019c0 2024-09-19T04:29:14.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.896+0000 7fbe17855740 1 Processor -- start 2024-09-19T04:29:14.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe17855740 1 -- start start 2024-09-19T04:29:14.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe17855740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5b9f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe17855740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 0x5607f5ba99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe17855740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5ba9f30 0x5607f5bac320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe17855740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5607f5b053e0 con 0x5607f5ba9f30 2024-09-19T04:29:14.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe17855740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5607f5aacd80 con 0x5607f5b9ed50 2024-09-19T04:29:14.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe17855740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5607f5b013a0 con 0x5607f5b9f670 2024-09-19T04:29:14.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe0e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5b9f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe0e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5b9f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39022/0 (socket says 172.21.15.107:39022) 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe0e800640 1 -- 172.21.15.107:0/559753960 learned_addr learned my addr 172.21.15.107:0/559753960 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe0de00640 1 --2- 172.21.15.107:0/559753960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 0x5607f5ba99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.897+0000 7fbe0d400640 1 --2- 172.21.15.107:0/559753960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5ba9f30 0x5607f5bac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.898+0000 7fbe0e800640 1 -- 172.21.15.107:0/559753960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 msgr2=0x5607f5ba99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.898+0000 7fbe0e800640 1 --2- 172.21.15.107:0/559753960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 0x5607f5ba99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.898+0000 7fbe0e800640 1 -- 172.21.15.107:0/559753960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5ba9f30 msgr2=0x5607f5bac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.898+0000 7fbe0e800640 1 --2- 172.21.15.107:0/559753960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5ba9f30 0x5607f5bac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.898+0000 7fbe0e800640 1 -- 172.21.15.107:0/559753960 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f5acbae0 con 0x5607f5b9ed50 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.898+0000 7fbe0e800640 1 --2- 172.21.15.107:0/559753960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5b9f130 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009be0 tx=0x7fbe08030b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26cbd6ed85e9194 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.898+0000 7fbe06a00640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbdf40a3480 0x7fbdf40a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.898+0000 7fbdfb400640 1 -- 172.21.15.107:0/559753960 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08040030 con 0x5607f5b9ed50 2024-09-19T04:29:14.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.898+0000 7fbdfb400640 1 -- 172.21.15.107:0/559753960 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe08030ca0 con 0x5607f5b9ed50 2024-09-19T04:29:14.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.898+0000 7fbdfb400640 1 -- 172.21.15.107:0/559753960 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08036040 con 0x5607f5b9ed50 2024-09-19T04:29:14.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.899+0000 7fbe06a00640 1 -- 172.21.15.107:0/1534719407 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7fbe000cffc0 con 0x7fbe101019c0 2024-09-19T04:29:14.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.899+0000 7fbe0de00640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbdf40a3480 0x7fbdf40a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.899+0000 7fbe17855740 1 -- 172.21.15.107:0/559753960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 msgr2=0x5607f5b9f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:14.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.899+0000 7fbe17855740 1 --2- 172.21.15.107:0/559753960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5b9f130 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009be0 tx=0x7fbe08030b30 comp rx=0 tx=0).stop 2024-09-19T04:29:14.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.899+0000 7fbe0de00640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbdf40a3480 0x7fbdf40a5940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbe000cd7e0 tx=0x7fbe000d3b10 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:14.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.899+0000 7fbe17855740 1 -- 172.21.15.107:0/559753960 shutdown_connections 2024-09-19T04:29:14.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.899+0000 7fbe17855740 1 --2- 172.21.15.107:0/559753960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5ba9f30 0x5607f5bac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.899+0000 7fbe17855740 1 --2- 172.21.15.107:0/559753960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 0x5607f5ba99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.899+0000 7fbe17855740 1 --2- 172.21.15.107:0/559753960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5b9f130 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.899+0000 7fbe17855740 1 -- 172.21.15.107:0/559753960 >> 172.21.15.107:0/559753960 conn(0x5607f5b9b2e0 msgr2=0x5607f5b9b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:14.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.900+0000 7fbe17855740 1 -- 172.21.15.107:0/559753960 shutdown_connections 2024-09-19T04:29:14.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.900+0000 7fbe17855740 1 -- 172.21.15.107:0/559753960 wait complete. 2024-09-19T04:29:14.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.900+0000 7fbe17855740 1 -- start start 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.900+0000 7fbe17855740 1 Processor -- start 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe17855740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9ed50 0x5607f5ba6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe17855740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9f670 0x5607f5ba73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe17855740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5ba9f30 0x5607f5ba78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe17855740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5607f5bb0fb0 con 0x5607f5ba9f30 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe17855740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5607f5ac00f0 con 0x5607f5b9f670 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe17855740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5607f5bb1130 con 0x5607f5b9ed50 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe17855740 10 client.0 _ll_register_callbacks cb 0x5607f5ab19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe0d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9f670 0x5607f5ba73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe0e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9ed50 0x5607f5ba6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe0d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9f670 0x5607f5ba73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39034/0 (socket says 172.21.15.107:39034) 2024-09-19T04:29:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe0d400640 1 -- 172.21.15.107:0/1509935382 learned_addr learned my addr 172.21.15.107:0/1509935382 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:14.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe0e800640 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5ba9f30 0x5607f5ba78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe0d400640 1 -- 172.21.15.107:0/1509935382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9ed50 msgr2=0x5607f5ba6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:14.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe0d400640 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9ed50 0x5607f5ba6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe0d400640 1 -- 172.21.15.107:0/1509935382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5ba9f30 msgr2=0x5607f5ba78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:14.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe0d400640 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5ba9f30 0x5607f5ba78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.901+0000 7fbe0d400640 1 -- 172.21.15.107:0/1509935382 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f58763c0 con 0x5607f5b9f670 2024-09-19T04:29:14.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.902+0000 7fbe0d400640 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9f670 0x5607f5ba73b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0116d0 tx=0x7fbdfc011b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a786986a4d0adf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:14.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.902+0000 7fbdefe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:29:14.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.902+0000 7fbdefe00640 1 -- 172.21.15.107:0/1509935382 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc01e060 con 0x5607f5b9f670 2024-09-19T04:29:14.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.903+0000 7fbdefe00640 1 -- 172.21.15.107:0/1509935382 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7fbdfc00fbf0 con 0x5607f5b9f670 2024-09-19T04:29:14.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.903+0000 7fbe17855740 1 -- 172.21.15.107:0/1509935382 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe08030ca0 con 0x5607f5b9f670 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.903+0000 7fbdefe00640 1 -- 172.21.15.107:0/1509935382 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbdfc00fdf0 con 0x5607f5b9f670 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.903+0000 7fbe17855740 10 client.26795 fetch_fsmap learned FSMap version 47 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.903+0000 7fbe17855740 1 -- 172.21.15.107:0/1509935382 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5607f5bbcd10 con 0x5607f5b9f670 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.903+0000 7fbdefe00640 1 -- 172.21.15.107:0/1509935382 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe08030ca0 con 0x5607f5b9f670 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.904+0000 7fbdefe00640 1 -- 172.21.15.107:0/1509935382 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fbdfc019070 con 0x5607f5b9f670 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.904+0000 7fbe17855740 10 client.26795 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.904+0000 7fbe17855740 10 client.26795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.904+0000 7fbe17855740 20 client.26795 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.904+0000 7fbe17855740 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5607f5bbe270 0x5607f5bc0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.904+0000 7fbe17855740 4 client.26795 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:14.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.904+0000 7fbe17855740 1 -- 172.21.15.107:0/1509935382 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607f5b12950 con 0x5607f5bbe270 2024-09-19T04:29:14.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.904+0000 7fbe0e800640 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5607f5bbe270 0x5607f5bc0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.905+0000 7fbe0e800640 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5607f5bbe270 0x5607f5bc0690 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88826ffcb504ec0e server_cookie=f626ed4c92b28c3d in_seq=0 out_seq=0 2024-09-19T04:29:14.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.905+0000 7fbdefe00640 10 client.26795 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:14.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.906+0000 7fbdefe00640 1 -- 172.21.15.107:0/1509935382 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607f5b12950 con 0x5607f5bbe270 2024-09-19T04:29:14.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.906+0000 7fbdefe00640 10 client.26795 handle_command_reply: tid=0 2024-09-19T04:29:14.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.906+0000 7fbe17855740 1 client.26795 shutdown 2024-09-19T04:29:14.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.907+0000 7fbe17855740 1 -- 172.21.15.107:0/1509935382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9f670 msgr2=0x5607f5ba73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:14.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.907+0000 7fbe17855740 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9f670 0x5607f5ba73b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0116d0 tx=0x7fbdfc011b90 comp rx=0 tx=0).stop 2024-09-19T04:29:14.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.907+0000 7fbe17855740 1 -- 172.21.15.107:0/1509935382 shutdown_connections 2024-09-19T04:29:14.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.907+0000 7fbe17855740 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5607f5bbe270 0x5607f5bc0690 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.907+0000 7fbe17855740 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5ba9f30 0x5607f5ba78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.907+0000 7fbe17855740 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9f670 0x5607f5ba73b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.907+0000 7fbe17855740 1 --2- 172.21.15.107:0/1509935382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9ed50 0x5607f5ba6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.907+0000 7fbe17855740 1 -- 172.21.15.107:0/1509935382 >> 172.21.15.107:0/1509935382 conn(0x5607f5b9b2e0 msgr2=0x5607f5ba0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:14.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.907+0000 7fbdefe00640 0 client.26795 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:14.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.907+0000 7fbe17855740 1 -- 172.21.15.107:0/1509935382 shutdown_connections 2024-09-19T04:29:14.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.908+0000 7fbe17855740 1 -- 172.21.15.107:0/1509935382 wait complete. 2024-09-19T04:29:14.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.908+0000 7fbe17855740 20 client.26795 trim_cache size 0 max 16384 2024-09-19T04:29:14.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.934+0000 7fbe17855740 1 Processor -- start 2024-09-19T04:29:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.934+0000 7fbe17855740 1 -- start start 2024-09-19T04:29:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.934+0000 7fbe17855740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5ba7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.934+0000 7fbe17855740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9f670 0x5607f5b9a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.934+0000 7fbe17855740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5ba9f30 0x5607f5bc5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.934+0000 7fbe17855740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101ab5c0 con 0x5607f5b9f670 2024-09-19T04:29:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.934+0000 7fbe17855740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101ad130 con 0x5607f5b9ed50 2024-09-19T04:29:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.934+0000 7fbe17855740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101abcc0 con 0x5607f5ba9f30 2024-09-19T04:29:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5ba7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9f670 0x5607f5b9a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5ba7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39040/0 (socket says 172.21.15.107:39040) 2024-09-19T04:29:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0e800640 1 -- 172.21.15.107:0/1069730395 learned_addr learned my addr 172.21.15.107:0/1069730395 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0e800640 1 --2- 172.21.15.107:0/1069730395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5ba9f30 0x5607f5bc5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0d400640 1 -- 172.21.15.107:0/1069730395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5ba9f30 msgr2=0x5607f5bc5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:14.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0d400640 1 --2- 172.21.15.107:0/1069730395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5ba9f30 0x5607f5bc5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0d400640 1 -- 172.21.15.107:0/1069730395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 msgr2=0x5607f5ba7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0d400640 1 --2- 172.21.15.107:0/1069730395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5ba7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0d400640 1 -- 172.21.15.107:0/1069730395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f5b9fef0 con 0x5607f5b9f670 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0e800640 1 --2- 172.21.15.107:0/1069730395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5ba7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.935+0000 7fbe0d400640 1 --2- 172.21.15.107:0/1069730395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9f670 0x5607f5b9a290 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009530 tx=0x7fbdfc00fd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277fb937c6b39c5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.936+0000 7fbdfaa00640 1 -- 172.21.15.107:0/1069730395 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc01a560 con 0x5607f5b9f670 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.936+0000 7fbdfaa00640 1 -- 172.21.15.107:0/1069730395 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdfc01a700 con 0x5607f5b9f670 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.936+0000 7fbdfaa00640 1 -- 172.21.15.107:0/1069730395 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc0038e0 con 0x5607f5b9f670 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.937+0000 7fbe17855740 1 -- 172.21.15.107:0/1069730395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9f670 msgr2=0x5607f5b9a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.937+0000 7fbe17855740 1 --2- 172.21.15.107:0/1069730395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9f670 0x5607f5b9a290 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009530 tx=0x7fbdfc00fd80 comp rx=0 tx=0).stop 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.937+0000 7fbe17855740 1 -- 172.21.15.107:0/1069730395 shutdown_connections 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.937+0000 7fbe17855740 1 --2- 172.21.15.107:0/1069730395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5ba9f30 0x5607f5bc5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.937+0000 7fbe17855740 1 --2- 172.21.15.107:0/1069730395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9f670 0x5607f5b9a290 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.937+0000 7fbe17855740 1 --2- 172.21.15.107:0/1069730395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5b9ed50 0x5607f5ba7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.937+0000 7fbe17855740 1 -- 172.21.15.107:0/1069730395 >> 172.21.15.107:0/1069730395 conn(0x5607f5b9b2e0 msgr2=0x5607f5b99b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:14.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.937+0000 7fbe17855740 1 -- 172.21.15.107:0/1069730395 shutdown_connections 2024-09-19T04:29:14.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.937+0000 7fbe17855740 1 -- 172.21.15.107:0/1069730395 wait complete. 2024-09-19T04:29:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.937+0000 7fbe17855740 1 -- start start 2024-09-19T04:29:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe17855740 1 Processor -- start 2024-09-19T04:29:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe17855740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9ed50 0x5607f5bb6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe17855740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 0x5607f5bb68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe17855740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5ba9f30 0x5607f5bb3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe17855740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101aca60 con 0x5607f5b9ed50 2024-09-19T04:29:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe17855740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101ac390 con 0x5607f5ba9f30 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe17855740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101a95c0 con 0x5607f5b9f670 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe17855740 10 client.0 _ll_register_callbacks cb 0x7fbdfc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe0d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 0x5607f5bb68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe0d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 0x5607f5bb68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:34860/0 (socket says 172.21.15.107:34860) 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe0d400640 1 -- 172.21.15.107:0/2667525234 learned_addr learned my addr 172.21.15.107:0/2667525234 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.938+0000 7fbe0e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9ed50 0x5607f5bb6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.939+0000 7fbe0e800640 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5ba9f30 0x5607f5bb3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.939+0000 7fbe0d400640 1 -- 172.21.15.107:0/2667525234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5ba9f30 msgr2=0x5607f5bb3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.939+0000 7fbe0d400640 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5ba9f30 0x5607f5bb3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.939+0000 7fbe0d400640 1 -- 172.21.15.107:0/2667525234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9ed50 msgr2=0x5607f5bb6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.939+0000 7fbe0d400640 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9ed50 0x5607f5bb6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.939+0000 7fbe0d400640 1 -- 172.21.15.107:0/2667525234 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f5bb6f30 con 0x5607f5b9f670 2024-09-19T04:29:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.939+0000 7fbe0d400640 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 0x5607f5bb68c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc01a410 tx=0x7fbdfc004230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed7bdf2109002846 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.939+0000 7fbdefe00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:29:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.940+0000 7fbdefe00640 1 -- 172.21.15.107:0/2667525234 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc01a510 con 0x5607f5b9f670 2024-09-19T04:29:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.940+0000 7fbdefe00640 1 -- 172.21.15.107:0/2667525234 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7fbdfc01a8b0 con 0x5607f5b9f670 2024-09-19T04:29:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.940+0000 7fbe17855740 1 -- 172.21.15.107:0/2667525234 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdfc01a700 con 0x5607f5b9f670 2024-09-19T04:29:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.940+0000 7fbdefe00640 1 -- 172.21.15.107:0/2667525234 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbdfc01aab0 con 0x5607f5b9f670 2024-09-19T04:29:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.940+0000 7fbe17855740 10 client.26770 fetch_fsmap learned FSMap version 47 2024-09-19T04:29:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.940+0000 7fbe17855740 1 -- 172.21.15.107:0/2667525234 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5607f5c294a0 con 0x5607f5b9f670 2024-09-19T04:29:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.940+0000 7fbdefe00640 1 -- 172.21.15.107:0/2667525234 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdfc01a700 con 0x5607f5b9f670 2024-09-19T04:29:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.941+0000 7fbdefe00640 1 -- 172.21.15.107:0/2667525234 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fbdfc019070 con 0x5607f5b9f670 2024-09-19T04:29:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.941+0000 7fbe17855740 10 client.26770 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T04:29:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.941+0000 7fbe17855740 10 client.26770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.941+0000 7fbe17855740 20 client.26770 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.941+0000 7fbe17855740 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5607f5c29b50 0x5607f5c2bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.941+0000 7fbe17855740 4 client.26770 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:29:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.941+0000 7fbe17855740 1 -- 172.21.15.107:0/2667525234 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607f5b0ce60 con 0x5607f5c29b50 2024-09-19T04:29:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.941+0000 7fbe0e800640 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5607f5c29b50 0x5607f5c2bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbe0e800640 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5607f5c29b50 0x5607f5c2bf70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=596e4d45cb6e8a91 server_cookie=ee0eac86528a78b4 in_seq=0 out_seq=0 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbdefe00640 10 client.26770 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbdefe00640 1 -- 172.21.15.107:0/2667525234 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5607f5b0ce60 con 0x5607f5c29b50 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbdefe00640 10 client.26770 handle_command_reply: tid=0 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbe17855740 1 client.26770 shutdown 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbe17855740 1 -- 172.21.15.107:0/2667525234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 msgr2=0x5607f5bb68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbe17855740 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 0x5607f5bb68c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc01a410 tx=0x7fbdfc004230 comp rx=0 tx=0).stop 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbe17855740 1 -- 172.21.15.107:0/2667525234 shutdown_connections 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbe17855740 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5607f5c29b50 0x5607f5c2bf70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbe17855740 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5607f5ba9f30 0x5607f5bb3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbe17855740 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5607f5b9f670 0x5607f5bb68c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbe17855740 1 --2- 172.21.15.107:0/2667525234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607f5b9ed50 0x5607f5bb6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbe17855740 1 -- 172.21.15.107:0/2667525234 >> 172.21.15.107:0/2667525234 conn(0x5607f5b9b2e0 msgr2=0x5607f5ba0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.942+0000 7fbdefe00640 0 client.26770 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.943+0000 7fbe17855740 1 -- 172.21.15.107:0/2667525234 shutdown_connections 2024-09-19T04:29:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.943+0000 7fbe17855740 1 -- 172.21.15.107:0/2667525234 wait complete. 2024-09-19T04:29:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.943+0000 7fbe17855740 20 client.26770 trim_cache size 0 max 16384 2024-09-19T04:29:14.948 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:29:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:29:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:29:14.948 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:29:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 -- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbdf40a3480 msgr2=0x7fbdf40a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbdf40a3480 0x7fbdf40a5940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbe000cd7e0 tx=0x7fbe000d3b10 comp rx=0 tx=0).stop 2024-09-19T04:29:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 -- 172.21.15.107:0/1534719407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe101019c0 msgr2=0x7fbe101982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe101019c0 0x7fbe101982a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000ea60 tx=0x7fbe0000ef20 comp rx=0 tx=0).stop 2024-09-19T04:29:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 -- 172.21.15.107:0/1534719407 shutdown_connections 2024-09-19T04:29:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbdf40a3480 0x7fbdf40a5940 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe101079c0 0x7fbe1019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe10102360 0x7fbe101987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 --2- 172.21.15.107:0/1534719407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe101019c0 0x7fbe101982a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 -- 172.21.15.107:0/1534719407 >> 172.21.15.107:0/1534719407 conn(0x7fbe100fd740 msgr2=0x7fbe100ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 -- 172.21.15.107:0/1534719407 shutdown_connections 2024-09-19T04:29:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:14.944+0000 7fbe0f200640 1 -- 172.21.15.107:0/1534719407 wait complete. 2024-09-19T04:29:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:14 smithi115 ceph-mon[35477]: pgmap v276: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-19T04:29:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:14 smithi145 ceph-mon[35096]: pgmap v276: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-19T04:29:15.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:14 smithi107 ceph-mon[30659]: pgmap v276: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-19T04:29:15.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 284356 2.53 MB/sec execute 333 sec latency 12.552 ms 2024-09-19T04:29:15.855 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:29:15.855 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:29:16.252 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:16.703 INFO:tasks.workunit.client.0.smithi107.stdout: 1 285476 2.53 MB/sec execute 334 sec latency 60.666 ms 2024-09-19T04:29:16.954 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:16 smithi107 ceph-mon[30659]: pgmap v277: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-19T04:29:16.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.967+0000 7f6ccf600640 1 -- 172.21.15.107:0/2917942433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cc81134e0 msgr2=0x7f6cc81158d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.967+0000 7f6ccf600640 1 --2- 172.21.15.107:0/2917942433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cc81134e0 0x7f6cc81158d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000adb0 tx=0x7f6cc002b7a0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.967+0000 7f6ccf600640 1 -- 172.21.15.107:0/2917942433 shutdown_connections 2024-09-19T04:29:16.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.967+0000 7f6ccf600640 1 --2- 172.21.15.107:0/2917942433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cc81134e0 0x7f6cc81158d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.967+0000 7f6ccf600640 1 --2- 172.21.15.107:0/2917942433 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc8109d70 0x7f6cc810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.967+0000 7f6ccf600640 1 --2- 172.21.15.107:0/2917942433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cc8069ab0 0x7f6cc810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.967+0000 7f6ccf600640 1 -- 172.21.15.107:0/2917942433 >> 172.21.15.107:0/2917942433 conn(0x7f6cc806e2d0 msgr2=0x7f6cc806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:16.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.967+0000 7f6ccf600640 1 -- 172.21.15.107:0/2917942433 shutdown_connections 2024-09-19T04:29:16.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.967+0000 7f6ccf600640 1 -- 172.21.15.107:0/2917942433 wait complete. 2024-09-19T04:29:16.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.968+0000 7f6ccf600640 1 Processor -- start 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.968+0000 7f6ccf600640 1 -- start start 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6ccf600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc8069ab0 0x7f6cc80708c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6ccf600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cc8109d70 0x7f6cc8070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6ccf600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cc81134e0 0x7f6cc8071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6ccf600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc81185d0 con 0x7f6cc8109d70 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6ccf600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc8118450 con 0x7f6cc81134e0 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6ccf600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc8118750 con 0x7f6cc8069ab0 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6cc7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cc8109d70 0x7f6cc8070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6cc7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cc8109d70 0x7f6cc8070e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46132/0 (socket says 172.21.15.107:46132) 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6cc7e00640 1 -- 172.21.15.107:0/489160307 learned_addr learned my addr 172.21.15.107:0/489160307 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6cccc00640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc8069ab0 0x7f6cc80708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6ccd600640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cc81134e0 0x7f6cc8071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6cc7e00640 1 -- 172.21.15.107:0/489160307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc8069ab0 msgr2=0x7f6cc80708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6cc7e00640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc8069ab0 0x7f6cc80708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6cc7e00640 1 -- 172.21.15.107:0/489160307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cc81134e0 msgr2=0x7f6cc8071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6cc7e00640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cc81134e0 0x7f6cc8071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6cc7e00640 1 -- 172.21.15.107:0/489160307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc81af140 con 0x7f6cc8109d70 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6cccc00640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc8069ab0 0x7f6cc80708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.969+0000 7f6ccd600640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cc81134e0 0x7f6cc8071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.970+0000 7f6cc7e00640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cc8109d70 0x7f6cc8070e00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400e9f0 tx=0x7f6cb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.970+0000 7f6cc5600640 1 -- 172.21.15.107:0/489160307 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400cd80 con 0x7f6cc8109d70 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.970+0000 7f6ccf600640 1 -- 172.21.15.107:0/489160307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc81af430 con 0x7f6cc8109d70 2024-09-19T04:29:16.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.970+0000 7f6cc5600640 1 -- 172.21.15.107:0/489160307 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cb4010910 con 0x7f6cc8109d70 2024-09-19T04:29:16.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.970+0000 7f6cc5600640 1 -- 172.21.15.107:0/489160307 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4018910 con 0x7f6cc8109d70 2024-09-19T04:29:16.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.970+0000 7f6ccf600640 1 -- 172.21.15.107:0/489160307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc81af970 con 0x7f6cc8109d70 2024-09-19T04:29:16.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.971+0000 7f6ccf600640 1 -- 172.21.15.107:0/489160307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc810e350 con 0x7f6cc8109d70 2024-09-19T04:29:16.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.973+0000 7f6cc5600640 1 -- 172.21.15.107:0/489160307 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f6cb4010ac0 con 0x7f6cc8109d70 2024-09-19T04:29:16.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.974+0000 7f6cc5600640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6cb00a33b0 0x7f6cb00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:16.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.975+0000 7f6cc5600640 1 -- 172.21.15.107:0/489160307 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f6cb40cffe0 con 0x7f6cc8109d70 2024-09-19T04:29:16.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.975+0000 7f6cccc00640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6cb00a33b0 0x7f6cb00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:16.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.975+0000 7f6cccc00640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6cb00a33b0 0x7f6cb00a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009fc0 tx=0x7f6cbc005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:16.978+0000 7f6cc5600640 1 -- 172.21.15.107:0/489160307 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6cb409c390 con 0x7f6cc8109d70 2024-09-19T04:29:17.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.189+0000 7f6ccf600640 1 -- 172.21.15.107:0/489160307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cc81afc20 con 0x7f6cc8109d70 2024-09-19T04:29:17.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.190+0000 7f6cc5600640 1 -- 172.21.15.107:0/489160307 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11993 (secure 0 0 0) 0x7f6cb40a0690 con 0x7f6cc8109d70 2024-09-19T04:29:17.192 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:29:17.192 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":47,"btime":"2024-09-19T04:29:01:865340+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:29:01.466436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:29:17.193 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 47 2024-09-19T04:29:17.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.192+0000 7f6ccf600640 1 -- 172.21.15.107:0/489160307 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6cb00a33b0 msgr2=0x7f6cb00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:17.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.192+0000 7f6ccf600640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6cb00a33b0 0x7f6cb00a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009fc0 tx=0x7f6cbc005f90 comp rx=0 tx=0).stop 2024-09-19T04:29:17.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.192+0000 7f6ccf600640 1 -- 172.21.15.107:0/489160307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cc8109d70 msgr2=0x7f6cc8070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:17.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.192+0000 7f6ccf600640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cc8109d70 0x7f6cc8070e00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400e9f0 tx=0x7f6cb400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:29:17.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.193+0000 7f6ccf600640 1 -- 172.21.15.107:0/489160307 shutdown_connections 2024-09-19T04:29:17.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.193+0000 7f6ccf600640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6cb00a33b0 0x7f6cb00a5870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:17.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.193+0000 7f6ccf600640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cc81134e0 0x7f6cc8071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:17.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.193+0000 7f6ccf600640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cc8109d70 0x7f6cc8070e00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:17.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.193+0000 7f6ccf600640 1 --2- 172.21.15.107:0/489160307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc8069ab0 0x7f6cc80708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:17.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.193+0000 7f6ccf600640 1 -- 172.21.15.107:0/489160307 >> 172.21.15.107:0/489160307 conn(0x7f6cc806e2d0 msgr2=0x7f6cc810c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:17.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.193+0000 7f6ccf600640 1 -- 172.21.15.107:0/489160307 shutdown_connections 2024-09-19T04:29:17.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:17.193+0000 7f6ccf600640 1 -- 172.21.15.107:0/489160307 wait complete. 2024-09-19T04:29:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:16 smithi115 ceph-mon[35477]: pgmap v277: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-19T04:29:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:16 smithi145 ceph-mon[35096]: pgmap v277: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-19T04:29:17.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 285476 2.53 MB/sec execute 335 sec latency 1049.905 ms 2024-09-19T04:29:18.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:29:18.146 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:17 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/489160307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:29:18.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:17 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/489160307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:29:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:17 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/489160307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:29:18.499 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:18.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 285476 2.52 MB/sec execute 336 sec latency 2049.965 ms 2024-09-19T04:29:18.946 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:18 smithi107 ceph-mon[30659]: pgmap v278: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-09-19T04:29:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:18 smithi115 ceph-mon[35477]: pgmap v278: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-09-19T04:29:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.172+0000 7f38b0200640 1 -- 172.21.15.107:0/3167018695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a8106f10 msgr2=0x7f38a810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.172+0000 7f38b0200640 1 --2- 172.21.15.107:0/3167018695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a8106f10 0x7f38a810ea90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009ca0 tx=0x7f38a002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.172+0000 7f38b0200640 1 -- 172.21.15.107:0/3167018695 shutdown_connections 2024-09-19T04:29:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.172+0000 7f38b0200640 1 --2- 172.21.15.107:0/3167018695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a8106f10 0x7f38a810ea90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.172+0000 7f38b0200640 1 --2- 172.21.15.107:0/3167018695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38a8102790 0x7f38a81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.172+0000 7f38b0200640 1 --2- 172.21.15.107:0/3167018695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38a8101de0 0x7f38a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.172+0000 7f38b0200640 1 -- 172.21.15.107:0/3167018695 >> 172.21.15.107:0/3167018695 conn(0x7f38a80fd8c0 msgr2=0x7f38a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.172+0000 7f38b0200640 1 -- 172.21.15.107:0/3167018695 shutdown_connections 2024-09-19T04:29:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.172+0000 7f38b0200640 1 -- 172.21.15.107:0/3167018695 wait complete. 2024-09-19T04:29:19.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.173+0000 7f38b0200640 1 Processor -- start 2024-09-19T04:29:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.173+0000 7f38b0200640 1 -- start start 2024-09-19T04:29:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.173+0000 7f38b0200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38a8101de0 0x7f38a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.173+0000 7f38b0200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38a8102790 0x7f38a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.173+0000 7f38b0200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a8106f10 0x7f38a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.173+0000 7f38b0200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8111fc0 con 0x7f38a8106f10 2024-09-19T04:29:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.173+0000 7f38b0200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8111e40 con 0x7f38a8101de0 2024-09-19T04:29:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.173+0000 7f38b0200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8112140 con 0x7f38a8102790 2024-09-19T04:29:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ad800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38a8101de0 0x7f38a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ae200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a8106f10 0x7f38a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ad800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38a8101de0 0x7f38a80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58592/0 (socket says 172.21.15.107:58592) 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ad800640 1 -- 172.21.15.107:0/260941006 learned_addr learned my addr 172.21.15.107:0/260941006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ace00640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38a8102790 0x7f38a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ad800640 1 -- 172.21.15.107:0/260941006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38a8102790 msgr2=0x7f38a8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ad800640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38a8102790 0x7f38a8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ad800640 1 -- 172.21.15.107:0/260941006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a8106f10 msgr2=0x7f38a8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ad800640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a8106f10 0x7f38a8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ad800640 1 -- 172.21.15.107:0/260941006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a81a68a0 con 0x7f38a8101de0 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ae200640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a8106f10 0x7f38a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.174+0000 7f38ad800640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38a8101de0 0x7f38a80744f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f389c00d950 tx=0x7f389c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.175+0000 7f38a6000640 1 -- 172.21.15.107:0/260941006 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c017030 con 0x7f38a8101de0 2024-09-19T04:29:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.175+0000 7f38b0200640 1 -- 172.21.15.107:0/260941006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a81a6b90 con 0x7f38a8101de0 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.175+0000 7f38a6000640 1 -- 172.21.15.107:0/260941006 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c004020 con 0x7f38a8101de0 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.175+0000 7f38a6000640 1 -- 172.21.15.107:0/260941006 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c0047a0 con 0x7f38a8101de0 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.175+0000 7f38b0200640 1 -- 172.21.15.107:0/260941006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a81a7120 con 0x7f38a8101de0 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.177+0000 7f38b2c4a740 1 Processor -- start 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38b2c4a740 1 -- start start 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535d5c70 0x55a8535d6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535e0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38a6000640 1 -- 172.21.15.107:0/260941006 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f389c0041d0 con 0x7f38a8101de0 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535e0e50 0x55a8535e3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a85353c3e0 con 0x55a8535d6590 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a8534e3d80 con 0x55a8535e0e50 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a8535383a0 con 0x55a8535d5c70 2024-09-19T04:29:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38ad800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38ace00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535e0e50 0x55a8535e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38ad800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535e0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38738/0 (socket says 172.21.15.107:38738) 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38ad800640 1 -- 172.21.15.107:0/3711692943 learned_addr learned my addr 172.21.15.107:0/3711692943 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.178+0000 7f38ae200640 1 --2- 172.21.15.107:0/3711692943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535d5c70 0x55a8535d6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.179+0000 7f38ace00640 1 -- 172.21.15.107:0/3711692943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535d5c70 msgr2=0x55a8535d6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.179+0000 7f38ace00640 1 --2- 172.21.15.107:0/3711692943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535d5c70 0x55a8535d6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.179+0000 7f38ace00640 1 -- 172.21.15.107:0/3711692943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 msgr2=0x55a8535e0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.179+0000 7f38ace00640 1 --2- 172.21.15.107:0/3711692943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535e0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.179+0000 7f38ace00640 1 -- 172.21.15.107:0/3711692943 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a853502ae0 con 0x55a8535e0e50 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.179+0000 7f38ace00640 1 --2- 172.21.15.107:0/3711692943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535e0e50 0x55a8535e3240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3894009bf0 tx=0x7f3894011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49c0c48c764e085d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.179+0000 7f389aa00640 1 -- 172.21.15.107:0/3711692943 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894004050 con 0x55a8535e0e50 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.179+0000 7f389aa00640 1 -- 172.21.15.107:0/3711692943 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3894005500 con 0x55a8535e0e50 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.179+0000 7f389aa00640 1 -- 172.21.15.107:0/3711692943 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894004d70 con 0x55a8535e0e50 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.179+0000 7f38a6000640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f38900a33b0 0x7f38900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38a6000640 1 -- 172.21.15.107:0/260941006 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f389c0cf690 con 0x7f38a8101de0 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38ae200640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f38900a33b0 0x7f38900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38b2c4a740 1 -- 172.21.15.107:0/3711692943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535e0e50 msgr2=0x55a8535e3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/3711692943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535e0e50 0x55a8535e3240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3894009bf0 tx=0x7f3894011810 comp rx=0 tx=0).stop 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38b2c4a740 1 -- 172.21.15.107:0/3711692943 shutdown_connections 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/3711692943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535e0e50 0x55a8535e3240 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/3711692943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/3711692943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535d5c70 0x55a8535d6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38b2c4a740 1 -- 172.21.15.107:0/3711692943 >> 172.21.15.107:0/3711692943 conn(0x55a8535d2200 msgr2=0x55a8535d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38b2c4a740 1 -- 172.21.15.107:0/3711692943 shutdown_connections 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.180+0000 7f38b2c4a740 1 -- 172.21.15.107:0/3711692943 wait complete. 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.181+0000 7f38ae200640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f38900a33b0 0x7f38900a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38a0006010 tx=0x7f38a0005d80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.181+0000 7f38b2c4a740 1 -- start start 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.181+0000 7f38b2c4a740 1 Processor -- start 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535d5c70 0x55a8535ddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 0x55a8535de180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535e0e50 0x55a8535de6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a8535e3a20 con 0x55a8535e0e50 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a8534f70f0 con 0x55a8535d6590 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a8535e3ba0 con 0x55a8535d5c70 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38b2c4a740 10 client.0 _ll_register_callbacks cb 0x55a8534debd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ae200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535e0e50 0x55a8535de6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ae200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535e0e50 0x55a8535de6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38746/0 (socket says 172.21.15.107:38746) 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ae200640 1 -- 172.21.15.107:0/2184596966 learned_addr learned my addr 172.21.15.107:0/2184596966 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ace00640 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 0x55a8535de180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ad800640 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535d5c70 0x55a8535ddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ae200640 1 -- 172.21.15.107:0/2184596966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535d5c70 msgr2=0x55a8535ddc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ae200640 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535d5c70 0x55a8535ddc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ae200640 1 -- 172.21.15.107:0/2184596966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 msgr2=0x55a8535de180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ae200640 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 0x55a8535de180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ae200640 1 -- 172.21.15.107:0/2184596966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8532ad3c0 con 0x55a8535e0e50 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.182+0000 7f38ace00640 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 0x55a8535de180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.183+0000 7f38ae200640 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535e0e50 0x55a8535de6c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f38a0035000 tx=0x7f38a002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5e4110fa8d3ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.183+0000 7f388f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.183+0000 7f388f400640 1 -- 172.21.15.107:0/2184596966 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a00319d0 con 0x55a8535e0e50 2024-09-19T04:29:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.183+0000 7f388f400640 1 -- 172.21.15.107:0/2184596966 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f38a0004250 con 0x55a8535e0e50 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.183+0000 7f38b2c4a740 1 -- 172.21.15.107:0/2184596966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3894005500 con 0x55a8535e0e50 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.183+0000 7f388f400640 1 -- 172.21.15.107:0/2184596966 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f38a00098a0 con 0x55a8535e0e50 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.183+0000 7f38b2c4a740 10 client.16779 fetch_fsmap learned FSMap version 47 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.183+0000 7f38b2c4a740 1 -- 172.21.15.107:0/2184596966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a8535ef990 con 0x55a8535e0e50 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.184+0000 7f388f400640 1 -- 172.21.15.107:0/2184596966 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3894005500 con 0x55a8535e0e50 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.184+0000 7f388f400640 1 -- 172.21.15.107:0/2184596966 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f38a005b070 con 0x55a8535e0e50 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.184+0000 7f38b2c4a740 10 client.16779 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.184+0000 7f38b2c4a740 10 client.16779 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.184+0000 7f38b2c4a740 20 client.16779 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.184+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55a8535f0e70 0x55a8535f3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.184+0000 7f38b2c4a740 4 client.16779 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:19.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.184+0000 7f38b2c4a740 1 -- 172.21.15.107:0/2184596966 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a853549950 con 0x55a8535f0e70 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.184+0000 7f38ad800640 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55a8535f0e70 0x55a8535f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.185+0000 7f38ad800640 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55a8535f0e70 0x55a8535f3310 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=52ae07f5f4628a84 server_cookie=d91049e51488fa56 in_seq=0 out_seq=0 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.185+0000 7f388f400640 10 client.16779 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f388f400640 1 -- 172.21.15.107:0/2184596966 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a853549950 con 0x55a8535f0e70 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f388f400640 10 client.16779 handle_command_reply: tid=0 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 client.16779 shutdown 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 -- 172.21.15.107:0/2184596966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535e0e50 msgr2=0x55a8535de6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535e0e50 0x55a8535de6c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f38a0035000 tx=0x7f38a002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 -- 172.21.15.107:0/2184596966 shutdown_connections 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55a8535f0e70 0x55a8535f3310 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535e0e50 0x55a8535de6c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 0x55a8535de180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/2184596966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535d5c70 0x55a8535ddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 -- 172.21.15.107:0/2184596966 >> 172.21.15.107:0/2184596966 conn(0x55a8535d2200 msgr2=0x55a8535d7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f388f400640 0 client.16779 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 -- 172.21.15.107:0/2184596966 shutdown_connections 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 1 -- 172.21.15.107:0/2184596966 wait complete. 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.186+0000 7f38b2c4a740 20 client.16779 trim_cache size 0 max 16384 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.200+0000 7f38b2c4a740 1 Processor -- start 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.200+0000 7f38b2c4a740 1 -- start start 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.200+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d5c70 0x55a8535d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.200+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 0x55a8535f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.200+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 0x55a8535f9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.200+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81af8f0 con 0x55a8535d5c70 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.200+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81b1460 con 0x55a8535d6590 2024-09-19T04:29:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.200+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81afff0 con 0x55a8535e0e50 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.200+0000 7f38ad800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d5c70 0x55a8535d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ace00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 0x55a8535f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ad800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d5c70 0x55a8535d8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38760/0 (socket says 172.21.15.107:38760) 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ad800640 1 -- 172.21.15.107:0/153512425 learned_addr learned my addr 172.21.15.107:0/153512425 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ae200640 1 --2- 172.21.15.107:0/153512425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 0x55a8535f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ad800640 1 -- 172.21.15.107:0/153512425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 msgr2=0x55a8535f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ad800640 1 --2- 172.21.15.107:0/153512425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 0x55a8535f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ad800640 1 -- 172.21.15.107:0/153512425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 msgr2=0x55a8535f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ad800640 1 --2- 172.21.15.107:0/153512425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 0x55a8535f8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ad800640 1 -- 172.21.15.107:0/153512425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8535d2030 con 0x55a8535d5c70 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ae200640 1 --2- 172.21.15.107:0/153512425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 0x55a8535f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f38ad800640 1 --2- 172.21.15.107:0/153512425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d5c70 0x55a8535d8ff0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f389c0a4070 tx=0x7f389c0d1ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec7f391f774376 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f389a000640 1 -- 172.21.15.107:0/153512425 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c09b470 con 0x55a8535d5c70 2024-09-19T04:29:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.201+0000 7f389a000640 1 -- 172.21.15.107:0/153512425 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c09c030 con 0x55a8535d5c70 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.202+0000 7f389a000640 1 -- 172.21.15.107:0/153512425 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c09b290 con 0x55a8535d5c70 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.202+0000 7f38b2c4a740 1 -- 172.21.15.107:0/153512425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d5c70 msgr2=0x55a8535d8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.202+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/153512425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d5c70 0x55a8535d8ff0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f389c0a4070 tx=0x7f389c0d1ae0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.202+0000 7f38b2c4a740 1 -- 172.21.15.107:0/153512425 shutdown_connections 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.202+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/153512425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 0x55a8535f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.202+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/153512425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d6590 0x55a8535f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.202+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/153512425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d5c70 0x55a8535d8ff0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.202+0000 7f38b2c4a740 1 -- 172.21.15.107:0/153512425 >> 172.21.15.107:0/153512425 conn(0x55a8535d2200 msgr2=0x55a8535dcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.202+0000 7f38b2c4a740 1 -- 172.21.15.107:0/153512425 shutdown_connections 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.202+0000 7f38b2c4a740 1 -- 172.21.15.107:0/153512425 wait complete. 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.203+0000 7f38b2c4a740 1 -- start start 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.203+0000 7f38b2c4a740 1 Processor -- start 2024-09-19T04:29:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d5c70 0x55a8535e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38b2c4a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 0x55a8535dae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81b0d90 con 0x55a8535d6590 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81b06c0 con 0x55a8535d5c70 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38b2c4a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81ad8f0 con 0x55a8535e0e50 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38b2c4a740 10 client.0 _ll_register_callbacks cb 0x7f38a002f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38ad800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d5c70 0x55a8535e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38ad800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d5c70 0x55a8535e94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58616/0 (socket says 172.21.15.107:58616) 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38ad800640 1 -- 172.21.15.107:0/1283036775 learned_addr learned my addr 172.21.15.107:0/1283036775 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535da950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38770/0 (socket says 172.21.15.107:38770) 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38ae200640 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 0x55a8535dae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38ad800640 1 -- 172.21.15.107:0/1283036775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 msgr2=0x55a8535dae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.204+0000 7f38ad800640 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 0x55a8535dae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f38ad800640 1 -- 172.21.15.107:0/1283036775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 msgr2=0x55a8535da950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f38ad800640 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535da950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f38ad800640 1 -- 172.21.15.107:0/1283036775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8535cd770 con 0x55a8535d5c70 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f38ace00640 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535da950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f38ad800640 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d5c70 0x55a8535e94d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f389c097d90 tx=0x7f389c0a4610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5deff20b703940c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f388f400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f388f400640 1 -- 172.21.15.107:0/1283036775 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c09fce0 con 0x55a8535d5c70 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f388f400640 1 -- 172.21.15.107:0/1283036775 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f389c0cd020 con 0x55a8535d5c70 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f38b2c4a740 1 -- 172.21.15.107:0/1283036775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389c09c030 con 0x55a8535d5c70 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f388f400640 1 -- 172.21.15.107:0/1283036775 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f389c0cd220 con 0x55a8535d5c70 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f38b2c4a740 10 client.26825 fetch_fsmap learned FSMap version 47 2024-09-19T04:29:19.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.205+0000 7f38b2c4a740 1 -- 172.21.15.107:0/1283036775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a8535e3900 con 0x55a8535d5c70 2024-09-19T04:29:19.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.206+0000 7f388f400640 1 -- 172.21.15.107:0/1283036775 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c09c030 con 0x55a8535d5c70 2024-09-19T04:29:19.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.206+0000 7f388f400640 1 -- 172.21.15.107:0/1283036775 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f389c0d1b50 con 0x55a8535d5c70 2024-09-19T04:29:19.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.206+0000 7f38b2c4a740 10 client.26825 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T04:29:19.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.206+0000 7f38b2c4a740 10 client.26825 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:29:19.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.206+0000 7f38b2c4a740 20 client.26825 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:19.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.206+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55a85365beb0 0x55a85365e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:19.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.206+0000 7f38b2c4a740 4 client.26825 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:29:19.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.206+0000 7f38b2c4a740 1 -- 172.21.15.107:0/1283036775 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a853543e60 con 0x55a85365beb0 2024-09-19T04:29:19.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.206+0000 7f38ace00640 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55a85365beb0 0x55a85365e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:19.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f38ace00640 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55a85365beb0 0x55a85365e2d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=367a42b84f7fc38 server_cookie=6ac1912ee940ba56 in_seq=0 out_seq=0 2024-09-19T04:29:19.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f388f400640 10 client.26825 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f388f400640 1 -- 172.21.15.107:0/1283036775 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a853543e60 con 0x55a85365beb0 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f388f400640 10 client.26825 handle_command_reply: tid=0 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f38b2c4a740 1 client.26825 shutdown 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f38b2c4a740 1 -- 172.21.15.107:0/1283036775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d5c70 msgr2=0x55a8535e94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d5c70 0x55a8535e94d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f389c097d90 tx=0x7f389c0a4610 comp rx=0 tx=0).stop 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f38b2c4a740 1 -- 172.21.15.107:0/1283036775 shutdown_connections 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55a85365beb0 0x55a85365e2d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8535e0e50 0x55a8535dae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a8535d6590 0x55a8535da950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f38b2c4a740 1 --2- 172.21.15.107:0/1283036775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8535d5c70 0x55a8535e94d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f388f400640 0 client.26825 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.207+0000 7f38b2c4a740 1 -- 172.21.15.107:0/1283036775 >> 172.21.15.107:0/1283036775 conn(0x55a8535d2200 msgr2=0x55a8535df4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:19.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.208+0000 7f38b2c4a740 1 -- 172.21.15.107:0/1283036775 shutdown_connections 2024-09-19T04:29:19.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.208+0000 7f38b2c4a740 1 -- 172.21.15.107:0/1283036775 wait complete. 2024-09-19T04:29:19.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.208+0000 7f38b2c4a740 20 client.26825 trim_cache size 0 max 16384 2024-09-19T04:29:19.226 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:29:19.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 -- 172.21.15.107:0/260941006 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f38900a33b0 msgr2=0x7f38900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f38900a33b0 0x7f38900a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38a0006010 tx=0x7f38a0005d80 comp rx=0 tx=0).stop 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 -- 172.21.15.107:0/260941006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38a8101de0 msgr2=0x7f38a80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38a8101de0 0x7f38a80744f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f389c00d950 tx=0x7f389c00de10 comp rx=0 tx=0).stop 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 -- 172.21.15.107:0/260941006 shutdown_connections 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f38900a33b0 0x7f38900a5870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a8106f10 0x7f38a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38a8102790 0x7f38a8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 --2- 172.21.15.107:0/260941006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38a8101de0 0x7f38a80744f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 -- 172.21.15.107:0/260941006 >> 172.21.15.107:0/260941006 conn(0x7f38a80fd8c0 msgr2=0x7f38a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.209+0000 7f38b0200640 1 -- 172.21.15.107:0/260941006 shutdown_connections 2024-09-19T04:29:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:19.210+0000 7f38b0200640 1 -- 172.21.15.107:0/260941006 wait complete. 2024-09-19T04:29:19.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:18 smithi145 ceph-mon[35096]: pgmap v278: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-09-19T04:29:19.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 286558 2.53 MB/sec execute 337 sec latency 2568.062 ms 2024-09-19T04:29:20.173 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:29:20.567 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:20.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 287925 2.54 MB/sec execute 338 sec latency 12.141 ms 2024-09-19T04:29:20.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:20 smithi107 ceph-mon[30659]: pgmap v279: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-19T04:29:20.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:20 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:29:21.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.141+0000 7fbd63200640 1 -- 172.21.15.107:0/2232899463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd64103e30 msgr2=0x7fbd64104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:21.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.141+0000 7fbd63200640 1 --2- 172.21.15.107:0/2232899463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd64103e30 0x7fbd64104210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009900 tx=0x7fbd5402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.141+0000 7fbd63200640 1 -- 172.21.15.107:0/2232899463 shutdown_connections 2024-09-19T04:29:21.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.142+0000 7fbd63200640 1 --2- 172.21.15.107:0/2232899463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd64108f60 0x7fbd64110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.142+0000 7fbd63200640 1 --2- 172.21.15.107:0/2232899463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd641047e0 0x7fbd64108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.142+0000 7fbd63200640 1 --2- 172.21.15.107:0/2232899463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd64103e30 0x7fbd64104210 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.142+0000 7fbd63200640 1 -- 172.21.15.107:0/2232899463 >> 172.21.15.107:0/2232899463 conn(0x7fbd640ff910 msgr2=0x7fbd64101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:21.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.142+0000 7fbd63200640 1 -- 172.21.15.107:0/2232899463 shutdown_connections 2024-09-19T04:29:21.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.142+0000 7fbd63200640 1 -- 172.21.15.107:0/2232899463 wait complete. 2024-09-19T04:29:21.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd63200640 1 Processor -- start 2024-09-19T04:29:21.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd63200640 1 -- start start 2024-09-19T04:29:21.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd63200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd64103e30 0x7fbd6419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd63200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd641047e0 0x7fbd6419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd63200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd64108f60 0x7fbd6419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd63200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbd64114010 con 0x7fbd64103e30 2024-09-19T04:29:21.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd63200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbd64113e90 con 0x7fbd641047e0 2024-09-19T04:29:21.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd63200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbd64114190 con 0x7fbd64108f60 2024-09-19T04:29:21.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd61400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd641047e0 0x7fbd6419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd62800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd64108f60 0x7fbd6419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.143+0000 7fbd61400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd641047e0 0x7fbd6419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58634/0 (socket says 172.21.15.107:58634) 2024-09-19T04:29:21.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd61400640 1 -- 172.21.15.107:0/1841624287 learned_addr learned my addr 172.21.15.107:0/1841624287 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:21.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd61e00640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd64103e30 0x7fbd6419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd61400640 1 -- 172.21.15.107:0/1841624287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd64108f60 msgr2=0x7fbd6419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:21.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd61400640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd64108f60 0x7fbd6419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd61400640 1 -- 172.21.15.107:0/1841624287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd64103e30 msgr2=0x7fbd6419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:21.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd61400640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd64103e30 0x7fbd6419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd61400640 1 -- 172.21.15.107:0/1841624287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6419f3c0 con 0x7fbd641047e0 2024-09-19T04:29:21.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd61400640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd641047e0 0x7fbd6419a950 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000ea60 tx=0x7fbd5000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:21.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd5aa00640 1 -- 172.21.15.107:0/1841624287 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000cde0 con 0x7fbd641047e0 2024-09-19T04:29:21.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd5aa00640 1 -- 172.21.15.107:0/1841624287 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd500035f0 con 0x7fbd641047e0 2024-09-19T04:29:21.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.144+0000 7fbd5aa00640 1 -- 172.21.15.107:0/1841624287 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50010670 con 0x7fbd641047e0 2024-09-19T04:29:21.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.145+0000 7fbd63200640 1 -- 172.21.15.107:0/1841624287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6419f6b0 con 0x7fbd641047e0 2024-09-19T04:29:21.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.145+0000 7fbd63200640 1 -- 172.21.15.107:0/1841624287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd641a6f90 con 0x7fbd641047e0 2024-09-19T04:29:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.148+0000 7fbd6b68d740 1 Processor -- start 2024-09-19T04:29:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.148+0000 7fbd5aa00640 1 -- 172.21.15.107:0/1841624287 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fbd500037a0 con 0x7fbd641047e0 2024-09-19T04:29:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.148+0000 7fbd6b68d740 1 -- start start 2024-09-19T04:29:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.148+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.148+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8daae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.148+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8db020 0x557dcb8dd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.148+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557dcb8363e0 con 0x557dcb8d0760 2024-09-19T04:29:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.148+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557dcb7ddd80 con 0x557dcb8cfe40 2024-09-19T04:29:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.148+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557dcb8323a0 con 0x557dcb8db020 2024-09-19T04:29:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd61e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8daae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd62800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd61e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8daae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38796/0 (socket says 172.21.15.107:38796) 2024-09-19T04:29:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd61e00640 1 -- 172.21.15.107:0/1362751502 learned_addr learned my addr 172.21.15.107:0/1362751502 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd61400640 1 --2- 172.21.15.107:0/1362751502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8db020 0x557dcb8dd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd62800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58636/0 (socket says 172.21.15.107:58636) 2024-09-19T04:29:21.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd61e00640 1 -- 172.21.15.107:0/1362751502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8db020 msgr2=0x557dcb8dd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:21.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd61e00640 1 --2- 172.21.15.107:0/1362751502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8db020 0x557dcb8dd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd61e00640 1 -- 172.21.15.107:0/1362751502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 msgr2=0x557dcb8d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:21.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd61e00640 1 --2- 172.21.15.107:0/1362751502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd61e00640 1 -- 172.21.15.107:0/1362751502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dcb7fcae0 con 0x557dcb8d0760 2024-09-19T04:29:21.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.149+0000 7fbd62800640 1 --2- 172.21.15.107:0/1362751502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.150+0000 7fbd5aa00640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbd480a3430 0x7fbd480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.150+0000 7fbd61e00640 1 --2- 172.21.15.107:0/1362751502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8daae0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009900 tx=0x7fbd54005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad41cac3bfc965eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.150+0000 7fbd5aa00640 1 -- 172.21.15.107:0/1841624287 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7fbd500cffd0 con 0x7fbd641047e0 2024-09-19T04:29:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.150+0000 7fbd62800640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbd480a3430 0x7fbd480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.150+0000 7fbd4f000640 1 -- 172.21.15.107:0/1362751502 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54035040 con 0x557dcb8d0760 2024-09-19T04:29:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.150+0000 7fbd4f000640 1 -- 172.21.15.107:0/1362751502 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd54004400 con 0x557dcb8d0760 2024-09-19T04:29:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.150+0000 7fbd4f000640 1 -- 172.21.15.107:0/1362751502 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd540035f0 con 0x557dcb8d0760 2024-09-19T04:29:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.151+0000 7fbd6b68d740 1 -- 172.21.15.107:0/1362751502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 msgr2=0x557dcb8daae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.151+0000 7fbd62800640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbd480a3430 0x7fbd480a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009fd0 tx=0x7fbd5c009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.151+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/1362751502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8daae0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009900 tx=0x7fbd54005de0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.151+0000 7fbd6b68d740 1 -- 172.21.15.107:0/1362751502 shutdown_connections 2024-09-19T04:29:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.151+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/1362751502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8db020 0x557dcb8dd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.151+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/1362751502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8daae0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.151+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/1362751502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.151+0000 7fbd6b68d740 1 -- 172.21.15.107:0/1362751502 >> 172.21.15.107:0/1362751502 conn(0x557dcb8cc3d0 msgr2=0x557dcb8cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.151+0000 7fbd6b68d740 1 -- 172.21.15.107:0/1362751502 shutdown_connections 2024-09-19T04:29:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.151+0000 7fbd6b68d740 1 -- 172.21.15.107:0/1362751502 wait complete. 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.152+0000 7fbd6b68d740 1 -- start start 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.152+0000 7fbd6b68d740 1 Processor -- start 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.152+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8cfe40 0x557dcb8d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.152+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 0x557dcb8d84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.152+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8d8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.152+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557dcb8ddbf0 con 0x557dcb8cfe40 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.152+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557dcb7f10f0 con 0x557dcb8db020 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.152+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557dcb8ddd70 con 0x557dcb8d0760 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd6b68d740 10 client.0 _ll_register_callbacks cb 0x557dcb7d8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd61e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8cfe40 0x557dcb8d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd62800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8d8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd61e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8cfe40 0x557dcb8d7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38802/0 (socket says 172.21.15.107:38802) 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd61e00640 1 -- 172.21.15.107:0/3068360992 learned_addr learned my addr 172.21.15.107:0/3068360992 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd62800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8d8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58648/0 (socket says 172.21.15.107:58648) 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd61400640 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 0x557dcb8d84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd62800640 1 -- 172.21.15.107:0/3068360992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 msgr2=0x557dcb8d84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd62800640 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 0x557dcb8d84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd62800640 1 -- 172.21.15.107:0/3068360992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8cfe40 msgr2=0x557dcb8d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd62800640 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8cfe40 0x557dcb8d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd62800640 1 -- 172.21.15.107:0/3068360992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dcb5a73c0 con 0x557dcb8db020 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd62800640 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8d8a00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00f610 tx=0x7fbd5c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5085376f739dd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.153+0000 7fbd43400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.154+0000 7fbd43400640 1 -- 172.21.15.107:0/3068360992 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c003c40 con 0x557dcb8db020 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.154+0000 7fbd43400640 1 -- 172.21.15.107:0/3068360992 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7fbd5c003de0 con 0x557dcb8db020 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.154+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3068360992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd54004400 con 0x557dcb8db020 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.154+0000 7fbd43400640 1 -- 172.21.15.107:0/3068360992 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fbd5c004030 con 0x557dcb8db020 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.154+0000 7fbd6b68d740 10 client.26837 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.154+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3068360992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557dcb8e9d40 con 0x557dcb8db020 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.154+0000 7fbd43400640 1 -- 172.21.15.107:0/3068360992 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd54004400 con 0x557dcb8db020 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.154+0000 7fbd43400640 1 -- 172.21.15.107:0/3068360992 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fbd5c02f020 con 0x557dcb8db020 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.155+0000 7fbd6b68d740 10 client.26837 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.155+0000 7fbd6b68d740 10 client.26837 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.155+0000 7fbd6b68d740 20 client.26837 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.155+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557dcb8eb150 0x557dcb8ed5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.155+0000 7fbd6b68d740 4 client.26837 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.155+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3068360992 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dcb843950 con 0x557dcb8eb150 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.155+0000 7fbd61e00640 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557dcb8eb150 0x557dcb8ed5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.156+0000 7fbd61e00640 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557dcb8eb150 0x557dcb8ed5a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dab0fa69474fcfcf server_cookie=2bf95487cc547612 in_seq=0 out_seq=0 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.156+0000 7fbd43400640 10 client.26837 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd43400640 1 -- 172.21.15.107:0/3068360992 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dcb843950 con 0x557dcb8eb150 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd43400640 10 client.26837 handle_command_reply: tid=0 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 client.26837 shutdown 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3068360992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 msgr2=0x557dcb8d8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8d8a00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00f610 tx=0x7fbd5c004ab0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3068360992 shutdown_connections 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557dcb8eb150 0x557dcb8ed5a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8d8a00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 0x557dcb8d84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd43400640 0 client.26837 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:29:21.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/3068360992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8cfe40 0x557dcb8d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3068360992 >> 172.21.15.107:0/3068360992 conn(0x557dcb8cc3d0 msgr2=0x557dcb8d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:21.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3068360992 shutdown_connections 2024-09-19T04:29:21.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3068360992 wait complete. 2024-09-19T04:29:21.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.157+0000 7fbd6b68d740 20 client.26837 trim_cache size 0 max 16384 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd6b68d740 1 Processor -- start 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd6b68d740 1 -- start start 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 0x557dcb8f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8db020 0x557dcb8d51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbd641af880 con 0x557dcb8db020 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbd641b13c0 con 0x557dcb8cfe40 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbd641aff50 con 0x557dcb8d0760 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd61e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd61400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 0x557dcb8f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd61e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58654/0 (socket says 172.21.15.107:58654) 2024-09-19T04:29:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd61e00640 1 -- 172.21.15.107:0/3514898036 learned_addr learned my addr 172.21.15.107:0/3514898036 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.172+0000 7fbd62800640 1 --2- 172.21.15.107:0/3514898036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8db020 0x557dcb8d51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.173+0000 7fbd61e00640 1 -- 172.21.15.107:0/3514898036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 msgr2=0x557dcb8f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.173+0000 7fbd61e00640 1 --2- 172.21.15.107:0/3514898036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 0x557dcb8f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.173+0000 7fbd61e00640 1 -- 172.21.15.107:0/3514898036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8db020 msgr2=0x557dcb8d51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.173+0000 7fbd61e00640 1 --2- 172.21.15.107:0/3514898036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8db020 0x557dcb8d51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.173+0000 7fbd61e00640 1 -- 172.21.15.107:0/3514898036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dcb8d0590 con 0x557dcb8cfe40 2024-09-19T04:29:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.173+0000 7fbd61400640 1 --2- 172.21.15.107:0/3514898036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 0x557dcb8f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.173+0000 7fbd61e00640 1 --2- 172.21.15.107:0/3514898036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8f2e70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbd54004bb0 tx=0x7fbd54004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ff2c301b61151a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.173+0000 7fbd4e600640 1 -- 172.21.15.107:0/3514898036 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54035030 con 0x557dcb8cfe40 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.173+0000 7fbd4e600640 1 -- 172.21.15.107:0/3514898036 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd54004400 con 0x557dcb8cfe40 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.173+0000 7fbd4e600640 1 -- 172.21.15.107:0/3514898036 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54003850 con 0x557dcb8cfe40 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.174+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3514898036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 msgr2=0x557dcb8f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.174+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/3514898036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8f2e70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbd54004bb0 tx=0x7fbd54004be0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.174+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3514898036 shutdown_connections 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.174+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/3514898036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8db020 0x557dcb8d51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.174+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/3514898036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8d0760 0x557dcb8f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.174+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/3514898036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8cfe40 0x557dcb8f2e70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.174+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3514898036 >> 172.21.15.107:0/3514898036 conn(0x557dcb8cc3d0 msgr2=0x557dcb8da000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:21.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.174+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3514898036 shutdown_connections 2024-09-19T04:29:21.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.174+0000 7fbd6b68d740 1 -- 172.21.15.107:0/3514898036 wait complete. 2024-09-19T04:29:21.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.174+0000 7fbd6b68d740 1 -- start start 2024-09-19T04:29:21.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd6b68d740 1 Processor -- start 2024-09-19T04:29:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8cfe40 0x557dcb8e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8deaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd6b68d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8df050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbd641b0cf0 con 0x557dcb8d0760 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbd641b0620 con 0x557dcb8db020 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd6b68d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbd641ad880 con 0x557dcb8cfe40 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd6b68d740 10 client.0 _ll_register_callbacks cb 0x7fbd5c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd61400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8deaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd61400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8deaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38824/0 (socket says 172.21.15.107:38824) 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd61400640 1 -- 172.21.15.107:0/4082159222 learned_addr learned my addr 172.21.15.107:0/4082159222 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd62800640 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.175+0000 7fbd61e00640 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8cfe40 0x557dcb8e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd61400640 1 -- 172.21.15.107:0/4082159222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8cfe40 msgr2=0x557dcb8e37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd61400640 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8cfe40 0x557dcb8e37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd61400640 1 -- 172.21.15.107:0/4082159222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 msgr2=0x557dcb8df050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd61400640 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8df050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd61400640 1 -- 172.21.15.107:0/4082159222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dcb8df880 con 0x557dcb8d0760 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd62800640 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd61400640 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8deaf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbd500cd740 tx=0x7fbd500985d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61585e745c14572 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd43400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd43400640 1 -- 172.21.15.107:0/4082159222 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5009bbd0 con 0x557dcb8d0760 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd43400640 1 -- 172.21.15.107:0/4082159222 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7fbd500cd960 con 0x557dcb8d0760 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.176+0000 7fbd6b68d740 1 -- 172.21.15.107:0/4082159222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd54004400 con 0x557dcb8d0760 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd43400640 1 -- 172.21.15.107:0/4082159222 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fbd500cdb60 con 0x557dcb8d0760 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd6b68d740 10 client.16803 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:21.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd6b68d740 1 -- 172.21.15.107:0/4082159222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557dcb955f30 con 0x557dcb8d0760 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd43400640 1 -- 172.21.15.107:0/4082159222 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd54004400 con 0x557dcb8d0760 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd43400640 1 -- 172.21.15.107:0/4082159222 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fbd500a4a50 con 0x557dcb8d0760 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd6b68d740 10 client.16803 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd6b68d740 10 client.16803 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd6b68d740 20 client.16803 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557dcb9566d0 0x557dcb958af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd6b68d740 4 client.16803 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd6b68d740 1 -- 172.21.15.107:0/4082159222 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557dcb83de60 con 0x557dcb9566d0 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.177+0000 7fbd61e00640 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557dcb9566d0 0x557dcb958af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.178+0000 7fbd61e00640 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557dcb9566d0 0x557dcb958af0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3929052a83b58db server_cookie=89f1f6ef31500720 in_seq=0 out_seq=0 2024-09-19T04:29:21.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.178+0000 7fbd43400640 10 client.16803 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:29:21.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.178+0000 7fbd43400640 1 -- 172.21.15.107:0/4082159222 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557dcb83de60 con 0x557dcb9566d0 2024-09-19T04:29:21.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.178+0000 7fbd43400640 10 client.16803 handle_command_reply: tid=0 2024-09-19T04:29:21.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.178+0000 7fbd6b68d740 1 client.16803 shutdown 2024-09-19T04:29:21.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 1 -- 172.21.15.107:0/4082159222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 msgr2=0x557dcb8deaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:21.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8deaf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbd500cd740 tx=0x7fbd500985d0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 1 -- 172.21.15.107:0/4082159222 shutdown_connections 2024-09-19T04:29:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557dcb9566d0 0x557dcb958af0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcb8db020 0x557dcb8df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557dcb8d0760 0x557dcb8deaf0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 1 --2- 172.21.15.107:0/4082159222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557dcb8cfe40 0x557dcb8e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd43400640 0 client.16803 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:29:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 1 -- 172.21.15.107:0/4082159222 >> 172.21.15.107:0/4082159222 conn(0x557dcb8cc3d0 msgr2=0x557dcb8d7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 1 -- 172.21.15.107:0/4082159222 shutdown_connections 2024-09-19T04:29:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 1 -- 172.21.15.107:0/4082159222 wait complete. 2024-09-19T04:29:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.179+0000 7fbd6b68d740 20 client.16803 trim_cache size 0 max 16384 2024-09-19T04:29:21.185 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:29:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 -- 172.21.15.107:0/1841624287 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbd480a3430 msgr2=0x7fbd480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbd480a3430 0x7fbd480a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009fd0 tx=0x7fbd5c009500 comp rx=0 tx=0).stop 2024-09-19T04:29:21.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 -- 172.21.15.107:0/1841624287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd641047e0 msgr2=0x7fbd6419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd641047e0 0x7fbd6419a950 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000ea60 tx=0x7fbd5000ef20 comp rx=0 tx=0).stop 2024-09-19T04:29:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 -- 172.21.15.107:0/1841624287 shutdown_connections 2024-09-19T04:29:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fbd480a3430 0x7fbd480a58f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd64108f60 0x7fbd6419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd641047e0 0x7fbd6419a950 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 --2- 172.21.15.107:0/1841624287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd64103e30 0x7fbd6419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 -- 172.21.15.107:0/1841624287 >> 172.21.15.107:0/1841624287 conn(0x7fbd640ff910 msgr2=0x7fbd64100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 -- 172.21.15.107:0/1841624287 shutdown_connections 2024-09-19T04:29:21.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:21.180+0000 7fbd63200640 1 -- 172.21.15.107:0/1841624287 wait complete. 2024-09-19T04:29:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:20 smithi115 ceph-mon[35477]: pgmap v279: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-19T04:29:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:20 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:29:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:20 smithi145 ceph-mon[35096]: pgmap v279: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-19T04:29:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:20 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:29:21.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 289024 2.55 MB/sec execute 339 sec latency 157.525 ms 2024-09-19T04:29:22.166 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:29:22.535 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:22.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 289024 2.55 MB/sec execute 340 sec latency 1157.666 ms 2024-09-19T04:29:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:22 smithi107 ceph-mon[30659]: pgmap v280: 97 pgs: 97 active+clean; 212 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-09-19T04:29:23.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.091+0000 7fdcb9c00640 1 -- 172.21.15.107:0/1969212238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcb4103e30 msgr2=0x7fdcb4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:23.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.091+0000 7fdcb9c00640 1 --2- 172.21.15.107:0/1969212238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcb4103e30 0x7fdcb4104210 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009900 tx=0x7fdca402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.092+0000 7fdcb9c00640 1 -- 172.21.15.107:0/1969212238 shutdown_connections 2024-09-19T04:29:23.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.092+0000 7fdcb9c00640 1 --2- 172.21.15.107:0/1969212238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcb4108f60 0x7fdcb4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.092+0000 7fdcb9c00640 1 --2- 172.21.15.107:0/1969212238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41047e0 0x7fdcb4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.092+0000 7fdcb9c00640 1 --2- 172.21.15.107:0/1969212238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcb4103e30 0x7fdcb4104210 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.092+0000 7fdcb9c00640 1 -- 172.21.15.107:0/1969212238 >> 172.21.15.107:0/1969212238 conn(0x7fdcb40ff8d0 msgr2=0x7fdcb4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:23.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.092+0000 7fdcb9c00640 1 -- 172.21.15.107:0/1969212238 shutdown_connections 2024-09-19T04:29:23.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.092+0000 7fdcb9c00640 1 -- 172.21.15.107:0/1969212238 wait complete. 2024-09-19T04:29:23.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.093+0000 7fdcb9c00640 1 Processor -- start 2024-09-19T04:29:23.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.093+0000 7fdcb9c00640 1 -- start start 2024-09-19T04:29:23.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcb9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcb4103e30 0x7fdcb419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcb9c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41047e0 0x7fdcb419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcb9c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcb4108f60 0x7fdcb419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcb9c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4114010 con 0x7fdcb4103e30 2024-09-19T04:29:23.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcb9c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4113e90 con 0x7fdcb41047e0 2024-09-19T04:29:23.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcb9c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4114190 con 0x7fdcb4108f60 2024-09-19T04:29:23.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41047e0 0x7fdcb419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41047e0 0x7fdcb419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58682/0 (socket says 172.21.15.107:58682) 2024-09-19T04:29:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcabe00640 1 -- 172.21.15.107:0/2213541369 learned_addr learned my addr 172.21.15.107:0/2213541369 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcb4103e30 0x7fdcb419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcb9200640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcb4108f60 0x7fdcb419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcabe00640 1 -- 172.21.15.107:0/2213541369 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcb4108f60 msgr2=0x7fdcb419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcabe00640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcb4108f60 0x7fdcb419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcabe00640 1 -- 172.21.15.107:0/2213541369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcb4103e30 msgr2=0x7fdcb419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcabe00640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcb4103e30 0x7fdcb419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.094+0000 7fdcabe00640 1 -- 172.21.15.107:0/2213541369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb419f400 con 0x7fdcb41047e0 2024-09-19T04:29:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.095+0000 7fdcabe00640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41047e0 0x7fdcb419a990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e9f0 tx=0x7fdc9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.095+0000 7fdcb1600640 1 -- 172.21.15.107:0/2213541369 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00cd80 con 0x7fdcb41047e0 2024-09-19T04:29:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.095+0000 7fdcb1600640 1 -- 172.21.15.107:0/2213541369 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c010910 con 0x7fdcb41047e0 2024-09-19T04:29:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.095+0000 7fdcb9c00640 1 -- 172.21.15.107:0/2213541369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb419f6f0 con 0x7fdcb41047e0 2024-09-19T04:29:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.095+0000 7fdcb1600640 1 -- 172.21.15.107:0/2213541369 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c018990 con 0x7fdcb41047e0 2024-09-19T04:29:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.095+0000 7fdcb9c00640 1 -- 172.21.15.107:0/2213541369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb41a6fd0 con 0x7fdcb41047e0 2024-09-19T04:29:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.098+0000 7fdcbdca5740 1 Processor -- start 2024-09-19T04:29:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.098+0000 7fdcb1600640 1 -- 172.21.15.107:0/2213541369 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fdc9c010ac0 con 0x7fdcb41047e0 2024-09-19T04:29:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.098+0000 7fdcbdca5740 1 -- start start 2024-09-19T04:29:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.098+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8131220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e813bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e813e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5610e80973e0 con 0x5610e813c020 2024-09-19T04:29:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5610e803ed80 con 0x5610e8131760 2024-09-19T04:29:23.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5610e80933a0 con 0x5610e8130e40 2024-09-19T04:29:23.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e813bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e813bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58690/0 (socket says 172.21.15.107:58690) 2024-09-19T04:29:23.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcb3e00640 1 -- 172.21.15.107:0/4062302190 learned_addr learned my addr 172.21.15.107:0/4062302190 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:23.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcabe00640 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e813e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcb9200640 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8131220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcb3e00640 1 -- 172.21.15.107:0/4062302190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 msgr2=0x5610e8131220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:23.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8131220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcb3e00640 1 -- 172.21.15.107:0/4062302190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 msgr2=0x5610e813e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:23.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e813e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcb3e00640 1 -- 172.21.15.107:0/4062302190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e805dae0 con 0x5610e8131760 2024-09-19T04:29:23.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcabe00640 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e813e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:23.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.099+0000 7fdcb9200640 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8131220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:23.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.100+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e813bae0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009900 tx=0x7fdca4005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33cbc05c3c7deb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:23.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.100+0000 7fdcb1600640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdc980a33b0 0x7fdc980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.100+0000 7fdcb1600640 1 -- 172.21.15.107:0/2213541369 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7fdc9c0d0070 con 0x7fdcb41047e0 2024-09-19T04:29:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.100+0000 7fdca9600640 1 -- 172.21.15.107:0/4062302190 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4035040 con 0x5610e8131760 2024-09-19T04:29:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.100+0000 7fdca9600640 1 -- 172.21.15.107:0/4062302190 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca4004400 con 0x5610e8131760 2024-09-19T04:29:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.100+0000 7fdca9600640 1 -- 172.21.15.107:0/4062302190 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca40035f0 con 0x5610e8131760 2024-09-19T04:29:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.100+0000 7fdcb9200640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdc980a33b0 0x7fdc980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcbdca5740 1 -- 172.21.15.107:0/4062302190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 msgr2=0x5610e813bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e813bae0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009900 tx=0x7fdca4005de0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcb9200640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdc980a33b0 0x7fdc980a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009fd0 tx=0x7fdcac009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcbdca5740 1 -- 172.21.15.107:0/4062302190 shutdown_connections 2024-09-19T04:29:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e813e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e813bae0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/4062302190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8131220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcbdca5740 1 -- 172.21.15.107:0/4062302190 >> 172.21.15.107:0/4062302190 conn(0x5610e812d3d0 msgr2=0x5610e812d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcbdca5740 1 -- 172.21.15.107:0/4062302190 shutdown_connections 2024-09-19T04:29:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcbdca5740 1 -- 172.21.15.107:0/4062302190 wait complete. 2024-09-19T04:29:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.101+0000 7fdcbdca5740 1 -- start start 2024-09-19T04:29:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.102+0000 7fdcbdca5740 1 Processor -- start 2024-09-19T04:29:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.102+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8138ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.102+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8139420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.102+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e8139960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.102+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5610e813ebf0 con 0x5610e813c020 2024-09-19T04:29:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.102+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5610e80520f0 con 0x5610e8131760 2024-09-19T04:29:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.102+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5610e813ed70 con 0x5610e8130e40 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.102+0000 7fdcbdca5740 10 client.0 _ll_register_callbacks cb 0x5610e8039bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.102+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8139420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8139420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58694/0 (socket says 172.21.15.107:58694) 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcb9200640 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e8139960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcabe00640 1 -- 172.21.15.107:0/227886239 learned_addr learned my addr 172.21.15.107:0/227886239 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8138ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcabe00640 1 -- 172.21.15.107:0/227886239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 msgr2=0x5610e8138ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcabe00640 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8138ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcabe00640 1 -- 172.21.15.107:0/227886239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 msgr2=0x5610e8139960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcabe00640 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e8139960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcabe00640 1 -- 172.21.15.107:0/227886239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e7e083c0 con 0x5610e8131760 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8138ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdcabe00640 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8139420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c6a0 tx=0x7fdc9c098750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23049e657c65261 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.103+0000 7fdca2000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdca2000640 1 -- 172.21.15.107:0/227886239 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0cdc00 con 0x5610e8131760 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdca2000640 1 -- 172.21.15.107:0/227886239 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7fdc9c0bc0e0 con 0x5610e8131760 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdcbdca5740 1 -- 172.21.15.107:0/227886239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdca4004400 con 0x5610e8131760 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdca2000640 1 -- 172.21.15.107:0/227886239 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdc9c09c3c0 con 0x5610e8131760 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdcbdca5740 10 client.26855 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdcbdca5740 1 -- 172.21.15.107:0/227886239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5610e814acc0 con 0x5610e8131760 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdca2000640 1 -- 172.21.15.107:0/227886239 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca4004400 con 0x5610e8131760 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdca2000640 1 -- 172.21.15.107:0/227886239 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fdc9c018b30 con 0x5610e8131760 2024-09-19T04:29:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdcbdca5740 10 client.26855 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:23.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdcbdca5740 10 client.26855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:29:23.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdcbdca5740 20 client.26855 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:23.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.104+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5610e814c1d0 0x5610e814e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.105+0000 7fdcbdca5740 4 client.26855 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:23.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.105+0000 7fdcbdca5740 1 -- 172.21.15.107:0/227886239 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610e80a4950 con 0x5610e814c1d0 2024-09-19T04:29:23.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.105+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5610e814c1d0 0x5610e814e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.105+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5610e814c1d0 0x5610e814e610 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d58a0562daaaa325 server_cookie=e3c03019ae2eaf9f in_seq=0 out_seq=0 2024-09-19T04:29:23.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.105+0000 7fdca2000640 10 client.26855 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:29:23.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.106+0000 7fdca2000640 1 -- 172.21.15.107:0/227886239 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610e80a4950 con 0x5610e814c1d0 2024-09-19T04:29:23.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.106+0000 7fdca2000640 10 client.26855 handle_command_reply: tid=0 2024-09-19T04:29:23.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.106+0000 7fdcbdca5740 1 client.26855 shutdown 2024-09-19T04:29:23.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.106+0000 7fdcbdca5740 1 -- 172.21.15.107:0/227886239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 msgr2=0x5610e8139420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:23.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.106+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8139420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c6a0 tx=0x7fdc9c098750 comp rx=0 tx=0).stop 2024-09-19T04:29:23.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.106+0000 7fdcbdca5740 1 -- 172.21.15.107:0/227886239 shutdown_connections 2024-09-19T04:29:23.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.107+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5610e814c1d0 0x5610e814e610 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.107+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e8139960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.107+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8139420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.107+0000 7fdca2000640 0 client.26855 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:29:23.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.107+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/227886239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8138ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.107+0000 7fdcbdca5740 1 -- 172.21.15.107:0/227886239 >> 172.21.15.107:0/227886239 conn(0x5610e812d3d0 msgr2=0x5610e81330b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:23.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.107+0000 7fdcbdca5740 1 -- 172.21.15.107:0/227886239 shutdown_connections 2024-09-19T04:29:23.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.107+0000 7fdcbdca5740 1 -- 172.21.15.107:0/227886239 wait complete. 2024-09-19T04:29:23.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.107+0000 7fdcbdca5740 20 client.26855 trim_cache size 0 max 16384 2024-09-19T04:29:23.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.120+0000 7fdcbdca5740 1 Processor -- start 2024-09-19T04:29:23.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.120+0000 7fdcbdca5740 1 -- start start 2024-09-19T04:29:23.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e81387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e8131760 0x5610e813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e813c020 0x5610e8153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb41af850 con 0x5610e8131760 2024-09-19T04:29:23.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb41b1390 con 0x5610e813c020 2024-09-19T04:29:23.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb41aff20 con 0x5610e8130e40 2024-09-19T04:29:23.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e8131760 0x5610e813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e8131760 0x5610e813ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38864/0 (socket says 172.21.15.107:38864) 2024-09-19T04:29:23.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcabe00640 1 -- 172.21.15.107:0/2717388735 learned_addr learned my addr 172.21.15.107:0/2717388735 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:23.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/2717388735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e81387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcb9200640 1 --2- 172.21.15.107:0/2717388735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e813c020 0x5610e8153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcabe00640 1 -- 172.21.15.107:0/2717388735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 msgr2=0x5610e81387a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:23.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcabe00640 1 --2- 172.21.15.107:0/2717388735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e81387a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcabe00640 1 -- 172.21.15.107:0/2717388735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e813c020 msgr2=0x5610e8153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:23.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcabe00640 1 --2- 172.21.15.107:0/2717388735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e813c020 0x5610e8153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcabe00640 1 -- 172.21.15.107:0/2717388735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e812d200 con 0x5610e8131760 2024-09-19T04:29:23.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.121+0000 7fdcb9200640 1 --2- 172.21.15.107:0/2717388735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e813c020 0x5610e8153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:23.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.122+0000 7fdcabe00640 1 --2- 172.21.15.107:0/2717388735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e8131760 0x5610e813ae50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0a4a50 tx=0x7fdc9c098660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689b33f5a875591 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:23.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.122+0000 7fdca8c00640 1 -- 172.21.15.107:0/2717388735 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0a4f80 con 0x5610e8131760 2024-09-19T04:29:23.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.122+0000 7fdca8c00640 1 -- 172.21.15.107:0/2717388735 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c0d3390 con 0x5610e8131760 2024-09-19T04:29:23.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.122+0000 7fdca8c00640 1 -- 172.21.15.107:0/2717388735 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0cda00 con 0x5610e8131760 2024-09-19T04:29:23.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.122+0000 7fdcbdca5740 1 -- 172.21.15.107:0/2717388735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e8131760 msgr2=0x5610e813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:23.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.122+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/2717388735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e8131760 0x5610e813ae50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0a4a50 tx=0x7fdc9c098660 comp rx=0 tx=0).stop 2024-09-19T04:29:23.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.123+0000 7fdcbdca5740 1 -- 172.21.15.107:0/2717388735 shutdown_connections 2024-09-19T04:29:23.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.123+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/2717388735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e813c020 0x5610e8153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.123+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/2717388735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e8131760 0x5610e813ae50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.123+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/2717388735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e81387a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.123+0000 7fdcbdca5740 1 -- 172.21.15.107:0/2717388735 >> 172.21.15.107:0/2717388735 conn(0x5610e812d3d0 msgr2=0x5610e813a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.123+0000 7fdcbdca5740 1 -- 172.21.15.107:0/2717388735 shutdown_connections 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.123+0000 7fdcbdca5740 1 -- 172.21.15.107:0/2717388735 wait complete. 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.123+0000 7fdcbdca5740 1 -- start start 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.123+0000 7fdcbdca5740 1 Processor -- start 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8135e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8136360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcbdca5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e81368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb41b0cc0 con 0x5610e813c020 2024-09-19T04:29:23.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb41b05f0 con 0x5610e8131760 2024-09-19T04:29:23.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcbdca5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb41ad850 con 0x5610e8130e40 2024-09-19T04:29:23.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcbdca5740 10 client.0 _ll_register_callbacks cb 0x7fdc9c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:23.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcb9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e81368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8136360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8135e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41936/0 (socket says 172.21.15.107:41936) 2024-09-19T04:29:23.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcb3e00640 1 -- 172.21.15.107:0/798891410 learned_addr learned my addr 172.21.15.107:0/798891410 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:23.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.124+0000 7fdcabe00640 1 -- 172.21.15.107:0/798891410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 msgr2=0x5610e8135e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:23.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdcabe00640 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8135e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdcabe00640 1 -- 172.21.15.107:0/798891410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 msgr2=0x5610e81368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdcabe00640 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e81368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdcabe00640 1 -- 172.21.15.107:0/798891410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e81b5eb0 con 0x5610e8131760 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8135e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdcb9200640 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e81368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdcabe00640 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8136360 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5610e8039d10 tx=0x7fdc9c0cd860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c523956cb1c31191 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdca2000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdca2000640 1 -- 172.21.15.107:0/798891410 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0b4020 con 0x5610e8131760 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdca2000640 1 -- 172.21.15.107:0/798891410 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7fdc9c010430 con 0x5610e8131760 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdcbdca5740 1 -- 172.21.15.107:0/798891410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc9c0d3390 con 0x5610e8131760 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdca2000640 1 -- 172.21.15.107:0/798891410 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdc9c09bf00 con 0x5610e8131760 2024-09-19T04:29:23.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdcbdca5740 10 client.26861 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.125+0000 7fdcbdca5740 1 -- 172.21.15.107:0/798891410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5610e81b70d0 con 0x5610e8131760 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.126+0000 7fdca2000640 1 -- 172.21.15.107:0/798891410 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c0d3390 con 0x5610e8131760 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.126+0000 7fdca2000640 1 -- 172.21.15.107:0/798891410 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fdc9c0d1070 con 0x5610e8131760 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.126+0000 7fdcbdca5740 10 client.26861 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.126+0000 7fdcbdca5740 10 client.26861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.126+0000 7fdcbdca5740 20 client.26861 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.126+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5610e81b75a0 0x5610e81b9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.126+0000 7fdcbdca5740 4 client.26861 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.126+0000 7fdcbdca5740 1 -- 172.21.15.107:0/798891410 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610e809ee60 con 0x5610e81b75a0 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.126+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5610e81b75a0 0x5610e81b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:23.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcb3e00640 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5610e81b75a0 0x5610e81b9990 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b415d68ebb1df840 server_cookie=75442862465fd481 in_seq=0 out_seq=0 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdca2000640 10 client.26861 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdca2000640 1 -- 172.21.15.107:0/798891410 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610e809ee60 con 0x5610e81b75a0 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdca2000640 10 client.26861 handle_command_reply: tid=0 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 client.26861 shutdown 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 -- 172.21.15.107:0/798891410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 msgr2=0x5610e8136360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8136360 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5610e8039d10 tx=0x7fdc9c0cd860 comp rx=0 tx=0).stop 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 -- 172.21.15.107:0/798891410 shutdown_connections 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x5610e81b75a0 0x5610e81b9990 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610e813c020 0x5610e81368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610e8131760 0x5610e8136360 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdca2000640 0 client.26861 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:29:23.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 --2- 172.21.15.107:0/798891410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5610e8130e40 0x5610e8135e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 -- 172.21.15.107:0/798891410 >> 172.21.15.107:0/798891410 conn(0x5610e812d3d0 msgr2=0x5610e81330b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:23.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 -- 172.21.15.107:0/798891410 shutdown_connections 2024-09-19T04:29:23.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 1 -- 172.21.15.107:0/798891410 wait complete. 2024-09-19T04:29:23.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.127+0000 7fdcbdca5740 20 client.26861 trim_cache size 0 max 16384 2024-09-19T04:29:23.136 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:29:23.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 -- 172.21.15.107:0/2213541369 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdc980a33b0 msgr2=0x7fdc980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:23.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdc980a33b0 0x7fdc980a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009fd0 tx=0x7fdcac009500 comp rx=0 tx=0).stop 2024-09-19T04:29:23.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 -- 172.21.15.107:0/2213541369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41047e0 msgr2=0x7fdcb419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:23.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41047e0 0x7fdcb419a990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e9f0 tx=0x7fdc9c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 -- 172.21.15.107:0/2213541369 shutdown_connections 2024-09-19T04:29:23.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdc980a33b0 0x7fdc980a5870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcb4108f60 0x7fdcb419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41047e0 0x7fdcb419a990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 --2- 172.21.15.107:0/2213541369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcb4103e30 0x7fdcb419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:23.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 -- 172.21.15.107:0/2213541369 >> 172.21.15.107:0/2213541369 conn(0x7fdcb40ff8d0 msgr2=0x7fdcb4100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:23.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 -- 172.21.15.107:0/2213541369 shutdown_connections 2024-09-19T04:29:23.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:23.128+0000 7fdcb9c00640 1 -- 172.21.15.107:0/2213541369 wait complete. 2024-09-19T04:29:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:22 smithi115 ceph-mon[35477]: pgmap v280: 97 pgs: 97 active+clean; 212 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-09-19T04:29:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:22 smithi145 ceph-mon[35096]: pgmap v280: 97 pgs: 97 active+clean; 212 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-09-19T04:29:23.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 289024 2.54 MB/sec execute 341 sec latency 2157.756 ms 2024-09-19T04:29:23.909 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:29:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:24.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 290092 2.55 MB/sec execute 342 sec latency 2678.282 ms 2024-09-19T04:29:25.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:24 smithi107 ceph-mon[30659]: pgmap v281: 97 pgs: 97 active+clean; 212 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-19T04:29:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.078+0000 7feecbc00640 1 -- 172.21.15.107:0/1242254492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feec410e470 msgr2=0x7feec4110860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.078+0000 7feecbc00640 1 --2- 172.21.15.107:0/1242254492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feec410e470 0x7feec4110860 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feebc009ca0 tx=0x7feebc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.079+0000 7feecbc00640 1 -- 172.21.15.107:0/1242254492 shutdown_connections 2024-09-19T04:29:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.079+0000 7feecbc00640 1 --2- 172.21.15.107:0/1242254492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feec410e470 0x7feec4110860 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.079+0000 7feecbc00640 1 --2- 172.21.15.107:0/1242254492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feec4101270 0x7feec410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.079+0000 7feecbc00640 1 --2- 172.21.15.107:0/1242254492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec41008c0 0x7feec4100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.079+0000 7feecbc00640 1 -- 172.21.15.107:0/1242254492 >> 172.21.15.107:0/1242254492 conn(0x7feec40fc790 msgr2=0x7feec40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.079+0000 7feecbc00640 1 -- 172.21.15.107:0/1242254492 shutdown_connections 2024-09-19T04:29:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.079+0000 7feecbc00640 1 -- 172.21.15.107:0/1242254492 wait complete. 2024-09-19T04:29:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.080+0000 7feecbc00640 1 Processor -- start 2024-09-19T04:29:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.080+0000 7feecbc00640 1 -- start start 2024-09-19T04:29:25.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.080+0000 7feecbc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feec41008c0 0x7feec419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.080+0000 7feecbc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101270 0x7feec419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.080+0000 7feecbc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feec410e470 0x7feec419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.080+0000 7feecbc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feec41141b0 con 0x7feec41008c0 2024-09-19T04:29:25.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.080+0000 7feecbc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feec4114030 con 0x7feec410e470 2024-09-19T04:29:25.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.080+0000 7feecbc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feec4114330 con 0x7feec4101270 2024-09-19T04:29:25.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101270 0x7feec419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101270 0x7feec419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41962/0 (socket says 172.21.15.107:41962) 2024-09-19T04:29:25.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feec41008c0 0x7feec419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec3e00640 1 -- 172.21.15.107:0/2501124141 learned_addr learned my addr 172.21.15.107:0/2501124141 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec9c00640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feec410e470 0x7feec419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec3e00640 1 -- 172.21.15.107:0/2501124141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feec410e470 msgr2=0x7feec419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec3e00640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feec410e470 0x7feec419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec3e00640 1 -- 172.21.15.107:0/2501124141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feec41008c0 msgr2=0x7feec419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec3e00640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feec41008c0 0x7feec419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec3e00640 1 -- 172.21.15.107:0/2501124141 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec419f670 con 0x7feec4101270 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.081+0000 7feec3e00640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101270 0x7feec419ac00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feeb000e9f0 tx=0x7feeb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.082+0000 7feec1600640 1 -- 172.21.15.107:0/2501124141 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb000cd80 con 0x7feec4101270 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.082+0000 7feec1600640 1 -- 172.21.15.107:0/2501124141 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb0010910 con 0x7feec4101270 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.082+0000 7feec1600640 1 -- 172.21.15.107:0/2501124141 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0018950 con 0x7feec4101270 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.082+0000 7feecbc00640 1 -- 172.21.15.107:0/2501124141 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec419f960 con 0x7feec4101270 2024-09-19T04:29:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.082+0000 7feecbc00640 1 -- 172.21.15.107:0/2501124141 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec4102ad0 con 0x7feec4101270 2024-09-19T04:29:25.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.085+0000 7feece6c0740 1 Processor -- start 2024-09-19T04:29:25.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.085+0000 7feec1600640 1 -- 172.21.15.107:0/2501124141 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7feeb0010ac0 con 0x7feec4101270 2024-09-19T04:29:25.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.085+0000 7feece6c0740 1 -- start start 2024-09-19T04:29:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.085+0000 7feece6c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b55e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.085+0000 7feece6c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 0x55570b568910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.085+0000 7feece6c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b568e50 0x55570b56b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.085+0000 7feece6c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55570b4c43e0 con 0x55570b568e50 2024-09-19T04:29:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.085+0000 7feece6c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55570b46bd80 con 0x55570b55dc70 2024-09-19T04:29:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.085+0000 7feece6c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55570b4c03a0 con 0x55570b55e590 2024-09-19T04:29:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.086+0000 7feec9c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b55e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.086+0000 7feec9c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b55e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58734/0 (socket says 172.21.15.107:58734) 2024-09-19T04:29:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.086+0000 7feec9c00640 1 -- 172.21.15.107:0/2724700268 learned_addr learned my addr 172.21.15.107:0/2724700268 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.086+0000 7feec9200640 1 --2- 172.21.15.107:0/2724700268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 0x55570b568910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.086+0000 7feec9c00640 1 -- 172.21.15.107:0/2724700268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 msgr2=0x55570b568910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.086+0000 7feec9c00640 1 --2- 172.21.15.107:0/2724700268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 0x55570b568910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.086+0000 7feec9c00640 1 -- 172.21.15.107:0/2724700268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b568e50 msgr2=0x55570b56b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.086+0000 7feec9c00640 1 --2- 172.21.15.107:0/2724700268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b568e50 0x55570b56b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.086+0000 7feec9c00640 1 -- 172.21.15.107:0/2724700268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570b48aae0 con 0x55570b55dc70 2024-09-19T04:29:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.086+0000 7feec9c00640 1 --2- 172.21.15.107:0/2724700268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b55e050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feec4102aa0 tx=0x7feebc005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84b794a44e233715 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.087+0000 7feeb6000640 1 -- 172.21.15.107:0/2724700268 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc050070 con 0x55570b55dc70 2024-09-19T04:29:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.087+0000 7feeb6000640 1 -- 172.21.15.107:0/2724700268 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feebc036040 con 0x55570b55dc70 2024-09-19T04:29:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.087+0000 7feeb6000640 1 -- 172.21.15.107:0/2724700268 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc031650 con 0x55570b55dc70 2024-09-19T04:29:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.087+0000 7feec1600640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feeac0a33b0 0x7feeac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.087+0000 7feec1600640 1 -- 172.21.15.107:0/2501124141 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7feeb00d00a0 con 0x7feec4101270 2024-09-19T04:29:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.087+0000 7feec9200640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feeac0a33b0 0x7feeac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.087+0000 7feece6c0740 1 -- 172.21.15.107:0/2724700268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 msgr2=0x55570b55e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.087+0000 7feece6c0740 1 --2- 172.21.15.107:0/2724700268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b55e050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feec4102aa0 tx=0x7feebc005f80 comp rx=0 tx=0).stop 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.088+0000 7feece6c0740 1 -- 172.21.15.107:0/2724700268 shutdown_connections 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.088+0000 7feece6c0740 1 --2- 172.21.15.107:0/2724700268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b568e50 0x55570b56b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.088+0000 7feece6c0740 1 --2- 172.21.15.107:0/2724700268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 0x55570b568910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.088+0000 7feece6c0740 1 --2- 172.21.15.107:0/2724700268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b55e050 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.088+0000 7feece6c0740 1 -- 172.21.15.107:0/2724700268 >> 172.21.15.107:0/2724700268 conn(0x55570b55a200 msgr2=0x55570b55a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.088+0000 7feece6c0740 1 -- 172.21.15.107:0/2724700268 shutdown_connections 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.088+0000 7feece6c0740 1 -- 172.21.15.107:0/2724700268 wait complete. 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.088+0000 7feec9200640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feeac0a33b0 0x7feeac0a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55570b55b060 tx=0x7feeb8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.088+0000 7feece6c0740 1 -- start start 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feece6c0740 1 Processor -- start 2024-09-19T04:29:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feece6c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 0x55570b565db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feece6c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 0x55570b5662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feece6c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b566830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feece6c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55570b56ba20 con 0x55570b55dc70 2024-09-19T04:29:25.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feece6c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55570b47f0f0 con 0x55570b568e50 2024-09-19T04:29:25.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feece6c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55570b56bba0 con 0x55570b55e590 2024-09-19T04:29:25.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feece6c0740 10 client.0 _ll_register_callbacks cb 0x55570b466bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:25.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feec9c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b566830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feec9c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b566830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58746/0 (socket says 172.21.15.107:58746) 2024-09-19T04:29:25.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.089+0000 7feec9c00640 1 -- 172.21.15.107:0/2858258132 learned_addr learned my addr 172.21.15.107:0/2858258132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:25.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feec3e00640 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 0x55570b5662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feec9200640 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 0x55570b565db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feec3e00640 1 -- 172.21.15.107:0/2858258132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 msgr2=0x55570b566830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feec3e00640 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b566830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feec3e00640 1 -- 172.21.15.107:0/2858258132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 msgr2=0x55570b565db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feec3e00640 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 0x55570b565db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feec3e00640 1 -- 172.21.15.107:0/2858258132 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570b2353c0 con 0x55570b55e590 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feec9200640 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 0x55570b565db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feec3e00640 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 0x55570b5662f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feeb00cd780 tx=0x7feeb009c060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b3efe121c52eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feeaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.090+0000 7feeaaa00640 1 -- 172.21.15.107:0/2858258132 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00cdc40 con 0x55570b55e590 2024-09-19T04:29:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feece6c0740 1 -- 172.21.15.107:0/2858258132 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feebc036040 con 0x55570b55e590 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feeaaa00640 1 -- 172.21.15.107:0/2858258132 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7feeb009b840 con 0x55570b55e590 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feeaaa00640 1 -- 172.21.15.107:0/2858258132 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feeb00d22d0 con 0x55570b55e590 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feece6c0740 10 client.26839 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feece6c0740 1 -- 172.21.15.107:0/2858258132 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55570b577b90 con 0x55570b55e590 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feeaaa00640 1 -- 172.21.15.107:0/2858258132 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feebc036040 con 0x55570b55e590 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feeaaa00640 1 -- 172.21.15.107:0/2858258132 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7feeb0018ca0 con 0x55570b55e590 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feece6c0740 10 client.26839 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feece6c0740 10 client.26839 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feece6c0740 20 client.26839 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feece6c0740 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55570b578ff0 0x55570b57b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feece6c0740 4 client.26839 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.091+0000 7feece6c0740 1 -- 172.21.15.107:0/2858258132 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55570b4d1950 con 0x55570b578ff0 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.092+0000 7feec9200640 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55570b578ff0 0x55570b57b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.092+0000 7feec9200640 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55570b578ff0 0x55570b57b3e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c1986d54f61afb2 server_cookie=84c9663c718ad6bd in_seq=0 out_seq=0 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.092+0000 7feeaaa00640 10 client.26839 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feeaaa00640 1 -- 172.21.15.107:0/2858258132 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55570b4d1950 con 0x55570b578ff0 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feeaaa00640 10 client.26839 handle_command_reply: tid=0 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 client.26839 shutdown 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 -- 172.21.15.107:0/2858258132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 msgr2=0x55570b5662f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 0x55570b5662f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feeb00cd780 tx=0x7feeb009c060 comp rx=0 tx=0).stop 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 -- 172.21.15.107:0/2858258132 shutdown_connections 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55570b578ff0 0x55570b57b3e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b566830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b55e590 0x55570b5662f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feeaaa00640 0 client.26839 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:29:25.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 --2- 172.21.15.107:0/2858258132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 0x55570b565db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 -- 172.21.15.107:0/2858258132 >> 172.21.15.107:0/2858258132 conn(0x55570b55a200 msgr2=0x55570b559380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:25.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 -- 172.21.15.107:0/2858258132 shutdown_connections 2024-09-19T04:29:25.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.093+0000 7feece6c0740 1 -- 172.21.15.107:0/2858258132 wait complete. 2024-09-19T04:29:25.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.094+0000 7feece6c0740 20 client.26839 trim_cache size 0 max 16384 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feece6c0740 1 Processor -- start 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feece6c0740 1 -- start start 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feece6c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b564680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feece6c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b568e50 0x55570b566680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feece6c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 0x55570b5810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feece6c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feec41ab670 con 0x55570b568e50 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feece6c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feec41ad1b0 con 0x55570b55dc70 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feece6c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feec41abd40 con 0x55570b580c90 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feec9200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b564680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feec9c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 0x55570b5810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feec9200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b564680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58756/0 (socket says 172.21.15.107:58756) 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feec9200640 1 -- 172.21.15.107:0/3177455129 learned_addr learned my addr 172.21.15.107:0/3177455129 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.108+0000 7feec3e00640 1 --2- 172.21.15.107:0/3177455129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b568e50 0x55570b566680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.109+0000 7feec9200640 1 -- 172.21.15.107:0/3177455129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 msgr2=0x55570b5810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:25.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.109+0000 7feec9200640 1 --2- 172.21.15.107:0/3177455129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 0x55570b5810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.109+0000 7feec9200640 1 -- 172.21.15.107:0/3177455129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b568e50 msgr2=0x55570b566680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:25.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.109+0000 7feec9200640 1 --2- 172.21.15.107:0/3177455129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b568e50 0x55570b566680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.109+0000 7feec9200640 1 -- 172.21.15.107:0/3177455129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570b55daa0 con 0x55570b55dc70 2024-09-19T04:29:25.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.109+0000 7feec9200640 1 --2- 172.21.15.107:0/3177455129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b564680 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feeb8013cc0 tx=0x7feeb8016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c37d3acb974c7a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:25.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.109+0000 7feeb5600640 1 -- 172.21.15.107:0/3177455129 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80295a0 con 0x55570b55dc70 2024-09-19T04:29:25.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.109+0000 7feeb5600640 1 -- 172.21.15.107:0/3177455129 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb802f070 con 0x55570b55dc70 2024-09-19T04:29:25.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.109+0000 7feeb5600640 1 -- 172.21.15.107:0/3177455129 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8020ad0 con 0x55570b55dc70 2024-09-19T04:29:25.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.110+0000 7feece6c0740 1 -- 172.21.15.107:0/3177455129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 msgr2=0x55570b564680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:25.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.110+0000 7feece6c0740 1 --2- 172.21.15.107:0/3177455129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b564680 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feeb8013cc0 tx=0x7feeb8016000 comp rx=0 tx=0).stop 2024-09-19T04:29:25.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.110+0000 7feece6c0740 1 -- 172.21.15.107:0/3177455129 shutdown_connections 2024-09-19T04:29:25.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.110+0000 7feece6c0740 1 --2- 172.21.15.107:0/3177455129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 0x55570b5810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.110+0000 7feece6c0740 1 --2- 172.21.15.107:0/3177455129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b568e50 0x55570b566680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.110+0000 7feece6c0740 1 --2- 172.21.15.107:0/3177455129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b55dc70 0x55570b564680 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.110+0000 7feece6c0740 1 -- 172.21.15.107:0/3177455129 >> 172.21.15.107:0/3177455129 conn(0x55570b55a200 msgr2=0x55570b567e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:25.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.110+0000 7feece6c0740 1 -- 172.21.15.107:0/3177455129 shutdown_connections 2024-09-19T04:29:25.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.110+0000 7feece6c0740 1 -- 172.21.15.107:0/3177455129 wait complete. 2024-09-19T04:29:25.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.110+0000 7feece6c0740 1 -- start start 2024-09-19T04:29:25.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.111+0000 7feece6c0740 1 Processor -- start 2024-09-19T04:29:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.111+0000 7feece6c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 0x55570b56ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.111+0000 7feece6c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b56d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.111+0000 7feece6c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 0x55570b56d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.111+0000 7feece6c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feec41acae0 con 0x55570b55dc70 2024-09-19T04:29:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.111+0000 7feece6c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feec41ac410 con 0x55570b568e50 2024-09-19T04:29:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.111+0000 7feece6c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feec41a9670 con 0x55570b580c90 2024-09-19T04:29:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.111+0000 7feece6c0740 10 client.0 _ll_register_callbacks cb 0x7feeb0009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.111+0000 7feec3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b56d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 0x55570b56ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b56d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:58758/0 (socket says 172.21.15.107:58758) 2024-09-19T04:29:25.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec3e00640 1 -- 172.21.15.107:0/1031127308 learned_addr learned my addr 172.21.15.107:0/1031127308 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:25.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec9c00640 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 0x55570b56d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec9200640 1 -- 172.21.15.107:0/1031127308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 msgr2=0x55570b56d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:25.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec9200640 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 0x55570b56d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec9200640 1 -- 172.21.15.107:0/1031127308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 msgr2=0x55570b56d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:25.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec9200640 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b56d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec9200640 1 -- 172.21.15.107:0/1031127308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570b555770 con 0x55570b55dc70 2024-09-19T04:29:25.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec3e00640 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b56d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:25.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.112+0000 7feec9c00640 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 0x55570b56d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:25.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.113+0000 7feec9200640 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 0x55570b56ccc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000c00 tx=0x7feeb80203f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1160de6a17d0dc1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:25.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.113+0000 7feeaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:29:25.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.113+0000 7feeaaa00640 1 -- 172.21.15.107:0/1031127308 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8005240 con 0x55570b55dc70 2024-09-19T04:29:25.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.113+0000 7feeaaa00640 1 -- 172.21.15.107:0/1031127308 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7feeb80053e0 con 0x55570b55dc70 2024-09-19T04:29:25.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.113+0000 7feece6c0740 1 -- 172.21.15.107:0/1031127308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb802f070 con 0x55570b55dc70 2024-09-19T04:29:25.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.113+0000 7feeaaa00640 1 -- 172.21.15.107:0/1031127308 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feeb80055e0 con 0x55570b55dc70 2024-09-19T04:29:25.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.113+0000 7feece6c0740 10 client.16833 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:25.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.113+0000 7feece6c0740 1 -- 172.21.15.107:0/1031127308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55570b5e3e90 con 0x55570b55dc70 2024-09-19T04:29:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.114+0000 7feeaaa00640 1 -- 172.21.15.107:0/1031127308 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb802f070 con 0x55570b55dc70 2024-09-19T04:29:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.114+0000 7feeaaa00640 1 -- 172.21.15.107:0/1031127308 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7feeb801d040 con 0x55570b55dc70 2024-09-19T04:29:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.114+0000 7feece6c0740 10 client.16833 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.114+0000 7feece6c0740 10 client.16833 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:29:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.114+0000 7feece6c0740 20 client.16833 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.114+0000 7feece6c0740 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55570b5e45b0 0x55570b5e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.114+0000 7feece6c0740 4 client.16833 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:29:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.114+0000 7feece6c0740 1 -- 172.21.15.107:0/1031127308 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55570b4cbe60 con 0x55570b5e45b0 2024-09-19T04:29:25.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.114+0000 7feec3e00640 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55570b5e45b0 0x55570b5e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:25.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feec3e00640 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55570b5e45b0 0x55570b5e69d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eadc0dd27eb27964 server_cookie=db80442995f8a3c9 in_seq=0 out_seq=0 2024-09-19T04:29:25.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feeaaa00640 10 client.16833 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:29:25.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feeaaa00640 1 -- 172.21.15.107:0/1031127308 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55570b4cbe60 con 0x55570b5e45b0 2024-09-19T04:29:25.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feeaaa00640 10 client.16833 handle_command_reply: tid=0 2024-09-19T04:29:25.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feece6c0740 1 client.16833 shutdown 2024-09-19T04:29:25.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feece6c0740 1 -- 172.21.15.107:0/1031127308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 msgr2=0x55570b56ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:25.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feece6c0740 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 0x55570b56ccc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000c00 tx=0x7feeb80203f0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feece6c0740 1 -- 172.21.15.107:0/1031127308 shutdown_connections 2024-09-19T04:29:25.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feece6c0740 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55570b5e45b0 0x55570b5e69d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feece6c0740 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55570b580c90 0x55570b56d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feece6c0740 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55570b568e50 0x55570b56d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feeaaa00640 0 client.16833 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:29:25.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feece6c0740 1 --2- 172.21.15.107:0/1031127308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55570b55dc70 0x55570b56ccc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.115+0000 7feece6c0740 1 -- 172.21.15.107:0/1031127308 >> 172.21.15.107:0/1031127308 conn(0x55570b55a200 msgr2=0x55570b5804a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:25.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.116+0000 7feece6c0740 1 -- 172.21.15.107:0/1031127308 shutdown_connections 2024-09-19T04:29:25.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.116+0000 7feece6c0740 1 -- 172.21.15.107:0/1031127308 wait complete. 2024-09-19T04:29:25.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.116+0000 7feece6c0740 20 client.16833 trim_cache size 0 max 16384 2024-09-19T04:29:25.126 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:29:25.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 -- 172.21.15.107:0/2501124141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feeac0a33b0 msgr2=0x7feeac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:25.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feeac0a33b0 0x7feeac0a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55570b55b060 tx=0x7feeb8005f90 comp rx=0 tx=0).stop 2024-09-19T04:29:25.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 -- 172.21.15.107:0/2501124141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101270 msgr2=0x7feec419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:25.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101270 0x7feec419ac00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feeb000e9f0 tx=0x7feeb000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 -- 172.21.15.107:0/2501124141 shutdown_connections 2024-09-19T04:29:25.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feeac0a33b0 0x7feeac0a5870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feec410e470 0x7feec419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101270 0x7feec419ac00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 --2- 172.21.15.107:0/2501124141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feec41008c0 0x7feec419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:25.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 -- 172.21.15.107:0/2501124141 >> 172.21.15.107:0/2501124141 conn(0x7feec40fc790 msgr2=0x7feec40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:25.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 -- 172.21.15.107:0/2501124141 shutdown_connections 2024-09-19T04:29:25.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:25.117+0000 7feecbc00640 1 -- 172.21.15.107:0/2501124141 wait complete. 2024-09-19T04:29:25.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:24 smithi115 ceph-mon[35477]: pgmap v281: 97 pgs: 97 active+clean; 212 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-19T04:29:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:24 smithi145 ceph-mon[35096]: pgmap v281: 97 pgs: 97 active+clean; 212 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-19T04:29:25.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 291446 2.56 MB/sec execute 343 sec latency 10.403 ms 2024-09-19T04:29:26.002 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:29:26.412 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:26.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 292567 2.57 MB/sec execute 344 sec latency 125.217 ms 2024-09-19T04:29:26.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.990+0000 7f7cba000640 1 -- 172.21.15.107:0/2985781770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb4102340 msgr2=0x7f7cb410e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:26.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.990+0000 7f7cba000640 1 --2- 172.21.15.107:0/2985781770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb4102340 0x7f7cb410e870 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009ca0 tx=0x7f7cac02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:29:26.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.990+0000 7f7cba000640 1 -- 172.21.15.107:0/2985781770 shutdown_connections 2024-09-19T04:29:26.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.990+0000 7f7cba000640 1 --2- 172.21.15.107:0/2985781770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb4102340 0x7f7cb410e870 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:26.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.990+0000 7f7cba000640 1 --2- 172.21.15.107:0/2985781770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb41019a0 0x7f7cb4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:26.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.990+0000 7f7cba000640 1 --2- 172.21.15.107:0/2985781770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb41079a0 0x7f7cb4107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:26.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.990+0000 7f7cba000640 1 -- 172.21.15.107:0/2985781770 >> 172.21.15.107:0/2985781770 conn(0x7f7cb40fd700 msgr2=0x7f7cb40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:26.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.990+0000 7f7cba000640 1 -- 172.21.15.107:0/2985781770 shutdown_connections 2024-09-19T04:29:26.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.990+0000 7f7cba000640 1 -- 172.21.15.107:0/2985781770 wait complete. 2024-09-19T04:29:26.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.991+0000 7f7cba000640 1 Processor -- start 2024-09-19T04:29:26.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.991+0000 7f7cba000640 1 -- start start 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cba000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb41019a0 0x7f7cb40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cba000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb4102340 0x7f7cb4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cba000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb41079a0 0x7f7cb4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cba000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4111df0 con 0x7f7cb41079a0 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cba000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4111c70 con 0x7f7cb4102340 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cba000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4111f70 con 0x7f7cb41019a0 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cb9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb41079a0 0x7f7cb4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cb3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb4102340 0x7f7cb4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cb8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb41019a0 0x7f7cb40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cb9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb41079a0 0x7f7cb4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38918/0 (socket says 172.21.15.107:38918) 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cb9600640 1 -- 172.21.15.107:0/3894882399 learned_addr learned my addr 172.21.15.107:0/3894882399 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:26.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cb8c00640 1 -- 172.21.15.107:0/3894882399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb4102340 msgr2=0x7f7cb4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cb8c00640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb4102340 0x7f7cb4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cb8c00640 1 -- 172.21.15.107:0/3894882399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb41079a0 msgr2=0x7f7cb4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cb8c00640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb41079a0 0x7f7cb4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.992+0000 7f7cb8c00640 1 -- 172.21.15.107:0/3894882399 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb41a66b0 con 0x7f7cb41019a0 2024-09-19T04:29:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.993+0000 7f7cb8c00640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb41019a0 0x7f7cb40744f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800e9f0 tx=0x7f7ca800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.993+0000 7f7cb1600640 1 -- 172.21.15.107:0/3894882399 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800ccc0 con 0x7f7cb41019a0 2024-09-19T04:29:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.993+0000 7f7cba000640 1 -- 172.21.15.107:0/3894882399 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb41a69a0 con 0x7f7cb41019a0 2024-09-19T04:29:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.993+0000 7f7cb1600640 1 -- 172.21.15.107:0/3894882399 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca8010910 con 0x7f7cb41019a0 2024-09-19T04:29:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.993+0000 7f7cb1600640 1 -- 172.21.15.107:0/3894882399 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8018960 con 0x7f7cb41019a0 2024-09-19T04:29:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.993+0000 7f7cba000640 1 -- 172.21.15.107:0/3894882399 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb41a6ee0 con 0x7f7cb41019a0 2024-09-19T04:29:26.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.996+0000 7f7cb1600640 1 -- 172.21.15.107:0/3894882399 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f7ca8010ac0 con 0x7f7cb41019a0 2024-09-19T04:29:26.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.996+0000 7f7cbe1fa740 1 Processor -- start 2024-09-19T04:29:26.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.996+0000 7f7cbe1fa740 1 -- start start 2024-09-19T04:29:26.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc429220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:26.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 0x560bcc433ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:26.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc434020 0x560bcc436410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:26.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560bcc38f3e0 con 0x560bcc428e40 2024-09-19T04:29:26.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560bcc336d80 con 0x560bcc434020 2024-09-19T04:29:26.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560bcc38b3a0 con 0x560bcc429760 2024-09-19T04:29:27.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cb9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc429220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cb8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 0x560bcc433ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cb9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc429220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38930/0 (socket says 172.21.15.107:38930) 2024-09-19T04:29:27.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cb9600640 1 -- 172.21.15.107:0/2996880297 learned_addr learned my addr 172.21.15.107:0/2996880297 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:27.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.997+0000 7f7cb3e00640 1 --2- 172.21.15.107:0/2996880297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc434020 0x560bcc436410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.998+0000 7f7cb9600640 1 -- 172.21.15.107:0/2996880297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 msgr2=0x560bcc433ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:27.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.998+0000 7f7cb9600640 1 --2- 172.21.15.107:0/2996880297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 0x560bcc433ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.998+0000 7f7cb9600640 1 -- 172.21.15.107:0/2996880297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc434020 msgr2=0x560bcc436410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:27.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.998+0000 7f7cb9600640 1 --2- 172.21.15.107:0/2996880297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc434020 0x560bcc436410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.998+0000 7f7cb9600640 1 -- 172.21.15.107:0/2996880297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcc355ae0 con 0x560bcc428e40 2024-09-19T04:29:27.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.998+0000 7f7cb9600640 1 --2- 172.21.15.107:0/2996880297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc429220 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7cac005fb0 tx=0x7f7cac009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dbd0f119780b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:27.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.998+0000 7f7ca5c00640 1 -- 172.21.15.107:0/2996880297 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac031da0 con 0x560bcc428e40 2024-09-19T04:29:27.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.998+0000 7f7ca5c00640 1 -- 172.21.15.107:0/2996880297 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac036030 con 0x560bcc428e40 2024-09-19T04:29:27.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.998+0000 7f7ca5c00640 1 -- 172.21.15.107:0/2996880297 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac004310 con 0x560bcc428e40 2024-09-19T04:29:27.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.999+0000 7f7cb1600640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f7c9c0a33b0 0x7f7c9c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.999+0000 7f7cb1600640 1 -- 172.21.15.107:0/3894882399 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f7ca80d0080 con 0x7f7cb41019a0 2024-09-19T04:29:27.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.999+0000 7f7cb8c00640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f7c9c0a33b0 0x7f7c9c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.999+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2996880297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 msgr2=0x560bcc429220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:27.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.999+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/2996880297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc429220 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7cac005fb0 tx=0x7f7cac009a10 comp rx=0 tx=0).stop 2024-09-19T04:29:27.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.999+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2996880297 shutdown_connections 2024-09-19T04:29:27.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.999+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/2996880297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc434020 0x560bcc436410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.999+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/2996880297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 0x560bcc433ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.999+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/2996880297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc429220 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:26.999+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2996880297 >> 172.21.15.107:0/2996880297 conn(0x560bcc4253d0 msgr2=0x560bcc4257b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.000+0000 7f7cb8c00640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f7c9c0a33b0 0x7f7c9c0a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x560bcc426230 tx=0x7f7ca80d3b30 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.000+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2996880297 shutdown_connections 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.000+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2996880297 wait complete. 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.000+0000 7f7cbe1fa740 1 -- start start 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.000+0000 7f7cbe1fa740 1 Processor -- start 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 0x560bcc430e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 0x560bcc4313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc434020 0x560bcc431900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560bcc436a70 con 0x560bcc434020 2024-09-19T04:29:27.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560bcc34a0f0 con 0x560bcc428e40 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560bcc436bf0 con 0x560bcc429760 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cbe1fa740 10 client.0 _ll_register_callbacks cb 0x560bcc331bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cb3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 0x560bcc4313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cb3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 0x560bcc4313c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41992/0 (socket says 172.21.15.107:41992) 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cb9600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 0x560bcc430e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cb3e00640 1 -- 172.21.15.107:0/4044616293 learned_addr learned my addr 172.21.15.107:0/4044616293 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cb9600640 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc434020 0x560bcc431900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cb3e00640 1 -- 172.21.15.107:0/4044616293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 msgr2=0x560bcc430e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.001+0000 7f7cb3e00640 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 0x560bcc430e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7cb3e00640 1 -- 172.21.15.107:0/4044616293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc434020 msgr2=0x560bcc431900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7cb3e00640 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc434020 0x560bcc431900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7cb3e00640 1 -- 172.21.15.107:0/4044616293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcc1003c0 con 0x560bcc429760 2024-09-19T04:29:27.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7cb9600640 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 0x560bcc430e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:27.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7cb3e00640 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 0x560bcc4313c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0012700 tx=0x7f7ca0012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41dfc9de8efe7ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:27.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7c9a000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:29:27.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7c9a000640 1 -- 172.21.15.107:0/4044616293 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000f8c0 con 0x560bcc429760 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/4044616293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cac036030 con 0x560bcc429760 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7c9a000640 1 -- 172.21.15.107:0/4044616293 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f7ca000fa60 con 0x560bcc429760 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7c9a000640 1 -- 172.21.15.107:0/4044616293 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7ca000fc60 con 0x560bcc429760 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7cbe1fa740 10 client.26857 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.002+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/4044616293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560bcc442a90 con 0x560bcc429760 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.003+0000 7f7c9a000640 1 -- 172.21.15.107:0/4044616293 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac036030 con 0x560bcc429760 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.003+0000 7f7c9a000640 1 -- 172.21.15.107:0/4044616293 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f7ca0010430 con 0x560bcc429760 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.003+0000 7f7cbe1fa740 10 client.26857 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.003+0000 7f7cbe1fa740 10 client.26857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.003+0000 7f7cbe1fa740 20 client.26857 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.003+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560bcc443ff0 0x560bcc446410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.003+0000 7f7cbe1fa740 4 client.26857 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:27.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.003+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/4044616293 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bcc39c950 con 0x560bcc443ff0 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.003+0000 7f7cb9600640 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560bcc443ff0 0x560bcc446410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.004+0000 7f7cb9600640 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560bcc443ff0 0x560bcc446410 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f0b61d92c5dab67 server_cookie=3252ee86fc08bef1 in_seq=0 out_seq=0 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.004+0000 7f7c9a000640 10 client.26857 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.004+0000 7f7c9a000640 1 -- 172.21.15.107:0/4044616293 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bcc39c950 con 0x560bcc443ff0 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.004+0000 7f7c9a000640 10 client.26857 handle_command_reply: tid=0 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.004+0000 7f7cbe1fa740 1 client.26857 shutdown 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/4044616293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 msgr2=0x560bcc4313c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 0x560bcc4313c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0012700 tx=0x7f7ca0012bc0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/4044616293 shutdown_connections 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560bcc443ff0 0x560bcc446410 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc434020 0x560bcc431900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc429760 0x560bcc4313c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/4044616293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 0x560bcc430e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/4044616293 >> 172.21.15.107:0/4044616293 conn(0x560bcc4253d0 msgr2=0x560bcc4244b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:27.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7c9a000640 0 client.26857 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:27.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/4044616293 shutdown_connections 2024-09-19T04:29:27.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/4044616293 wait complete. 2024-09-19T04:29:27.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.005+0000 7f7cbe1fa740 20 client.26857 trim_cache size 0 max 16384 2024-09-19T04:29:27.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.018+0000 7f7cbe1fa740 1 Processor -- start 2024-09-19T04:29:27.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cbe1fa740 1 -- start start 2024-09-19T04:29:27.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 0x560bcc44bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc429760 0x560bcc44c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 0x560bcc42e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41af7c0 con 0x560bcc429760 2024-09-19T04:29:27.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41b1300 con 0x560bcc428e40 2024-09-19T04:29:27.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41afe90 con 0x560bcc434020 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cb3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc429760 0x560bcc44c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cb3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc429760 0x560bcc44c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38950/0 (socket says 172.21.15.107:38950) 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cb3e00640 1 -- 172.21.15.107:0/1966853156 learned_addr learned my addr 172.21.15.107:0/1966853156 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cb9600640 1 --2- 172.21.15.107:0/1966853156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 0x560bcc44bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.019+0000 7f7cb9600640 1 --2- 172.21.15.107:0/1966853156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 0x560bcc42e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.020+0000 7f7cb3e00640 1 -- 172.21.15.107:0/1966853156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 msgr2=0x560bcc42e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.020+0000 7f7cb3e00640 1 --2- 172.21.15.107:0/1966853156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 0x560bcc42e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.020+0000 7f7cb3e00640 1 -- 172.21.15.107:0/1966853156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 msgr2=0x560bcc44bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.020+0000 7f7cb3e00640 1 --2- 172.21.15.107:0/1966853156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 0x560bcc44bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.020+0000 7f7cb3e00640 1 -- 172.21.15.107:0/1966853156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcc42bcb0 con 0x560bcc429760 2024-09-19T04:29:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.020+0000 7f7cb9600640 1 --2- 172.21.15.107:0/1966853156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 0x560bcc44bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:27.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.020+0000 7f7cb3e00640 1 --2- 172.21.15.107:0/1966853156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc429760 0x560bcc44c200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0012700 tx=0x7f7ca000ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606e5f9fdad7edc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:27.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.020+0000 7f7ca5200640 1 -- 172.21.15.107:0/1966853156 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0029070 con 0x560bcc429760 2024-09-19T04:29:27.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.020+0000 7f7ca5200640 1 -- 172.21.15.107:0/1966853156 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca0010b10 con 0x560bcc429760 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.020+0000 7f7ca5200640 1 -- 172.21.15.107:0/1966853156 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca001c950 con 0x560bcc429760 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.021+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/1966853156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc429760 msgr2=0x560bcc44c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.021+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/1966853156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc429760 0x560bcc44c200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0012700 tx=0x7f7ca000ff90 comp rx=0 tx=0).stop 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.021+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/1966853156 shutdown_connections 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.021+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/1966853156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 0x560bcc42e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.021+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/1966853156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc429760 0x560bcc44c200 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.021+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/1966853156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc428e40 0x560bcc44bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.021+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/1966853156 >> 172.21.15.107:0/1966853156 conn(0x560bcc4253d0 msgr2=0x560bcc4244b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.021+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/1966853156 shutdown_connections 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.021+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/1966853156 wait complete. 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.021+0000 7f7cbe1fa740 1 -- start start 2024-09-19T04:29:27.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cbe1fa740 1 Processor -- start 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc42deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc429760 0x560bcc42e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cbe1fa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 0x560bcc42e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41b0c30 con 0x560bcc428e40 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41b0560 con 0x560bcc429760 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cbe1fa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41ad7c0 con 0x560bcc434020 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cbe1fa740 10 client.0 _ll_register_callbacks cb 0x7f7ca0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cb9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc42deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cb3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc429760 0x560bcc42e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cb9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc42deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38952/0 (socket says 172.21.15.107:38952) 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cb9600640 1 -- 172.21.15.107:0/2396733531 learned_addr learned my addr 172.21.15.107:0/2396733531 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.022+0000 7f7cb9600640 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 0x560bcc42e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7cb3e00640 1 -- 172.21.15.107:0/2396733531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 msgr2=0x560bcc42e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:27.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7cb3e00640 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 0x560bcc42e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7cb3e00640 1 -- 172.21.15.107:0/2396733531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 msgr2=0x560bcc42deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7cb3e00640 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc42deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7cb3e00640 1 -- 172.21.15.107:0/2396733531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcc4adc90 con 0x560bcc429760 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7cb9600640 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 0x560bcc42e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7cb3e00640 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc429760 0x560bcc42e3f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000a010 tx=0x7f7ca0005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de93f9738f6fec55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7c9a000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7c9a000640 1 -- 172.21.15.107:0/2396733531 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0029070 con 0x560bcc429760 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7c9a000640 1 -- 172.21.15.107:0/2396733531 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f7ca00050e0 con 0x560bcc429760 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2396733531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca0010b10 con 0x560bcc429760 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7c9a000640 1 -- 172.21.15.107:0/2396733531 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7ca0005300 con 0x560bcc429760 2024-09-19T04:29:27.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7cbe1fa740 10 client.26897 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:27.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.023+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2396733531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560bcc4aef50 con 0x560bcc429760 2024-09-19T04:29:27.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7c9a000640 1 -- 172.21.15.107:0/2396733531 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca0010b10 con 0x560bcc429760 2024-09-19T04:29:27.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7c9a000640 1 -- 172.21.15.107:0/2396733531 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f7ca0035030 con 0x560bcc429760 2024-09-19T04:29:27.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7cbe1fa740 10 client.26897 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:27.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7cbe1fa740 10 client.26897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:27.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7cbe1fa740 20 client.26897 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:27.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560bcc4af550 0x560bcc4b1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:27.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7cbe1fa740 4 client.26897 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:29:27.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2396733531 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560bcc396e60 con 0x560bcc4af550 2024-09-19T04:29:27.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7cb9600640 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560bcc4af550 0x560bcc4b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:27.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7cb9600640 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560bcc4af550 0x560bcc4b1940 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed9686f4b4bff46 server_cookie=49cdfab1bcd49e1c in_seq=0 out_seq=0 2024-09-19T04:29:27.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.024+0000 7f7c9a000640 10 client.26897 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7c9a000640 1 -- 172.21.15.107:0/2396733531 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560bcc396e60 con 0x560bcc4af550 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7c9a000640 10 client.26897 handle_command_reply: tid=0 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 client.26897 shutdown 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2396733531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc429760 msgr2=0x560bcc42e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc429760 0x560bcc42e3f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000a010 tx=0x7f7ca0005fa0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2396733531 shutdown_connections 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x560bcc4af550 0x560bcc4b1940 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560bcc434020 0x560bcc42e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560bcc429760 0x560bcc42e3f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 --2- 172.21.15.107:0/2396733531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bcc428e40 0x560bcc42deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2396733531 >> 172.21.15.107:0/2396733531 conn(0x560bcc4253d0 msgr2=0x560bcc42f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7c9a000640 0 client.26897 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2396733531 shutdown_connections 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 1 -- 172.21.15.107:0/2396733531 wait complete. 2024-09-19T04:29:27.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.025+0000 7f7cbe1fa740 20 client.26897 trim_cache size 0 max 16384 2024-09-19T04:29:27.032 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 -- 172.21.15.107:0/3894882399 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f7c9c0a33b0 msgr2=0x7f7c9c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f7c9c0a33b0 0x7f7c9c0a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x560bcc426230 tx=0x7f7ca80d3b30 comp rx=0 tx=0).stop 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 -- 172.21.15.107:0/3894882399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb41019a0 msgr2=0x7f7cb40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb41019a0 0x7f7cb40744f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800e9f0 tx=0x7f7ca800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 -- 172.21.15.107:0/3894882399 shutdown_connections 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f7c9c0a33b0 0x7f7c9c0a5870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb41079a0 0x7f7cb4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb4102340 0x7f7cb4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 --2- 172.21.15.107:0/3894882399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb41019a0 0x7f7cb40744f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 -- 172.21.15.107:0/3894882399 >> 172.21.15.107:0/3894882399 conn(0x7f7cb40fd700 msgr2=0x7f7cb40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 -- 172.21.15.107:0/3894882399 shutdown_connections 2024-09-19T04:29:27.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:27.026+0000 7f7cba000640 1 -- 172.21.15.107:0/3894882399 wait complete. 2024-09-19T04:29:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:26 smithi115 ceph-mon[35477]: pgmap v282: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-09-19T04:29:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:26 smithi145 ceph-mon[35096]: pgmap v282: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-09-19T04:29:27.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:26 smithi107 ceph-mon[30659]: pgmap v282: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-09-19T04:29:27.704 INFO:tasks.workunit.client.0.smithi107.stdout: 1 292567 2.56 MB/sec execute 345 sec latency 1125.294 ms 2024-09-19T04:29:28.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 292567 2.56 MB/sec execute 346 sec latency 2132.640 ms 2024-09-19T04:29:28.890 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:29:28.891 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:29:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:28 smithi115 ceph-mon[35477]: pgmap v283: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-19T04:29:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:28 smithi145 ceph-mon[35096]: pgmap v283: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-19T04:29:29.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:28 smithi107 ceph-mon[30659]: pgmap v283: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-19T04:29:29.286 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:29.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 293896 2.57 MB/sec execute 347 sec latency 2464.898 ms 2024-09-19T04:29:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.908+0000 7f95efe00640 1 -- 172.21.15.107:0/405711134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f01034b0 msgr2=0x7f95f010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.908+0000 7f95efe00640 1 --2- 172.21.15.107:0/405711134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f01034b0 0x7f95f010b9a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009860 tx=0x7f95dc02a3d0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.908+0000 7f95efe00640 1 -- 172.21.15.107:0/405711134 shutdown_connections 2024-09-19T04:29:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.908+0000 7f95efe00640 1 --2- 172.21.15.107:0/405711134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f010bfc0 0x7f95f010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.908+0000 7f95efe00640 1 --2- 172.21.15.107:0/405711134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f01034b0 0x7f95f010b9a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.908+0000 7f95efe00640 1 --2- 172.21.15.107:0/405711134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f0102b90 0x7f95f0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.908+0000 7f95efe00640 1 -- 172.21.15.107:0/405711134 >> 172.21.15.107:0/405711134 conn(0x7f95f00fc610 msgr2=0x7f95f00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.908+0000 7f95efe00640 1 -- 172.21.15.107:0/405711134 shutdown_connections 2024-09-19T04:29:29.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.908+0000 7f95efe00640 1 -- 172.21.15.107:0/405711134 wait complete. 2024-09-19T04:29:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.909+0000 7f95efe00640 1 Processor -- start 2024-09-19T04:29:29.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.909+0000 7f95efe00640 1 -- start start 2024-09-19T04:29:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.910+0000 7f95efe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f0102b90 0x7f95f019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.910+0000 7f95efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f01034b0 0x7f95f019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.910+0000 7f95efe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f010bfc0 0x7f95f019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.910+0000 7f95efe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0111e10 con 0x7f95f01034b0 2024-09-19T04:29:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.910+0000 7f95efe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0111c90 con 0x7f95f010bfc0 2024-09-19T04:29:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.910+0000 7f95efe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0111f90 con 0x7f95f0102b90 2024-09-19T04:29:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.910+0000 7f95ef400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f010bfc0 0x7f95f019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.910+0000 7f95ef400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f010bfc0 0x7f95f019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51306/0 (socket says 172.21.15.107:51306) 2024-09-19T04:29:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.910+0000 7f95ef400640 1 -- 172.21.15.107:0/3892365132 learned_addr learned my addr 172.21.15.107:0/3892365132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:29.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.910+0000 7f95ee000640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f01034b0 0x7f95f019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95ef400640 1 -- 172.21.15.107:0/3892365132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f0102b90 msgr2=0x7f95f019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:29.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95eea00640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f0102b90 0x7f95f019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95ef400640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f0102b90 0x7f95f019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95ef400640 1 -- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f01034b0 msgr2=0x7f95f019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95ef400640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f01034b0 0x7f95f019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95ef400640 1 -- 172.21.15.107:0/3892365132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f019f4f0 con 0x7f95f010bfc0 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95ee000640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f01034b0 0x7f95f019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95eea00640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f0102b90 0x7f95f019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95ef400640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f010bfc0 0x7f95f019ee10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009fd0 tx=0x7f95e800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95f4a00640 1 -- 172.21.15.107:0/3892365132 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8004b20 con 0x7f95f010bfc0 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.911+0000 7f95efe00640 1 -- 172.21.15.107:0/3892365132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f019f7e0 con 0x7f95f010bfc0 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.912+0000 7f95efe00640 1 -- 172.21.15.107:0/3892365132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f01a70c0 con 0x7f95f010bfc0 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.912+0000 7f95f4a00640 1 -- 172.21.15.107:0/3892365132 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e8003e50 con 0x7f95f010bfc0 2024-09-19T04:29:29.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.912+0000 7f95f4a00640 1 -- 172.21.15.107:0/3892365132 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e800b630 con 0x7f95f010bfc0 2024-09-19T04:29:29.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.916+0000 7f95f4a00640 1 -- 172.21.15.107:0/3892365132 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f95e801d020 con 0x7f95f010bfc0 2024-09-19T04:29:29.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.916+0000 7f95f8a65740 1 Processor -- start 2024-09-19T04:29:29.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95f8a65740 1 -- start start 2024-09-19T04:29:29.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33c54c0 0x5574f33c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c5de0 0x5574f33d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33d06a0 0x5574f33d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95f8a65740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5574f3323290 con 0x5574f33c5de0 2024-09-19T04:29:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95f8a65740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574f32cab40 con 0x5574f33c54c0 2024-09-19T04:29:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95f8a65740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5574f331f250 con 0x5574f33d06a0 2024-09-19T04:29:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95ee000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33c54c0 0x5574f33c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95eea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c5de0 0x5574f33d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95eea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c5de0 0x5574f33d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45244/0 (socket says 172.21.15.107:45244) 2024-09-19T04:29:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.917+0000 7f95eea00640 1 -- 172.21.15.107:0/161132602 learned_addr learned my addr 172.21.15.107:0/161132602 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.918+0000 7f95ee000640 1 --2- 172.21.15.107:0/161132602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33d06a0 0x5574f33d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.918+0000 7f95eea00640 1 -- 172.21.15.107:0/161132602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33d06a0 msgr2=0x5574f33d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.918+0000 7f95eea00640 1 --2- 172.21.15.107:0/161132602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33d06a0 0x5574f33d2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.918+0000 7f95eea00640 1 -- 172.21.15.107:0/161132602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33c54c0 msgr2=0x5574f33c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.918+0000 7f95eea00640 1 --2- 172.21.15.107:0/161132602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33c54c0 0x5574f33c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.918+0000 7f95eea00640 1 -- 172.21.15.107:0/161132602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f30946a0 con 0x5574f33c5de0 2024-09-19T04:29:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.918+0000 7f95f4a00640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d40a34b0 0x7f95d40a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.919+0000 7f95ee000640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d40a34b0 0x7f95d40a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.919+0000 7f95f4a00640 1 -- 172.21.15.107:0/3892365132 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f95e80cfc00 con 0x7f95f010bfc0 2024-09-19T04:29:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.919+0000 7f95eea00640 1 --2- 172.21.15.107:0/161132602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c5de0 0x5574f33d0160 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009c30 tx=0x7f95e0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13a24799b7c0e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.920+0000 7f95ee000640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d40a34b0 0x7f95d40a5970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f95f01028c0 tx=0x7f95dc030c50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.920+0000 7f95dbe00640 1 -- 172.21.15.107:0/161132602 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0004020 con 0x5574f33c5de0 2024-09-19T04:29:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.920+0000 7f95dbe00640 1 -- 172.21.15.107:0/161132602 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e00041c0 con 0x5574f33c5de0 2024-09-19T04:29:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.920+0000 7f95dbe00640 1 -- 172.21.15.107:0/161132602 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e00126b0 con 0x5574f33c5de0 2024-09-19T04:29:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.920+0000 7f95f8a65740 1 -- 172.21.15.107:0/161132602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c5de0 msgr2=0x5574f33d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.920+0000 7f95f8a65740 1 --2- 172.21.15.107:0/161132602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c5de0 0x5574f33d0160 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009c30 tx=0x7f95e0010740 comp rx=0 tx=0).stop 2024-09-19T04:29:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.921+0000 7f95f8a65740 1 -- 172.21.15.107:0/161132602 shutdown_connections 2024-09-19T04:29:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.921+0000 7f95f8a65740 1 --2- 172.21.15.107:0/161132602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33d06a0 0x5574f33d2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.921+0000 7f95f8a65740 1 --2- 172.21.15.107:0/161132602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c5de0 0x5574f33d0160 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.921+0000 7f95f8a65740 1 --2- 172.21.15.107:0/161132602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33c54c0 0x5574f33c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.921+0000 7f95f8a65740 1 -- 172.21.15.107:0/161132602 >> 172.21.15.107:0/161132602 conn(0x5574f33c1a50 msgr2=0x5574f33c1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.921+0000 7f95f8a65740 1 -- 172.21.15.107:0/161132602 shutdown_connections 2024-09-19T04:29:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.921+0000 7f95f8a65740 1 -- 172.21.15.107:0/161132602 wait complete. 2024-09-19T04:29:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.922+0000 7f95f8a65740 1 -- start start 2024-09-19T04:29:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.922+0000 7f95f8a65740 1 Processor -- start 2024-09-19T04:29:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.923+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c54c0 0x5574f33cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.923+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.923+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.923+0000 7f95f8a65740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5574f33d3310 con 0x5574f33c54c0 2024-09-19T04:29:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.923+0000 7f95f8a65740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574f32ddfa0 con 0x5574f33d06a0 2024-09-19T04:29:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.923+0000 7f95f8a65740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5574f33d3490 con 0x5574f33c5de0 2024-09-19T04:29:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.923+0000 7f95f8a65740 10 client.0 _ll_register_callbacks cb 0x5574f3031ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.923+0000 7f95ef400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.924+0000 7f95ef400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51322/0 (socket says 172.21.15.107:51322) 2024-09-19T04:29:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.924+0000 7f95ee000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.924+0000 7f95ee000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59484/0 (socket says 172.21.15.107:59484) 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.924+0000 7f95ee000640 1 -- 172.21.15.107:0/3773990346 learned_addr learned my addr 172.21.15.107:0/3773990346 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.924+0000 7f95eea00640 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c54c0 0x5574f33cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.924+0000 7f95eea00640 1 -- 172.21.15.107:0/3773990346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 msgr2=0x5574f33cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.924+0000 7f95eea00640 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.924+0000 7f95eea00640 1 -- 172.21.15.107:0/3773990346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 msgr2=0x5574f33cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.924+0000 7f95eea00640 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.924+0000 7f95eea00640 1 -- 172.21.15.107:0/3773990346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f33cdea0 con 0x5574f33c54c0 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.925+0000 7f95ef400640 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.925+0000 7f95eea00640 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c54c0 0x5574f33cff00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f95e00093d0 tx=0x7f95e0005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c64cc8309ff387 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.925+0000 7f95d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:29:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.925+0000 7f95d8c00640 1 -- 172.21.15.107:0/3773990346 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e00035f0 con 0x5574f33c54c0 2024-09-19T04:29:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.925+0000 7f95d8c00640 1 -- 172.21.15.107:0/3773990346 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f95e0003790 con 0x5574f33c54c0 2024-09-19T04:29:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.925+0000 7f95f8a65740 1 -- 172.21.15.107:0/3773990346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e00041c0 con 0x5574f33c54c0 2024-09-19T04:29:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.926+0000 7f95d8c00640 1 -- 172.21.15.107:0/3773990346 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f95e0003990 con 0x5574f33c54c0 2024-09-19T04:29:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.926+0000 7f95f8a65740 10 client.16863 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.926+0000 7f95f8a65740 1 -- 172.21.15.107:0/3773990346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5574f33f01c0 con 0x5574f33c54c0 2024-09-19T04:29:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.927+0000 7f95d8c00640 1 -- 172.21.15.107:0/3773990346 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e00041c0 con 0x5574f33c54c0 2024-09-19T04:29:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.927+0000 7f95d8c00640 1 -- 172.21.15.107:0/3773990346 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f95e002c070 con 0x5574f33c54c0 2024-09-19T04:29:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.927+0000 7f95f8a65740 10 client.16863 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.927+0000 7f95f8a65740 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.927+0000 7f95f8a65740 20 client.16863 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.927+0000 7f95f8a65740 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574f33f1620 0x5574f33f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.927+0000 7f95f8a65740 4 client.16863 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.927+0000 7f95f8a65740 1 -- 172.21.15.107:0/3773990346 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574f3330800 con 0x5574f33f1620 2024-09-19T04:29:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.927+0000 7f95ee000640 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574f33f1620 0x5574f33f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.928+0000 7f95d8c00640 10 client.16863 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:29.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.928+0000 7f95ee000640 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574f33f1620 0x5574f33f3a10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13fbecd4b103cab9 server_cookie=4a7d377d9b9895c8 in_seq=0 out_seq=0 2024-09-19T04:29:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.929+0000 7f95d8c00640 1 -- 172.21.15.107:0/3773990346 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574f3330800 con 0x5574f33f1620 2024-09-19T04:29:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.929+0000 7f95d8c00640 10 client.16863 handle_command_reply: tid=0 2024-09-19T04:29:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.929+0000 7f95f8a65740 1 client.16863 shutdown 2024-09-19T04:29:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.930+0000 7f95f8a65740 1 -- 172.21.15.107:0/3773990346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c54c0 msgr2=0x5574f33cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:29.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.930+0000 7f95f8a65740 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c54c0 0x5574f33cff00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f95e00093d0 tx=0x7f95e0005900 comp rx=0 tx=0).stop 2024-09-19T04:29:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.930+0000 7f95f8a65740 1 -- 172.21.15.107:0/3773990346 shutdown_connections 2024-09-19T04:29:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.930+0000 7f95f8a65740 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574f33f1620 0x5574f33f3a10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.930+0000 7f95f8a65740 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.930+0000 7f95f8a65740 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.930+0000 7f95f8a65740 1 --2- 172.21.15.107:0/3773990346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33c54c0 0x5574f33cff00 secure :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f95e00093d0 tx=0x7f95e0005900 comp rx=0 tx=0).stop 2024-09-19T04:29:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.930+0000 7f95f8a65740 1 -- 172.21.15.107:0/3773990346 >> 172.21.15.107:0/3773990346 conn(0x5574f33c1a50 msgr2=0x5574f33c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.930+0000 7f95d8c00640 0 client.16863 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.931+0000 7f95f8a65740 1 -- 172.21.15.107:0/3773990346 shutdown_connections 2024-09-19T04:29:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.931+0000 7f95f8a65740 1 -- 172.21.15.107:0/3773990346 wait complete. 2024-09-19T04:29:29.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.931+0000 7f95f8a65740 20 client.16863 trim_cache size 0 max 16384 2024-09-19T04:29:29.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.953+0000 7f95f8a65740 1 Processor -- start 2024-09-19T04:29:29.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.953+0000 7f95f8a65740 1 -- start start 2024-09-19T04:29:29.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.953+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.953+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33f9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.953+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 0x5574f33ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.953+0000 7f95f8a65740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95d000e690 con 0x5574f33f9750 2024-09-19T04:29:29.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.953+0000 7f95f8a65740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95d00101d0 con 0x5574f33d06a0 2024-09-19T04:29:29.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.953+0000 7f95f8a65740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f95d000ed60 con 0x5574f33c5de0 2024-09-19T04:29:29.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ef400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 0x5574f33ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95eea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ee000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ee000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33f9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51334/0 (socket says 172.21.15.107:51334) 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ef400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 0x5574f33ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45272/0 (socket says 172.21.15.107:45272) 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95eea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59498/0 (socket says 172.21.15.107:59498) 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ee000640 1 -- 172.21.15.107:0/2182262356 learned_addr learned my addr 172.21.15.107:0/2182262356 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ee000640 1 -- 172.21.15.107:0/2182262356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 msgr2=0x5574f33cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ee000640 1 --2- 172.21.15.107:0/2182262356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ee000640 1 -- 172.21.15.107:0/2182262356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 msgr2=0x5574f33ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ee000640 1 --2- 172.21.15.107:0/2182262356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 0x5574f33ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ee000640 1 -- 172.21.15.107:0/2182262356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f33c5c10 con 0x5574f33d06a0 2024-09-19T04:29:29.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95eea00640 1 --2- 172.21.15.107:0/2182262356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:29.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.954+0000 7f95ef400640 1 --2- 172.21.15.107:0/2182262356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 0x5574f33ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:29.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.955+0000 7f95ee000640 1 --2- 172.21.15.107:0/2182262356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33f9210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f95dc000c00 tx=0x7f95dc03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6d63d5f1b725bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:29.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.955+0000 7f95db400640 1 -- 172.21.15.107:0/2182262356 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc05c070 con 0x5574f33d06a0 2024-09-19T04:29:29.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.955+0000 7f95db400640 1 -- 172.21.15.107:0/2182262356 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95dc0046c0 con 0x5574f33d06a0 2024-09-19T04:29:29.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.955+0000 7f95db400640 1 -- 172.21.15.107:0/2182262356 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc003830 con 0x5574f33d06a0 2024-09-19T04:29:29.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.956+0000 7f95f8a65740 1 -- 172.21.15.107:0/2182262356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 msgr2=0x5574f33f9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:29.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.956+0000 7f95f8a65740 1 --2- 172.21.15.107:0/2182262356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33f9210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f95dc000c00 tx=0x7f95dc03f000 comp rx=0 tx=0).stop 2024-09-19T04:29:29.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.956+0000 7f95f8a65740 1 -- 172.21.15.107:0/2182262356 shutdown_connections 2024-09-19T04:29:29.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.956+0000 7f95f8a65740 1 --2- 172.21.15.107:0/2182262356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 0x5574f33ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.956+0000 7f95f8a65740 1 --2- 172.21.15.107:0/2182262356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33f9210 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.956+0000 7f95f8a65740 1 --2- 172.21.15.107:0/2182262356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.956+0000 7f95f8a65740 1 -- 172.21.15.107:0/2182262356 >> 172.21.15.107:0/2182262356 conn(0x5574f33c1a50 msgr2=0x5574f33cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:29.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.956+0000 7f95f8a65740 1 -- 172.21.15.107:0/2182262356 shutdown_connections 2024-09-19T04:29:29.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.956+0000 7f95f8a65740 1 -- 172.21.15.107:0/2182262356 wait complete. 2024-09-19T04:29:29.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.957+0000 7f95f8a65740 1 -- start start 2024-09-19T04:29:29.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.957+0000 7f95f8a65740 1 Processor -- start 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.957+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33d65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.957+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.957+0000 7f95f8a65740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 0x5574f33d7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.958+0000 7f95f8a65740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95d000fb00 con 0x5574f33f9750 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.958+0000 7f95f8a65740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95d000f430 con 0x5574f33d06a0 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.958+0000 7f95f8a65740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f95d000c690 con 0x5574f33c5de0 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.958+0000 7f95f8a65740 10 client.0 _ll_register_callbacks cb 0x7f95e8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.958+0000 7f95eea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33d65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.958+0000 7f95eea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33d65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59512/0 (socket says 172.21.15.107:59512) 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.958+0000 7f95eea00640 1 -- 172.21.15.107:0/2331824446 learned_addr learned my addr 172.21.15.107:0/2331824446 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:29.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.958+0000 7f95ef400640 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 0x5574f33d7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.959+0000 7f95eea00640 1 -- 172.21.15.107:0/2331824446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 msgr2=0x5574f33d6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:29.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.959+0000 7f95ee000640 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.959+0000 7f95eea00640 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33d6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.959+0000 7f95eea00640 1 -- 172.21.15.107:0/2331824446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 msgr2=0x5574f33d7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:29.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.959+0000 7f95eea00640 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 0x5574f33d7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.959+0000 7f95eea00640 1 -- 172.21.15.107:0/2331824446 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f345b350 con 0x5574f33c5de0 2024-09-19T04:29:29.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.959+0000 7f95eea00640 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33d65f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5574f33cca10 tx=0x7f95e0005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=632c9da7ff943889 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:29.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.959+0000 7f95d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:29:29.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.959+0000 7f95d8c00640 1 -- 172.21.15.107:0/2331824446 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0032070 con 0x5574f33c5de0 2024-09-19T04:29:29.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.960+0000 7f95f8a65740 1 -- 172.21.15.107:0/2331824446 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95dc0046c0 con 0x5574f33c5de0 2024-09-19T04:29:29.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.960+0000 7f95d8c00640 1 -- 172.21.15.107:0/2331824446 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f95e001ad60 con 0x5574f33c5de0 2024-09-19T04:29:29.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.960+0000 7f95d8c00640 1 -- 172.21.15.107:0/2331824446 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f95e0009ab0 con 0x5574f33c5de0 2024-09-19T04:29:29.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.960+0000 7f95f8a65740 10 client.26878 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:29.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.960+0000 7f95f8a65740 1 -- 172.21.15.107:0/2331824446 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5574f345c630 con 0x5574f33c5de0 2024-09-19T04:29:29.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.961+0000 7f95d8c00640 1 -- 172.21.15.107:0/2331824446 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95dc0046c0 con 0x5574f33c5de0 2024-09-19T04:29:29.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.961+0000 7f95d8c00640 1 -- 172.21.15.107:0/2331824446 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f95e002c040 con 0x5574f33c5de0 2024-09-19T04:29:29.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.961+0000 7f95f8a65740 10 client.26878 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:29.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.961+0000 7f95f8a65740 10 client.26878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:29.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.961+0000 7f95f8a65740 20 client.26878 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:29.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.961+0000 7f95f8a65740 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574f345cc90 0x5574f345f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:29.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.961+0000 7f95f8a65740 4 client.26878 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:29:29.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.961+0000 7f95f8a65740 1 -- 172.21.15.107:0/2331824446 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5574f332ad10 con 0x5574f345cc90 2024-09-19T04:29:29.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.962+0000 7f95ee000640 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574f345cc90 0x5574f345f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:29.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.962+0000 7f95ee000640 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574f345cc90 0x5574f345f0b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a768b96efe48a1 server_cookie=b9db0249cb7d5339 in_seq=0 out_seq=0 2024-09-19T04:29:29.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.962+0000 7f95d8c00640 10 client.26878 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:29.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.963+0000 7f95d8c00640 1 -- 172.21.15.107:0/2331824446 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5574f332ad10 con 0x5574f345cc90 2024-09-19T04:29:29.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.963+0000 7f95d8c00640 10 client.26878 handle_command_reply: tid=0 2024-09-19T04:29:29.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95f8a65740 1 client.26878 shutdown 2024-09-19T04:29:29.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95f8a65740 1 -- 172.21.15.107:0/2331824446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 msgr2=0x5574f33d65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:29.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95f8a65740 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33d65f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5574f33cca10 tx=0x7f95e0005db0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95f8a65740 1 -- 172.21.15.107:0/2331824446 shutdown_connections 2024-09-19T04:29:29.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95f8a65740 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574f345cc90 0x5574f345f0b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95f8a65740 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574f33f9750 0x5574f33d7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95f8a65740 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f33d06a0 0x5574f33d6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95f8a65740 1 --2- 172.21.15.107:0/2331824446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574f33c5de0 0x5574f33d65f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95d8c00640 0 client.26878 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:29.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95f8a65740 1 -- 172.21.15.107:0/2331824446 >> 172.21.15.107:0/2331824446 conn(0x5574f33c1a50 msgr2=0x5574f33cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:29.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.964+0000 7f95f8a65740 1 -- 172.21.15.107:0/2331824446 shutdown_connections 2024-09-19T04:29:29.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.965+0000 7f95f8a65740 1 -- 172.21.15.107:0/2331824446 wait complete. 2024-09-19T04:29:29.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.965+0000 7f95f8a65740 20 client.26878 trim_cache size 0 max 16384 2024-09-19T04:29:29.968 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:29:29.968 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:29:29.968 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "0f4269e0-2e78-45a2-b541-2a8bd0c98982", 2024-09-19T04:29:29.968 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:29:29.968 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:29:29.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.965+0000 7f95efe00640 1 -- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d40a34b0 msgr2=0x7f95d40a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:29.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.965+0000 7f95efe00640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d40a34b0 0x7f95d40a5970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f95f01028c0 tx=0x7f95dc030c50 comp rx=0 tx=0).stop 2024-09-19T04:29:29.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.965+0000 7f95efe00640 1 -- 172.21.15.107:0/3892365132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f010bfc0 msgr2=0x7f95f019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:29.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.965+0000 7f95efe00640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f010bfc0 0x7f95f019ee10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009fd0 tx=0x7f95e800c800 comp rx=0 tx=0).stop 2024-09-19T04:29:29.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.966+0000 7f95efe00640 1 -- 172.21.15.107:0/3892365132 shutdown_connections 2024-09-19T04:29:29.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.966+0000 7f95efe00640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f95d40a34b0 0x7f95d40a5970 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.966+0000 7f95efe00640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f010bfc0 0x7f95f019ee10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.966+0000 7f95efe00640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f01034b0 0x7f95f019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.966+0000 7f95efe00640 1 --2- 172.21.15.107:0/3892365132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95f0102b90 0x7f95f019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:29.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.966+0000 7f95efe00640 1 -- 172.21.15.107:0/3892365132 >> 172.21.15.107:0/3892365132 conn(0x7f95f00fc610 msgr2=0x7f95f00fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:29.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.966+0000 7f95efe00640 1 -- 172.21.15.107:0/3892365132 shutdown_connections 2024-09-19T04:29:29.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:29.966+0000 7f95efe00640 1 -- 172.21.15.107:0/3892365132 wait complete. 2024-09-19T04:29:30.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 295215 2.58 MB/sec execute 348 sec latency 14.669 ms 2024-09-19T04:29:30.998 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:29:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: pgmap v284: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-19T04:29:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:29:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:30 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: pgmap v284: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-19T04:29:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:29:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:30 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: pgmap v284: 97 pgs: 97 active+clean; 216 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-19T04:29:31.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:30 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:31.402 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:29:31.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 296117 2.59 MB/sec execute 349 sec latency 301.655 ms 2024-09-19T04:29:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.990+0000 7f0527000640 1 -- 172.21.15.107:0/1871428993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05201079c0 msgr2=0x7f0520107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.990+0000 7f0527000640 1 --2- 172.21.15.107:0/1871428993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05201079c0 0x7f0520107da0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0514009ca0 tx=0x7f051402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:29:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.990+0000 7f0527000640 1 -- 172.21.15.107:0/1871428993 shutdown_connections 2024-09-19T04:29:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.990+0000 7f0527000640 1 --2- 172.21.15.107:0/1871428993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0520102360 0x7f052010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:31.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.990+0000 7f0527000640 1 --2- 172.21.15.107:0/1871428993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05201019c0 0x7f0520101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.990+0000 7f0527000640 1 --2- 172.21.15.107:0/1871428993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05201079c0 0x7f0520107da0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.990+0000 7f0527000640 1 -- 172.21.15.107:0/1871428993 >> 172.21.15.107:0/1871428993 conn(0x7f05200fd740 msgr2=0x7f05200ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.990+0000 7f0527000640 1 -- 172.21.15.107:0/1871428993 shutdown_connections 2024-09-19T04:29:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.990+0000 7f0527000640 1 -- 172.21.15.107:0/1871428993 wait complete. 2024-09-19T04:29:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.991+0000 7f0527000640 1 Processor -- start 2024-09-19T04:29:31.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.991+0000 7f0527000640 1 -- start start 2024-09-19T04:29:32.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.991+0000 7f0527000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05201019c0 0x7f052019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.991+0000 7f0527000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0520102360 0x7f052019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.991+0000 7f0527000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05201079c0 0x7f052019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0527000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0520111e10 con 0x7f05201019c0 2024-09-19T04:29:32.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0527000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0520111c90 con 0x7f0520102360 2024-09-19T04:29:32.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0527000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0520111f90 con 0x7f05201079c0 2024-09-19T04:29:32.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0525c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05201019c0 0x7f052019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0525c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05201019c0 0x7f052019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45296/0 (socket says 172.21.15.107:45296) 2024-09-19T04:29:32.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0525c00640 1 -- 172.21.15.107:0/1524028215 learned_addr learned my addr 172.21.15.107:0/1524028215 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:32.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0525200640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0520102360 0x7f052019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0526600640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05201079c0 0x7f052019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0525c00640 1 -- 172.21.15.107:0/1524028215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05201079c0 msgr2=0x7f052019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:32.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0525c00640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05201079c0 0x7f052019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0525c00640 1 -- 172.21.15.107:0/1524028215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0520102360 msgr2=0x7f052019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0525c00640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0520102360 0x7f052019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0525c00640 1 -- 172.21.15.107:0/1524028215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052019f460 con 0x7f05201019c0 2024-09-19T04:29:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.992+0000 7f0525200640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0520102360 0x7f052019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.993+0000 7f0525c00640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05201019c0 0x7f052019a4b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0514005e30 tx=0x7f0514003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:32.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.993+0000 7f051e000640 1 -- 172.21.15.107:0/1524028215 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0514031650 con 0x7f05201019c0 2024-09-19T04:29:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.993+0000 7f051e000640 1 -- 172.21.15.107:0/1524028215 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05140317f0 con 0x7f05201019c0 2024-09-19T04:29:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.993+0000 7f051e000640 1 -- 172.21.15.107:0/1524028215 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051402faa0 con 0x7f05201019c0 2024-09-19T04:29:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.993+0000 7f0527000640 1 -- 172.21.15.107:0/1524028215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052019f6f0 con 0x7f05201019c0 2024-09-19T04:29:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.993+0000 7f0527000640 1 -- 172.21.15.107:0/1524028215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05201a7030 con 0x7f05201019c0 2024-09-19T04:29:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.996+0000 7f051e000640 1 -- 172.21.15.107:0/1524028215 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f051402fc40 con 0x7f05201019c0 2024-09-19T04:29:32.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.996+0000 7f052b0b9740 1 Processor -- start 2024-09-19T04:29:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.996+0000 7f052b0b9740 1 -- start start 2024-09-19T04:29:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1dd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1e7f30 0x55c44f1ea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f052b0b9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c44f1433e0 con 0x55c44f1dcd50 2024-09-19T04:29:32.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f052b0b9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c44f0ead80 con 0x55c44f1dd670 2024-09-19T04:29:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f052b0b9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c44f13f3a0 con 0x55c44f1e7f30 2024-09-19T04:29:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0526600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0526600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1dd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45306/0 (socket says 172.21.15.107:45306) 2024-09-19T04:29:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0526600640 1 -- 172.21.15.107:0/4171864526 learned_addr learned my addr 172.21.15.107:0/4171864526 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0525c00640 1 --2- 172.21.15.107:0/4171864526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0525200640 1 --2- 172.21.15.107:0/4171864526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1e7f30 0x55c44f1ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0526600640 1 -- 172.21.15.107:0/4171864526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1e7f30 msgr2=0x55c44f1ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0526600640 1 --2- 172.21.15.107:0/4171864526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1e7f30 0x55c44f1ea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0526600640 1 -- 172.21.15.107:0/4171864526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 msgr2=0x55c44f1e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0526600640 1 --2- 172.21.15.107:0/4171864526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0526600640 1 -- 172.21.15.107:0/4171864526 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44f109ae0 con 0x55c44f1dcd50 2024-09-19T04:29:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.997+0000 7f0525c00640 1 --2- 172.21.15.107:0/4171864526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:29:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.998+0000 7f051e000640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f05080a3430 0x7f05080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.998+0000 7f0526600640 1 --2- 172.21.15.107:0/4171864526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1dd130 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0518009b90 tx=0x7f05180117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e02d588a017350a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.998+0000 7f051e000640 1 -- 172.21.15.107:0/1524028215 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f05140f0fc0 con 0x7f05201019c0 2024-09-19T04:29:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.998+0000 7f0525c00640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f05080a3430 0x7f05080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.998+0000 7f0512a00640 1 -- 172.21.15.107:0/4171864526 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518012450 con 0x55c44f1dcd50 2024-09-19T04:29:32.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.998+0000 7f0512a00640 1 -- 172.21.15.107:0/4171864526 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0518012ad0 con 0x55c44f1dcd50 2024-09-19T04:29:32.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.998+0000 7f0512a00640 1 -- 172.21.15.107:0/4171864526 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051800fa70 con 0x55c44f1dcd50 2024-09-19T04:29:32.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.999+0000 7f052b0b9740 1 -- 172.21.15.107:0/4171864526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 msgr2=0x55c44f1dd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.999+0000 7f052b0b9740 1 --2- 172.21.15.107:0/4171864526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1dd130 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0518009b90 tx=0x7f05180117b0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.999+0000 7f0525c00640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f05080a3430 0x7f05080a58f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55c44f1da140 tx=0x7f05140f4a70 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.999+0000 7f052b0b9740 1 -- 172.21.15.107:0/4171864526 shutdown_connections 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.999+0000 7f052b0b9740 1 --2- 172.21.15.107:0/4171864526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1e7f30 0x55c44f1ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.999+0000 7f052b0b9740 1 --2- 172.21.15.107:0/4171864526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.999+0000 7f052b0b9740 1 --2- 172.21.15.107:0/4171864526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1dd130 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.999+0000 7f052b0b9740 1 -- 172.21.15.107:0/4171864526 >> 172.21.15.107:0/4171864526 conn(0x55c44f1d92e0 msgr2=0x55c44f1d96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.999+0000 7f052b0b9740 1 -- 172.21.15.107:0/4171864526 shutdown_connections 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:31.999+0000 7f052b0b9740 1 -- 172.21.15.107:0/4171864526 wait complete. 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.000+0000 7f052b0b9740 1 -- start start 2024-09-19T04:29:32.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.000+0000 7f052b0b9740 1 Processor -- start 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.000+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 0x55c44f1e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.000+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dd670 0x55c44f1e5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.000+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 0x55c44f1e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.000+0000 7f052b0b9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c44f1eaba0 con 0x55c44f1dd670 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.000+0000 7f052b0b9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c44f0fe0f0 con 0x55c44f1e7f30 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.000+0000 7f052b0b9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c44f1ead20 con 0x55c44f1dcd50 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.000+0000 7f052b0b9740 10 client.0 _ll_register_callbacks cb 0x55c44f0ef9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0525200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dd670 0x55c44f1e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0525200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dd670 0x55c44f1e5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45322/0 (socket says 172.21.15.107:45322) 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0525200640 1 -- 172.21.15.107:0/643897824 learned_addr learned my addr 172.21.15.107:0/643897824 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0526600640 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 0x55c44f1e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0526600640 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 0x55c44f1e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0525200640 1 -- 172.21.15.107:0/643897824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 msgr2=0x55c44f1e4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:32.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0525200640 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 0x55c44f1e4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0525200640 1 -- 172.21.15.107:0/643897824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 msgr2=0x55c44f1e5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:32.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0525200640 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 0x55c44f1e5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0525200640 1 -- 172.21.15.107:0/643897824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44eeb43c0 con 0x55c44f1dd670 2024-09-19T04:29:32.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0526600640 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 0x55c44f1e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:32.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0525200640 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dd670 0x55c44f1e5350 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f050c012d50 tx=0x7f050c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d98169ec1a62a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:32.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.001+0000 7f0507400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:29:32.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.002+0000 7f0507400640 1 -- 172.21.15.107:0/643897824 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c012110 con 0x55c44f1dd670 2024-09-19T04:29:32.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.002+0000 7f0507400640 1 -- 172.21.15.107:0/643897824 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f050c004050 con 0x55c44f1dd670 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.002+0000 7f052b0b9740 1 -- 172.21.15.107:0/643897824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0518012ad0 con 0x55c44f1dd670 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.002+0000 7f0507400640 1 -- 172.21.15.107:0/643897824 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f050c0122b0 con 0x55c44f1dd670 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.002+0000 7f052b0b9740 10 client.16881 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.002+0000 7f052b0b9740 1 -- 172.21.15.107:0/643897824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c44f1f68b0 con 0x55c44f1dd670 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.002+0000 7f0507400640 1 -- 172.21.15.107:0/643897824 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0518012ad0 con 0x55c44f1dd670 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.003+0000 7f0507400640 1 -- 172.21.15.107:0/643897824 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f050c023020 con 0x55c44f1dd670 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.003+0000 7f052b0b9740 10 client.16881 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.003+0000 7f052b0b9740 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.003+0000 7f052b0b9740 20 client.16881 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.003+0000 7f052b0b9740 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55c44f1f7d60 0x55c44f1fa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.003+0000 7f052b0b9740 4 client.16881 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:29:32.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.003+0000 7f052b0b9740 1 -- 172.21.15.107:0/643897824 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c44f150950 con 0x55c44f1f7d60 2024-09-19T04:29:32.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.003+0000 7f0526600640 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55c44f1f7d60 0x55c44f1fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.003+0000 7f0526600640 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55c44f1f7d60 0x55c44f1fa150 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b0cece06efdf9c server_cookie=bbcf771b73ee4e49 in_seq=0 out_seq=0 2024-09-19T04:29:32.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.003+0000 7f0507400640 10 client.16881 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:32.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.004+0000 7f0507400640 1 -- 172.21.15.107:0/643897824 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c44f150950 con 0x55c44f1f7d60 2024-09-19T04:29:32.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.004+0000 7f0507400640 10 client.16881 handle_command_reply: tid=0 2024-09-19T04:29:32.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.004+0000 7f052b0b9740 1 client.16881 shutdown 2024-09-19T04:29:32.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 1 -- 172.21.15.107:0/643897824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dd670 msgr2=0x55c44f1e5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:32.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dd670 0x55c44f1e5350 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f050c012d50 tx=0x7f050c00f790 comp rx=0 tx=0).stop 2024-09-19T04:29:32.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 1 -- 172.21.15.107:0/643897824 shutdown_connections 2024-09-19T04:29:32.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55c44f1f7d60 0x55c44f1fa150 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 0x55c44f1e5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f0507400640 0 client.16881 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:32.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dd670 0x55c44f1e5350 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 1 --2- 172.21.15.107:0/643897824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 0x55c44f1e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 1 -- 172.21.15.107:0/643897824 >> 172.21.15.107:0/643897824 conn(0x55c44f1d92e0 msgr2=0x55c44f1df010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:32.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 1 -- 172.21.15.107:0/643897824 shutdown_connections 2024-09-19T04:29:32.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 1 -- 172.21.15.107:0/643897824 wait complete. 2024-09-19T04:29:32.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.005+0000 7f052b0b9740 20 client.16881 trim_cache size 0 max 16384 2024-09-19T04:29:32.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.022+0000 7f052b0b9740 1 Processor -- start 2024-09-19T04:29:32.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.022+0000 7f052b0b9740 1 -- start start 2024-09-19T04:29:32.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 0x55c44f1e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1e7f30 0x55c44f1ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f052b0b9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f05201af870 con 0x55c44f1e7f30 2024-09-19T04:29:32.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f052b0b9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f05201b13b0 con 0x55c44f1dd670 2024-09-19T04:29:32.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f052b0b9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f05201aff40 con 0x55c44f1dcd50 2024-09-19T04:29:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0525200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0525200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:51380/0 (socket says 172.21.15.107:51380) 2024-09-19T04:29:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0525200640 1 -- 172.21.15.107:0/446993523 learned_addr learned my addr 172.21.15.107:0/446993523 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0526600640 1 --2- 172.21.15.107:0/446993523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 0x55c44f1e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0526600640 1 --2- 172.21.15.107:0/446993523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1e7f30 0x55c44f1ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0525200640 1 -- 172.21.15.107:0/446993523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 msgr2=0x55c44f1e46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0525200640 1 --2- 172.21.15.107:0/446993523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 0x55c44f1e46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0525200640 1 -- 172.21.15.107:0/446993523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1e7f30 msgr2=0x55c44f1ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0525200640 1 --2- 172.21.15.107:0/446993523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1e7f30 0x55c44f1ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0525200640 1 -- 172.21.15.107:0/446993523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44f1e3dc0 con 0x55c44f1dd670 2024-09-19T04:29:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.023+0000 7f0526600640 1 --2- 172.21.15.107:0/446993523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1e7f30 0x55c44f1ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f0525200640 1 --2- 172.21.15.107:0/446993523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e6d80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f050c0255a0 tx=0x7f050c0057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1642123e3e9717f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f0512000640 1 -- 172.21.15.107:0/446993523 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c0050e0 con 0x55c44f1dd670 2024-09-19T04:29:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f0512000640 1 -- 172.21.15.107:0/446993523 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c02a020 con 0x55c44f1dd670 2024-09-19T04:29:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f0512000640 1 -- 172.21.15.107:0/446993523 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c025620 con 0x55c44f1dd670 2024-09-19T04:29:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f052b0b9740 1 -- 172.21.15.107:0/446993523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 msgr2=0x55c44f1e6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f052b0b9740 1 --2- 172.21.15.107:0/446993523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e6d80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f050c0255a0 tx=0x7f050c0057d0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f052b0b9740 1 -- 172.21.15.107:0/446993523 shutdown_connections 2024-09-19T04:29:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f052b0b9740 1 --2- 172.21.15.107:0/446993523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1e7f30 0x55c44f1ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f052b0b9740 1 --2- 172.21.15.107:0/446993523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1dd670 0x55c44f1e6d80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f052b0b9740 1 --2- 172.21.15.107:0/446993523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dcd50 0x55c44f1e46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.024+0000 7f052b0b9740 1 -- 172.21.15.107:0/446993523 >> 172.21.15.107:0/446993523 conn(0x55c44f1d92e0 msgr2=0x55c44f1e6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.025+0000 7f052b0b9740 1 -- 172.21.15.107:0/446993523 shutdown_connections 2024-09-19T04:29:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.025+0000 7f052b0b9740 1 -- 172.21.15.107:0/446993523 wait complete. 2024-09-19T04:29:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.025+0000 7f052b0b9740 1 -- start start 2024-09-19T04:29:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.025+0000 7f052b0b9740 1 Processor -- start 2024-09-19T04:29:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.025+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1f0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.025+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dd670 0x55c44f1e1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.025+0000 7f052b0b9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 0x55c44f1e2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.025+0000 7f052b0b9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f05201b0ce0 con 0x55c44f1dcd50 2024-09-19T04:29:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.025+0000 7f052b0b9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f05201b0610 con 0x55c44f1e7f30 2024-09-19T04:29:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.025+0000 7f052b0b9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f05201ad870 con 0x55c44f1dd670 2024-09-19T04:29:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f052b0b9740 10 client.0 _ll_register_callbacks cb 0x7f050c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:29:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0526600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0526600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1f0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45332/0 (socket says 172.21.15.107:45332) 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0525200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dd670 0x55c44f1e1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0526600640 1 -- 172.21.15.107:0/3074930444 learned_addr learned my addr 172.21.15.107:0/3074930444 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0526600640 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 0x55c44f1e2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0525200640 1 -- 172.21.15.107:0/3074930444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 msgr2=0x55c44f1e2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0525200640 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 0x55c44f1e2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0525200640 1 -- 172.21.15.107:0/3074930444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 msgr2=0x55c44f1f0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0525200640 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1f0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0525200640 1 -- 172.21.15.107:0/3074930444 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44f1fa4c0 con 0x55c44f1dd670 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.026+0000 7f0526600640 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 0x55c44f1e2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.027+0000 7f0525200640 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dd670 0x55c44f1e1b10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f050c012d50 tx=0x7f050c003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4168d8b28f3344ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:29:32.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.027+0000 7f0507400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:29:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.027+0000 7f0507400640 1 -- 172.21.15.107:0/3074930444 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c005280 con 0x55c44f1dd670 2024-09-19T04:29:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.027+0000 7f0507400640 1 -- 172.21.15.107:0/3074930444 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8484+0+0 (secure 0 0 0) 0x7f050c018040 con 0x55c44f1dd670 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.027+0000 7f052b0b9740 1 -- 172.21.15.107:0/3074930444 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f050c02a020 con 0x55c44f1dd670 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.027+0000 7f0507400640 1 -- 172.21.15.107:0/3074930444 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f050c005420 con 0x55c44f1dd670 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.027+0000 7f052b0b9740 10 client.26896 fetch_fsmap learned FSMap version 48 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.027+0000 7f052b0b9740 1 -- 172.21.15.107:0/3074930444 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c44f2627e0 con 0x55c44f1dd670 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.028+0000 7f0507400640 1 -- 172.21.15.107:0/3074930444 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c02a020 con 0x55c44f1dd670 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.028+0000 7f0507400640 1 -- 172.21.15.107:0/3074930444 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f050c036070 con 0x55c44f1dd670 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.028+0000 7f052b0b9740 10 client.26896 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.028+0000 7f052b0b9740 10 client.26896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.028+0000 7f052b0b9740 20 client.26896 populate_metadata read hostname 'smithi107' 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.028+0000 7f052b0b9740 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55c44f262e40 0x55c44f265260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.028+0000 7f052b0b9740 4 client.26896 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:29:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.028+0000 7f052b0b9740 1 -- 172.21.15.107:0/3074930444 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c44f14ae60 con 0x55c44f262e40 2024-09-19T04:29:32.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.028+0000 7f0526600640 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55c44f262e40 0x55c44f265260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:29:32.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f0526600640 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55c44f262e40 0x55c44f265260 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08cbe230dfea7ff server_cookie=24935cd9fb1d0dd5 in_seq=0 out_seq=0 2024-09-19T04:29:32.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f0507400640 10 client.26896 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:32.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f0507400640 1 -- 172.21.15.107:0/3074930444 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c44f14ae60 con 0x55c44f262e40 2024-09-19T04:29:32.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f0507400640 10 client.26896 handle_command_reply: tid=0 2024-09-19T04:29:32.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 client.26896 shutdown 2024-09-19T04:29:32.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 -- 172.21.15.107:0/3074930444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dd670 msgr2=0x55c44f1e1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:32.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dd670 0x55c44f1e1b10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f050c012d50 tx=0x7f050c003820 comp rx=0 tx=0).stop 2024-09-19T04:29:32.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 -- 172.21.15.107:0/3074930444 shutdown_connections 2024-09-19T04:29:32.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55c44f262e40 0x55c44f265260 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c44f1e7f30 0x55c44f1e2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f0507400640 0 client.26896 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:29:32.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c44f1dd670 0x55c44f1e1b10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 --2- 172.21.15.107:0/3074930444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c44f1dcd50 0x55c44f1f0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 -- 172.21.15.107:0/3074930444 >> 172.21.15.107:0/3074930444 conn(0x55c44f1d92e0 msgr2=0x55c44f1e4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:32.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 -- 172.21.15.107:0/3074930444 shutdown_connections 2024-09-19T04:29:32.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 1 -- 172.21.15.107:0/3074930444 wait complete. 2024-09-19T04:29:32.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.029+0000 7f052b0b9740 20 client.26896 trim_cache size 0 max 16384 2024-09-19T04:29:32.038 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:29:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:29:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:29:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "0f4269e0-2e78-45a2-b541-2a8bd0c98982": { 2024-09-19T04:29:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:29:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "0f4269e0-2e78-45a2-b541-2a8bd0c98982", 2024-09-19T04:29:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:29:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:29:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:29:32.039 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:29:32.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 -- 172.21.15.107:0/1524028215 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f05080a3430 msgr2=0x7f05080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:32.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f05080a3430 0x7f05080a58f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55c44f1da140 tx=0x7f05140f4a70 comp rx=0 tx=0).stop 2024-09-19T04:29:32.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 -- 172.21.15.107:0/1524028215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05201019c0 msgr2=0x7f052019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:29:32.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05201019c0 0x7f052019a4b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0514005e30 tx=0x7f0514003f90 comp rx=0 tx=0).stop 2024-09-19T04:29:32.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 -- 172.21.15.107:0/1524028215 shutdown_connections 2024-09-19T04:29:32.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f05080a3430 0x7f05080a58f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05201079c0 0x7f052019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0520102360 0x7f052019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 --2- 172.21.15.107:0/1524028215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05201019c0 0x7f052019a4b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:29:32.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 -- 172.21.15.107:0/1524028215 >> 172.21.15.107:0/1524028215 conn(0x7f05200fd740 msgr2=0x7f05200fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:29:32.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 -- 172.21.15.107:0/1524028215 shutdown_connections 2024-09-19T04:29:32.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:29:32.030+0000 7f0527000640 1 -- 172.21.15.107:0/1524028215 wait complete. 2024-09-19T04:29:32.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:31 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:29:32.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:31 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:32.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:31 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:29:32.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:31 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:32.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:31 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:29:32.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:31 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:32.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:31 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:29:32.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:31 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:32.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:31 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:29:32.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:31 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:32.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:31 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:29:32.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:31 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:29:32.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 296117 2.58 MB/sec execute 350 sec latency 1301.771 ms 2024-09-19T04:29:32.891 INFO:tasks.cephfs.filesystem:scrub status for tag:0f4269e0-2e78-45a2-b541-2a8bd0c98982 - {'path': '/', 'tag': '0f4269e0-2e78-45a2-b541-2a8bd0c98982', 'options': 'recursive,force'} 2024-09-19T04:29:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:32 smithi115 ceph-mon[35477]: pgmap v285: 97 pgs: 97 active+clean; 219 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-19T04:29:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:32 smithi145 ceph-mon[35096]: pgmap v285: 97 pgs: 97 active+clean; 219 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-19T04:29:33.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:32 smithi107 ceph-mon[30659]: pgmap v285: 97 pgs: 97 active+clean; 219 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-19T04:29:33.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 296117 2.57 MB/sec execute 351 sec latency 2301.810 ms 2024-09-19T04:29:34.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 297216 2.58 MB/sec execute 352 sec latency 2814.140 ms 2024-09-19T04:29:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:34 smithi115 ceph-mon[35477]: pgmap v286: 97 pgs: 97 active+clean; 219 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-19T04:29:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:34 smithi145 ceph-mon[35096]: pgmap v286: 97 pgs: 97 active+clean; 219 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-19T04:29:35.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:34 smithi107 ceph-mon[30659]: pgmap v286: 97 pgs: 97 active+clean; 219 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-19T04:29:35.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 298564 2.60 MB/sec execute 353 sec latency 12.405 ms 2024-09-19T04:29:36.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 299665 2.61 MB/sec execute 354 sec latency 129.283 ms 2024-09-19T04:29:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:36 smithi115 ceph-mon[35477]: pgmap v287: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-09-19T04:29:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:36 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:29:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:36 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:29:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:36 smithi145 ceph-mon[35096]: pgmap v287: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-09-19T04:29:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:36 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:29:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:36 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:29:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:36 smithi107 ceph-mon[30659]: pgmap v287: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-09-19T04:29:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:36 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:29:37.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:36 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:29:37.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 299665 2.60 MB/sec execute 355 sec latency 1129.365 ms 2024-09-19T04:29:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:37 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:29:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:37 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:29:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:37 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:29:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:37 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:29:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:37 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:29:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:37 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:29:38.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:37 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:29:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:37 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:29:38.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:37 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:29:38.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 299665 2.59 MB/sec execute 356 sec latency 2129.438 ms 2024-09-19T04:29:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:38 smithi115 ceph-mon[35477]: pgmap v288: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-09-19T04:29:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:38 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:29:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:38 smithi145 ceph-mon[35096]: pgmap v288: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-09-19T04:29:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:38 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:29:39.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:38 smithi107 ceph-mon[30659]: pgmap v288: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-09-19T04:29:39.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:38 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:29:39.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 300382 2.59 MB/sec execute 357 sec latency 2380.057 ms 2024-09-19T04:29:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:39 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:29:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:39 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:29:40.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:39 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:29:40.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:39 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:29:40.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:39 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:29:40.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:39 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:29:40.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:39 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:29:40.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:39 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:29:40.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:39 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:29:40.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 301915 2.62 MB/sec execute 358 sec latency 45.957 ms 2024-09-19T04:29:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:40 smithi145 ceph-mon[35096]: pgmap v289: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-09-19T04:29:41.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:40 smithi107 ceph-mon[30659]: pgmap v289: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-09-19T04:29:41.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:40 smithi115 ceph-mon[35477]: pgmap v289: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-09-19T04:29:41.712 INFO:tasks.workunit.client.0.smithi107.stdout: 1 303113 2.62 MB/sec execute 359 sec latency 17.910 ms 2024-09-19T04:29:42.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 303214 2.62 MB/sec execute 360 sec latency 877.207 ms 2024-09-19T04:29:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:42 smithi145 ceph-mon[35096]: pgmap v290: 97 pgs: 97 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-19T04:29:43.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:42 smithi107 ceph-mon[30659]: pgmap v290: 97 pgs: 97 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-19T04:29:43.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:42 smithi115 ceph-mon[35477]: pgmap v290: 97 pgs: 97 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-19T04:29:43.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 303214 2.61 MB/sec execute 361 sec latency 1877.321 ms 2024-09-19T04:29:44.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 303856 2.61 MB/sec execute 362 sec latency 2660.942 ms 2024-09-19T04:29:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:44 smithi145 ceph-mon[35096]: pgmap v291: 97 pgs: 97 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 956 KiB/s wr, 107 op/s 2024-09-19T04:29:45.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:44 smithi107 ceph-mon[30659]: pgmap v291: 97 pgs: 97 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 956 KiB/s wr, 107 op/s 2024-09-19T04:29:45.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:44 smithi115 ceph-mon[35477]: pgmap v291: 97 pgs: 97 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 956 KiB/s wr, 107 op/s 2024-09-19T04:29:45.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 305447 2.63 MB/sec execute 363 sec latency 11.533 ms 2024-09-19T04:29:46.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 306599 2.64 MB/sec execute 364 sec latency 18.395 ms 2024-09-19T04:29:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:46 smithi145 ceph-mon[35096]: pgmap v292: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-09-19T04:29:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:46 smithi107 ceph-mon[30659]: pgmap v292: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-09-19T04:29:47.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:46 smithi115 ceph-mon[35477]: pgmap v292: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-09-19T04:29:47.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 306757 2.63 MB/sec execute 365 sec latency 834.266 ms 2024-09-19T04:29:48.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 306757 2.63 MB/sec execute 366 sec latency 1834.390 ms 2024-09-19T04:29:49.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:48 smithi107 ceph-mon[30659]: pgmap v293: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-09-19T04:29:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:48 smithi145 ceph-mon[35096]: pgmap v293: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-09-19T04:29:49.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:48 smithi115 ceph-mon[35477]: pgmap v293: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-09-19T04:29:49.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 306757 2.62 MB/sec execute 367 sec latency 2834.438 ms 2024-09-19T04:29:50.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 308749 2.65 MB/sec execute 368 sec latency 2889.731 ms 2024-09-19T04:29:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:50 smithi145 ceph-mon[35096]: pgmap v294: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-09-19T04:29:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:50 smithi107 ceph-mon[30659]: pgmap v294: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-09-19T04:29:51.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:50 smithi115 ceph-mon[35477]: pgmap v294: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-09-19T04:29:51.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 309852 2.65 MB/sec execute 369 sec latency 17.680 ms 2024-09-19T04:29:52.713 INFO:tasks.workunit.client.0.smithi107.stdout: 1 310305 2.65 MB/sec execute 370 sec latency 539.269 ms 2024-09-19T04:29:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:52 smithi145 ceph-mon[35096]: pgmap v295: 97 pgs: 97 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-09-19T04:29:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:52 smithi107 ceph-mon[30659]: pgmap v295: 97 pgs: 97 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-09-19T04:29:53.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:52 smithi115 ceph-mon[35477]: pgmap v295: 97 pgs: 97 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-09-19T04:29:53.714 INFO:tasks.workunit.client.0.smithi107.stdout: 1 310305 2.64 MB/sec execute 371 sec latency 1539.407 ms 2024-09-19T04:29:54.714 INFO:tasks.workunit.client.0.smithi107.stdout: 1 310474 2.63 MB/sec execute 372 sec latency 1784.141 ms 2024-09-19T04:29:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:54 smithi145 ceph-mon[35096]: pgmap v296: 97 pgs: 97 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-09-19T04:29:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:54 smithi107 ceph-mon[30659]: pgmap v296: 97 pgs: 97 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-09-19T04:29:55.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:54 smithi115 ceph-mon[35477]: pgmap v296: 97 pgs: 97 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-09-19T04:29:55.714 INFO:tasks.workunit.client.0.smithi107.stdout: 1 312396 2.66 MB/sec execute 373 sec latency 599.385 ms 2024-09-19T04:29:56.714 INFO:tasks.workunit.client.0.smithi107.stdout: 1 313565 2.67 MB/sec execute 374 sec latency 15.933 ms 2024-09-19T04:29:57.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:56 smithi107 ceph-mon[30659]: pgmap v297: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 167 op/s 2024-09-19T04:29:57.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:56 smithi115 ceph-mon[35477]: pgmap v297: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 167 op/s 2024-09-19T04:29:57.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:56 smithi145 ceph-mon[35096]: pgmap v297: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 167 op/s 2024-09-19T04:29:57.714 INFO:tasks.workunit.client.0.smithi107.stdout: 1 313847 2.66 MB/sec execute 375 sec latency 620.164 ms 2024-09-19T04:29:58.714 INFO:tasks.workunit.client.0.smithi107.stdout: 1 313847 2.66 MB/sec execute 376 sec latency 1620.352 ms 2024-09-19T04:29:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:29:58 smithi107 ceph-mon[30659]: pgmap v298: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-19T04:29:59.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:29:58 smithi115 ceph-mon[35477]: pgmap v298: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-19T04:29:59.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:29:58 smithi145 ceph-mon[35096]: pgmap v298: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-19T04:29:59.714 INFO:tasks.workunit.client.0.smithi107.stdout: 1 313919 2.65 MB/sec execute 377 sec latency 2127.347 ms 2024-09-19T04:30:00.714 INFO:tasks.workunit.client.0.smithi107.stdout: 1 315395 2.67 MB/sec execute 378 sec latency 378.793 ms 2024-09-19T04:30:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:00 smithi107 ceph-mon[30659]: pgmap v299: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-19T04:30:01.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:00 smithi107 ceph-mon[30659]: overall HEALTH_OK 2024-09-19T04:30:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:00 smithi115 ceph-mon[35477]: pgmap v299: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-19T04:30:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:00 smithi115 ceph-mon[35477]: overall HEALTH_OK 2024-09-19T04:30:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:00 smithi145 ceph-mon[35096]: pgmap v299: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-19T04:30:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:00 smithi145 ceph-mon[35096]: overall HEALTH_OK 2024-09-19T04:30:01.714 INFO:tasks.workunit.client.0.smithi107.stdout: 1 316211 2.67 MB/sec execute 379 sec latency 30.280 ms 2024-09-19T04:30:02.714 INFO:tasks.workunit.client.0.smithi107.stdout: 1 317396 2.68 MB/sec execute 380 sec latency 16.172 ms 2024-09-19T04:30:02.892 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:30:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:02 smithi107 ceph-mon[30659]: pgmap v300: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-09-19T04:30:03.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:02 smithi107 ceph-mon[30659]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:30:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:03.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:02 smithi115 ceph-mon[35477]: pgmap v300: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-09-19T04:30:03.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:02 smithi115 ceph-mon[35477]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:30:03.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:02 smithi145 ceph-mon[35096]: pgmap v300: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-09-19T04:30:03.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:02 smithi145 ceph-mon[35096]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:30:03.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 317396 2.67 MB/sec execute 381 sec latency 1002.739 ms 2024-09-19T04:30:03.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.809+0000 7fd525e00640 1 -- 172.21.15.107:0/320478382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd520106f10 msgr2=0x7fd52010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.809+0000 7fd525e00640 1 --2- 172.21.15.107:0/320478382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd520106f10 0x7fd52010ea90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd518009ca0 tx=0x7fd51802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.809+0000 7fd525e00640 1 -- 172.21.15.107:0/320478382 shutdown_connections 2024-09-19T04:30:03.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.809+0000 7fd525e00640 1 --2- 172.21.15.107:0/320478382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd520106f10 0x7fd52010ea90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.809+0000 7fd525e00640 1 --2- 172.21.15.107:0/320478382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd520102790 0x7fd5201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.809+0000 7fd525e00640 1 --2- 172.21.15.107:0/320478382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd520101de0 0x7fd5201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.809+0000 7fd525e00640 1 -- 172.21.15.107:0/320478382 >> 172.21.15.107:0/320478382 conn(0x7fd5200fd8c0 msgr2=0x7fd5200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.809+0000 7fd525e00640 1 -- 172.21.15.107:0/320478382 shutdown_connections 2024-09-19T04:30:03.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.809+0000 7fd525e00640 1 -- 172.21.15.107:0/320478382 wait complete. 2024-09-19T04:30:03.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.810+0000 7fd525e00640 1 Processor -- start 2024-09-19T04:30:03.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.810+0000 7fd525e00640 1 -- start start 2024-09-19T04:30:03.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd525e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd520101de0 0x7fd5200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd525e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd520102790 0x7fd520074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd525e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd520106f10 0x7fd520074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd525e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd520111fc0 con 0x7fd520106f10 2024-09-19T04:30:03.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd525e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd520111e40 con 0x7fd520102790 2024-09-19T04:30:03.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd525e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd520112140 con 0x7fd520101de0 2024-09-19T04:30:03.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd51f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd520106f10 0x7fd520074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd51f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd520106f10 0x7fd520074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40514/0 (socket says 172.21.15.107:40514) 2024-09-19T04:30:03.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd51f400640 1 -- 172.21.15.107:0/1652962268 learned_addr learned my addr 172.21.15.107:0/1652962268 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:03.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd516000640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd520102790 0x7fd520074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.811+0000 7fd51ea00640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd520101de0 0x7fd5200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd51f400640 1 -- 172.21.15.107:0/1652962268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd520101de0 msgr2=0x7fd5200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd51f400640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd520101de0 0x7fd5200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd51f400640 1 -- 172.21.15.107:0/1652962268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd520102790 msgr2=0x7fd520074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd51f400640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd520102790 0x7fd520074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd51f400640 1 -- 172.21.15.107:0/1652962268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5201a68e0 con 0x7fd520106f10 2024-09-19T04:30:03.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd516000640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd520102790 0x7fd520074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:03.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd51f400640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd520106f10 0x7fd520074f70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd518005e30 tx=0x7fd518003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd524a00640 1 -- 172.21.15.107:0/1652962268 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518031650 con 0x7fd520106f10 2024-09-19T04:30:03.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd525e00640 1 -- 172.21.15.107:0/1652962268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5201a6b70 con 0x7fd520106f10 2024-09-19T04:30:03.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd524a00640 1 -- 172.21.15.107:0/1652962268 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5180317f0 con 0x7fd520106f10 2024-09-19T04:30:03.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.812+0000 7fd524a00640 1 -- 172.21.15.107:0/1652962268 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51802f9d0 con 0x7fd520106f10 2024-09-19T04:30:03.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.813+0000 7fd525e00640 1 -- 172.21.15.107:0/1652962268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5201a70b0 con 0x7fd520106f10 2024-09-19T04:30:03.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.815+0000 7fd524a00640 1 -- 172.21.15.107:0/1652962268 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fd51802fbb0 con 0x7fd520106f10 2024-09-19T04:30:03.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd528845740 1 Processor -- start 2024-09-19T04:30:03.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd528845740 1 -- start start 2024-09-19T04:30:03.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd528845740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562064852b90 0x562064852f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd528845740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620648534b0 0x56206485d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd528845740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x562064860160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd528845740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5620647b93e0 con 0x562064852b90 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd528845740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562064760d80 con 0x56206485dd70 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd528845740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5620647b53a0 con 0x5620648534b0 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd516000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562064852b90 0x562064852f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd516000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562064852b90 0x562064852f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40524/0 (socket says 172.21.15.107:40524) 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.816+0000 7fd516000640 1 -- 172.21.15.107:0/2768612469 learned_addr learned my addr 172.21.15.107:0/2768612469 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.817+0000 7fd51ea00640 1 --2- 172.21.15.107:0/2768612469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620648534b0 0x56206485d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.817+0000 7fd516000640 1 --2- 172.21.15.107:0/2768612469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x562064860160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.817+0000 7fd516000640 1 -- 172.21.15.107:0/2768612469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620648534b0 msgr2=0x56206485d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.817+0000 7fd516000640 1 --2- 172.21.15.107:0/2768612469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620648534b0 0x56206485d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.817+0000 7fd516000640 1 -- 172.21.15.107:0/2768612469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 msgr2=0x562064860160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.817+0000 7fd516000640 1 --2- 172.21.15.107:0/2768612469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x562064860160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.817+0000 7fd516000640 1 -- 172.21.15.107:0/2768612469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56206477fae0 con 0x562064852b90 2024-09-19T04:30:03.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.817+0000 7fd516000640 1 --2- 172.21.15.107:0/2768612469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562064852b90 0x562064852f70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x5620648504a0 tx=0x7fd508005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223f9dc4741f8d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.817+0000 7fd524a00640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5040a33b0 0x7fd5040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.817+0000 7fd50fe00640 1 -- 172.21.15.107:0/2768612469 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508013170 con 0x562064852b90 2024-09-19T04:30:03.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.818+0000 7fd524a00640 1 -- 172.21.15.107:0/1652962268 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8528+0+0 (secure 0 0 0) 0x7fd5180f0ff0 con 0x7fd520106f10 2024-09-19T04:30:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.818+0000 7fd50fe00640 1 -- 172.21.15.107:0/2768612469 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd508005500 con 0x562064852b90 2024-09-19T04:30:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.818+0000 7fd50fe00640 1 -- 172.21.15.107:0/2768612469 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508004a00 con 0x562064852b90 2024-09-19T04:30:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.818+0000 7fd51ea00640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5040a33b0 0x7fd5040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.818+0000 7fd528845740 1 -- 172.21.15.107:0/2768612469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562064852b90 msgr2=0x562064852f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.818+0000 7fd51ea00640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5040a33b0 0x7fd5040a5870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x56206484ff80 tx=0x7fd510005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.818+0000 7fd528845740 1 --2- 172.21.15.107:0/2768612469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562064852b90 0x562064852f70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x5620648504a0 tx=0x7fd508005850 comp rx=0 tx=0).stop 2024-09-19T04:30:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.819+0000 7fd528845740 1 -- 172.21.15.107:0/2768612469 shutdown_connections 2024-09-19T04:30:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.819+0000 7fd528845740 1 --2- 172.21.15.107:0/2768612469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x562064860160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.819+0000 7fd528845740 1 --2- 172.21.15.107:0/2768612469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620648534b0 0x56206485d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.819+0000 7fd528845740 1 --2- 172.21.15.107:0/2768612469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562064852b90 0x562064852f70 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.819+0000 7fd528845740 1 -- 172.21.15.107:0/2768612469 >> 172.21.15.107:0/2768612469 conn(0x56206484f120 msgr2=0x56206484f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.819+0000 7fd528845740 1 -- 172.21.15.107:0/2768612469 shutdown_connections 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.819+0000 7fd528845740 1 -- 172.21.15.107:0/2768612469 wait complete. 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.819+0000 7fd528845740 1 -- start start 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.819+0000 7fd528845740 1 Processor -- start 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd528845740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x56206485ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd528845740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd528845740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x56206485b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd528845740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5620648609e0 con 0x5620648534b0 2024-09-19T04:30:03.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd528845740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5620647740f0 con 0x56206485dd70 2024-09-19T04:30:03.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd528845740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562064860b60 con 0x562064852b90 2024-09-19T04:30:03.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd528845740 10 client.0 _ll_register_callbacks cb 0x5620647659e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd516000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd516000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40534/0 (socket says 172.21.15.107:40534) 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd516000640 1 -- 172.21.15.107:0/2969313159 learned_addr learned my addr 172.21.15.107:0/2969313159 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd51ea00640 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x56206485ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.820+0000 7fd51f400640 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x56206485b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.821+0000 7fd516000640 1 -- 172.21.15.107:0/2969313159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 msgr2=0x56206485ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.821+0000 7fd516000640 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x56206485ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.821+0000 7fd516000640 1 -- 172.21.15.107:0/2969313159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 msgr2=0x56206485b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.821+0000 7fd516000640 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x56206485b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.821+0000 7fd516000640 1 -- 172.21.15.107:0/2969313159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206452a3c0 con 0x5620648534b0 2024-09-19T04:30:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.821+0000 7fd516000640 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b1c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd50800fda0 tx=0x7fd508013950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e237cee5f329632 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.821+0000 7fd50cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.821+0000 7fd50cc00640 1 -- 172.21.15.107:0/2969313159 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508024070 con 0x5620648534b0 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.821+0000 7fd528845740 1 -- 172.21.15.107:0/2969313159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd508005500 con 0x5620648534b0 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.821+0000 7fd50cc00640 1 -- 172.21.15.107:0/2969313159 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8528+0+0 (secure 0 0 0) 0x7fd508005230 con 0x5620648534b0 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd50cc00640 1 -- 172.21.15.107:0/2969313159 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd508030050 con 0x5620648534b0 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd528845740 10 client.16899 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd528845740 1 -- 172.21.15.107:0/2969313159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56206486c6f0 con 0x5620648534b0 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd50cc00640 1 -- 172.21.15.107:0/2969313159 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd508005500 con 0x5620648534b0 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd50cc00640 1 -- 172.21.15.107:0/2969313159 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fd508004be0 con 0x5620648534b0 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd528845740 10 client.16899 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd528845740 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd528845740 20 client.16899 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd528845740 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56206486da40 0x56206486fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd528845740 4 client.16899 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.822+0000 7fd528845740 1 -- 172.21.15.107:0/2969313159 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620647c6950 con 0x56206486da40 2024-09-19T04:30:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.823+0000 7fd51ea00640 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56206486da40 0x56206486fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.823+0000 7fd50cc00640 10 client.16899 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.823+0000 7fd51ea00640 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56206486da40 0x56206486fe30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1486cb87e91dc405 server_cookie=9f7cdb557e56965a in_seq=0 out_seq=0 2024-09-19T04:30:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.824+0000 7fd50cc00640 1 -- 172.21.15.107:0/2969313159 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620647c6950 con 0x56206486da40 2024-09-19T04:30:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.824+0000 7fd50cc00640 10 client.16899 handle_command_reply: tid=0 2024-09-19T04:30:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.824+0000 7fd528845740 1 client.16899 shutdown 2024-09-19T04:30:03.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 1 -- 172.21.15.107:0/2969313159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 msgr2=0x56206485b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b1c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd50800fda0 tx=0x7fd508013950 comp rx=0 tx=0).stop 2024-09-19T04:30:03.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 1 -- 172.21.15.107:0/2969313159 shutdown_connections 2024-09-19T04:30:03.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56206486da40 0x56206486fe30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x56206485b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b1c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 1 --2- 172.21.15.107:0/2969313159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x56206485ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 1 -- 172.21.15.107:0/2969313159 >> 172.21.15.107:0/2969313159 conn(0x56206484f120 msgr2=0x562064854e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd50cc00640 0 client.16899 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 1 -- 172.21.15.107:0/2969313159 shutdown_connections 2024-09-19T04:30:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 1 -- 172.21.15.107:0/2969313159 wait complete. 2024-09-19T04:30:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.825+0000 7fd528845740 20 client.16899 trim_cache size 0 max 16384 2024-09-19T04:30:03.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.842+0000 7fd528845740 1 Processor -- start 2024-09-19T04:30:03.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.842+0000 7fd528845740 1 -- start start 2024-09-19T04:30:03.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd528845740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x56206485c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd528845740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd528845740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x5620648755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd528845740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201afa60 con 0x5620648534b0 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd528845740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201b15a0 con 0x56206485dd70 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd528845740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201b0130 con 0x562064852b90 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd516000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd516000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40538/0 (socket says 172.21.15.107:40538) 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd516000640 1 -- 172.21.15.107:0/2012902918 learned_addr learned my addr 172.21.15.107:0/2012902918 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd51f400640 1 --2- 172.21.15.107:0/2012902918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x5620648755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd51ea00640 1 --2- 172.21.15.107:0/2012902918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x56206485c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd516000640 1 -- 172.21.15.107:0/2012902918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 msgr2=0x56206485c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd516000640 1 --2- 172.21.15.107:0/2012902918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x56206485c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd516000640 1 -- 172.21.15.107:0/2012902918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 msgr2=0x5620648755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.843+0000 7fd516000640 1 --2- 172.21.15.107:0/2012902918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x5620648755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.844+0000 7fd516000640 1 -- 172.21.15.107:0/2012902918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620648501d0 con 0x5620648534b0 2024-09-19T04:30:03.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.844+0000 7fd516000640 1 --2- 172.21.15.107:0/2012902918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b550 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd508003640 tx=0x7fd508005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6bec53d124ae75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.844+0000 7fd50f400640 1 -- 172.21.15.107:0/2012902918 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508024040 con 0x5620648534b0 2024-09-19T04:30:03.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.844+0000 7fd50f400640 1 -- 172.21.15.107:0/2012902918 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd508003b40 con 0x5620648534b0 2024-09-19T04:30:03.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.844+0000 7fd50f400640 1 -- 172.21.15.107:0/2012902918 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5080206d0 con 0x5620648534b0 2024-09-19T04:30:03.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 -- 172.21.15.107:0/2012902918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 msgr2=0x56206485b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 --2- 172.21.15.107:0/2012902918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b550 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd508003640 tx=0x7fd508005fd0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 -- 172.21.15.107:0/2012902918 shutdown_connections 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 --2- 172.21.15.107:0/2012902918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x5620648755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 --2- 172.21.15.107:0/2012902918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x56206485b550 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 --2- 172.21.15.107:0/2012902918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x56206485c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 -- 172.21.15.107:0/2012902918 >> 172.21.15.107:0/2012902918 conn(0x56206484f120 msgr2=0x562064859c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 -- 172.21.15.107:0/2012902918 shutdown_connections 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 -- 172.21.15.107:0/2012902918 wait complete. 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 -- start start 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.845+0000 7fd528845740 1 Processor -- start 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd528845740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x562064866430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd528845740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x562064861760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd528845740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x562064861ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd528845740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201b0ed0 con 0x5620648534b0 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd528845740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201b0800 con 0x56206485dd70 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd528845740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201ada40 con 0x562064852b90 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd528845740 10 client.0 _ll_register_callbacks cb 0x7fd508013a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd516000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x562064861760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd516000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x562064861760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40546/0 (socket says 172.21.15.107:40546) 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd516000640 1 -- 172.21.15.107:0/1269278815 learned_addr learned my addr 172.21.15.107:0/1269278815 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd51f400640 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x562064861ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.846+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x562064866430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd516000640 1 -- 172.21.15.107:0/1269278815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 msgr2=0x562064866430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd516000640 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x562064866430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd516000640 1 -- 172.21.15.107:0/1269278815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 msgr2=0x562064861ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:03.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd516000640 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x562064861ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd516000640 1 -- 172.21.15.107:0/1269278815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620648624d0 con 0x5620648534b0 2024-09-19T04:30:03.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd516000640 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x562064861760 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd508009700 tx=0x7fd508003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cfdb171537a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:03.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd50cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:30:03.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd50cc00640 1 -- 172.21.15.107:0/1269278815 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508024040 con 0x5620648534b0 2024-09-19T04:30:03.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd50cc00640 1 -- 172.21.15.107:0/1269278815 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8528+0+0 (secure 0 0 0) 0x7fd508005230 con 0x5620648534b0 2024-09-19T04:30:03.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd528845740 1 -- 172.21.15.107:0/1269278815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd508003b40 con 0x5620648534b0 2024-09-19T04:30:03.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd50cc00640 1 -- 172.21.15.107:0/1269278815 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd508042050 con 0x5620648534b0 2024-09-19T04:30:03.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd528845740 10 client.16905 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.847+0000 7fd528845740 1 -- 172.21.15.107:0/1269278815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5620648d84a0 con 0x5620648534b0 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.848+0000 7fd50cc00640 1 -- 172.21.15.107:0/1269278815 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd508003b40 con 0x5620648534b0 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.848+0000 7fd50cc00640 1 -- 172.21.15.107:0/1269278815 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fd508020c20 con 0x5620648534b0 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.848+0000 7fd528845740 10 client.16905 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.848+0000 7fd528845740 10 client.16905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.848+0000 7fd528845740 20 client.16905 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.848+0000 7fd528845740 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5620648d8aa0 0x5620648daec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.848+0000 7fd528845740 4 client.16905 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.848+0000 7fd528845740 1 -- 172.21.15.107:0/1269278815 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620647c0e60 con 0x5620648d8aa0 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.849+0000 7fd51ea00640 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5620648d8aa0 0x5620648daec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:03.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.849+0000 7fd51ea00640 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5620648d8aa0 0x5620648daec0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a2864707b145f91 server_cookie=f9573accb651b828 in_seq=0 out_seq=0 2024-09-19T04:30:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.849+0000 7fd50cc00640 10 client.16905 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.849+0000 7fd50cc00640 1 -- 172.21.15.107:0/1269278815 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5620647c0e60 con 0x5620648d8aa0 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.849+0000 7fd50cc00640 10 client.16905 handle_command_reply: tid=0 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.849+0000 7fd528845740 1 client.16905 shutdown 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.849+0000 7fd528845740 1 -- 172.21.15.107:0/1269278815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 msgr2=0x562064861760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.849+0000 7fd528845740 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x562064861760 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd508009700 tx=0x7fd508003730 comp rx=0 tx=0).stop 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd528845740 1 -- 172.21.15.107:0/1269278815 shutdown_connections 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd528845740 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5620648d8aa0 0x5620648daec0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd528845740 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56206485dd70 0x562064861ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd528845740 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620648534b0 0x562064861760 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd528845740 1 --2- 172.21.15.107:0/1269278815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562064852b90 0x562064866430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd528845740 1 -- 172.21.15.107:0/1269278815 >> 172.21.15.107:0/1269278815 conn(0x56206484f120 msgr2=0x5620648545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd50cc00640 0 client.16905 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd528845740 1 -- 172.21.15.107:0/1269278815 shutdown_connections 2024-09-19T04:30:03.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd528845740 1 -- 172.21.15.107:0/1269278815 wait complete. 2024-09-19T04:30:03.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd528845740 20 client.16905 trim_cache size 0 max 16384 2024-09-19T04:30:03.857 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:30:03.857 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:30:03.857 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:30:03.857 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:30:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd525e00640 1 -- 172.21.15.107:0/1652962268 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5040a33b0 msgr2=0x7fd5040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd525e00640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5040a33b0 0x7fd5040a5870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x56206484ff80 tx=0x7fd510005f90 comp rx=0 tx=0).stop 2024-09-19T04:30:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd525e00640 1 -- 172.21.15.107:0/1652962268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd520106f10 msgr2=0x7fd520074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.850+0000 7fd525e00640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd520106f10 0x7fd520074f70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd518005e30 tx=0x7fd518003d00 comp rx=0 tx=0).stop 2024-09-19T04:30:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.851+0000 7fd525e00640 1 -- 172.21.15.107:0/1652962268 shutdown_connections 2024-09-19T04:30:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.851+0000 7fd525e00640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd5040a33b0 0x7fd5040a5870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.851+0000 7fd525e00640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd520106f10 0x7fd520074f70 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.851+0000 7fd525e00640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd520102790 0x7fd520074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.851+0000 7fd525e00640 1 --2- 172.21.15.107:0/1652962268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd520101de0 0x7fd5200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:03.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.851+0000 7fd525e00640 1 -- 172.21.15.107:0/1652962268 >> 172.21.15.107:0/1652962268 conn(0x7fd5200fd8c0 msgr2=0x7fd5200feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:03.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.851+0000 7fd525e00640 1 -- 172.21.15.107:0/1652962268 shutdown_connections 2024-09-19T04:30:03.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:03.851+0000 7fd525e00640 1 -- 172.21.15.107:0/1652962268 wait complete. 2024-09-19T04:30:04.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 317684 2.67 MB/sec execute 382 sec latency 1511.301 ms 2024-09-19T04:30:04.846 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:30:04.847 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:30:05.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:04 smithi107 ceph-mon[30659]: pgmap v302: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-09-19T04:30:05.243 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:05.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:04 smithi115 ceph-mon[35477]: pgmap v302: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-09-19T04:30:05.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:04 smithi145 ceph-mon[35096]: pgmap v302: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-09-19T04:30:05.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 319245 2.69 MB/sec execute 383 sec latency 597.939 ms 2024-09-19T04:30:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.826+0000 7f4e4d000640 1 -- 172.21.15.107:0/3131614003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e481068d0 msgr2=0x7f4e48106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.826+0000 7f4e4d000640 1 --2- 172.21.15.107:0/3131614003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e481068d0 0x7f4e48106cb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009900 tx=0x7f4e3802a400 comp rx=0 tx=0).stop 2024-09-19T04:30:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.827+0000 7f4e4d000640 1 -- 172.21.15.107:0/3131614003 shutdown_connections 2024-09-19T04:30:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.827+0000 7f4e4d000640 1 --2- 172.21.15.107:0/3131614003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e48101270 0x7f4e4810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.827+0000 7f4e4d000640 1 --2- 172.21.15.107:0/3131614003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e481008d0 0x7f4e48100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.827+0000 7f4e4d000640 1 --2- 172.21.15.107:0/3131614003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e481068d0 0x7f4e48106cb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.827+0000 7f4e4d000640 1 -- 172.21.15.107:0/3131614003 >> 172.21.15.107:0/3131614003 conn(0x7f4e480fc610 msgr2=0x7f4e480fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.827+0000 7f4e4d000640 1 -- 172.21.15.107:0/3131614003 shutdown_connections 2024-09-19T04:30:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.827+0000 7f4e4d000640 1 -- 172.21.15.107:0/3131614003 wait complete. 2024-09-19T04:30:05.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.827+0000 7f4e4d000640 1 Processor -- start 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e4d000640 1 -- start start 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e4d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e481008d0 0x7f4e4819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e4d000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e48101270 0x7f4e4819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e4d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e481068d0 0x7f4e4819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e4d000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48110d20 con 0x7f4e481008d0 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e4d000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48110ba0 con 0x7f4e48101270 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e4d000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48110ea0 con 0x7f4e481068d0 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e481008d0 0x7f4e4819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e46a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e48101270 0x7f4e4819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e46a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e48101270 0x7f4e4819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44892/0 (socket says 172.21.15.107:44892) 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e46a00640 1 -- 172.21.15.107:0/341739820 learned_addr learned my addr 172.21.15.107:0/341739820 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e481008d0 0x7f4e4819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40570/0 (socket says 172.21.15.107:40570) 2024-09-19T04:30:05.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.828+0000 7f4e47e00640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e481068d0 0x7f4e4819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:05.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.829+0000 7f4e46a00640 1 -- 172.21.15.107:0/341739820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e481068d0 msgr2=0x7f4e4819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:05.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.829+0000 7f4e46a00640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e481068d0 0x7f4e4819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:05.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.829+0000 7f4e46a00640 1 -- 172.21.15.107:0/341739820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e481008d0 msgr2=0x7f4e4819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:05.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.829+0000 7f4e46a00640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e481008d0 0x7f4e4819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:05.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.829+0000 7f4e46a00640 1 -- 172.21.15.107:0/341739820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4819f490 con 0x7f4e48101270 2024-09-19T04:30:05.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.829+0000 7f4e47e00640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e481068d0 0x7f4e4819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:05.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.829+0000 7f4e46a00640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e48101270 0x7f4e4819aa20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400ea60 tx=0x7f4e3400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:05.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.829+0000 7f4e3fe00640 1 -- 172.21.15.107:0/341739820 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400cde0 con 0x7f4e48101270 2024-09-19T04:30:05.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.829+0000 7f4e3fe00640 1 -- 172.21.15.107:0/341739820 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e340035f0 con 0x7f4e48101270 2024-09-19T04:30:05.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.830+0000 7f4e3fe00640 1 -- 172.21.15.107:0/341739820 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34010670 con 0x7f4e48101270 2024-09-19T04:30:05.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.830+0000 7f4e4d000640 1 -- 172.21.15.107:0/341739820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4819f780 con 0x7f4e48101270 2024-09-19T04:30:05.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.830+0000 7f4e4d000640 1 -- 172.21.15.107:0/341739820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e481a7060 con 0x7f4e48101270 2024-09-19T04:30:05.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.831+0000 7f4e4d000640 1 -- 172.21.15.107:0/341739820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e48100d30 con 0x7f4e48101270 2024-09-19T04:30:05.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.833+0000 7f4e3fe00640 1 -- 172.21.15.107:0/341739820 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f4e340037a0 con 0x7f4e48101270 2024-09-19T04:30:05.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.835+0000 7f4e3fe00640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4e2c0a3430 0x7f4e2c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:05.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.835+0000 7f4e3fe00640 1 -- 172.21.15.107:0/341739820 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8528+0+0 (secure 0 0 0) 0x7f4e340d02b0 con 0x7f4e48101270 2024-09-19T04:30:05.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.835+0000 7f4e47400640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4e2c0a3430 0x7f4e2c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:05.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.836+0000 7f4e47400640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4e2c0a3430 0x7f4e2c0a58f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4e38000c00 tx=0x7f4e38005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:05.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:05.836+0000 7f4e3fe00640 1 -- 172.21.15.107:0/341739820 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e3409cc50 con 0x7f4e48101270 2024-09-19T04:30:06.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.088+0000 7f4e4d000640 1 -- 172.21.15.107:0/341739820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e4819b840 con 0x7f4e48101270 2024-09-19T04:30:06.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.089+0000 7f4e3fe00640 1 -- 172.21.15.107:0/341739820 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11992 (secure 0 0 0) 0x7f4e3409c550 con 0x7f4e48101270 2024-09-19T04:30:06.091 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:30:06.091 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":48,"btime":"2024-09-19T04:29:19:894497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:29:18.944591+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24490,24496,24500,24539,24517]},"id":1}]} 2024-09-19T04:30:06.092 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 48 2024-09-19T04:30:06.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 -- 172.21.15.107:0/341739820 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4e2c0a3430 msgr2=0x7f4e2c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:06.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4e2c0a3430 0x7f4e2c0a58f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4e38000c00 tx=0x7f4e38005b50 comp rx=0 tx=0).stop 2024-09-19T04:30:06.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 -- 172.21.15.107:0/341739820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e48101270 msgr2=0x7f4e4819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:06.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e48101270 0x7f4e4819aa20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400ea60 tx=0x7f4e3400ef20 comp rx=0 tx=0).stop 2024-09-19T04:30:06.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 -- 172.21.15.107:0/341739820 shutdown_connections 2024-09-19T04:30:06.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4e2c0a3430 0x7f4e2c0a58f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e481068d0 0x7f4e4819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e48101270 0x7f4e4819aa20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 --2- 172.21.15.107:0/341739820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e481008d0 0x7f4e4819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:06.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 -- 172.21.15.107:0/341739820 >> 172.21.15.107:0/341739820 conn(0x7f4e480fc610 msgr2=0x7f4e48069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 -- 172.21.15.107:0/341739820 shutdown_connections 2024-09-19T04:30:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:06.092+0000 7f4e4d000640 1 -- 172.21.15.107:0/341739820 wait complete. 2024-09-19T04:30:06.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 320305 2.69 MB/sec execute 384 sec latency 15.605 ms 2024-09-19T04:30:06.870 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:30:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:06 smithi107 ceph-mon[30659]: pgmap v303: 97 pgs: 97 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-19T04:30:07.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:06 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/341739820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:30:07.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:06 smithi107 ceph-mon[30659]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:30:07.264 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:06 smithi115 ceph-mon[35477]: pgmap v303: 97 pgs: 97 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-19T04:30:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:06 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/341739820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:30:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:06 smithi115 ceph-mon[35477]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:30:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:06 smithi145 ceph-mon[35096]: pgmap v303: 97 pgs: 97 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-19T04:30:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:06 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/341739820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:30:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:06 smithi145 ceph-mon[35096]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:30:07.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 320938 2.69 MB/sec execute 385 sec latency 606.828 ms 2024-09-19T04:30:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.878+0000 7f7768600640 1 -- 172.21.15.107:0/3269042305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77601049c0 msgr2=0x7f7760108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.878+0000 7f7768600640 1 --2- 172.21.15.107:0/3269042305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77601049c0 0x7f7760108a10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f774c009900 tx=0x7f774c02a410 comp rx=0 tx=0).stop 2024-09-19T04:30:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.878+0000 7f7768600640 1 -- 172.21.15.107:0/3269042305 shutdown_connections 2024-09-19T04:30:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.878+0000 7f7768600640 1 --2- 172.21.15.107:0/3269042305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7760109140 0x7f7760110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.878+0000 7f7768600640 1 --2- 172.21.15.107:0/3269042305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77601049c0 0x7f7760108a10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.878+0000 7f7768600640 1 --2- 172.21.15.107:0/3269042305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7760104010 0x7f77601043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.878+0000 7f7768600640 1 -- 172.21.15.107:0/3269042305 >> 172.21.15.107:0/3269042305 conn(0x7f77600ffa50 msgr2=0x7f7760101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.878+0000 7f7768600640 1 -- 172.21.15.107:0/3269042305 shutdown_connections 2024-09-19T04:30:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.878+0000 7f7768600640 1 -- 172.21.15.107:0/3269042305 wait complete. 2024-09-19T04:30:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.879+0000 7f7768600640 1 Processor -- start 2024-09-19T04:30:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.879+0000 7f7768600640 1 -- start start 2024-09-19T04:30:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.879+0000 7f7768600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7760104010 0x7f776019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7768600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77601049c0 0x7f776019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7768600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7760109140 0x7f776019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7768600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7760114160 con 0x7f7760109140 2024-09-19T04:30:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7768600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7760113fe0 con 0x7f77601049c0 2024-09-19T04:30:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7768600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77601142e0 con 0x7f7760104010 2024-09-19T04:30:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7765c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7760104010 0x7f776019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7765c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7760104010 0x7f776019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46104/0 (socket says 172.21.15.107:46104) 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7765c00640 1 -- 172.21.15.107:0/3398749052 learned_addr learned my addr 172.21.15.107:0/3398749052 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7765200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77601049c0 0x7f776019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7766600640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7760109140 0x7f776019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7765c00640 1 -- 172.21.15.107:0/3398749052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77601049c0 msgr2=0x7f776019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7765c00640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77601049c0 0x7f776019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7765c00640 1 -- 172.21.15.107:0/3398749052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7760109140 msgr2=0x7f776019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7765c00640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7760109140 0x7f776019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.880+0000 7f7765c00640 1 -- 172.21.15.107:0/3398749052 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776019f690 con 0x7f7760104010 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.881+0000 7f7765c00640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7760104010 0x7f776019a6e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f775400e9f0 tx=0x7f775400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.881+0000 7f775e000640 1 -- 172.21.15.107:0/3398749052 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775400cd80 con 0x7f7760104010 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.881+0000 7f775e000640 1 -- 172.21.15.107:0/3398749052 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7754005500 con 0x7f7760104010 2024-09-19T04:30:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.881+0000 7f7768600640 1 -- 172.21.15.107:0/3398749052 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776019f980 con 0x7f7760104010 2024-09-19T04:30:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.881+0000 7f775e000640 1 -- 172.21.15.107:0/3398749052 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754010690 con 0x7f7760104010 2024-09-19T04:30:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.881+0000 7f7768600640 1 -- 172.21.15.107:0/3398749052 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77601a7260 con 0x7f7760104010 2024-09-19T04:30:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.885+0000 7f776b02a740 1 Processor -- start 2024-09-19T04:30:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.885+0000 7f776b02a740 1 -- start start 2024-09-19T04:30:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.885+0000 7f775e000640 1 -- 172.21.15.107:0/3398749052 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f77540046c0 con 0x7f7760104010 2024-09-19T04:30:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.885+0000 7f776b02a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 0x55d707d24050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.885+0000 7f776b02a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d24590 0x55d707d2e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.885+0000 7f776b02a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d2ee50 0x55d707d31240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f776b02a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d707c8a3e0 con 0x55d707d24590 2024-09-19T04:30:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f776b02a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d707c31d80 con 0x55d707d23c70 2024-09-19T04:30:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f776b02a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d707c863a0 con 0x55d707d2ee50 2024-09-19T04:30:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f7765c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d24590 0x55d707d2e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f7766600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 0x55d707d24050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f7766600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 0x55d707d24050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44938/0 (socket says 172.21.15.107:44938) 2024-09-19T04:30:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f7766600640 1 -- 172.21.15.107:0/2866229197 learned_addr learned my addr 172.21.15.107:0/2866229197 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f7765200640 1 --2- 172.21.15.107:0/2866229197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d2ee50 0x55d707d31240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f7766600640 1 -- 172.21.15.107:0/2866229197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d2ee50 msgr2=0x55d707d31240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f7766600640 1 --2- 172.21.15.107:0/2866229197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d2ee50 0x55d707d31240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f7766600640 1 -- 172.21.15.107:0/2866229197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d24590 msgr2=0x55d707d2e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f7766600640 1 --2- 172.21.15.107:0/2866229197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d24590 0x55d707d2e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.886+0000 7f7766600640 1 -- 172.21.15.107:0/2866229197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d707c50ae0 con 0x55d707d23c70 2024-09-19T04:30:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.887+0000 7f7766600640 1 --2- 172.21.15.107:0/2866229197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 0x55d707d24050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7758009f50 tx=0x7f77580118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcd0262317b30639 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.887+0000 7f7752a00640 1 -- 172.21.15.107:0/2866229197 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758004b20 con 0x55d707d23c70 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.887+0000 7f7752a00640 1 -- 172.21.15.107:0/2866229197 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7758004cc0 con 0x55d707d23c70 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.887+0000 7f7752a00640 1 -- 172.21.15.107:0/2866229197 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758005020 con 0x55d707d23c70 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.887+0000 7f775e000640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77480a3430 0x7f77480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.887+0000 7f775e000640 1 -- 172.21.15.107:0/3398749052 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f77540d00e0 con 0x7f7760104010 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.887+0000 7f7765c00640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77480a3430 0x7f77480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.888+0000 7f776b02a740 1 -- 172.21.15.107:0/2866229197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 msgr2=0x55d707d24050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.888+0000 7f776b02a740 1 --2- 172.21.15.107:0/2866229197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 0x55d707d24050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7758009f50 tx=0x7f77580118d0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.888+0000 7f776b02a740 1 -- 172.21.15.107:0/2866229197 shutdown_connections 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.888+0000 7f776b02a740 1 --2- 172.21.15.107:0/2866229197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d2ee50 0x55d707d31240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.888+0000 7f776b02a740 1 --2- 172.21.15.107:0/2866229197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d24590 0x55d707d2e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.888+0000 7f776b02a740 1 --2- 172.21.15.107:0/2866229197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 0x55d707d24050 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.888+0000 7f776b02a740 1 -- 172.21.15.107:0/2866229197 >> 172.21.15.107:0/2866229197 conn(0x55d707d20200 msgr2=0x55d707d205e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:07.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.888+0000 7f7765c00640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77480a3430 0x7f77480a58f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55d707d21060 tx=0x7f77540bc040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:07.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.888+0000 7f776b02a740 1 -- 172.21.15.107:0/2866229197 shutdown_connections 2024-09-19T04:30:07.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.888+0000 7f776b02a740 1 -- 172.21.15.107:0/2866229197 wait complete. 2024-09-19T04:30:07.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.889+0000 7f776b02a740 1 -- start start 2024-09-19T04:30:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.889+0000 7f776b02a740 1 Processor -- start 2024-09-19T04:30:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.889+0000 7f776b02a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 0x55d707d2bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.889+0000 7f776b02a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d2c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.889+0000 7f776b02a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d2ee50 0x55d707d2c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.889+0000 7f776b02a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d707d31a20 con 0x55d707d2ee50 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.889+0000 7f776b02a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d707c450f0 con 0x55d707d23c70 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.889+0000 7f776b02a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d707d31ba0 con 0x55d707d24590 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.889+0000 7f776b02a740 10 client.0 _ll_register_callbacks cb 0x55d707c2cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.890+0000 7f7765200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d2c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.890+0000 7f7766600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d2ee50 0x55d707d2c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.890+0000 7f7765200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d2c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46134/0 (socket says 172.21.15.107:46134) 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.890+0000 7f7765200640 1 -- 172.21.15.107:0/1229086119 learned_addr learned my addr 172.21.15.107:0/1229086119 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.890+0000 7f7766600640 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 0x55d707d2bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.890+0000 7f7765200640 1 -- 172.21.15.107:0/1229086119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 msgr2=0x55d707d2bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.890+0000 7f7765200640 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 0x55d707d2bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.890+0000 7f7765200640 1 -- 172.21.15.107:0/1229086119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d2ee50 msgr2=0x55d707d2c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.890+0000 7f7765200640 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d2ee50 0x55d707d2c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.890+0000 7f7765200640 1 -- 172.21.15.107:0/1229086119 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7079fb3c0 con 0x55d707d24590 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.891+0000 7f7765200640 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d2c230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f774c02a930 tx=0x7f774c0317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c043665910de766e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.891+0000 7f7747400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.891+0000 7f7747400640 1 -- 172.21.15.107:0/1229086119 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c035040 con 0x55d707d24590 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.891+0000 7f776b02a740 1 -- 172.21.15.107:0/1229086119 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7758004cc0 con 0x55d707d24590 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.891+0000 7f7747400640 1 -- 172.21.15.107:0/1229086119 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f774c004240 con 0x55d707d24590 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.891+0000 7f7747400640 1 -- 172.21.15.107:0/1229086119 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f774c004440 con 0x55d707d24590 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.891+0000 7f776b02a740 10 client.26932 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.891+0000 7f776b02a740 1 -- 172.21.15.107:0/1229086119 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d707d3da60 con 0x55d707d24590 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.892+0000 7f7747400640 1 -- 172.21.15.107:0/1229086119 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7758004cc0 con 0x55d707d24590 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.892+0000 7f7747400640 1 -- 172.21.15.107:0/1229086119 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f774c04d070 con 0x55d707d24590 2024-09-19T04:30:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.892+0000 7f776b02a740 10 client.26932 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.892+0000 7f776b02a740 10 client.26932 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:30:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.892+0000 7f776b02a740 20 client.26932 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.892+0000 7f776b02a740 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55d707d3ef30 0x55d707d41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.892+0000 7f776b02a740 4 client.26932 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.892+0000 7f776b02a740 1 -- 172.21.15.107:0/1229086119 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d707c97950 con 0x55d707d3ef30 2024-09-19T04:30:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.892+0000 7f7766600640 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55d707d3ef30 0x55d707d41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.893+0000 7f7766600640 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55d707d3ef30 0x55d707d41350 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d7cf3e456ecbd0a server_cookie=b6ae32bc7253e890 in_seq=0 out_seq=0 2024-09-19T04:30:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.893+0000 7f7747400640 10 client.26932 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:30:07.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.893+0000 7f7747400640 1 -- 172.21.15.107:0/1229086119 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d707c97950 con 0x55d707d3ef30 2024-09-19T04:30:07.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.893+0000 7f7747400640 10 client.26932 handle_command_reply: tid=0 2024-09-19T04:30:07.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 client.26932 shutdown 2024-09-19T04:30:07.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 -- 172.21.15.107:0/1229086119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 msgr2=0x55d707d2c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d2c230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f774c02a930 tx=0x7f774c0317e0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 -- 172.21.15.107:0/1229086119 shutdown_connections 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55d707d3ef30 0x55d707d41350 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d2ee50 0x55d707d2c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f7747400640 0 client.26932 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d2c230 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 --2- 172.21.15.107:0/1229086119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d23c70 0x55d707d2bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 -- 172.21.15.107:0/1229086119 >> 172.21.15.107:0/1229086119 conn(0x55d707d20200 msgr2=0x55d707d25f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 -- 172.21.15.107:0/1229086119 shutdown_connections 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 1 -- 172.21.15.107:0/1229086119 wait complete. 2024-09-19T04:30:07.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.894+0000 7f776b02a740 20 client.26932 trim_cache size 0 max 16384 2024-09-19T04:30:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.908+0000 7f776b02a740 1 Processor -- start 2024-09-19T04:30:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.908+0000 7f776b02a740 1 -- start start 2024-09-19T04:30:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.908+0000 7f776b02a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 0x55d707d256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.908+0000 7f776b02a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d24590 0x55d707d2c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.908+0000 7f776b02a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d2ee50 0x55d707d46c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.908+0000 7f776b02a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77601afa30 con 0x55d707d23c70 2024-09-19T04:30:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.908+0000 7f776b02a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77601b1570 con 0x55d707d24590 2024-09-19T04:30:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.908+0000 7f776b02a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77601b0100 con 0x55d707d2ee50 2024-09-19T04:30:07.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7766600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 0x55d707d256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7766600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 0x55d707d256c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40604/0 (socket says 172.21.15.107:40604) 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7766600640 1 -- 172.21.15.107:0/851229736 learned_addr learned my addr 172.21.15.107:0/851229736 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7765200640 1 --2- 172.21.15.107:0/851229736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d24590 0x55d707d2c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7766600640 1 --2- 172.21.15.107:0/851229736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d2ee50 0x55d707d46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7766600640 1 -- 172.21.15.107:0/851229736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d2ee50 msgr2=0x55d707d46c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7766600640 1 --2- 172.21.15.107:0/851229736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d2ee50 0x55d707d46c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7766600640 1 -- 172.21.15.107:0/851229736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d24590 msgr2=0x55d707d2c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7766600640 1 --2- 172.21.15.107:0/851229736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d24590 0x55d707d2c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7766600640 1 -- 172.21.15.107:0/851229736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d707d20030 con 0x55d707d23c70 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.909+0000 7f7766600640 1 --2- 172.21.15.107:0/851229736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 0x55d707d256c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7758009270 tx=0x7f7758009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698e20aabdce654d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.910+0000 7f7752000640 1 -- 172.21.15.107:0/851229736 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758034070 con 0x55d707d23c70 2024-09-19T04:30:07.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.910+0000 7f7752000640 1 -- 172.21.15.107:0/851229736 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7758004ca0 con 0x55d707d23c70 2024-09-19T04:30:07.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.910+0000 7f7752000640 1 -- 172.21.15.107:0/851229736 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758039aa0 con 0x55d707d23c70 2024-09-19T04:30:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.910+0000 7f776b02a740 1 -- 172.21.15.107:0/851229736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 msgr2=0x55d707d256c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.910+0000 7f776b02a740 1 --2- 172.21.15.107:0/851229736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 0x55d707d256c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7758009270 tx=0x7f7758009500 comp rx=0 tx=0).stop 2024-09-19T04:30:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.910+0000 7f776b02a740 1 -- 172.21.15.107:0/851229736 shutdown_connections 2024-09-19T04:30:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.910+0000 7f776b02a740 1 --2- 172.21.15.107:0/851229736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d2ee50 0x55d707d46c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.910+0000 7f776b02a740 1 --2- 172.21.15.107:0/851229736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d24590 0x55d707d2c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.910+0000 7f776b02a740 1 --2- 172.21.15.107:0/851229736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 0x55d707d256c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.910+0000 7f776b02a740 1 -- 172.21.15.107:0/851229736 >> 172.21.15.107:0/851229736 conn(0x55d707d20200 msgr2=0x55d707d2acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.911+0000 7f776b02a740 1 -- 172.21.15.107:0/851229736 shutdown_connections 2024-09-19T04:30:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.911+0000 7f776b02a740 1 -- 172.21.15.107:0/851229736 wait complete. 2024-09-19T04:30:07.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.911+0000 7f776b02a740 1 -- start start 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.911+0000 7f776b02a740 1 Processor -- start 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f776b02a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 0x55d707d37400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f776b02a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d28980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f776b02a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d2ee50 0x55d707d28ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f776b02a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77601b0ea0 con 0x55d707d23c70 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f776b02a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77601b07d0 con 0x55d707d2ee50 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f776b02a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77601ada30 con 0x55d707d24590 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f776b02a740 10 client.0 _ll_register_callbacks cb 0x7f774c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7765200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d28980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7766600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 0x55d707d37400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7765200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d28980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46166/0 (socket says 172.21.15.107:46166) 2024-09-19T04:30:07.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7765200640 1 -- 172.21.15.107:0/844108840 learned_addr learned my addr 172.21.15.107:0/844108840 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7766600640 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d2ee50 0x55d707d28ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7765200640 1 -- 172.21.15.107:0/844108840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d2ee50 msgr2=0x55d707d28ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7765200640 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d2ee50 0x55d707d28ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7765200640 1 -- 172.21.15.107:0/844108840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 msgr2=0x55d707d37400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7765200640 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 0x55d707d37400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7765200640 1 -- 172.21.15.107:0/844108840 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d707d1b770 con 0x55d707d24590 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.912+0000 7f7766600640 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d2ee50 0x55d707d28ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.913+0000 7f7765200640 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d28980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f774c005940 tx=0x7f774c03da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33439c526a8a1f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.913+0000 7f7747400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.913+0000 7f7747400640 1 -- 172.21.15.107:0/844108840 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c05c060 con 0x55d707d24590 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.913+0000 7f7747400640 1 -- 172.21.15.107:0/844108840 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f774c005240 con 0x55d707d24590 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.913+0000 7f776b02a740 1 -- 172.21.15.107:0/844108840 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7758004ca0 con 0x55d707d24590 2024-09-19T04:30:07.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.913+0000 7f7747400640 1 -- 172.21.15.107:0/844108840 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f774c053040 con 0x55d707d24590 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.913+0000 7f776b02a740 10 client.26938 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.913+0000 7f776b02a740 1 -- 172.21.15.107:0/844108840 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d707da9940 con 0x55d707d24590 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.914+0000 7f7747400640 1 -- 172.21.15.107:0/844108840 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7758004ca0 con 0x55d707d24590 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.914+0000 7f7747400640 1 -- 172.21.15.107:0/844108840 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f774c04d040 con 0x55d707d24590 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.914+0000 7f776b02a740 10 client.26938 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.914+0000 7f776b02a740 10 client.26938 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.914+0000 7f776b02a740 20 client.26938 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.914+0000 7f776b02a740 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55d707daa040 0x55d707dac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.914+0000 7f776b02a740 4 client.26938 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.914+0000 7f776b02a740 1 -- 172.21.15.107:0/844108840 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d707c91e60 con 0x55d707daa040 2024-09-19T04:30:07.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.914+0000 7f7766600640 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55d707daa040 0x55d707dac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:07.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f7766600640 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55d707daa040 0x55d707dac460 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8607975e20a93032 server_cookie=deba3c1cae9bac56 in_seq=0 out_seq=0 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f7747400640 10 client.26938 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f7747400640 1 -- 172.21.15.107:0/844108840 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d707c91e60 con 0x55d707daa040 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f7747400640 10 client.26938 handle_command_reply: tid=0 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 client.26938 shutdown 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 -- 172.21.15.107:0/844108840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 msgr2=0x55d707d28980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d28980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f774c005940 tx=0x7f774c03da30 comp rx=0 tx=0).stop 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 -- 172.21.15.107:0/844108840 shutdown_connections 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55d707daa040 0x55d707dac460 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d707d2ee50 0x55d707d28ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d707d24590 0x55d707d28980 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f7747400640 0 client.26938 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:30:07.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 --2- 172.21.15.107:0/844108840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d707d23c70 0x55d707d37400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 -- 172.21.15.107:0/844108840 >> 172.21.15.107:0/844108840 conn(0x55d707d20200 msgr2=0x55d707d2acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 -- 172.21.15.107:0/844108840 shutdown_connections 2024-09-19T04:30:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.915+0000 7f776b02a740 1 -- 172.21.15.107:0/844108840 wait complete. 2024-09-19T04:30:07.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f776b02a740 20 client.26938 trim_cache size 0 max 16384 2024-09-19T04:30:07.922 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:30:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f7768600640 1 -- 172.21.15.107:0/3398749052 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77480a3430 msgr2=0x7f77480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f7768600640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77480a3430 0x7f77480a58f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55d707d21060 tx=0x7f77540bc040 comp rx=0 tx=0).stop 2024-09-19T04:30:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f7768600640 1 -- 172.21.15.107:0/3398749052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7760104010 msgr2=0x7f776019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f7768600640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7760104010 0x7f776019a6e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f775400e9f0 tx=0x7f775400edf0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f7768600640 1 -- 172.21.15.107:0/3398749052 shutdown_connections 2024-09-19T04:30:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f7768600640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f77480a3430 0x7f77480a58f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f7768600640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7760109140 0x7f776019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f7768600640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77601049c0 0x7f776019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f7768600640 1 --2- 172.21.15.107:0/3398749052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7760104010 0x7f776019a6e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:07.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.916+0000 7f7768600640 1 -- 172.21.15.107:0/3398749052 >> 172.21.15.107:0/3398749052 conn(0x7f77600ffa50 msgr2=0x7f7760100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:07.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.917+0000 7f7768600640 1 -- 172.21.15.107:0/3398749052 shutdown_connections 2024-09-19T04:30:07.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:07.917+0000 7f7768600640 1 -- 172.21.15.107:0/3398749052 wait complete. 2024-09-19T04:30:08.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 320938 2.69 MB/sec execute 386 sec latency 1606.943 ms 2024-09-19T04:30:08.827 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:30:09.199 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:09.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:08 smithi107 ceph-mon[30659]: pgmap v305: 97 pgs: 97 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-19T04:30:09.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:08 smithi115 ceph-mon[35477]: pgmap v305: 97 pgs: 97 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-19T04:30:09.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:08 smithi145 ceph-mon[35096]: pgmap v305: 97 pgs: 97 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-19T04:30:09.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 321648 2.69 MB/sec execute 387 sec latency 2112.162 ms 2024-09-19T04:30:09.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- 172.21.15.107:0/3632616317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75540745c0 msgr2=0x7f75540749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:09.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 --2- 172.21.15.107:0/3632616317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75540745c0 0x7f75540749a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7544009900 tx=0x7f754402a480 comp rx=0 tx=0).stop 2024-09-19T04:30:09.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- 172.21.15.107:0/3632616317 shutdown_connections 2024-09-19T04:30:09.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 --2- 172.21.15.107:0/3632616317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f755410e260 0x7f7554110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 --2- 172.21.15.107:0/3632616317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7554074ee0 0x7f755410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 --2- 172.21.15.107:0/3632616317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75540745c0 0x7f75540749a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- 172.21.15.107:0/3632616317 >> 172.21.15.107:0/3632616317 conn(0x7f75540fd740 msgr2=0x7f75540ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:09.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- 172.21.15.107:0/3632616317 shutdown_connections 2024-09-19T04:30:09.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- 172.21.15.107:0/3632616317 wait complete. 2024-09-19T04:30:09.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 Processor -- start 2024-09-19T04:30:09.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- start start 2024-09-19T04:30:09.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75540745c0 0x7f755419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7554074ee0 0x7f755419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755410e260 0x7f755419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7554114000 con 0x7f755410e260 2024-09-19T04:30:09.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7554113e80 con 0x7f7554074ee0 2024-09-19T04:30:09.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7554114180 con 0x7f75540745c0 2024-09-19T04:30:09.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7553400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7554074ee0 0x7f755419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755410e260 0x7f755419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7553400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7554074ee0 0x7f755419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47484/0 (socket says 172.21.15.107:47484) 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7553400640 1 -- 172.21.15.107:0/2852894882 learned_addr learned my addr 172.21.15.107:0/2852894882 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7553e00640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75540745c0 0x7f755419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7553400640 1 -- 172.21.15.107:0/2852894882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75540745c0 msgr2=0x7f755419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7553400640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75540745c0 0x7f755419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7553400640 1 -- 172.21.15.107:0/2852894882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755410e260 msgr2=0x7f755419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7553400640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755410e260 0x7f755419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7553400640 1 -- 172.21.15.107:0/2852894882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755419f450 con 0x7f7554074ee0 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7553400640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7554074ee0 0x7f755419a9e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f754000e9f0 tx=0x7f754000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:09.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7550c00640 1 -- 172.21.15.107:0/2852894882 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754000cd80 con 0x7f7554074ee0 2024-09-19T04:30:09.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- 172.21.15.107:0/2852894882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755419f740 con 0x7f7554074ee0 2024-09-19T04:30:09.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7559c00640 1 -- 172.21.15.107:0/2852894882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75541a7020 con 0x7f7554074ee0 2024-09-19T04:30:09.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7550c00640 1 -- 172.21.15.107:0/2852894882 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7540010910 con 0x7f7554074ee0 2024-09-19T04:30:09.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.830+0000 7f7550c00640 1 -- 172.21.15.107:0/2852894882 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540018980 con 0x7f7554074ee0 2024-09-19T04:30:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f755dc05740 1 Processor -- start 2024-09-19T04:30:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f755dc05740 1 -- start start 2024-09-19T04:30:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f7550c00640 1 -- 172.21.15.107:0/2852894882 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f7540010ac0 con 0x7f7554074ee0 2024-09-19T04:30:09.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f755dc05740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f755dc05740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f755dc05740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0dc020 0x55a3ef0de410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f755dc05740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a3ef0373e0 con 0x55a3ef0d1760 2024-09-19T04:30:09.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f755dc05740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a3eefded80 con 0x55a3ef0d0e40 2024-09-19T04:30:09.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f755dc05740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a3ef0333a0 con 0x55a3ef0dc020 2024-09-19T04:30:09.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f7553e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f7559200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f7553e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0dbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60738/0 (socket says 172.21.15.107:60738) 2024-09-19T04:30:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f7559200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47494/0 (socket says 172.21.15.107:47494) 2024-09-19T04:30:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f7553e00640 1 -- 172.21.15.107:0/3653727209 learned_addr learned my addr 172.21.15.107:0/3653727209 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.836+0000 7f7553400640 1 --2- 172.21.15.107:0/3653727209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0dc020 0x55a3ef0de410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553e00640 1 -- 172.21.15.107:0/3653727209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0dc020 msgr2=0x55a3ef0de410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553e00640 1 --2- 172.21.15.107:0/3653727209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0dc020 0x55a3ef0de410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553e00640 1 -- 172.21.15.107:0/3653727209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 msgr2=0x55a3ef0d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553e00640 1 --2- 172.21.15.107:0/3653727209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553e00640 1 -- 172.21.15.107:0/3653727209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3eeffdae0 con 0x55a3ef0d1760 2024-09-19T04:30:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553e00640 1 --2- 172.21.15.107:0/3653727209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0dbae0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7544009900 tx=0x7f7544005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb49ec7a17642a47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7550c00640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75380a33b0 0x7f75380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7549600640 1 -- 172.21.15.107:0/3653727209 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544035030 con 0x55a3ef0d1760 2024-09-19T04:30:09.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7550c00640 1 -- 172.21.15.107:0/2852894882 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f75400d0060 con 0x7f7554074ee0 2024-09-19T04:30:09.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7549600640 1 -- 172.21.15.107:0/3653727209 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75440043c0 con 0x55a3ef0d1760 2024-09-19T04:30:09.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7549600640 1 -- 172.21.15.107:0/3653727209 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75440035f0 con 0x55a3ef0d1760 2024-09-19T04:30:09.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7559200640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75380a33b0 0x7f75380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- 172.21.15.107:0/3653727209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 msgr2=0x55a3ef0dbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7559200640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75380a33b0 0x7f75380a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f755406a6f0 tx=0x7f754c009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 --2- 172.21.15.107:0/3653727209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0dbae0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7544009900 tx=0x7f7544005e60 comp rx=0 tx=0).stop 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- 172.21.15.107:0/3653727209 shutdown_connections 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 --2- 172.21.15.107:0/3653727209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0dc020 0x55a3ef0de410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 --2- 172.21.15.107:0/3653727209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0dbae0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 --2- 172.21.15.107:0/3653727209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- 172.21.15.107:0/3653727209 >> 172.21.15.107:0/3653727209 conn(0x55a3ef0cd3d0 msgr2=0x55a3ef0cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- 172.21.15.107:0/3653727209 shutdown_connections 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- 172.21.15.107:0/3653727209 wait complete. 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- start start 2024-09-19T04:30:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 Processor -- start 2024-09-19T04:30:09.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0d8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d1760 0x55a3ef0d9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 0x55a3ef0d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a3ef0debf0 con 0x55a3ef0dc020 2024-09-19T04:30:09.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a3eeff20f0 con 0x55a3ef0d1760 2024-09-19T04:30:09.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a3ef0ded70 con 0x55a3ef0d0e40 2024-09-19T04:30:09.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 10 client.0 _ll_register_callbacks cb 0x55a3eefd9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:09.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d1760 0x55a3ef0d9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d1760 0x55a3ef0d9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47498/0 (socket says 172.21.15.107:47498) 2024-09-19T04:30:09.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553400640 1 -- 172.21.15.107:0/3034719282 learned_addr learned my addr 172.21.15.107:0/3034719282 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553e00640 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0d8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7559200640 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 0x55a3ef0d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553400640 1 -- 172.21.15.107:0/3034719282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d0e40 msgr2=0x55a3ef0d8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553400640 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0d8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553400640 1 -- 172.21.15.107:0/3034719282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 msgr2=0x55a3ef0d9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553400640 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 0x55a3ef0d9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553400640 1 -- 172.21.15.107:0/3034719282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3eeda83c0 con 0x55a3ef0d1760 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7559200640 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 0x55a3ef0d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f7553400640 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d1760 0x55a3ef0d9420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7540018530 tx=0x7f7540097e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21376a17f54f56a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f753e000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f753e000640 1 -- 172.21.15.107:0/3034719282 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754009be40 con 0x55a3ef0d1760 2024-09-19T04:30:09.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f753e000640 1 -- 172.21.15.107:0/3034719282 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f75400cd9f0 con 0x55a3ef0d1760 2024-09-19T04:30:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- 172.21.15.107:0/3034719282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75440043c0 con 0x55a3ef0d1760 2024-09-19T04:30:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f753e000640 1 -- 172.21.15.107:0/3034719282 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f754009bc60 con 0x55a3ef0d1760 2024-09-19T04:30:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 10 client.26984 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:09.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.840+0000 7f755dc05740 1 -- 172.21.15.107:0/3034719282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a3ef0ead20 con 0x55a3ef0d1760 2024-09-19T04:30:09.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f753e000640 1 -- 172.21.15.107:0/3034719282 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75440043c0 con 0x55a3ef0d1760 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f753e000640 1 -- 172.21.15.107:0/3034719282 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f75400bc0d0 con 0x55a3ef0d1760 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 10 client.26984 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 10 client.26984 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 20 client.26984 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3ef0ec1d0 0x55a3ef0ee650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 4 client.26984 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 -- 172.21.15.107:0/3034719282 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3ef044950 con 0x55a3ef0ec1d0 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f7553e00640 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3ef0ec1d0 0x55a3ef0ee650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f7553e00640 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3ef0ec1d0 0x55a3ef0ee650 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=20a1f7e1d0aab074 server_cookie=5f9a2cf8079591ff in_seq=0 out_seq=0 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f753e000640 10 client.26984 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f753e000640 1 -- 172.21.15.107:0/3034719282 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3ef044950 con 0x55a3ef0ec1d0 2024-09-19T04:30:09.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f753e000640 10 client.26984 handle_command_reply: tid=0 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 client.26984 shutdown 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 -- 172.21.15.107:0/3034719282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d1760 msgr2=0x55a3ef0d9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d1760 0x55a3ef0d9420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7540018530 tx=0x7f7540097e80 comp rx=0 tx=0).stop 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 -- 172.21.15.107:0/3034719282 shutdown_connections 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3ef0ec1d0 0x55a3ef0ee650 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 0x55a3ef0d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d1760 0x55a3ef0d9420 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 --2- 172.21.15.107:0/3034719282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0d8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 -- 172.21.15.107:0/3034719282 >> 172.21.15.107:0/3034719282 conn(0x55a3ef0cd3d0 msgr2=0x55a3ef0d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f753e000640 0 client.26984 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 -- 172.21.15.107:0/3034719282 shutdown_connections 2024-09-19T04:30:09.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 1 -- 172.21.15.107:0/3034719282 wait complete. 2024-09-19T04:30:09.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.846+0000 7f755dc05740 20 client.26984 trim_cache size 0 max 16384 2024-09-19T04:30:09.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f755dc05740 1 Processor -- start 2024-09-19T04:30:09.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f755dc05740 1 -- start start 2024-09-19T04:30:09.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f755dc05740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f755dc05740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d1760 0x55a3ef0f4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f755dc05740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 0x55a3ef0d6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f755dc05740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f753400e6e0 con 0x55a3ef0dc020 2024-09-19T04:30:09.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f755dc05740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7534010220 con 0x55a3ef0d0e40 2024-09-19T04:30:09.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f755dc05740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f753400edb0 con 0x55a3ef0d1760 2024-09-19T04:30:09.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7553e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7553400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d1760 0x55a3ef0f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7553e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0f3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47500/0 (socket says 172.21.15.107:47500) 2024-09-19T04:30:09.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7553400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d1760 0x55a3ef0f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46162/0 (socket says 172.21.15.107:46162) 2024-09-19T04:30:09.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7553e00640 1 -- 172.21.15.107:0/3345463092 learned_addr learned my addr 172.21.15.107:0/3345463092 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:09.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7559200640 1 --2- 172.21.15.107:0/3345463092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 0x55a3ef0d6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7553e00640 1 -- 172.21.15.107:0/3345463092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d1760 msgr2=0x55a3ef0f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7553e00640 1 --2- 172.21.15.107:0/3345463092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d1760 0x55a3ef0f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7553e00640 1 -- 172.21.15.107:0/3345463092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 msgr2=0x55a3ef0d6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7553e00640 1 --2- 172.21.15.107:0/3345463092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 0x55a3ef0d6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.861+0000 7f7553e00640 1 -- 172.21.15.107:0/3345463092 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3ef0cd200 con 0x55a3ef0d0e40 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553e00640 1 --2- 172.21.15.107:0/3345463092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0f3f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f754402aa70 tx=0x7f75440058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=814b78b6fe9bac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7548c00640 1 -- 172.21.15.107:0/3345463092 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544035040 con 0x55a3ef0d0e40 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7548c00640 1 -- 172.21.15.107:0/3345463092 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7544004340 con 0x55a3ef0d0e40 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7548c00640 1 -- 172.21.15.107:0/3345463092 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544031810 con 0x55a3ef0d0e40 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- 172.21.15.107:0/3345463092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 msgr2=0x55a3ef0f3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 --2- 172.21.15.107:0/3345463092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0f3f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f754402aa70 tx=0x7f75440058f0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- 172.21.15.107:0/3345463092 shutdown_connections 2024-09-19T04:30:09.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 --2- 172.21.15.107:0/3345463092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0dc020 0x55a3ef0d6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 --2- 172.21.15.107:0/3345463092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0d1760 0x55a3ef0f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 --2- 172.21.15.107:0/3345463092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0f3f20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- 172.21.15.107:0/3345463092 >> 172.21.15.107:0/3345463092 conn(0x55a3ef0cd3d0 msgr2=0x55a3ef0daf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:09.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- 172.21.15.107:0/3345463092 shutdown_connections 2024-09-19T04:30:09.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- 172.21.15.107:0/3345463092 wait complete. 2024-09-19T04:30:09.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- start start 2024-09-19T04:30:09.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 Processor -- start 2024-09-19T04:30:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0e4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0dc020 0x55a3ef0e1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f753400fb50 con 0x55a3ef0d1760 2024-09-19T04:30:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f753400f480 con 0x55a3ef0d0e40 2024-09-19T04:30:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f753400c6e0 con 0x55a3ef0dc020 2024-09-19T04:30:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 10 client.0 _ll_register_callbacks cb 0x7f75400d3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:09.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0e4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47512/0 (socket says 172.21.15.107:47512) 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553e00640 1 -- 172.21.15.107:0/404096841 learned_addr learned my addr 172.21.15.107:0/404096841 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553400640 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553e00640 1 -- 172.21.15.107:0/404096841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0dc020 msgr2=0x55a3ef0e1020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553e00640 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0dc020 0x55a3ef0e1020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553e00640 1 -- 172.21.15.107:0/404096841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 msgr2=0x55a3ef0e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553e00640 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553e00640 1 -- 172.21.15.107:0/404096841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3ef0e1560 con 0x55a3ef0d0e40 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553400640 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553e00640 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0e4050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7544009900 tx=0x7f754402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcba69e472165eec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f753e000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:09.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f753e000640 1 -- 172.21.15.107:0/404096841 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544060060 con 0x55a3ef0d0e40 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f753e000640 1 -- 172.21.15.107:0/404096841 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7544003800 con 0x55a3ef0d0e40 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- 172.21.15.107:0/404096841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7544004340 con 0x55a3ef0d0e40 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f753e000640 1 -- 172.21.15.107:0/404096841 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7544035040 con 0x55a3ef0d0e40 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 10 client.26990 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- 172.21.15.107:0/404096841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a3ef157610 con 0x55a3ef0d0e40 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f753e000640 1 -- 172.21.15.107:0/404096841 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7544004340 con 0x55a3ef0d0e40 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f753e000640 1 -- 172.21.15.107:0/404096841 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f754403b430 con 0x55a3ef0d0e40 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 10 client.26990 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 10 client.26990 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 20 client.26990 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:09.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3ef157840 0x55a3ef159c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 4 client.26990 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f755dc05740 1 -- 172.21.15.107:0/404096841 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3ef03ee60 con 0x55a3ef157840 2024-09-19T04:30:09.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.864+0000 7f7553400640 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3ef157840 0x55a3ef159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f753e000640 10 client.26990 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:30:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7553400640 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3ef157840 0x55a3ef159c30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=957ca6b3ff58b0e9 server_cookie=9b5372aa19cecfa8 in_seq=0 out_seq=0 2024-09-19T04:30:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f753e000640 1 -- 172.21.15.107:0/404096841 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3ef03ee60 con 0x55a3ef157840 2024-09-19T04:30:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f753e000640 10 client.26990 handle_command_reply: tid=0 2024-09-19T04:30:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 client.26990 shutdown 2024-09-19T04:30:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 -- 172.21.15.107:0/404096841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 msgr2=0x55a3ef0e4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0e4050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7544009900 tx=0x7f754402a480 comp rx=0 tx=0).stop 2024-09-19T04:30:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 -- 172.21.15.107:0/404096841 shutdown_connections 2024-09-19T04:30:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55a3ef157840 0x55a3ef159c30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a3ef0dc020 0x55a3ef0e1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3ef0d1760 0x55a3ef0e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 --2- 172.21.15.107:0/404096841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef0d0e40 0x55a3ef0e4050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f753e000640 0 client.26990 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:30:09.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 -- 172.21.15.107:0/404096841 >> 172.21.15.107:0/404096841 conn(0x55a3ef0cd3d0 msgr2=0x55a3ef0cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:09.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 -- 172.21.15.107:0/404096841 shutdown_connections 2024-09-19T04:30:09.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 1 -- 172.21.15.107:0/404096841 wait complete. 2024-09-19T04:30:09.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f755dc05740 20 client.26990 trim_cache size 0 max 16384 2024-09-19T04:30:09.875 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 -- 172.21.15.107:0/2852894882 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75380a33b0 msgr2=0x7f75380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75380a33b0 0x7f75380a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f755406a6f0 tx=0x7f754c009500 comp rx=0 tx=0).stop 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 -- 172.21.15.107:0/2852894882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7554074ee0 msgr2=0x7f755419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7554074ee0 0x7f755419a9e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f754000e9f0 tx=0x7f754000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 -- 172.21.15.107:0/2852894882 shutdown_connections 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f75380a33b0 0x7f75380a5870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755410e260 0x7f755419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7554074ee0 0x7f755419a9e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 --2- 172.21.15.107:0/2852894882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75540745c0 0x7f755419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 -- 172.21.15.107:0/2852894882 >> 172.21.15.107:0/2852894882 conn(0x7f75540fd740 msgr2=0x7f75540fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:09.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 -- 172.21.15.107:0/2852894882 shutdown_connections 2024-09-19T04:30:09.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:09.869+0000 7f7559c00640 1 -- 172.21.15.107:0/2852894882 wait complete. 2024-09-19T04:30:10.682 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:30:10.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 322880 2.71 MB/sec execute 388 sec latency 475.853 ms 2024-09-19T04:30:11.080 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:11.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:10 smithi107 ceph-mon[30659]: pgmap v306: 97 pgs: 97 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-09-19T04:30:11.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:10 smithi115 ceph-mon[35477]: pgmap v306: 97 pgs: 97 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-09-19T04:30:11.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:10 smithi145 ceph-mon[35096]: pgmap v306: 97 pgs: 97 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-09-19T04:30:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.665+0000 7f76a3e00640 1 -- 172.21.15.107:0/555091386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a41034b0 msgr2=0x7f76a410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.665+0000 7f76a3e00640 1 --2- 172.21.15.107:0/555091386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a41034b0 0x7f76a410b9a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7690009900 tx=0x7f769002a480 comp rx=0 tx=0).stop 2024-09-19T04:30:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.665+0000 7f76a3e00640 1 -- 172.21.15.107:0/555091386 shutdown_connections 2024-09-19T04:30:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.665+0000 7f76a3e00640 1 --2- 172.21.15.107:0/555091386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76a410bfc0 0x7f76a410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.665+0000 7f76a3e00640 1 --2- 172.21.15.107:0/555091386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a41034b0 0x7f76a410b9a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.665+0000 7f76a3e00640 1 --2- 172.21.15.107:0/555091386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76a4102b90 0x7f76a4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.665+0000 7f76a3e00640 1 -- 172.21.15.107:0/555091386 >> 172.21.15.107:0/555091386 conn(0x7f76a40fc610 msgr2=0x7f76a40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.665+0000 7f76a3e00640 1 -- 172.21.15.107:0/555091386 shutdown_connections 2024-09-19T04:30:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.665+0000 7f76a3e00640 1 -- 172.21.15.107:0/555091386 wait complete. 2024-09-19T04:30:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.666+0000 7f76a3e00640 1 Processor -- start 2024-09-19T04:30:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.666+0000 7f76a3e00640 1 -- start start 2024-09-19T04:30:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a4102b90 0x7f76a41007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76a41034b0 0x7f76a4100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76a410bfc0 0x7f76a4101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76a4111e10 con 0x7f76a4102b90 2024-09-19T04:30:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a3e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76a4111c90 con 0x7f76a41034b0 2024-09-19T04:30:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f76a4111f90 con 0x7f76a410bfc0 2024-09-19T04:30:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a4102b90 0x7f76a41007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76a410bfc0 0x7f76a4101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a2000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76a41034b0 0x7f76a4100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76a410bfc0 0x7f76a4101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46196/0 (socket says 172.21.15.107:46196) 2024-09-19T04:30:11.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.667+0000 7f76a3400640 1 -- 172.21.15.107:0/3888260166 learned_addr learned my addr 172.21.15.107:0/3888260166 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.668+0000 7f76a3400640 1 -- 172.21.15.107:0/3888260166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76a41034b0 msgr2=0x7f76a4100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.668+0000 7f76a3400640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76a41034b0 0x7f76a4100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.668+0000 7f76a3400640 1 -- 172.21.15.107:0/3888260166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a4102b90 msgr2=0x7f76a41007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.668+0000 7f76a3400640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a4102b90 0x7f76a41007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.668+0000 7f76a3400640 1 -- 172.21.15.107:0/3888260166 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a41a6760 con 0x7f76a410bfc0 2024-09-19T04:30:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.668+0000 7f76a3400640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76a410bfc0 0x7f76a4101270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f769c00eaa0 tx=0x7f769c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.668+0000 7f769b400640 1 -- 172.21.15.107:0/3888260166 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c00cdb0 con 0x7f76a410bfc0 2024-09-19T04:30:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.668+0000 7f76a3e00640 1 -- 172.21.15.107:0/3888260166 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a41a6a50 con 0x7f76a410bfc0 2024-09-19T04:30:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.668+0000 7f769b400640 1 -- 172.21.15.107:0/3888260166 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f769c005020 con 0x7f76a410bfc0 2024-09-19T04:30:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.668+0000 7f769b400640 1 -- 172.21.15.107:0/3888260166 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c0116b0 con 0x7f76a410bfc0 2024-09-19T04:30:11.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.669+0000 7f76a3e00640 1 -- 172.21.15.107:0/3888260166 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a41a6f60 con 0x7f76a410bfc0 2024-09-19T04:30:11.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.673+0000 7f76ac49c740 1 Processor -- start 2024-09-19T04:30:11.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.673+0000 7f769b400640 1 -- 172.21.15.107:0/3888260166 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f769c005560 con 0x7f76a410bfc0 2024-09-19T04:30:11.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.673+0000 7f76ac49c740 1 -- start start 2024-09-19T04:30:11.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.673+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e93c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.673+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e946ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.673+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e949410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.673+0000 7f76ac49c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561b2e8a23e0 con 0x561b2e93c760 2024-09-19T04:30:11.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.673+0000 7f76ac49c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561b2e849d80 con 0x561b2e93be40 2024-09-19T04:30:11.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.673+0000 7f76ac49c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561b2e89e3a0 con 0x561b2e947020 2024-09-19T04:30:11.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.674+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e946ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.674+0000 7f76a2000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e93c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.674+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e946ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60780/0 (socket says 172.21.15.107:60780) 2024-09-19T04:30:11.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.674+0000 7f76a2a00640 1 -- 172.21.15.107:0/4034345273 learned_addr learned my addr 172.21.15.107:0/4034345273 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:11.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.674+0000 7f76a2000640 1 --2- 172.21.15.107:0/4034345273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e949410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.674+0000 7f76a2a00640 1 -- 172.21.15.107:0/4034345273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 msgr2=0x561b2e949410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:11.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.674+0000 7f76a2a00640 1 --2- 172.21.15.107:0/4034345273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e949410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.674+0000 7f76a2a00640 1 -- 172.21.15.107:0/4034345273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 msgr2=0x561b2e93c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:11.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.674+0000 7f76a2a00640 1 --2- 172.21.15.107:0/4034345273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e93c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.674+0000 7f76a2a00640 1 -- 172.21.15.107:0/4034345273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2e868ae0 con 0x561b2e93c760 2024-09-19T04:30:11.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.675+0000 7f769b400640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76880a3430 0x7f76880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.675+0000 7f76a2a00640 1 --2- 172.21.15.107:0/4034345273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e946ae0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7694009cf0 tx=0x7f7694011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c71e1a2bf636a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:11.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.675+0000 7f769b400640 1 -- 172.21.15.107:0/3888260166 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f769c0cfdd0 con 0x7f76a410bfc0 2024-09-19T04:30:11.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.675+0000 7f768fe00640 1 -- 172.21.15.107:0/4034345273 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694004230 con 0x561b2e93c760 2024-09-19T04:30:11.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.675+0000 7f76a2000640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76880a3430 0x7f76880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.675+0000 7f768fe00640 1 -- 172.21.15.107:0/4034345273 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76940043d0 con 0x561b2e93c760 2024-09-19T04:30:11.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.675+0000 7f768fe00640 1 -- 172.21.15.107:0/4034345273 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694019840 con 0x561b2e93c760 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.676+0000 7f76ac49c740 1 -- 172.21.15.107:0/4034345273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 msgr2=0x561b2e946ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.676+0000 7f76a2000640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76880a3430 0x7f76880a58f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7690000c00 tx=0x7f7690005bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.676+0000 7f76ac49c740 1 --2- 172.21.15.107:0/4034345273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e946ae0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7694009cf0 tx=0x7f7694011810 comp rx=0 tx=0).stop 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.676+0000 7f76ac49c740 1 -- 172.21.15.107:0/4034345273 shutdown_connections 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.676+0000 7f76ac49c740 1 --2- 172.21.15.107:0/4034345273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e949410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.676+0000 7f76ac49c740 1 --2- 172.21.15.107:0/4034345273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e946ae0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.676+0000 7f76ac49c740 1 --2- 172.21.15.107:0/4034345273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e93c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.676+0000 7f76ac49c740 1 -- 172.21.15.107:0/4034345273 >> 172.21.15.107:0/4034345273 conn(0x561b2e9383d0 msgr2=0x561b2e9387b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.676+0000 7f76ac49c740 1 -- 172.21.15.107:0/4034345273 shutdown_connections 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.676+0000 7f76ac49c740 1 -- 172.21.15.107:0/4034345273 wait complete. 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.677+0000 7f76ac49c740 1 -- start start 2024-09-19T04:30:11.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.677+0000 7f76ac49c740 1 Processor -- start 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e943eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e93c760 0x561b2e9443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e947020 0x561b2e944930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76ac49c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561b2e949bf0 con 0x561b2e947020 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76ac49c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561b2e85d0f0 con 0x561b2e93be40 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76ac49c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561b2e949d70 con 0x561b2e93c760 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76ac49c740 10 client.0 _ll_register_callbacks cb 0x561b2e844bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e943eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e943eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47560/0 (socket says 172.21.15.107:47560) 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a2a00640 1 -- 172.21.15.107:0/1779733520 learned_addr learned my addr 172.21.15.107:0/1779733520 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a3400640 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e947020 0x561b2e944930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a2000640 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e93c760 0x561b2e9443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a2a00640 1 -- 172.21.15.107:0/1779733520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e93c760 msgr2=0x561b2e9443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a2a00640 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e93c760 0x561b2e9443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a2a00640 1 -- 172.21.15.107:0/1779733520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e947020 msgr2=0x561b2e944930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a2a00640 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e947020 0x561b2e944930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a2a00640 1 -- 172.21.15.107:0/1779733520 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2e6133c0 con 0x561b2e93be40 2024-09-19T04:30:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.678+0000 7f76a3400640 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e947020 0x561b2e944930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.679+0000 7f76a2a00640 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e943eb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7694011d10 tx=0x7f7694003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=669ecaa46b9a728f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.679+0000 7f768cc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.679+0000 7f768cc00640 1 -- 172.21.15.107:0/1779733520 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694027070 con 0x561b2e93be40 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.679+0000 7f76ac49c740 1 -- 172.21.15.107:0/1779733520 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76940043d0 con 0x561b2e93be40 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.679+0000 7f768cc00640 1 -- 172.21.15.107:0/1779733520 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7694005240 con 0x561b2e93be40 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.679+0000 7f768cc00640 1 -- 172.21.15.107:0/1779733520 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7694033050 con 0x561b2e93be40 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.679+0000 7f76ac49c740 10 client.27002 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.679+0000 7f76ac49c740 1 -- 172.21.15.107:0/1779733520 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561b2e955bc0 con 0x561b2e93be40 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.680+0000 7f768cc00640 1 -- 172.21.15.107:0/1779733520 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76940043d0 con 0x561b2e93be40 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.680+0000 7f768cc00640 1 -- 172.21.15.107:0/1779733520 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f7694004ba0 con 0x561b2e93be40 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.680+0000 7f76ac49c740 10 client.27002 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.680+0000 7f76ac49c740 10 client.27002 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.680+0000 7f76ac49c740 20 client.27002 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.680+0000 7f76ac49c740 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x561b2e956fd0 0x561b2e9593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.680+0000 7f76ac49c740 4 client.27002 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:11.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.680+0000 7f76ac49c740 1 -- 172.21.15.107:0/1779733520 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b2e8af950 con 0x561b2e956fd0 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.680+0000 7f76a2000640 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x561b2e956fd0 0x561b2e9593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.681+0000 7f76a2000640 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x561b2e956fd0 0x561b2e9593c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e8915b819c1fb0 server_cookie=534b77f3b233a2ec in_seq=0 out_seq=0 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.681+0000 7f768cc00640 10 client.27002 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f768cc00640 1 -- 172.21.15.107:0/1779733520 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b2e8af950 con 0x561b2e956fd0 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f768cc00640 10 client.27002 handle_command_reply: tid=0 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 client.27002 shutdown 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 -- 172.21.15.107:0/1779733520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 msgr2=0x561b2e943eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e943eb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7694011d10 tx=0x7f7694003820 comp rx=0 tx=0).stop 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 -- 172.21.15.107:0/1779733520 shutdown_connections 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x561b2e956fd0 0x561b2e9593c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e947020 0x561b2e944930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e93c760 0x561b2e9443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 --2- 172.21.15.107:0/1779733520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e943eb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 -- 172.21.15.107:0/1779733520 >> 172.21.15.107:0/1779733520 conn(0x561b2e9383d0 msgr2=0x561b2e93e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f768cc00640 0 client.27002 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:30:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 -- 172.21.15.107:0/1779733520 shutdown_connections 2024-09-19T04:30:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 1 -- 172.21.15.107:0/1779733520 wait complete. 2024-09-19T04:30:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.682+0000 7f76ac49c740 20 client.27002 trim_cache size 0 max 16384 2024-09-19T04:30:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.697+0000 7f76ac49c740 1 Processor -- start 2024-09-19T04:30:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.697+0000 7f76ac49c740 1 -- start start 2024-09-19T04:30:11.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.697+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e93e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.697+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e943790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.697+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e95eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.697+0000 7f76ac49c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41af8b0 con 0x561b2e93c760 2024-09-19T04:30:11.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.697+0000 7f76ac49c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41b13f0 con 0x561b2e93be40 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.697+0000 7f76ac49c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41aff80 con 0x561b2e947020 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e93e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e93e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47572/0 (socket says 172.21.15.107:47572) 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e943790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a2a00640 1 -- 172.21.15.107:0/3149973652 learned_addr learned my addr 172.21.15.107:0/3149973652 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a3400640 1 --2- 172.21.15.107:0/3149973652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e95eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a2a00640 1 -- 172.21.15.107:0/3149973652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 msgr2=0x561b2e95eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a2a00640 1 --2- 172.21.15.107:0/3149973652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e95eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a2a00640 1 -- 172.21.15.107:0/3149973652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 msgr2=0x561b2e943790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a2a00640 1 --2- 172.21.15.107:0/3149973652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e943790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a2a00640 1 -- 172.21.15.107:0/3149973652 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2e938200 con 0x561b2e93be40 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.698+0000 7f76a2a00640 1 --2- 172.21.15.107:0/3149973652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e93e860 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7694009820 tx=0x7f7694005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dba40f1c713b0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.699+0000 7f768f400640 1 -- 172.21.15.107:0/3149973652 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694027070 con 0x561b2e93be40 2024-09-19T04:30:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.699+0000 7f768f400640 1 -- 172.21.15.107:0/3149973652 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7694015020 con 0x561b2e93be40 2024-09-19T04:30:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.699+0000 7f768f400640 1 -- 172.21.15.107:0/3149973652 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694019c50 con 0x561b2e93be40 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.699+0000 7f76ac49c740 1 -- 172.21.15.107:0/3149973652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 msgr2=0x561b2e93e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.699+0000 7f76ac49c740 1 --2- 172.21.15.107:0/3149973652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e93e860 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7694009820 tx=0x7f7694005db0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.699+0000 7f76ac49c740 1 -- 172.21.15.107:0/3149973652 shutdown_connections 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.699+0000 7f76ac49c740 1 --2- 172.21.15.107:0/3149973652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e95eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.699+0000 7f76ac49c740 1 --2- 172.21.15.107:0/3149973652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e943790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.699+0000 7f76ac49c740 1 --2- 172.21.15.107:0/3149973652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e93e860 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.699+0000 7f76ac49c740 1 -- 172.21.15.107:0/3149973652 >> 172.21.15.107:0/3149973652 conn(0x561b2e9383d0 msgr2=0x561b2e945f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.700+0000 7f76ac49c740 1 -- 172.21.15.107:0/3149973652 shutdown_connections 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.700+0000 7f76ac49c740 1 -- 172.21.15.107:0/3149973652 wait complete. 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.700+0000 7f76ac49c740 1 -- start start 2024-09-19T04:30:11.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.700+0000 7f76ac49c740 1 Processor -- start 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e94f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e94f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76ac49c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e94a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76ac49c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41b0d20 con 0x561b2e93c760 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76ac49c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41b0650 con 0x561b2e93be40 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76ac49c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41ad8b0 con 0x561b2e947020 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76ac49c740 10 client.0 _ll_register_callbacks cb 0x7f7694005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e94f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e94f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47582/0 (socket says 172.21.15.107:47582) 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a2a00640 1 -- 172.21.15.107:0/2438769021 learned_addr learned my addr 172.21.15.107:0/2438769021 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:11.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a3400640 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e94a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a2000640 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e94f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a2a00640 1 -- 172.21.15.107:0/2438769021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 msgr2=0x561b2e94a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a2a00640 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e94a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a2a00640 1 -- 172.21.15.107:0/2438769021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 msgr2=0x561b2e94f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a2a00640 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e94f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a2a00640 1 -- 172.21.15.107:0/2438769021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2e94ad90 con 0x561b2e93be40 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a3400640 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e94a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.701+0000 7f76a2000640 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e94f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.702+0000 7f76a2a00640 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e94f060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7694006010 tx=0x7f76940054c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d8c01028efeeef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.702+0000 7f768cc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.702+0000 7f768cc00640 1 -- 172.21.15.107:0/2438769021 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694027040 con 0x561b2e93be40 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.702+0000 7f768cc00640 1 -- 172.21.15.107:0/2438769021 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7694019c70 con 0x561b2e93be40 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.702+0000 7f76ac49c740 1 -- 172.21.15.107:0/2438769021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7694015020 con 0x561b2e93be40 2024-09-19T04:30:11.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.702+0000 7f768cc00640 1 -- 172.21.15.107:0/2438769021 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7694009910 con 0x561b2e93be40 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.702+0000 7f76ac49c740 10 client.27008 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.702+0000 7f76ac49c740 1 -- 172.21.15.107:0/2438769021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561b2e9c1530 con 0x561b2e93be40 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.703+0000 7f768cc00640 1 -- 172.21.15.107:0/2438769021 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7694015020 con 0x561b2e93be40 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.703+0000 7f768cc00640 1 -- 172.21.15.107:0/2438769021 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f7694023ab0 con 0x561b2e93be40 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.703+0000 7f76ac49c740 10 client.27008 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.703+0000 7f76ac49c740 10 client.27008 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.703+0000 7f76ac49c740 20 client.27008 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.703+0000 7f76ac49c740 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x561b2e9c1b40 0x561b2e9c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.703+0000 7f76ac49c740 4 client.27008 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.703+0000 7f76ac49c740 1 -- 172.21.15.107:0/2438769021 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b2e8a9e60 con 0x561b2e9c1b40 2024-09-19T04:30:11.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.703+0000 7f76a2000640 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x561b2e9c1b40 0x561b2e9c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76a2000640 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x561b2e9c1b40 0x561b2e9c3f50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bd9138c61c25365 server_cookie=df00416b179060cf in_seq=0 out_seq=0 2024-09-19T04:30:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f768cc00640 10 client.27008 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:30:11.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f768cc00640 1 -- 172.21.15.107:0/2438769021 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b2e8a9e60 con 0x561b2e9c1b40 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f768cc00640 10 client.27008 handle_command_reply: tid=0 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 client.27008 shutdown 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 -- 172.21.15.107:0/2438769021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 msgr2=0x561b2e94f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e94f060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7694006010 tx=0x7f76940054c0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 -- 172.21.15.107:0/2438769021 shutdown_connections 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x561b2e9c1b40 0x561b2e9c3f50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2e947020 0x561b2e94a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2e93c760 0x561b2e94f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 --2- 172.21.15.107:0/2438769021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e93be40 0x561b2e94f060 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 -- 172.21.15.107:0/2438769021 >> 172.21.15.107:0/2438769021 conn(0x561b2e9383d0 msgr2=0x561b2e951e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f768cc00640 0 client.27008 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 -- 172.21.15.107:0/2438769021 shutdown_connections 2024-09-19T04:30:11.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 1 -- 172.21.15.107:0/2438769021 wait complete. 2024-09-19T04:30:11.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.704+0000 7f76ac49c740 20 client.27008 trim_cache size 0 max 16384 2024-09-19T04:30:11.711 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:30:11.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 -- 172.21.15.107:0/3888260166 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76880a3430 msgr2=0x7f76880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76880a3430 0x7f76880a58f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7690000c00 tx=0x7f7690005bc0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 -- 172.21.15.107:0/3888260166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76a410bfc0 msgr2=0x7f76a4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76a410bfc0 0x7f76a4101270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f769c00eaa0 tx=0x7f769c00ef60 comp rx=0 tx=0).stop 2024-09-19T04:30:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 -- 172.21.15.107:0/3888260166 shutdown_connections 2024-09-19T04:30:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76880a3430 0x7f76880a58f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76a410bfc0 0x7f76a4101270 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76a41034b0 0x7f76a4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 --2- 172.21.15.107:0/3888260166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a4102b90 0x7f76a41007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:11.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 -- 172.21.15.107:0/3888260166 >> 172.21.15.107:0/3888260166 conn(0x7f76a40fc610 msgr2=0x7f76a40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:11.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.705+0000 7f76a3e00640 1 -- 172.21.15.107:0/3888260166 shutdown_connections 2024-09-19T04:30:11.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:11.706+0000 7f76a3e00640 1 -- 172.21.15.107:0/3888260166 wait complete. 2024-09-19T04:30:11.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 323930 2.71 MB/sec execute 389 sec latency 12.741 ms 2024-09-19T04:30:12.606 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:30:12.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 324487 2.71 MB/sec execute 390 sec latency 635.054 ms 2024-09-19T04:30:13.016 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:13.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:13 smithi107 ceph-mon[30659]: pgmap v307: 97 pgs: 97 active+clean; 246 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 199 op/s 2024-09-19T04:30:13.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:13 smithi115 ceph-mon[35477]: pgmap v307: 97 pgs: 97 active+clean; 246 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 199 op/s 2024-09-19T04:30:13.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:13 smithi145 ceph-mon[35096]: pgmap v307: 97 pgs: 97 active+clean; 246 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 199 op/s 2024-09-19T04:30:13.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.664+0000 7f070a600640 1 -- 172.21.15.107:0/3771641118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07041019d0 msgr2=0x7f0704101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:13.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.664+0000 7f070a600640 1 --2- 172.21.15.107:0/3771641118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07041019d0 0x7f0704101e30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009890 tx=0x7f06f002a360 comp rx=0 tx=0).stop 2024-09-19T04:30:13.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.664+0000 7f070a600640 1 -- 172.21.15.107:0/3771641118 shutdown_connections 2024-09-19T04:30:13.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.664+0000 7f070a600640 1 --2- 172.21.15.107:0/3771641118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0704102370 0x7f070410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.664+0000 7f070a600640 1 --2- 172.21.15.107:0/3771641118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07041019d0 0x7f0704101e30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.664+0000 7f070a600640 1 --2- 172.21.15.107:0/3771641118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07041079d0 0x7f0704107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.664+0000 7f070a600640 1 -- 172.21.15.107:0/3771641118 >> 172.21.15.107:0/3771641118 conn(0x7f07040fd710 msgr2=0x7f07040ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:13.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.664+0000 7f070a600640 1 -- 172.21.15.107:0/3771641118 shutdown_connections 2024-09-19T04:30:13.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.664+0000 7f070a600640 1 -- 172.21.15.107:0/3771641118 wait complete. 2024-09-19T04:30:13.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.665+0000 7f070a600640 1 Processor -- start 2024-09-19T04:30:13.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.665+0000 7f070a600640 1 -- start start 2024-09-19T04:30:13.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.665+0000 7f070a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07041019d0 0x7f07041982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f070a600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0704102370 0x7f0704198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f070a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07041079d0 0x7f070419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f070a600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0704111dd0 con 0x7f07041019d0 2024-09-19T04:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f070a600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0704111c50 con 0x7f0704102370 2024-09-19T04:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f070a600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0704111f50 con 0x7f07041079d0 2024-09-19T04:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0709200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07041019d0 0x7f07041982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0709200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07041019d0 0x7f07041982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60830/0 (socket says 172.21.15.107:60830) 2024-09-19T04:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0709200640 1 -- 172.21.15.107:0/1330007432 learned_addr learned my addr 172.21.15.107:0/1330007432 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0703e00640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0704102370 0x7f0704198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0709c00640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07041079d0 0x7f070419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0709200640 1 -- 172.21.15.107:0/1330007432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07041079d0 msgr2=0x7f070419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0709200640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07041079d0 0x7f070419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0709200640 1 -- 172.21.15.107:0/1330007432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0704102370 msgr2=0x7f0704198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0709200640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0704102370 0x7f0704198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0709200640 1 -- 172.21.15.107:0/1330007432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070419d2a0 con 0x7f07041019d0 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.666+0000 7f0709c00640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07041079d0 0x7f070419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.667+0000 7f0709200640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07041019d0 0x7f07041982f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f06f800e9f0 tx=0x7f06f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.667+0000 7f0701600640 1 -- 172.21.15.107:0/1330007432 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f800ccc0 con 0x7f07041019d0 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.667+0000 7f0701600640 1 -- 172.21.15.107:0/1330007432 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f8005500 con 0x7f07041019d0 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.667+0000 7f070a600640 1 -- 172.21.15.107:0/1330007432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070419d590 con 0x7f07041019d0 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.667+0000 7f0701600640 1 -- 172.21.15.107:0/1330007432 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8010690 con 0x7f07041019d0 2024-09-19T04:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.667+0000 7f070a600640 1 -- 172.21.15.107:0/1330007432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07041a4e70 con 0x7f07041019d0 2024-09-19T04:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.670+0000 7f070e6bf740 1 Processor -- start 2024-09-19T04:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.670+0000 7f0701600640 1 -- 172.21.15.107:0/1330007432 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f06f80046c0 con 0x7f07041019d0 2024-09-19T04:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.670+0000 7f070e6bf740 1 -- start start 2024-09-19T04:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e23ce0 0x558df5e240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 0x558df5e32ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e33430 0x558df5e35820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f070e6bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558df5d8a3e0 con 0x558df5e23ce0 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f070e6bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558df5d31d80 con 0x558df5e33430 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f070e6bf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558df5d863a0 con 0x558df5e24600 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f0709c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e23ce0 0x558df5e240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f0709c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e23ce0 0x558df5e240c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60846/0 (socket says 172.21.15.107:60846) 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f0709c00640 1 -- 172.21.15.107:0/4224834014 learned_addr learned my addr 172.21.15.107:0/4224834014 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f0709200640 1 --2- 172.21.15.107:0/4224834014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 0x558df5e32ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f0703e00640 1 --2- 172.21.15.107:0/4224834014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e33430 0x558df5e35820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f0709c00640 1 -- 172.21.15.107:0/4224834014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 msgr2=0x558df5e32ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f0709c00640 1 --2- 172.21.15.107:0/4224834014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 0x558df5e32ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f0709c00640 1 -- 172.21.15.107:0/4224834014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e33430 msgr2=0x558df5e35820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f0709c00640 1 --2- 172.21.15.107:0/4224834014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e33430 0x558df5e35820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.671+0000 7f0709c00640 1 -- 172.21.15.107:0/4224834014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df5d50ae0 con 0x558df5e23ce0 2024-09-19T04:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.672+0000 7f0709c00640 1 --2- 172.21.15.107:0/4224834014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e23ce0 0x558df5e240c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009fd0 tx=0x7f06fc0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1df67575cad0bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.672+0000 7f06f6000640 1 -- 172.21.15.107:0/4224834014 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc011760 con 0x558df5e23ce0 2024-09-19T04:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.672+0000 7f0701600640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06ec0a3430 0x7f06ec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.672+0000 7f06f6000640 1 -- 172.21.15.107:0/4224834014 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06fc011de0 con 0x558df5e23ce0 2024-09-19T04:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.672+0000 7f06f6000640 1 -- 172.21.15.107:0/4224834014 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc00e3c0 con 0x558df5e23ce0 2024-09-19T04:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.672+0000 7f0701600640 1 -- 172.21.15.107:0/1330007432 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f06f80cff80 con 0x7f07041019d0 2024-09-19T04:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.672+0000 7f0709200640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06ec0a3430 0x7f06ec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.673+0000 7f070e6bf740 1 -- 172.21.15.107:0/4224834014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e23ce0 msgr2=0x558df5e240c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:13.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.673+0000 7f070e6bf740 1 --2- 172.21.15.107:0/4224834014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e23ce0 0x558df5e240c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009fd0 tx=0x7f06fc0108e0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.673+0000 7f070e6bf740 1 -- 172.21.15.107:0/4224834014 shutdown_connections 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.673+0000 7f070e6bf740 1 --2- 172.21.15.107:0/4224834014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e33430 0x558df5e35820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.673+0000 7f070e6bf740 1 --2- 172.21.15.107:0/4224834014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 0x558df5e32ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.673+0000 7f070e6bf740 1 --2- 172.21.15.107:0/4224834014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e23ce0 0x558df5e240c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.673+0000 7f070e6bf740 1 -- 172.21.15.107:0/4224834014 >> 172.21.15.107:0/4224834014 conn(0x558df5e203d0 msgr2=0x558df5e207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.673+0000 7f0709200640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06ec0a3430 0x7f06ec0a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x558df5e21230 tx=0x7f06f80bc040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.673+0000 7f070e6bf740 1 -- 172.21.15.107:0/4224834014 shutdown_connections 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.673+0000 7f070e6bf740 1 -- 172.21.15.107:0/4224834014 wait complete. 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.674+0000 7f070e6bf740 1 -- start start 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.674+0000 7f070e6bf740 1 Processor -- start 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.674+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 0x558df5e30370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.674+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.674+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e30df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f070e6bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558df5e36000 con 0x558df5e33430 2024-09-19T04:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f070e6bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558df5d450f0 con 0x558df5e24600 2024-09-19T04:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f070e6bf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558df5e36180 con 0x558df5e23ce0 2024-09-19T04:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f070e6bf740 10 client.0 _ll_register_callbacks cb 0x558df5d2cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0703e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0709c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 0x558df5e30370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0703e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e308b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47622/0 (socket says 172.21.15.107:47622) 2024-09-19T04:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0709c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 0x558df5e30370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46258/0 (socket says 172.21.15.107:46258) 2024-09-19T04:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0703e00640 1 -- 172.21.15.107:0/219450404 learned_addr learned my addr 172.21.15.107:0/219450404 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0709c00640 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e30df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0703e00640 1 -- 172.21.15.107:0/219450404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 msgr2=0x558df5e30370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0703e00640 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 0x558df5e30370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0703e00640 1 -- 172.21.15.107:0/219450404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 msgr2=0x558df5e30df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0703e00640 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e30df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0703e00640 1 -- 172.21.15.107:0/219450404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df5afb3c0 con 0x558df5e24600 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0709c00640 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e30df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.675+0000 7f0703e00640 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e308b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009890 tx=0x7f06f0030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16edfc4168e0c89b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.676+0000 7f06eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.676+0000 7f06eaa00640 1 -- 172.21.15.107:0/219450404 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0037040 con 0x558df5e24600 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.676+0000 7f06eaa00640 1 -- 172.21.15.107:0/219450404 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f06f0030580 con 0x558df5e24600 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.676+0000 7f070e6bf740 1 -- 172.21.15.107:0/219450404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06fc011de0 con 0x558df5e24600 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.676+0000 7f06eaa00640 1 -- 172.21.15.107:0/219450404 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f06f004c050 con 0x558df5e24600 2024-09-19T04:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.676+0000 7f070e6bf740 10 client.27020 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.676+0000 7f070e6bf740 1 -- 172.21.15.107:0/219450404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558df5e420d0 con 0x558df5e24600 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.676+0000 7f06eaa00640 1 -- 172.21.15.107:0/219450404 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06fc011de0 con 0x558df5e24600 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.677+0000 7f06eaa00640 1 -- 172.21.15.107:0/219450404 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f06f003d450 con 0x558df5e24600 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.677+0000 7f070e6bf740 10 client.27020 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.677+0000 7f070e6bf740 10 client.27020 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.677+0000 7f070e6bf740 20 client.27020 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.677+0000 7f070e6bf740 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x558df5e43630 0x558df5e45a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.677+0000 7f070e6bf740 4 client.27020 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.677+0000 7f070e6bf740 1 -- 172.21.15.107:0/219450404 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558df5d97950 con 0x558df5e43630 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.677+0000 7f0709c00640 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x558df5e43630 0x558df5e45a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.678+0000 7f0709c00640 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x558df5e43630 0x558df5e45a50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=985e383cc205e078 server_cookie=25d09ee12f9bba20 in_seq=0 out_seq=0 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.678+0000 7f06eaa00640 10 client.27020 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f06eaa00640 1 -- 172.21.15.107:0/219450404 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558df5d97950 con 0x558df5e43630 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f06eaa00640 10 client.27020 handle_command_reply: tid=0 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 client.27020 shutdown 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 -- 172.21.15.107:0/219450404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 msgr2=0x558df5e308b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e308b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009890 tx=0x7f06f0030430 comp rx=0 tx=0).stop 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 -- 172.21.15.107:0/219450404 shutdown_connections 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x558df5e43630 0x558df5e45a50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e30df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e308b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 --2- 172.21.15.107:0/219450404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 0x558df5e30370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f06eaa00640 0 client.27020 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 -- 172.21.15.107:0/219450404 >> 172.21.15.107:0/219450404 conn(0x558df5e203d0 msgr2=0x558df5e26130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 -- 172.21.15.107:0/219450404 shutdown_connections 2024-09-19T04:30:13.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 1 -- 172.21.15.107:0/219450404 wait complete. 2024-09-19T04:30:13.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.679+0000 7f070e6bf740 20 client.27020 trim_cache size 0 max 16384 2024-09-19T04:30:13.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f070e6bf740 1 Processor -- start 2024-09-19T04:30:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f070e6bf740 1 -- start start 2024-09-19T04:30:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e23ce0 0x558df5e2ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 0x558df5e30c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e4b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f070e6bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07041ab460 con 0x558df5e33430 2024-09-19T04:30:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f070e6bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f07041acfa0 con 0x558df5e23ce0 2024-09-19T04:30:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f070e6bf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f07041abb30 con 0x558df5e24600 2024-09-19T04:30:13.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f0709c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e23ce0 0x558df5e2ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f0709c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e23ce0 0x558df5e2ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47626/0 (socket says 172.21.15.107:47626) 2024-09-19T04:30:13.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f0709c00640 1 -- 172.21.15.107:0/3049002244 learned_addr learned my addr 172.21.15.107:0/3049002244 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:13.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.693+0000 7f0703e00640 1 --2- 172.21.15.107:0/3049002244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 0x558df5e30c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.694+0000 7f0709c00640 1 --2- 172.21.15.107:0/3049002244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e4b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.694+0000 7f0703e00640 1 -- 172.21.15.107:0/3049002244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e23ce0 msgr2=0x558df5e2ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:13.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.694+0000 7f0703e00640 1 --2- 172.21.15.107:0/3049002244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e23ce0 0x558df5e2ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.694+0000 7f0703e00640 1 -- 172.21.15.107:0/3049002244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 msgr2=0x558df5e4b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:13.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.694+0000 7f0703e00640 1 --2- 172.21.15.107:0/3049002244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e4b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.694+0000 7f0703e00640 1 -- 172.21.15.107:0/3049002244 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df5e21060 con 0x558df5e24600 2024-09-19T04:30:13.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.694+0000 7f0703e00640 1 --2- 172.21.15.107:0/3049002244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 0x558df5e30c40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f06f003da10 tx=0x7f06f002a360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b94e9111c4f4f079 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:13.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.694+0000 7f06f5600640 1 -- 172.21.15.107:0/3049002244 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0037040 con 0x558df5e24600 2024-09-19T04:30:13.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.694+0000 7f06f5600640 1 -- 172.21.15.107:0/3049002244 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f003d450 con 0x558df5e24600 2024-09-19T04:30:13.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.694+0000 7f06f5600640 1 -- 172.21.15.107:0/3049002244 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f00052d0 con 0x558df5e24600 2024-09-19T04:30:13.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.695+0000 7f070e6bf740 1 -- 172.21.15.107:0/3049002244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 msgr2=0x558df5e30c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:13.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.695+0000 7f070e6bf740 1 --2- 172.21.15.107:0/3049002244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 0x558df5e30c40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f06f003da10 tx=0x7f06f002a360 comp rx=0 tx=0).stop 2024-09-19T04:30:13.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.695+0000 7f070e6bf740 1 -- 172.21.15.107:0/3049002244 shutdown_connections 2024-09-19T04:30:13.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.695+0000 7f070e6bf740 1 --2- 172.21.15.107:0/3049002244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e4b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.695+0000 7f070e6bf740 1 --2- 172.21.15.107:0/3049002244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e24600 0x558df5e30c40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.695+0000 7f070e6bf740 1 --2- 172.21.15.107:0/3049002244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e23ce0 0x558df5e2ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.695+0000 7f070e6bf740 1 -- 172.21.15.107:0/3049002244 >> 172.21.15.107:0/3049002244 conn(0x558df5e203d0 msgr2=0x558df5e323f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.695+0000 7f070e6bf740 1 -- 172.21.15.107:0/3049002244 shutdown_connections 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.695+0000 7f070e6bf740 1 -- 172.21.15.107:0/3049002244 wait complete. 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.696+0000 7f070e6bf740 1 -- start start 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.696+0000 7f070e6bf740 1 Processor -- start 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.696+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 0x558df5e37180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.696+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.696+0000 7f070e6bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e37c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.696+0000 7f070e6bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07041ac8d0 con 0x558df5e33430 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.696+0000 7f070e6bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f07041ac200 con 0x558df5e24600 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.696+0000 7f070e6bf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f07041a9440 con 0x558df5e23ce0 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.696+0000 7f070e6bf740 10 client.0 _ll_register_callbacks cb 0x7f06f8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0709c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 0x558df5e37180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0709c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e37c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0703e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0709c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e37c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60878/0 (socket says 172.21.15.107:60878) 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0709c00640 1 -- 172.21.15.107:0/4141589777 learned_addr learned my addr 172.21.15.107:0/4141589777 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0709c00640 1 -- 172.21.15.107:0/4141589777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 msgr2=0x558df5e37180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0709c00640 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 0x558df5e37180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0709c00640 1 -- 172.21.15.107:0/4141589777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 msgr2=0x558df5e376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:13.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0709c00640 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0709c00640 1 -- 172.21.15.107:0/4141589777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df5ead300 con 0x558df5e33430 2024-09-19T04:30:13.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0703e00640 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:13.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.697+0000 7f0709c00640 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e37c00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f06fc000f80 tx=0x7f06fc01bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c7952b9508ca5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:13.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f06eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:30:13.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f06eaa00640 1 -- 172.21.15.107:0/4141589777 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc03e060 con 0x558df5e33430 2024-09-19T04:30:13.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f06eaa00640 1 -- 172.21.15.107:0/4141589777 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f06fc011560 con 0x558df5e33430 2024-09-19T04:30:13.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f070e6bf740 1 -- 172.21.15.107:0/4141589777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f003d450 con 0x558df5e33430 2024-09-19T04:30:13.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f06eaa00640 1 -- 172.21.15.107:0/4141589777 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f06fc024040 con 0x558df5e33430 2024-09-19T04:30:13.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f070e6bf740 10 client.16980 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:13.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f070e6bf740 1 -- 172.21.15.107:0/4141589777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558df5eae5c0 con 0x558df5e33430 2024-09-19T04:30:13.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f06eaa00640 1 -- 172.21.15.107:0/4141589777 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f003d450 con 0x558df5e33430 2024-09-19T04:30:13.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f06eaa00640 1 -- 172.21.15.107:0/4141589777 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f06fc034070 con 0x558df5e33430 2024-09-19T04:30:13.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f070e6bf740 10 client.16980 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:13.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f070e6bf740 10 client.16980 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:30:13.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.698+0000 7f070e6bf740 20 client.16980 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:13.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.699+0000 7f070e6bf740 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x558df5eaea90 0x558df5eb0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:13.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.699+0000 7f070e6bf740 4 client.16980 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:13.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.699+0000 7f070e6bf740 1 -- 172.21.15.107:0/4141589777 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558df5d91e60 con 0x558df5eaea90 2024-09-19T04:30:13.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.699+0000 7f0703e00640 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x558df5eaea90 0x558df5eb0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:13.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.699+0000 7f0703e00640 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x558df5eaea90 0x558df5eb0e80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2405315611bfed1b server_cookie=68b7400c0ac05b20 in_seq=0 out_seq=0 2024-09-19T04:30:13.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.699+0000 7f06eaa00640 10 client.16980 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f06eaa00640 1 -- 172.21.15.107:0/4141589777 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558df5d91e60 con 0x558df5eaea90 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f06eaa00640 10 client.16980 handle_command_reply: tid=0 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 client.16980 shutdown 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 -- 172.21.15.107:0/4141589777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 msgr2=0x558df5e37c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e37c00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f06fc000f80 tx=0x7f06fc01bb30 comp rx=0 tx=0).stop 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 -- 172.21.15.107:0/4141589777 shutdown_connections 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x558df5eaea90 0x558df5eb0e80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df5e33430 0x558df5e37c00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5e24600 0x558df5e376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f06eaa00640 0 client.16980 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:30:13.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 --2- 172.21.15.107:0/4141589777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558df5e23ce0 0x558df5e37180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 -- 172.21.15.107:0/4141589777 >> 172.21.15.107:0/4141589777 conn(0x558df5e203d0 msgr2=0x558df5e1ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 -- 172.21.15.107:0/4141589777 shutdown_connections 2024-09-19T04:30:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 1 -- 172.21.15.107:0/4141589777 wait complete. 2024-09-19T04:30:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.700+0000 7f070e6bf740 20 client.16980 trim_cache size 0 max 16384 2024-09-19T04:30:13.710 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:30:13.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 -- 172.21.15.107:0/1330007432 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06ec0a3430 msgr2=0x7f06ec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06ec0a3430 0x7f06ec0a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x558df5e21230 tx=0x7f06f80bc040 comp rx=0 tx=0).stop 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 -- 172.21.15.107:0/1330007432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07041019d0 msgr2=0x7f07041982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07041019d0 0x7f07041982f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f06f800e9f0 tx=0x7f06f800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 -- 172.21.15.107:0/1330007432 shutdown_connections 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f06ec0a3430 0x7f06ec0a58f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07041079d0 0x7f070419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0704102370 0x7f0704198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 --2- 172.21.15.107:0/1330007432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07041019d0 0x7f07041982f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 -- 172.21.15.107:0/1330007432 >> 172.21.15.107:0/1330007432 conn(0x7f07040fd710 msgr2=0x7f07040fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 -- 172.21.15.107:0/1330007432 shutdown_connections 2024-09-19T04:30:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:13.701+0000 7f070a600640 1 -- 172.21.15.107:0/1330007432 wait complete. 2024-09-19T04:30:13.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 324487 2.70 MB/sec execute 391 sec latency 1635.096 ms 2024-09-19T04:30:14.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 325792 2.71 MB/sec execute 392 sec latency 1879.796 ms 2024-09-19T04:30:15.270 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:15 smithi107 ceph-mon[30659]: pgmap v308: 97 pgs: 97 active+clean; 246 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 186 op/s 2024-09-19T04:30:15.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:30:15.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:15 smithi115 ceph-mon[35477]: pgmap v308: 97 pgs: 97 active+clean; 246 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 186 op/s 2024-09-19T04:30:15.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:15 smithi145 ceph-mon[35096]: pgmap v308: 97 pgs: 97 active+clean; 246 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 186 op/s 2024-09-19T04:30:15.676 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:15.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 326608 2.72 MB/sec execute 393 sec latency 621.262 ms 2024-09-19T04:30:16.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.356+0000 7fb878400640 1 -- 172.21.15.107:0/1152959440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb870101de0 msgr2=0x7fb8701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:16.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.356+0000 7fb878400640 1 --2- 172.21.15.107:0/1152959440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb870101de0 0x7fb8701021c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb864009900 tx=0x7fb86402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.357+0000 7fb878400640 1 -- 172.21.15.107:0/1152959440 shutdown_connections 2024-09-19T04:30:16.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.357+0000 7fb878400640 1 --2- 172.21.15.107:0/1152959440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb870106f10 0x7fb87010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.357+0000 7fb878400640 1 --2- 172.21.15.107:0/1152959440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb870102790 0x7fb8701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.357+0000 7fb878400640 1 --2- 172.21.15.107:0/1152959440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb870101de0 0x7fb8701021c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.357+0000 7fb878400640 1 -- 172.21.15.107:0/1152959440 >> 172.21.15.107:0/1152959440 conn(0x7fb8700fd8c0 msgr2=0x7fb8700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:16.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.357+0000 7fb878400640 1 -- 172.21.15.107:0/1152959440 shutdown_connections 2024-09-19T04:30:16.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.357+0000 7fb878400640 1 -- 172.21.15.107:0/1152959440 wait complete. 2024-09-19T04:30:16.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.358+0000 7fb878400640 1 Processor -- start 2024-09-19T04:30:16.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.358+0000 7fb878400640 1 -- start start 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb878400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb870101de0 0x7fb8700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb878400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb870102790 0x7fb870074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb878400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb870106f10 0x7fb870074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb878400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb870111fc0 con 0x7fb870106f10 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb878400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb870111e40 con 0x7fb870102790 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb878400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb870112140 con 0x7fb870101de0 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb875000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb870102790 0x7fb870074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb876400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb870106f10 0x7fb870074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb875000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb870102790 0x7fb870074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47652/0 (socket says 172.21.15.107:47652) 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb875000640 1 -- 172.21.15.107:0/3924196815 learned_addr learned my addr 172.21.15.107:0/3924196815 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb875a00640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb870101de0 0x7fb8700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb875000640 1 -- 172.21.15.107:0/3924196815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb870101de0 msgr2=0x7fb8700744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb875000640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb870101de0 0x7fb8700744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb875000640 1 -- 172.21.15.107:0/3924196815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb870106f10 msgr2=0x7fb870074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb875000640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb870106f10 0x7fb870074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.359+0000 7fb875000640 1 -- 172.21.15.107:0/3924196815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8701a68e0 con 0x7fb870102790 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.360+0000 7fb875000640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb870102790 0x7fb870074a30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9f0 tx=0x7fb85c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.360+0000 7fb86e000640 1 -- 172.21.15.107:0/3924196815 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00cd80 con 0x7fb870102790 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.360+0000 7fb86e000640 1 -- 172.21.15.107:0/3924196815 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c010910 con 0x7fb870102790 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.360+0000 7fb878400640 1 -- 172.21.15.107:0/3924196815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8701a6bd0 con 0x7fb870102790 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.360+0000 7fb86e000640 1 -- 172.21.15.107:0/3924196815 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c018990 con 0x7fb870102790 2024-09-19T04:30:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.360+0000 7fb878400640 1 -- 172.21.15.107:0/3924196815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8701a7160 con 0x7fb870102790 2024-09-19T04:30:16.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb87aeb2740 1 Processor -- start 2024-09-19T04:30:16.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb87aeb2740 1 -- start start 2024-09-19T04:30:16.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8f2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8f2590 0x555d0c8fc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c8ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555d0c8583e0 con 0x555d0c8f2590 2024-09-19T04:30:16.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555d0c7ffd80 con 0x555d0c8fce50 2024-09-19T04:30:16.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555d0c8543a0 con 0x555d0c8f1c70 2024-09-19T04:30:16.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb86e000640 1 -- 172.21.15.107:0/3924196815 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fb85c010ac0 con 0x7fb870102790 2024-09-19T04:30:16.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb876400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb876400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8f2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46282/0 (socket says 172.21.15.107:46282) 2024-09-19T04:30:16.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.365+0000 7fb876400640 1 -- 172.21.15.107:0/1465271468 learned_addr learned my addr 172.21.15.107:0/1465271468 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:16.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.364+0000 7fb875a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8f2590 0x555d0c8fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.365+0000 7fb875000640 1 --2- 172.21.15.107:0/1465271468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c8ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.365+0000 7fb876400640 1 -- 172.21.15.107:0/1465271468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 msgr2=0x555d0c8ff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:16.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.365+0000 7fb876400640 1 --2- 172.21.15.107:0/1465271468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c8ff240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.365+0000 7fb876400640 1 -- 172.21.15.107:0/1465271468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8f2590 msgr2=0x555d0c8fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:16.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.365+0000 7fb876400640 1 --2- 172.21.15.107:0/1465271468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8f2590 0x555d0c8fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.365+0000 7fb876400640 1 -- 172.21.15.107:0/1465271468 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0c81eae0 con 0x555d0c8f1c70 2024-09-19T04:30:16.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.365+0000 7fb875000640 1 --2- 172.21.15.107:0/1465271468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c8ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:16.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.365+0000 7fb876400640 1 --2- 172.21.15.107:0/1465271468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8f2050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb868009f50 tx=0x7fb8680118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d2ecd1dc27eb8cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:16.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.366+0000 7fb862a00640 1 -- 172.21.15.107:0/1465271468 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868004b20 con 0x555d0c8f1c70 2024-09-19T04:30:16.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.366+0000 7fb862a00640 1 -- 172.21.15.107:0/1465271468 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb868004cc0 con 0x555d0c8f1c70 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.366+0000 7fb862a00640 1 -- 172.21.15.107:0/1465271468 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868005020 con 0x555d0c8f1c70 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.366+0000 7fb86e000640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb8580a3430 0x7fb8580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.366+0000 7fb86e000640 1 -- 172.21.15.107:0/3924196815 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fb85c0cff70 con 0x7fb870102790 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.366+0000 7fb875a00640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb8580a3430 0x7fb8580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb87aeb2740 1 -- 172.21.15.107:0/1465271468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 msgr2=0x555d0c8f2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/1465271468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8f2050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb868009f50 tx=0x7fb8680118d0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb875a00640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb8580a3430 0x7fb8580a58f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb864000c00 tx=0x7fb864005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb87aeb2740 1 -- 172.21.15.107:0/1465271468 shutdown_connections 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/1465271468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c8ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/1465271468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8f2590 0x555d0c8fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/1465271468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8f2050 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb87aeb2740 1 -- 172.21.15.107:0/1465271468 >> 172.21.15.107:0/1465271468 conn(0x555d0c8ee200 msgr2=0x555d0c8ee5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb87aeb2740 1 -- 172.21.15.107:0/1465271468 shutdown_connections 2024-09-19T04:30:16.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb87aeb2740 1 -- 172.21.15.107:0/1465271468 wait complete. 2024-09-19T04:30:16.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.367+0000 7fb87aeb2740 1 -- start start 2024-09-19T04:30:16.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.368+0000 7fb87aeb2740 1 Processor -- start 2024-09-19T04:30:16.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.368+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f1c70 0x555d0c8f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.368+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 0x555d0c8fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.368+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 0x555d0c8fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.368+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555d0c8ffa20 con 0x555d0c8fce50 2024-09-19T04:30:16.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.368+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555d0c8130f0 con 0x555d0c8f1c70 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.368+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555d0c8ffba0 con 0x555d0c8f2590 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.368+0000 7fb87aeb2740 10 client.0 _ll_register_callbacks cb 0x555d0c7fabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb875a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f1c70 0x555d0c8f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb875a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f1c70 0x555d0c8f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47674/0 (socket says 172.21.15.107:47674) 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb875a00640 1 -- 172.21.15.107:0/3731384577 learned_addr learned my addr 172.21.15.107:0/3731384577 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb875000640 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 0x555d0c8fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb876400640 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 0x555d0c8fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb875a00640 1 -- 172.21.15.107:0/3731384577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 msgr2=0x555d0c8fa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb875a00640 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 0x555d0c8fa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb875a00640 1 -- 172.21.15.107:0/3731384577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 msgr2=0x555d0c8fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb875a00640 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 0x555d0c8fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb875a00640 1 -- 172.21.15.107:0/3731384577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0c5c93c0 con 0x555d0c8f1c70 2024-09-19T04:30:16.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb876400640 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 0x555d0c8fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb875a00640 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f1c70 0x555d0c8f9d30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb864009900 tx=0x7fb864009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70ad6133cadd70db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.369+0000 7fb857400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.370+0000 7fb857400640 1 -- 172.21.15.107:0/3731384577 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86404e070 con 0x555d0c8f1c70 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.370+0000 7fb857400640 1 -- 172.21.15.107:0/3731384577 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fb864004b90 con 0x555d0c8f1c70 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.370+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3731384577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb868004cc0 con 0x555d0c8f1c70 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.370+0000 7fb857400640 1 -- 172.21.15.107:0/3731384577 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb86405a050 con 0x555d0c8f1c70 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.370+0000 7fb87aeb2740 10 client.27038 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.370+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3731384577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555d0c90ba60 con 0x555d0c8f1c70 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.370+0000 7fb857400640 1 -- 172.21.15.107:0/3731384577 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb868004cc0 con 0x555d0c8f1c70 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.370+0000 7fb857400640 1 -- 172.21.15.107:0/3731384577 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fb8640036f0 con 0x555d0c8f1c70 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.371+0000 7fb87aeb2740 10 client.27038 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:16.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.371+0000 7fb87aeb2740 10 client.27038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:16.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.371+0000 7fb87aeb2740 20 client.27038 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:16.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.371+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555d0c90cec0 0x555d0c90f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.371+0000 7fb87aeb2740 4 client.27038 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:16.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.371+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3731384577 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d0c865950 con 0x555d0c90cec0 2024-09-19T04:30:16.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.371+0000 7fb875000640 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555d0c90cec0 0x555d0c90f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.371+0000 7fb875000640 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555d0c90cec0 0x555d0c90f2f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14897d204afaf74 server_cookie=e511cf00d416d4bf in_seq=0 out_seq=0 2024-09-19T04:30:16.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.371+0000 7fb857400640 10 client.27038 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:16.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb857400640 1 -- 172.21.15.107:0/3731384577 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d0c865950 con 0x555d0c90cec0 2024-09-19T04:30:16.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb857400640 10 client.27038 handle_command_reply: tid=0 2024-09-19T04:30:16.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 client.27038 shutdown 2024-09-19T04:30:16.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3731384577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f1c70 msgr2=0x555d0c8f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:16.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f1c70 0x555d0c8f9d30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb864009900 tx=0x7fb864009260 comp rx=0 tx=0).stop 2024-09-19T04:30:16.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3731384577 shutdown_connections 2024-09-19T04:30:16.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555d0c90cec0 0x555d0c90f2f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 0x555d0c8fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 0x555d0c8fa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/3731384577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f1c70 0x555d0c8f9d30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb857400640 0 client.27038 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:16.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3731384577 >> 172.21.15.107:0/3731384577 conn(0x555d0c8ee200 msgr2=0x555d0c8f3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:16.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3731384577 shutdown_connections 2024-09-19T04:30:16.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.372+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3731384577 wait complete. 2024-09-19T04:30:16.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.373+0000 7fb87aeb2740 20 client.27038 trim_cache size 0 max 16384 2024-09-19T04:30:16.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.387+0000 7fb87aeb2740 1 Processor -- start 2024-09-19T04:30:16.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.387+0000 7fb87aeb2740 1 -- start start 2024-09-19T04:30:16.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8ed270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f2590 0x555d0c914bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 0x555d0c915100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701af9d0 con 0x555d0c8fce50 2024-09-19T04:30:16.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701b1510 con 0x555d0c8f2590 2024-09-19T04:30:16.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701b00a0 con 0x555d0c8f1c70 2024-09-19T04:30:16.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb875000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f2590 0x555d0c914bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb875000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f2590 0x555d0c914bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47688/0 (socket says 172.21.15.107:47688) 2024-09-19T04:30:16.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb876400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 0x555d0c915100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb875000640 1 -- 172.21.15.107:0/3157657606 learned_addr learned my addr 172.21.15.107:0/3157657606 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:16.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb875a00640 1 --2- 172.21.15.107:0/3157657606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8ed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb875a00640 1 -- 172.21.15.107:0/3157657606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f2590 msgr2=0x555d0c914bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:16.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb875a00640 1 --2- 172.21.15.107:0/3157657606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f2590 0x555d0c914bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb875a00640 1 -- 172.21.15.107:0/3157657606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 msgr2=0x555d0c915100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:16.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb875a00640 1 --2- 172.21.15.107:0/3157657606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 0x555d0c915100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.388+0000 7fb875a00640 1 -- 172.21.15.107:0/3157657606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0c8ee030 con 0x555d0c8f1c70 2024-09-19T04:30:16.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.389+0000 7fb875a00640 1 --2- 172.21.15.107:0/3157657606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8ed270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb86402a910 tx=0x7fb8640058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5affad8923f1a3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:16.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.389+0000 7fb862000640 1 -- 172.21.15.107:0/3157657606 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864062070 con 0x555d0c8f1c70 2024-09-19T04:30:16.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.389+0000 7fb862000640 1 -- 172.21.15.107:0/3157657606 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb864004e70 con 0x555d0c8f1c70 2024-09-19T04:30:16.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.389+0000 7fb862000640 1 -- 172.21.15.107:0/3157657606 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86404e070 con 0x555d0c8f1c70 2024-09-19T04:30:16.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.390+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3157657606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 msgr2=0x555d0c8ed270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:16.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.390+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/3157657606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8ed270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb86402a910 tx=0x7fb8640058f0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.390+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3157657606 shutdown_connections 2024-09-19T04:30:16.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.390+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/3157657606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8fce50 0x555d0c915100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.390+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/3157657606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8f2590 0x555d0c914bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.390+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/3157657606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f1c70 0x555d0c8ed270 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.390+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3157657606 >> 172.21.15.107:0/3157657606 conn(0x555d0c8ee200 msgr2=0x555d0c8ecb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:16.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.390+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3157657606 shutdown_connections 2024-09-19T04:30:16.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.390+0000 7fb87aeb2740 1 -- 172.21.15.107:0/3157657606 wait complete. 2024-09-19T04:30:16.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.390+0000 7fb87aeb2740 1 -- start start 2024-09-19T04:30:16.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb87aeb2740 1 Processor -- start 2024-09-19T04:30:16.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8f1c70 0x555d0c900bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 0x555d0c901100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb87aeb2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c901640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701b0e40 con 0x555d0c8f1c70 2024-09-19T04:30:16.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701b0770 con 0x555d0c8fce50 2024-09-19T04:30:16.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb87aeb2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701ad9d0 con 0x555d0c8f2590 2024-09-19T04:30:16.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb87aeb2740 10 client.0 _ll_register_callbacks cb 0x7fb864009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:16.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb875000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 0x555d0c901100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb876400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c901640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb875000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 0x555d0c901100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46316/0 (socket says 172.21.15.107:46316) 2024-09-19T04:30:16.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb876400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c901640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47696/0 (socket says 172.21.15.107:47696) 2024-09-19T04:30:16.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb875000640 1 -- 172.21.15.107:0/2640535926 learned_addr learned my addr 172.21.15.107:0/2640535926 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:16.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.391+0000 7fb875a00640 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8f1c70 0x555d0c900bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.392+0000 7fb876400640 1 -- 172.21.15.107:0/2640535926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 msgr2=0x555d0c901100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:16.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.392+0000 7fb876400640 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 0x555d0c901100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.392+0000 7fb876400640 1 -- 172.21.15.107:0/2640535926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8f1c70 msgr2=0x555d0c900bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:16.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.392+0000 7fb876400640 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8f1c70 0x555d0c900bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.392+0000 7fb876400640 1 -- 172.21.15.107:0/2640535926 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0c8eee90 con 0x555d0c8fce50 2024-09-19T04:30:16.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.392+0000 7fb876400640 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c901640 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb86801b040 tx=0x7fb868005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23630947e860ecc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:16.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.392+0000 7fb857400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:16.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.392+0000 7fb857400640 1 -- 172.21.15.107:0/2640535926 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868009b70 con 0x555d0c8fce50 2024-09-19T04:30:16.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.392+0000 7fb87aeb2740 1 -- 172.21.15.107:0/2640535926 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb864004e70 con 0x555d0c8fce50 2024-09-19T04:30:16.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.392+0000 7fb857400640 1 -- 172.21.15.107:0/2640535926 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fb8680055f0 con 0x555d0c8fce50 2024-09-19T04:30:16.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb857400640 1 -- 172.21.15.107:0/2640535926 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb8680057f0 con 0x555d0c8fce50 2024-09-19T04:30:16.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb87aeb2740 10 client.27044 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:16.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb87aeb2740 1 -- 172.21.15.107:0/2640535926 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555d0c8e9770 con 0x555d0c8fce50 2024-09-19T04:30:16.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb857400640 1 -- 172.21.15.107:0/2640535926 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb864004e70 con 0x555d0c8fce50 2024-09-19T04:30:16.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb857400640 1 -- 172.21.15.107:0/2640535926 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fb86802d410 con 0x555d0c8fce50 2024-09-19T04:30:16.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb87aeb2740 10 client.27044 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:16.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb87aeb2740 10 client.27044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:16.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb87aeb2740 20 client.27044 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:16.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555d0c978080 0x555d0c97a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:16.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb87aeb2740 4 client.27044 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:16.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb87aeb2740 1 -- 172.21.15.107:0/2640535926 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d0c85fe60 con 0x555d0c978080 2024-09-19T04:30:16.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.393+0000 7fb875a00640 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555d0c978080 0x555d0c97a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:16.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb875a00640 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555d0c978080 0x555d0c97a470 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d0f57af205041d2 server_cookie=4038da3f281e9baf in_seq=0 out_seq=0 2024-09-19T04:30:16.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb857400640 10 client.27044 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:16.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb857400640 1 -- 172.21.15.107:0/2640535926 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d0c85fe60 con 0x555d0c978080 2024-09-19T04:30:16.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb857400640 10 client.27044 handle_command_reply: tid=0 2024-09-19T04:30:16.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb87aeb2740 1 client.27044 shutdown 2024-09-19T04:30:16.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb87aeb2740 1 -- 172.21.15.107:0/2640535926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 msgr2=0x555d0c901640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:16.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c901640 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb86801b040 tx=0x7fb868005270 comp rx=0 tx=0).stop 2024-09-19T04:30:16.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb87aeb2740 1 -- 172.21.15.107:0/2640535926 shutdown_connections 2024-09-19T04:30:16.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555d0c978080 0x555d0c97a470 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d0c8fce50 0x555d0c901640 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d0c8f2590 0x555d0c901100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb857400640 0 client.27044 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:16.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb87aeb2740 1 --2- 172.21.15.107:0/2640535926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d0c8f1c70 0x555d0c900bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.394+0000 7fb87aeb2740 1 -- 172.21.15.107:0/2640535926 >> 172.21.15.107:0/2640535926 conn(0x555d0c8ee200 msgr2=0x555d0c8f3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:16.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb87aeb2740 1 -- 172.21.15.107:0/2640535926 shutdown_connections 2024-09-19T04:30:16.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb87aeb2740 1 -- 172.21.15.107:0/2640535926 wait complete. 2024-09-19T04:30:16.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb87aeb2740 20 client.27044 trim_cache size 0 max 16384 2024-09-19T04:30:16.405 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:30:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb878400640 1 -- 172.21.15.107:0/3924196815 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb8580a3430 msgr2=0x7fb8580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb878400640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb8580a3430 0x7fb8580a58f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb864000c00 tx=0x7fb864005b50 comp rx=0 tx=0).stop 2024-09-19T04:30:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb878400640 1 -- 172.21.15.107:0/3924196815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb870102790 msgr2=0x7fb870074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb878400640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb870102790 0x7fb870074a30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9f0 tx=0x7fb85c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb878400640 1 -- 172.21.15.107:0/3924196815 shutdown_connections 2024-09-19T04:30:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb878400640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb8580a3430 0x7fb8580a58f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb878400640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb870106f10 0x7fb870074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb878400640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb870102790 0x7fb870074a30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb878400640 1 --2- 172.21.15.107:0/3924196815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb870101de0 0x7fb8700744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.395+0000 7fb878400640 1 -- 172.21.15.107:0/3924196815 >> 172.21.15.107:0/3924196815 conn(0x7fb8700fd8c0 msgr2=0x7fb8700fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.396+0000 7fb878400640 1 -- 172.21.15.107:0/3924196815 shutdown_connections 2024-09-19T04:30:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:16.396+0000 7fb878400640 1 -- 172.21.15.107:0/3924196815 wait complete. 2024-09-19T04:30:16.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 327790 2.73 MB/sec execute 394 sec latency 28.153 ms 2024-09-19T04:30:17.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:17 smithi107 ceph-mon[30659]: pgmap v309: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-09-19T04:30:17.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:17 smithi115 ceph-mon[35477]: pgmap v309: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-09-19T04:30:17.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:17 smithi145 ceph-mon[35096]: pgmap v309: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-09-19T04:30:17.715 INFO:tasks.workunit.client.0.smithi107.stdout: 1 328029 2.72 MB/sec execute 395 sec latency 799.473 ms 2024-09-19T04:30:18.263 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:30:18.264 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:30:18.634 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:18.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 328029 2.72 MB/sec execute 396 sec latency 1799.582 ms 2024-09-19T04:30:19.087 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:19 smithi107 ceph-mon[30659]: pgmap v310: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-09-19T04:30:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.200+0000 7fc6fca00640 1 -- 172.21.15.107:0/3141641584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f81019c0 msgr2=0x7fc6f8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.200+0000 7fc6fca00640 1 --2- 172.21.15.107:0/3141641584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f81019c0 0x7fc6f8101e20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009930 tx=0x7fc6e002a420 comp rx=0 tx=0).stop 2024-09-19T04:30:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.200+0000 7fc6fca00640 1 -- 172.21.15.107:0/3141641584 shutdown_connections 2024-09-19T04:30:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.200+0000 7fc6fca00640 1 --2- 172.21.15.107:0/3141641584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8102360 0x7fc6f810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.200+0000 7fc6fca00640 1 --2- 172.21.15.107:0/3141641584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f81019c0 0x7fc6f8101e20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.200+0000 7fc6fca00640 1 --2- 172.21.15.107:0/3141641584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f81079c0 0x7fc6f8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.200+0000 7fc6fca00640 1 -- 172.21.15.107:0/3141641584 >> 172.21.15.107:0/3141641584 conn(0x7fc6f80fd740 msgr2=0x7fc6f80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.200+0000 7fc6fca00640 1 -- 172.21.15.107:0/3141641584 shutdown_connections 2024-09-19T04:30:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.200+0000 7fc6fca00640 1 -- 172.21.15.107:0/3141641584 wait complete. 2024-09-19T04:30:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.201+0000 7fc6fca00640 1 Processor -- start 2024-09-19T04:30:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.201+0000 7fc6fca00640 1 -- start start 2024-09-19T04:30:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.201+0000 7fc6fca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f81019c0 0x7fc6f8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.201+0000 7fc6fca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f8102360 0x7fc6f8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6fca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f81079c0 0x7fc6f819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6fca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f8111dc0 con 0x7fc6f8102360 2024-09-19T04:30:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6fca00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f8111c40 con 0x7fc6f81079c0 2024-09-19T04:30:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6fca00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f8111f40 con 0x7fc6f81019c0 2024-09-19T04:30:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f8102360 0x7fc6f8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f8102360 0x7fc6f8198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46746/0 (socket says 172.21.15.107:46746) 2024-09-19T04:30:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6efe00640 1 -- 172.21.15.107:0/2073512956 learned_addr learned my addr 172.21.15.107:0/2073512956 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6f7400640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f81019c0 0x7fc6f8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6f7e00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f81079c0 0x7fc6f819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6efe00640 1 -- 172.21.15.107:0/2073512956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f81019c0 msgr2=0x7fc6f8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6efe00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f81019c0 0x7fc6f8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6efe00640 1 -- 172.21.15.107:0/2073512956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f81079c0 msgr2=0x7fc6f819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6efe00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f81079c0 0x7fc6f819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6efe00640 1 -- 172.21.15.107:0/2073512956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f819d2d0 con 0x7fc6f8102360 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.202+0000 7fc6f7e00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f81079c0 0x7fc6f819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.203+0000 7fc6efe00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f8102360 0x7fc6f8198860 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009230 tx=0x7fc6e0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.203+0000 7fc6f4c00640 1 -- 172.21.15.107:0/2073512956 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e003f030 con 0x7fc6f8102360 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.203+0000 7fc6f4c00640 1 -- 172.21.15.107:0/2073512956 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e0004440 con 0x7fc6f8102360 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.203+0000 7fc6fca00640 1 -- 172.21.15.107:0/2073512956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f819d560 con 0x7fc6f8102360 2024-09-19T04:30:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.203+0000 7fc6f4c00640 1 -- 172.21.15.107:0/2073512956 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0035070 con 0x7fc6f8102360 2024-09-19T04:30:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.203+0000 7fc6fca00640 1 -- 172.21.15.107:0/2073512956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f81a4ea0 con 0x7fc6f8102360 2024-09-19T04:30:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.207+0000 7fc700bbb740 1 Processor -- start 2024-09-19T04:30:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.207+0000 7fc700bbb740 1 -- start start 2024-09-19T04:30:19.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.207+0000 7fc6f4c00640 1 -- 172.21.15.107:0/2073512956 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fc6e0042030 con 0x7fc6f8102360 2024-09-19T04:30:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e748838090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 0x55e748842950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 0x55e748845280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc700bbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e74879e290 con 0x55e748837cb0 2024-09-19T04:30:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc700bbb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e748745b40 con 0x55e748842e90 2024-09-19T04:30:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc700bbb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e74879a250 con 0x55e7488385d0 2024-09-19T04:30:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc6f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e748838090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 0x55e748845280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc6f7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 0x55e748842950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 0x55e748845280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:34340/0 (socket says 172.21.15.107:34340) 2024-09-19T04:30:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc6efe00640 1 -- 172.21.15.107:0/791209545 learned_addr learned my addr 172.21.15.107:0/791209545 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.208+0000 7fc6f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e748838090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46750/0 (socket says 172.21.15.107:46750) 2024-09-19T04:30:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6f7400640 1 -- 172.21.15.107:0/791209545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 msgr2=0x55e748845280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6f7400640 1 --2- 172.21.15.107:0/791209545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 0x55e748845280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6f7400640 1 -- 172.21.15.107:0/791209545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 msgr2=0x55e748838090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6f7400640 1 --2- 172.21.15.107:0/791209545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e748838090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6f7400640 1 -- 172.21.15.107:0/791209545 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74850f6a0 con 0x55e7488385d0 2024-09-19T04:30:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6f7400640 1 --2- 172.21.15.107:0/791209545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 0x55e748842950 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009c00 tx=0x7fc6e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95031ce6aab2b285 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6f4c00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc6dc0a3430 0x7fc6dc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6f4c00640 1 -- 172.21.15.107:0/2073512956 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fc6e00efff0 con 0x7fc6f8102360 2024-09-19T04:30:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6e7e00640 1 -- 172.21.15.107:0/791209545 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8004050 con 0x55e7488385d0 2024-09-19T04:30:19.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6e7e00640 1 -- 172.21.15.107:0/791209545 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e8005500 con 0x55e7488385d0 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6f7e00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc6dc0a3430 0x7fc6dc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.209+0000 7fc6e7e00640 1 -- 172.21.15.107:0/791209545 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8004da0 con 0x55e7488385d0 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.210+0000 7fc700bbb740 1 -- 172.21.15.107:0/791209545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 msgr2=0x55e748842950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.210+0000 7fc700bbb740 1 --2- 172.21.15.107:0/791209545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 0x55e748842950 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009c00 tx=0x7fc6e8011810 comp rx=0 tx=0).stop 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.210+0000 7fc6f7e00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc6dc0a3430 0x7fc6dc0a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0005fd0 tx=0x7fc6f0009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.210+0000 7fc700bbb740 1 -- 172.21.15.107:0/791209545 shutdown_connections 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.210+0000 7fc700bbb740 1 --2- 172.21.15.107:0/791209545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 0x55e748845280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.210+0000 7fc700bbb740 1 --2- 172.21.15.107:0/791209545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 0x55e748842950 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.210+0000 7fc700bbb740 1 --2- 172.21.15.107:0/791209545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e748838090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.210+0000 7fc700bbb740 1 -- 172.21.15.107:0/791209545 >> 172.21.15.107:0/791209545 conn(0x55e748834240 msgr2=0x55e748834620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.210+0000 7fc700bbb740 1 -- 172.21.15.107:0/791209545 shutdown_connections 2024-09-19T04:30:19.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.210+0000 7fc700bbb740 1 -- 172.21.15.107:0/791209545 wait complete. 2024-09-19T04:30:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.211+0000 7fc700bbb740 1 -- start start 2024-09-19T04:30:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.211+0000 7fc700bbb740 1 Processor -- start 2024-09-19T04:30:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e748842680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 0x55e74883f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 0x55e74883fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc700bbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e748845b50 con 0x55e748837cb0 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc700bbb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e748758fa0 con 0x55e748842e90 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc700bbb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e748845cd0 con 0x55e7488385d0 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc700bbb740 10 client.0 _ll_register_callbacks cb 0x55e7484acba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc6f7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 0x55e74883fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc6f7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 0x55e74883fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:34348/0 (socket says 172.21.15.107:34348) 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc6f7e00640 1 -- 172.21.15.107:0/1277830374 learned_addr learned my addr 172.21.15.107:0/1277830374 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc6efe00640 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 0x55e74883f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.212+0000 7fc6f7400640 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e748842680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.213+0000 7fc6efe00640 1 -- 172.21.15.107:0/1277830374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 msgr2=0x55e74883fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.213+0000 7fc6efe00640 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 0x55e74883fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.213+0000 7fc6efe00640 1 -- 172.21.15.107:0/1277830374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 msgr2=0x55e748842680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.213+0000 7fc6efe00640 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e748842680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.213+0000 7fc6efe00640 1 -- 172.21.15.107:0/1277830374 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7488406a0 con 0x55e7488385d0 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.213+0000 7fc6efe00640 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 0x55e74883f930 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00edd60 tx=0x7fc6e00f3730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2ec609fede1521 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.213+0000 7fc6e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.214+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1277830374 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0049050 con 0x55e7488385d0 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.214+0000 7fc700bbb740 1 -- 172.21.15.107:0/1277830374 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e8005500 con 0x55e7488385d0 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.214+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1277830374 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fc6e00b8960 con 0x55e7488385d0 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.214+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1277830374 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc6e00b6e60 con 0x55e7488385d0 2024-09-19T04:30:19.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.214+0000 7fc700bbb740 10 client.27019 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.214+0000 7fc700bbb740 1 -- 172.21.15.107:0/1277830374 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e748845ba0 con 0x55e7488385d0 2024-09-19T04:30:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.215+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1277830374 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e8005500 con 0x55e7488385d0 2024-09-19T04:30:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.215+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1277830374 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fc6e00f2110 con 0x55e7488385d0 2024-09-19T04:30:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.215+0000 7fc700bbb740 10 client.27019 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.215+0000 7fc700bbb740 10 client.27019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.215+0000 7fc700bbb740 20 client.27019 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.215+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e7488571f0 0x55e748859640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.215+0000 7fc700bbb740 4 client.27019 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.215+0000 7fc700bbb740 1 -- 172.21.15.107:0/1277830374 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7487ab800 con 0x55e7488571f0 2024-09-19T04:30:19.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.215+0000 7fc6f7400640 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e7488571f0 0x55e748859640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.216+0000 7fc6f7400640 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e7488571f0 0x55e748859640 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72a83f533d38e8a server_cookie=b7d278c5d6a704fb in_seq=0 out_seq=0 2024-09-19T04:30:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.216+0000 7fc6e4c00640 10 client.27019 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1277830374 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7487ab800 con 0x55e7488571f0 2024-09-19T04:30:19.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc6e4c00640 10 client.27019 handle_command_reply: tid=0 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 client.27019 shutdown 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 -- 172.21.15.107:0/1277830374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 msgr2=0x55e74883f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 0x55e74883f930 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00edd60 tx=0x7fc6e00f3730 comp rx=0 tx=0).stop 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 -- 172.21.15.107:0/1277830374 shutdown_connections 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e7488571f0 0x55e748859640 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748842e90 0x55e74883fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e7488385d0 0x55e74883f930 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1277830374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e748842680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc6e4c00640 0 client.27019 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 -- 172.21.15.107:0/1277830374 >> 172.21.15.107:0/1277830374 conn(0x55e748834240 msgr2=0x55e748839f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 -- 172.21.15.107:0/1277830374 shutdown_connections 2024-09-19T04:30:19.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 1 -- 172.21.15.107:0/1277830374 wait complete. 2024-09-19T04:30:19.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.217+0000 7fc700bbb740 20 client.27019 trim_cache size 0 max 16384 2024-09-19T04:30:19.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc700bbb740 1 Processor -- start 2024-09-19T04:30:19.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc700bbb740 1 -- start start 2024-09-19T04:30:19.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748837cb0 0x55e74885ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7488385d0 0x55e74885f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e748842e90 0x55e74883cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc700bbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f81ab3b0 con 0x55e7488385d0 2024-09-19T04:30:19.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc700bbb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f81acef0 con 0x55e748837cb0 2024-09-19T04:30:19.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc700bbb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f81aba80 con 0x55e748842e90 2024-09-19T04:30:19.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc6f7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748837cb0 0x55e74885ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc6f7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748837cb0 0x55e74885ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:34364/0 (socket says 172.21.15.107:34364) 2024-09-19T04:30:19.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.244+0000 7fc6f7400640 1 -- 172.21.15.107:0/1774624713 learned_addr learned my addr 172.21.15.107:0/1774624713 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:19.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.245+0000 7fc6f7e00640 1 --2- 172.21.15.107:0/1774624713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e748842e90 0x55e74883cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.245+0000 7fc6efe00640 1 --2- 172.21.15.107:0/1774624713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7488385d0 0x55e74885f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.245+0000 7fc6f7400640 1 -- 172.21.15.107:0/1774624713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e748842e90 msgr2=0x55e74883cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:19.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.245+0000 7fc6f7400640 1 --2- 172.21.15.107:0/1774624713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e748842e90 0x55e74883cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.245+0000 7fc6f7400640 1 -- 172.21.15.107:0/1774624713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7488385d0 msgr2=0x55e74885f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:19.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.245+0000 7fc6f7400640 1 --2- 172.21.15.107:0/1774624713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7488385d0 0x55e74885f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.245+0000 7fc6f7400640 1 -- 172.21.15.107:0/1774624713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74883e920 con 0x55e748837cb0 2024-09-19T04:30:19.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.245+0000 7fc6f7400640 1 --2- 172.21.15.107:0/1774624713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748837cb0 0x55e74885ef50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009c00 tx=0x7fc6e8004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578dadd8ceebac22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:19.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.245+0000 7fc6e7400640 1 -- 172.21.15.107:0/1774624713 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8009db0 con 0x55e748837cb0 2024-09-19T04:30:19.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.246+0000 7fc6e7400640 1 -- 172.21.15.107:0/1774624713 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e80035f0 con 0x55e748837cb0 2024-09-19T04:30:19.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.246+0000 7fc6e7400640 1 -- 172.21.15.107:0/1774624713 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8004520 con 0x55e748837cb0 2024-09-19T04:30:19.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.246+0000 7fc700bbb740 1 -- 172.21.15.107:0/1774624713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748837cb0 msgr2=0x55e74885ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:19.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.246+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1774624713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748837cb0 0x55e74885ef50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009c00 tx=0x7fc6e8004a80 comp rx=0 tx=0).stop 2024-09-19T04:30:19.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.246+0000 7fc700bbb740 1 -- 172.21.15.107:0/1774624713 shutdown_connections 2024-09-19T04:30:19.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.246+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1774624713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e748842e90 0x55e74883cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.246+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1774624713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7488385d0 0x55e74885f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.246+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1774624713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e748837cb0 0x55e74885ef50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.246+0000 7fc700bbb740 1 -- 172.21.15.107:0/1774624713 >> 172.21.15.107:0/1774624713 conn(0x55e748834240 msgr2=0x55e7488419b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:19.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.247+0000 7fc700bbb740 1 -- 172.21.15.107:0/1774624713 shutdown_connections 2024-09-19T04:30:19.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.247+0000 7fc700bbb740 1 -- 172.21.15.107:0/1774624713 wait complete. 2024-09-19T04:30:19.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.247+0000 7fc700bbb740 1 -- start start 2024-09-19T04:30:19.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.247+0000 7fc700bbb740 1 Processor -- start 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e74884f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7488385d0 0x55e74884c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc700bbb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e748842e90 0x55e74884c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc700bbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f81ac820 con 0x55e748837cb0 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc700bbb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f81ac150 con 0x55e7488385d0 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc700bbb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f81a93b0 con 0x55e748842e90 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc700bbb740 10 client.0 _ll_register_callbacks cb 0x7fc6e0009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc6f7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e74884f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7488385d0 0x55e74884c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc6f7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e74884f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46782/0 (socket says 172.21.15.107:46782) 2024-09-19T04:30:19.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.248+0000 7fc6f7400640 1 -- 172.21.15.107:0/1663907712 learned_addr learned my addr 172.21.15.107:0/1663907712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6f7e00640 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e748842e90 0x55e74884c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6efe00640 1 -- 172.21.15.107:0/1663907712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e748842e90 msgr2=0x55e74884c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6efe00640 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e748842e90 0x55e74884c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6efe00640 1 -- 172.21.15.107:0/1663907712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 msgr2=0x55e74884f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6efe00640 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e74884f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6efe00640 1 -- 172.21.15.107:0/1663907712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74884cf30 con 0x55e7488385d0 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6efe00640 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7488385d0 0x55e74884c380 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55e748840ee0 tx=0x7fc6e0005cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4a27cedbce85b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1663907712 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0049070 con 0x55e7488385d0 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1663907712 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fc6e00c4ad0 con 0x55e7488385d0 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc700bbb740 1 -- 172.21.15.107:0/1663907712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e80035f0 con 0x55e7488385d0 2024-09-19T04:30:19.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1663907712 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc6e00bc2f0 con 0x55e7488385d0 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.249+0000 7fc700bbb740 10 client.27062 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.250+0000 7fc700bbb740 1 -- 172.21.15.107:0/1663907712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e7488c2de0 con 0x55e7488385d0 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.250+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1663907712 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e80035f0 con 0x55e7488385d0 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.250+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1663907712 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fc6e00bb740 con 0x55e7488385d0 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.250+0000 7fc700bbb740 10 client.27062 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.250+0000 7fc700bbb740 10 client.27062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.250+0000 7fc700bbb740 20 client.27062 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.250+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e7488c3380 0x55e7488c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.250+0000 7fc700bbb740 4 client.27062 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.250+0000 7fc700bbb740 1 -- 172.21.15.107:0/1663907712 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e7487a5d10 con 0x55e7488c3380 2024-09-19T04:30:19.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.250+0000 7fc6f7400640 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e7488c3380 0x55e7488c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:19.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.251+0000 7fc6f7400640 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e7488c3380 0x55e7488c57a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a709ff64e6973aba server_cookie=dddbbcee93365e26 in_seq=0 out_seq=0 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.251+0000 7fc6e4c00640 10 client.27062 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.252+0000 7fc6e4c00640 1 -- 172.21.15.107:0/1663907712 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e7487a5d10 con 0x55e7488c3380 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.252+0000 7fc6e4c00640 10 client.27062 handle_command_reply: tid=0 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.252+0000 7fc700bbb740 1 client.27062 shutdown 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.252+0000 7fc700bbb740 1 -- 172.21.15.107:0/1663907712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7488385d0 msgr2=0x55e74884c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.252+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7488385d0 0x55e74884c380 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55e748840ee0 tx=0x7fc6e0005cb0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.253+0000 7fc700bbb740 1 -- 172.21.15.107:0/1663907712 shutdown_connections 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.253+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55e7488c3380 0x55e7488c57a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.253+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e748842e90 0x55e74884c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.253+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7488385d0 0x55e74884c380 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.253+0000 7fc700bbb740 1 --2- 172.21.15.107:0/1663907712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e748837cb0 0x55e74884f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.253+0000 7fc700bbb740 1 -- 172.21.15.107:0/1663907712 >> 172.21.15.107:0/1663907712 conn(0x55e748834240 msgr2=0x55e748832a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.253+0000 7fc6e4c00640 0 client.27062 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.253+0000 7fc700bbb740 1 -- 172.21.15.107:0/1663907712 shutdown_connections 2024-09-19T04:30:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.253+0000 7fc700bbb740 1 -- 172.21.15.107:0/1663907712 wait complete. 2024-09-19T04:30:19.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.253+0000 7fc700bbb740 20 client.27062 trim_cache size 0 max 16384 2024-09-19T04:30:19.257 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:30:19.257 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:30:19.257 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "93ace02e-ce18-4836-9969-7cea8150688b", 2024-09-19T04:30:19.257 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:30:19.257 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:30:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 -- 172.21.15.107:0/2073512956 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc6dc0a3430 msgr2=0x7fc6dc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc6dc0a3430 0x7fc6dc0a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0005fd0 tx=0x7fc6f0009500 comp rx=0 tx=0).stop 2024-09-19T04:30:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 -- 172.21.15.107:0/2073512956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f8102360 msgr2=0x7fc6f8198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f8102360 0x7fc6f8198860 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009230 tx=0x7fc6e0004040 comp rx=0 tx=0).stop 2024-09-19T04:30:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 -- 172.21.15.107:0/2073512956 shutdown_connections 2024-09-19T04:30:19.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc6dc0a3430 0x7fc6dc0a58f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f81079c0 0x7fc6f819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f8102360 0x7fc6f8198860 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 --2- 172.21.15.107:0/2073512956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f81019c0 0x7fc6f8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 -- 172.21.15.107:0/2073512956 >> 172.21.15.107:0/2073512956 conn(0x7fc6f80fd740 msgr2=0x7fc6f80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 -- 172.21.15.107:0/2073512956 shutdown_connections 2024-09-19T04:30:19.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:19.254+0000 7fc6fca00640 1 -- 172.21.15.107:0/2073512956 wait complete. 2024-09-19T04:30:19.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:19 smithi145 ceph-mon[35096]: pgmap v310: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-09-19T04:30:19.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:19 smithi115 ceph-mon[35477]: pgmap v310: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-09-19T04:30:19.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 329097 2.72 MB/sec execute 397 sec latency 2307.538 ms 2024-09-19T04:30:20.234 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:30:20.289 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:30:20.289 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.289 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:30:20.289 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:30:20.289 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:30:20.289 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:30:20.289 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.290 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.290 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.290 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:30:20.290 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.290 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:30:20.290 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.290 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:20 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:30:20.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:30:20.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:30:20.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:30:20.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:30:20.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:30:20.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.325 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:30:20.325 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.325 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:20 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:30:20.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:20 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:20.652 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:20.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 330086 2.74 MB/sec execute 398 sec latency 315.622 ms 2024-09-19T04:30:21.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:21 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:30:21.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:21 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:21.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:21 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:30:21.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:21 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:21.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:21 smithi107 ceph-mon[30659]: pgmap v311: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-19T04:30:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.242+0000 7f770d800640 1 -- 172.21.15.107:0/2904187033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7708106f20 msgr2=0x7f770810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.242+0000 7f770d800640 1 --2- 172.21.15.107:0/2904187033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7708106f20 0x7f770810eaa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7700009900 tx=0x7f770002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.242+0000 7f770d800640 1 -- 172.21.15.107:0/2904187033 shutdown_connections 2024-09-19T04:30:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.242+0000 7f770d800640 1 --2- 172.21.15.107:0/2904187033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7708106f20 0x7f770810eaa0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.242+0000 7f770d800640 1 --2- 172.21.15.107:0/2904187033 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77081027a0 0x7f77081067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.242+0000 7f770d800640 1 --2- 172.21.15.107:0/2904187033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7708101df0 0x7f77081021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.242+0000 7f770d800640 1 -- 172.21.15.107:0/2904187033 >> 172.21.15.107:0/2904187033 conn(0x7f77080fd8b0 msgr2=0x7f77080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.242+0000 7f770d800640 1 -- 172.21.15.107:0/2904187033 shutdown_connections 2024-09-19T04:30:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.242+0000 7f770d800640 1 -- 172.21.15.107:0/2904187033 wait complete. 2024-09-19T04:30:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.243+0000 7f770d800640 1 Processor -- start 2024-09-19T04:30:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.243+0000 7f770d800640 1 -- start start 2024-09-19T04:30:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.243+0000 7f770d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7708101df0 0x7f770819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.243+0000 7f770d800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77081027a0 0x7f770819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.243+0000 7f770d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7708106f20 0x7f770819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.243+0000 7f770d800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7708111fd0 con 0x7f7708106f20 2024-09-19T04:30:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.243+0000 7f770d800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7708111e50 con 0x7f77081027a0 2024-09-19T04:30:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.243+0000 7f770d800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7708112150 con 0x7f7708101df0 2024-09-19T04:30:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7708101df0 0x7f770819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77081027a0 0x7f770819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77081027a0 0x7f770819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:34390/0 (socket says 172.21.15.107:34390) 2024-09-19T04:30:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7708101df0 0x7f770819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53202/0 (socket says 172.21.15.107:53202) 2024-09-19T04:30:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706000640 1 -- 172.21.15.107:0/3259421976 learned_addr learned my addr 172.21.15.107:0/3259421976 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7707400640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7708106f20 0x7f770819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706000640 1 -- 172.21.15.107:0/3259421976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7708101df0 msgr2=0x7f770819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706000640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7708101df0 0x7f770819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706000640 1 -- 172.21.15.107:0/3259421976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7708106f20 msgr2=0x7f770819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706000640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7708106f20 0x7f770819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706000640 1 -- 172.21.15.107:0/3259421976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770819f570 con 0x7f77081027a0 2024-09-19T04:30:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f7706000640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77081027a0 0x7f770819ab00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f76f000d770 tx=0x7f76f000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:21.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f76ff400640 1 -- 172.21.15.107:0/3259421976 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0017070 con 0x7f77081027a0 2024-09-19T04:30:21.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f76ff400640 1 -- 172.21.15.107:0/3259421976 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f0005500 con 0x7f77081027a0 2024-09-19T04:30:21.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.244+0000 7f76ff400640 1 -- 172.21.15.107:0/3259421976 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0004e00 con 0x7f77081027a0 2024-09-19T04:30:21.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.245+0000 7f770d800640 1 -- 172.21.15.107:0/3259421976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770819f860 con 0x7f77081027a0 2024-09-19T04:30:21.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.245+0000 7f770d800640 1 -- 172.21.15.107:0/3259421976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77081a7140 con 0x7f77081027a0 2024-09-19T04:30:21.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f76ff400640 1 -- 172.21.15.107:0/3259421976 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f76f0004fa0 con 0x7f77081027a0 2024-09-19T04:30:21.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f77101da740 1 Processor -- start 2024-09-19T04:30:21.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f77101da740 1 -- start start 2024-09-19T04:30:21.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f77101da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd86b90 0x56104fd86f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f77101da740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fd91830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f77101da740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 0x56104fd94160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f77101da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56104fced3e0 con 0x56104fd86b90 2024-09-19T04:30:21.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f77101da740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56104fc94d80 con 0x56104fd874b0 2024-09-19T04:30:21.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f77101da740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56104fce93a0 con 0x56104fd91d70 2024-09-19T04:30:21.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f7706a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fd91830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.248+0000 7f7706a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fd91830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:34398/0 (socket says 172.21.15.107:34398) 2024-09-19T04:30:21.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f7706a00640 1 -- 172.21.15.107:0/515746534 learned_addr learned my addr 172.21.15.107:0/515746534 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:21.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f7706000640 1 --2- 172.21.15.107:0/515746534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 0x56104fd94160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f7707400640 1 --2- 172.21.15.107:0/515746534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd86b90 0x56104fd86f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f7706a00640 1 -- 172.21.15.107:0/515746534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 msgr2=0x56104fd94160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:21.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f7706a00640 1 --2- 172.21.15.107:0/515746534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 0x56104fd94160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f7706a00640 1 -- 172.21.15.107:0/515746534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd86b90 msgr2=0x56104fd86f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:21.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f7706a00640 1 --2- 172.21.15.107:0/515746534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd86b90 0x56104fd86f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f7706a00640 1 -- 172.21.15.107:0/515746534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104fcb3ae0 con 0x56104fd874b0 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f7707400640 1 --2- 172.21.15.107:0/515746534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd86b90 0x56104fd86f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f7706a00640 1 --2- 172.21.15.107:0/515746534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fd91830 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f76f80114f0 tx=0x7f76f80119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe69643878c3b8ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f76f7e00640 1 -- 172.21.15.107:0/515746534 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f801e070 con 0x56104fd874b0 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f76f7e00640 1 -- 172.21.15.107:0/515746534 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f800fb00 con 0x56104fd874b0 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.249+0000 7f76f7e00640 1 -- 172.21.15.107:0/515746534 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8019bf0 con 0x56104fd874b0 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f76ff400640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76ec0a33b0 0x7f76ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f76ff400640 1 -- 172.21.15.107:0/3259421976 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f76f00cf6d0 con 0x7f77081027a0 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f7707400640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76ec0a33b0 0x7f76ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f77101da740 1 -- 172.21.15.107:0/515746534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 msgr2=0x56104fd91830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f77101da740 1 --2- 172.21.15.107:0/515746534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fd91830 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f76f80114f0 tx=0x7f76f80119b0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f77101da740 1 -- 172.21.15.107:0/515746534 shutdown_connections 2024-09-19T04:30:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f77101da740 1 --2- 172.21.15.107:0/515746534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 0x56104fd94160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f77101da740 1 --2- 172.21.15.107:0/515746534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fd91830 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f77101da740 1 --2- 172.21.15.107:0/515746534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd86b90 0x56104fd86f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f77101da740 1 -- 172.21.15.107:0/515746534 >> 172.21.15.107:0/515746534 conn(0x56104fd83120 msgr2=0x56104fd83500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.250+0000 7f77101da740 1 -- 172.21.15.107:0/515746534 shutdown_connections 2024-09-19T04:30:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.251+0000 7f77101da740 1 -- 172.21.15.107:0/515746534 wait complete. 2024-09-19T04:30:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.251+0000 7f7707400640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76ec0a33b0 0x7f76ec0a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7700031550 tx=0x7f77000046c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.251+0000 7f77101da740 1 -- start start 2024-09-19T04:30:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.251+0000 7f77101da740 1 Processor -- start 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f77101da740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd8ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f77101da740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd874b0 0x56104fd8f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f77101da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd91d70 0x56104fd8f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f77101da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56104fd949e0 con 0x56104fd91d70 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f77101da740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56104fca80f0 con 0x56104fd86b90 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f77101da740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56104fd94b60 con 0x56104fd874b0 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f77101da740 10 client.0 _ll_register_callbacks cb 0x56104fc999e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7706a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd8ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7706a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd8ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:34406/0 (socket says 172.21.15.107:34406) 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7706a00640 1 -- 172.21.15.107:0/988722370 learned_addr learned my addr 172.21.15.107:0/988722370 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7706000640 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd874b0 0x56104fd8f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7707400640 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd91d70 0x56104fd8f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7706a00640 1 -- 172.21.15.107:0/988722370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd874b0 msgr2=0x56104fd8f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7706a00640 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd874b0 0x56104fd8f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7706a00640 1 -- 172.21.15.107:0/988722370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd91d70 msgr2=0x56104fd8f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7706a00640 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd91d70 0x56104fd8f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7706a00640 1 -- 172.21.15.107:0/988722370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104fa5e3c0 con 0x56104fd86b90 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.252+0000 7f7706000640 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd874b0 0x56104fd8f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.253+0000 7f7706a00640 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd8ec60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f76f8019850 tx=0x7f76f8019410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d43af45f842d216 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.253+0000 7f76f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.253+0000 7f76f4c00640 1 -- 172.21.15.107:0/988722370 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f801e070 con 0x56104fd86b90 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.253+0000 7f76f4c00640 1 -- 172.21.15.107:0/988722370 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f76f8004c50 con 0x56104fd86b90 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.253+0000 7f77101da740 1 -- 172.21.15.107:0/988722370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76f800fb00 con 0x56104fd86b90 2024-09-19T04:30:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.253+0000 7f76f4c00640 1 -- 172.21.15.107:0/988722370 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f76f8004e50 con 0x56104fd86b90 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.253+0000 7f77101da740 10 client.27074 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.253+0000 7f77101da740 1 -- 172.21.15.107:0/988722370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56104fda0600 con 0x56104fd86b90 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.253+0000 7f76f4c00640 1 -- 172.21.15.107:0/988722370 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f800fb00 con 0x56104fd86b90 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.254+0000 7f76f4c00640 1 -- 172.21.15.107:0/988722370 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f76f80055a0 con 0x56104fd86b90 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.254+0000 7f77101da740 10 client.27074 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.254+0000 7f77101da740 10 client.27074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.254+0000 7f77101da740 20 client.27074 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.254+0000 7f77101da740 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56104fda1a10 0x56104fda3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.254+0000 7f77101da740 4 client.27074 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.254+0000 7f77101da740 1 -- 172.21.15.107:0/988722370 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56104fcfa950 con 0x56104fda1a10 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.254+0000 7f7706000640 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56104fda1a10 0x56104fda3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.254+0000 7f7706000640 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56104fda1a10 0x56104fda3e00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c05923ff3af250 server_cookie=76ab4e6bf6a8763 in_seq=0 out_seq=0 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.254+0000 7f76f4c00640 10 client.27074 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.255+0000 7f76f4c00640 1 -- 172.21.15.107:0/988722370 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56104fcfa950 con 0x56104fda1a10 2024-09-19T04:30:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.255+0000 7f76f4c00640 10 client.27074 handle_command_reply: tid=0 2024-09-19T04:30:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.255+0000 7f77101da740 1 client.27074 shutdown 2024-09-19T04:30:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.255+0000 7f77101da740 1 -- 172.21.15.107:0/988722370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 msgr2=0x56104fd8ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.255+0000 7f77101da740 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd8ec60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f76f8019850 tx=0x7f76f8019410 comp rx=0 tx=0).stop 2024-09-19T04:30:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.256+0000 7f77101da740 1 -- 172.21.15.107:0/988722370 shutdown_connections 2024-09-19T04:30:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.256+0000 7f77101da740 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56104fda1a10 0x56104fda3e00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.256+0000 7f77101da740 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd91d70 0x56104fd8f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.256+0000 7f77101da740 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd874b0 0x56104fd8f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.256+0000 7f77101da740 1 --2- 172.21.15.107:0/988722370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd8ec60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.256+0000 7f76f4c00640 0 client.27074 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:21.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.256+0000 7f77101da740 1 -- 172.21.15.107:0/988722370 >> 172.21.15.107:0/988722370 conn(0x56104fd83120 msgr2=0x56104fd88e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:21.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.256+0000 7f77101da740 1 -- 172.21.15.107:0/988722370 shutdown_connections 2024-09-19T04:30:21.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.256+0000 7f77101da740 1 -- 172.21.15.107:0/988722370 wait complete. 2024-09-19T04:30:21.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.256+0000 7f77101da740 20 client.27074 trim_cache size 0 max 16384 2024-09-19T04:30:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.275+0000 7f77101da740 1 Processor -- start 2024-09-19T04:30:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.275+0000 7f77101da740 1 -- start start 2024-09-19T04:30:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.275+0000 7f77101da740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd86b90 0x56104fd89fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.275+0000 7f77101da740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fda95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.275+0000 7f77101da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd91d70 0x56104fda9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.275+0000 7f77101da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77081afa50 con 0x56104fd91d70 2024-09-19T04:30:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.275+0000 7f77101da740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77081b1590 con 0x56104fd874b0 2024-09-19T04:30:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.275+0000 7f77101da740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77081b0120 con 0x56104fd86b90 2024-09-19T04:30:21.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fda95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd86b90 0x56104fd89fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fda95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:34408/0 (socket says 172.21.15.107:34408) 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706000640 1 -- 172.21.15.107:0/2410373594 learned_addr learned my addr 172.21.15.107:0/2410373594 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd86b90 0x56104fd89fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53234/0 (socket says 172.21.15.107:53234) 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7707400640 1 --2- 172.21.15.107:0/2410373594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd91d70 0x56104fda9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706000640 1 -- 172.21.15.107:0/2410373594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd86b90 msgr2=0x56104fd89fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706000640 1 --2- 172.21.15.107:0/2410373594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd86b90 0x56104fd89fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706000640 1 -- 172.21.15.107:0/2410373594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd91d70 msgr2=0x56104fda9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706000640 1 --2- 172.21.15.107:0/2410373594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd91d70 0x56104fda9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706000640 1 -- 172.21.15.107:0/2410373594 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104fda4390 con 0x56104fd874b0 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706a00640 1 --2- 172.21.15.107:0/2410373594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd86b90 0x56104fd89fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.276+0000 7f7706000640 1 --2- 172.21.15.107:0/2410373594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fda95c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76f00983f0 tx=0x7f76f00d1b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e5a9778e1b3773c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:21.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.277+0000 7f76f7400640 1 -- 172.21.15.107:0/2410373594 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0020040 con 0x56104fd874b0 2024-09-19T04:30:21.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.277+0000 7f76f7400640 1 -- 172.21.15.107:0/2410373594 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f00a45e0 con 0x56104fd874b0 2024-09-19T04:30:21.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.277+0000 7f76f7400640 1 -- 172.21.15.107:0/2410373594 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0098090 con 0x56104fd874b0 2024-09-19T04:30:21.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.277+0000 7f77101da740 1 -- 172.21.15.107:0/2410373594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 msgr2=0x56104fda95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:21.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.277+0000 7f77101da740 1 --2- 172.21.15.107:0/2410373594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fda95c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76f00983f0 tx=0x7f76f00d1b20 comp rx=0 tx=0).stop 2024-09-19T04:30:21.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.277+0000 7f77101da740 1 -- 172.21.15.107:0/2410373594 shutdown_connections 2024-09-19T04:30:21.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.277+0000 7f77101da740 1 --2- 172.21.15.107:0/2410373594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd91d70 0x56104fda9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.277+0000 7f77101da740 1 --2- 172.21.15.107:0/2410373594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd874b0 0x56104fda95c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.277+0000 7f77101da740 1 --2- 172.21.15.107:0/2410373594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd86b90 0x56104fd89fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.277+0000 7f77101da740 1 -- 172.21.15.107:0/2410373594 >> 172.21.15.107:0/2410373594 conn(0x56104fd83120 msgr2=0x56104fd90ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:21.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.278+0000 7f77101da740 1 -- 172.21.15.107:0/2410373594 shutdown_connections 2024-09-19T04:30:21.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.278+0000 7f77101da740 1 -- 172.21.15.107:0/2410373594 wait complete. 2024-09-19T04:30:21.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.278+0000 7f77101da740 1 -- start start 2024-09-19T04:30:21.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.278+0000 7f77101da740 1 Processor -- start 2024-09-19T04:30:21.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f77101da740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd99db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f77101da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd874b0 0x56104fd9a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f77101da740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 0x56104fd8b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f77101da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77081b0ec0 con 0x56104fd874b0 2024-09-19T04:30:21.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f77101da740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77081b07f0 con 0x56104fd86b90 2024-09-19T04:30:21.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f77101da740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77081ada50 con 0x56104fd91d70 2024-09-19T04:30:21.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f77101da740 10 client.0 _ll_register_callbacks cb 0x7f76f8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f7707400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 0x56104fd8b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f7706a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd99db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f7707400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 0x56104fd8b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53248/0 (socket says 172.21.15.107:53248) 2024-09-19T04:30:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f7707400640 1 -- 172.21.15.107:0/1703194344 learned_addr learned my addr 172.21.15.107:0/1703194344 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.279+0000 7f7706000640 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd874b0 0x56104fd9a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.280+0000 7f7706a00640 1 -- 172.21.15.107:0/1703194344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 msgr2=0x56104fd8b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.280+0000 7f7706a00640 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 0x56104fd8b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.280+0000 7f7706a00640 1 -- 172.21.15.107:0/1703194344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd874b0 msgr2=0x56104fd9a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.280+0000 7f7706a00640 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd874b0 0x56104fd9a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.280+0000 7f7706a00640 1 -- 172.21.15.107:0/1703194344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104fd8b800 con 0x56104fd86b90 2024-09-19T04:30:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.280+0000 7f7706a00640 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd99db0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f76f8005970 tx=0x7f76f800fa40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46145f95aece7032 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.280+0000 7f76f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.280+0000 7f76f4c00640 1 -- 172.21.15.107:0/1703194344 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f803c060 con 0x56104fd86b90 2024-09-19T04:30:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.280+0000 7f76f4c00640 1 -- 172.21.15.107:0/1703194344 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f76f8003b70 con 0x56104fd86b90 2024-09-19T04:30:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.281+0000 7f77101da740 1 -- 172.21.15.107:0/1703194344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76f00a45e0 con 0x56104fd86b90 2024-09-19T04:30:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.281+0000 7f76f4c00640 1 -- 172.21.15.107:0/1703194344 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f76f8004e60 con 0x56104fd86b90 2024-09-19T04:30:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.281+0000 7f77101da740 10 client.27080 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.281+0000 7f77101da740 1 -- 172.21.15.107:0/1703194344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56104fe0bde0 con 0x56104fd86b90 2024-09-19T04:30:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f76f4c00640 1 -- 172.21.15.107:0/1703194344 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f00a45e0 con 0x56104fd86b90 2024-09-19T04:30:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f76f4c00640 1 -- 172.21.15.107:0/1703194344 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f76f80247e0 con 0x56104fd86b90 2024-09-19T04:30:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f77101da740 10 client.27080 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f77101da740 10 client.27080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f77101da740 20 client.27080 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f77101da740 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56104fe0c4a0 0x56104fe0e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f77101da740 4 client.27080 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:30:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f77101da740 1 -- 172.21.15.107:0/1703194344 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56104fcf4e60 con 0x56104fe0c4a0 2024-09-19T04:30:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f7706000640 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56104fe0c4a0 0x56104fe0e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f7706000640 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56104fe0c4a0 0x56104fe0e890 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ac05bbf6aca86d server_cookie=71ba380baa83d2b9 in_seq=0 out_seq=0 2024-09-19T04:30:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.282+0000 7f76f4c00640 10 client.27080 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f76f4c00640 1 -- 172.21.15.107:0/1703194344 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56104fcf4e60 con 0x56104fe0c4a0 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f76f4c00640 10 client.27080 handle_command_reply: tid=0 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 client.27080 shutdown 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 -- 172.21.15.107:0/1703194344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 msgr2=0x56104fd99db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd99db0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f76f8005970 tx=0x7f76f800fa40 comp rx=0 tx=0).stop 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 -- 172.21.15.107:0/1703194344 shutdown_connections 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56104fe0c4a0 0x56104fe0e890 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56104fd91d70 0x56104fd8b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56104fd874b0 0x56104fd9a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f76f4c00640 0 client.27080 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:21.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 --2- 172.21.15.107:0/1703194344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56104fd86b90 0x56104fd99db0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 -- 172.21.15.107:0/1703194344 >> 172.21.15.107:0/1703194344 conn(0x56104fd83120 msgr2=0x56104fd8d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 -- 172.21.15.107:0/1703194344 shutdown_connections 2024-09-19T04:30:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.285+0000 7f77101da740 1 -- 172.21.15.107:0/1703194344 wait complete. 2024-09-19T04:30:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f77101da740 20 client.27080 trim_cache size 0 max 16384 2024-09-19T04:30:21.291 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:30:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:30:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:30:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "93ace02e-ce18-4836-9969-7cea8150688b": { 2024-09-19T04:30:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:30:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "93ace02e-ce18-4836-9969-7cea8150688b", 2024-09-19T04:30:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:30:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:30:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:30:21.292 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:30:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f770d800640 1 -- 172.21.15.107:0/3259421976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76ec0a33b0 msgr2=0x7f76ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:21.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f770d800640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76ec0a33b0 0x7f76ec0a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7700031550 tx=0x7f77000046c0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f770d800640 1 -- 172.21.15.107:0/3259421976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77081027a0 msgr2=0x7f770819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:21.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f770d800640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77081027a0 0x7f770819ab00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f76f000d770 tx=0x7f76f000dc30 comp rx=0 tx=0).stop 2024-09-19T04:30:21.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f770d800640 1 -- 172.21.15.107:0/3259421976 shutdown_connections 2024-09-19T04:30:21.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f770d800640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f76ec0a33b0 0x7f76ec0a5870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f770d800640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7708106f20 0x7f770819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f770d800640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77081027a0 0x7f770819ab00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f770d800640 1 --2- 172.21.15.107:0/3259421976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7708101df0 0x7f770819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:21.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.286+0000 7f770d800640 1 -- 172.21.15.107:0/3259421976 >> 172.21.15.107:0/3259421976 conn(0x7f77080fd8b0 msgr2=0x7f77080fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:21.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.287+0000 7f770d800640 1 -- 172.21.15.107:0/3259421976 shutdown_connections 2024-09-19T04:30:21.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:21.287+0000 7f770d800640 1 -- 172.21.15.107:0/3259421976 wait complete. 2024-09-19T04:30:21.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:21 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:30:21.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:21 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:21.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:21 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:30:21.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:21 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:21.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:21 smithi115 ceph-mon[35477]: pgmap v311: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-19T04:30:21.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:21 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:30:21.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:21 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:21.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:21 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:30:21.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:21 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:30:21.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:21 smithi145 ceph-mon[35096]: pgmap v311: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-19T04:30:21.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 331270 2.74 MB/sec execute 399 sec latency 14.131 ms 2024-09-19T04:30:22.258 INFO:tasks.cephfs.filesystem:scrub status for tag:93ace02e-ce18-4836-9969-7cea8150688b - {'path': '/', 'tag': '93ace02e-ce18-4836-9969-7cea8150688b', 'options': 'recursive,force'} 2024-09-19T04:30:22.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 331577 2.74 MB/sec execute 400 sec latency 772.087 ms 2024-09-19T04:30:23.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:23 smithi115 ceph-mon[35477]: pgmap v312: 97 pgs: 97 active+clean; 255 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-09-19T04:30:23.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:23 smithi145 ceph-mon[35096]: pgmap v312: 97 pgs: 97 active+clean; 255 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-09-19T04:30:23.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:23 smithi107 ceph-mon[30659]: pgmap v312: 97 pgs: 97 active+clean; 255 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-09-19T04:30:23.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 331577 2.73 MB/sec execute 401 sec latency 1772.214 ms 2024-09-19T04:30:24.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 332652 2.74 MB/sec execute 402 sec latency 2280.547 ms 2024-09-19T04:30:25.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:25 smithi107 ceph-mon[30659]: pgmap v313: 97 pgs: 97 active+clean; 255 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-19T04:30:25.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:25 smithi115 ceph-mon[35477]: pgmap v313: 97 pgs: 97 active+clean; 255 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-19T04:30:25.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:25 smithi145 ceph-mon[35096]: pgmap v313: 97 pgs: 97 active+clean; 255 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-19T04:30:25.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 333537 2.75 MB/sec execute 403 sec latency 487.879 ms 2024-09-19T04:30:26.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 334650 2.75 MB/sec execute 404 sec latency 11.525 ms 2024-09-19T04:30:27.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:27 smithi115 ceph-mon[35477]: pgmap v314: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-09-19T04:30:27.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:27 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:30:27.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:27 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:30:27.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:27 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:30:27.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:27 smithi145 ceph-mon[35096]: pgmap v314: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-09-19T04:30:27.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:27 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:30:27.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:27 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:30:27.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:27 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:30:27.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:27 smithi107 ceph-mon[30659]: pgmap v314: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-09-19T04:30:27.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:27 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:30:27.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:27 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:30:27.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:27 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:30:27.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 335120 2.75 MB/sec execute 405 sec latency 737.244 ms 2024-09-19T04:30:28.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:28 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:30:28.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:28 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:30:28.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:28 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:30:28.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:28 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:30:28.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:28 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:30:28.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:28 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:30:28.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 335120 2.74 MB/sec execute 406 sec latency 1737.361 ms 2024-09-19T04:30:29.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:29 smithi115 ceph-mon[35477]: pgmap v315: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T04:30:29.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:29 smithi145 ceph-mon[35096]: pgmap v315: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T04:30:29.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:29 smithi107 ceph-mon[30659]: pgmap v315: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T04:30:29.716 INFO:tasks.workunit.client.0.smithi107.stdout: 1 336187 2.75 MB/sec execute 407 sec latency 2242.693 ms 2024-09-19T04:30:30.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 337201 2.76 MB/sec execute 408 sec latency 256.166 ms 2024-09-19T04:30:31.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:31 smithi115 ceph-mon[35477]: pgmap v316: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T04:30:31.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:31 smithi145 ceph-mon[35096]: pgmap v316: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T04:30:31.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:31 smithi107 ceph-mon[30659]: pgmap v316: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-09-19T04:30:31.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 338362 2.77 MB/sec execute 409 sec latency 13.233 ms 2024-09-19T04:30:32.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 338670 2.76 MB/sec execute 410 sec latency 749.081 ms 2024-09-19T04:30:33.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:33 smithi115 ceph-mon[35477]: pgmap v317: 97 pgs: 97 active+clean; 264 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 183 op/s 2024-09-19T04:30:33.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:33 smithi145 ceph-mon[35096]: pgmap v317: 97 pgs: 97 active+clean; 264 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 183 op/s 2024-09-19T04:30:33.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:33 smithi107 ceph-mon[30659]: pgmap v317: 97 pgs: 97 active+clean; 264 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 183 op/s 2024-09-19T04:30:33.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 338670 2.76 MB/sec execute 411 sec latency 1749.199 ms 2024-09-19T04:30:34.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 340244 2.77 MB/sec execute 412 sec latency 1994.338 ms 2024-09-19T04:30:35.317 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:35 smithi107 ceph-mon[30659]: pgmap v318: 97 pgs: 97 active+clean; 264 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-09-19T04:30:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:35 smithi115 ceph-mon[35477]: pgmap v318: 97 pgs: 97 active+clean; 264 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-09-19T04:30:35.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:35 smithi145 ceph-mon[35096]: pgmap v318: 97 pgs: 97 active+clean; 264 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-09-19T04:30:35.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 340889 2.78 MB/sec execute 413 sec latency 448.254 ms 2024-09-19T04:30:36.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 342048 2.78 MB/sec execute 414 sec latency 63.668 ms 2024-09-19T04:30:37.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:37 smithi115 ceph-mon[35477]: pgmap v319: 97 pgs: 97 active+clean; 266 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-09-19T04:30:37.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:37 smithi145 ceph-mon[35096]: pgmap v319: 97 pgs: 97 active+clean; 266 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-09-19T04:30:37.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:37 smithi107 ceph-mon[30659]: pgmap v319: 97 pgs: 97 active+clean; 266 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-09-19T04:30:37.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 342210 2.78 MB/sec execute 415 sec latency 906.300 ms 2024-09-19T04:30:38.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 342210 2.77 MB/sec execute 416 sec latency 1906.427 ms 2024-09-19T04:30:39.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:39 smithi115 ceph-mon[35477]: pgmap v320: 97 pgs: 97 active+clean; 266 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-19T04:30:39.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:39 smithi145 ceph-mon[35096]: pgmap v320: 97 pgs: 97 active+clean; 266 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-19T04:30:39.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:39 smithi107 ceph-mon[30659]: pgmap v320: 97 pgs: 97 active+clean; 266 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-19T04:30:39.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 343115 2.77 MB/sec execute 417 sec latency 2414.229 ms 2024-09-19T04:30:40.337 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:40 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:30:40.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:40 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:30:40.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:40 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:30:40.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 344182 2.79 MB/sec execute 418 sec latency 458.709 ms 2024-09-19T04:30:41.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:41 smithi115 ceph-mon[35477]: pgmap v321: 97 pgs: 97 active+clean; 266 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-19T04:30:41.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:41 smithi145 ceph-mon[35096]: pgmap v321: 97 pgs: 97 active+clean; 266 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-19T04:30:41.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:41 smithi107 ceph-mon[30659]: pgmap v321: 97 pgs: 97 active+clean; 266 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-19T04:30:41.717 INFO:tasks.workunit.client.0.smithi107.stdout: 1 344735 2.78 MB/sec execute 419 sec latency 669.949 ms 2024-09-19T04:30:42.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: pgmap v322: 97 pgs: 97 active+clean; 269 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-19T04:30:42.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-19T04:30:42.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: Unable to set osd_memory_target on smithi145 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-19T04:30:42.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-19T04:30:42.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: Unable to set osd_memory_target on smithi115 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:30:42.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:42 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 345759 2.79 MB/sec execute 420 sec latency 338.673 ms 2024-09-19T04:30:42.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: pgmap v322: 97 pgs: 97 active+clean; 269 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-19T04:30:42.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: Unable to set osd_memory_target on smithi145 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-19T04:30:42.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: Unable to set osd_memory_target on smithi115 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:30:42.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:42 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: pgmap v322: 97 pgs: 97 active+clean; 269 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-19T04:30:42.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-19T04:30:42.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: Unable to set osd_memory_target on smithi145 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-19T04:30:42.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:42.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-19T04:30:42.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: Unable to set osd_memory_target on smithi115 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T04:30:42.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:42.732 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:42 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:43.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:43 smithi115 ceph-mon[35477]: Adjusting osd_memory_target on smithi107 to 176.4M 2024-09-19T04:30:43.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:43 smithi115 ceph-mon[35477]: Unable to set osd_memory_target on smithi107 to 184978124: error parsing value: Value '184978124' is below minimum 939524096 2024-09-19T04:30:43.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:30:43.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:30:43.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:43.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 345759 2.78 MB/sec execute 421 sec latency 1338.774 ms 2024-09-19T04:30:43.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:43 smithi145 ceph-mon[35096]: Adjusting osd_memory_target on smithi107 to 176.4M 2024-09-19T04:30:43.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:43 smithi145 ceph-mon[35096]: Unable to set osd_memory_target on smithi107 to 184978124: error parsing value: Value '184978124' is below minimum 939524096 2024-09-19T04:30:43.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:30:43.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:30:43.725 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:43.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:30:43.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:43 smithi107 ceph-mon[30659]: Adjusting osd_memory_target on smithi107 to 176.4M 2024-09-19T04:30:43.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:43 smithi107 ceph-mon[30659]: Unable to set osd_memory_target on smithi107 to 184978124: error parsing value: Value '184978124' is below minimum 939524096 2024-09-19T04:30:43.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:30:43.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:30:43.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:30:44.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 346674 2.78 MB/sec execute 422 sec latency 1847.104 ms 2024-09-19T04:30:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:44 smithi115 ceph-mon[35477]: pgmap v323: 97 pgs: 97 active+clean; 269 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-09-19T04:30:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:44 smithi145 ceph-mon[35096]: pgmap v323: 97 pgs: 97 active+clean; 269 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-09-19T04:30:45.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:44 smithi107 ceph-mon[30659]: pgmap v323: 97 pgs: 97 active+clean; 269 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-09-19T04:30:45.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 347845 2.80 MB/sec execute 423 sec latency 382.186 ms 2024-09-19T04:30:46.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 348590 2.80 MB/sec execute 424 sec latency 617.234 ms 2024-09-19T04:30:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:46 smithi115 ceph-mon[35477]: pgmap v324: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-09-19T04:30:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:46 smithi145 ceph-mon[35096]: pgmap v324: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-09-19T04:30:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:46 smithi107 ceph-mon[30659]: pgmap v324: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-09-19T04:30:47.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 349302 2.80 MB/sec execute 425 sec latency 578.106 ms 2024-09-19T04:30:48.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 349302 2.79 MB/sec execute 426 sec latency 1578.245 ms 2024-09-19T04:30:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:48 smithi115 ceph-mon[35477]: pgmap v325: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-19T04:30:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:48 smithi145 ceph-mon[35096]: pgmap v325: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-19T04:30:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:48 smithi107 ceph-mon[30659]: pgmap v325: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-19T04:30:49.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 350265 2.80 MB/sec execute 427 sec latency 2082.466 ms 2024-09-19T04:30:50.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 351526 2.81 MB/sec execute 428 sec latency 228.075 ms 2024-09-19T04:30:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:50 smithi115 ceph-mon[35477]: pgmap v326: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-19T04:30:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:50 smithi145 ceph-mon[35096]: pgmap v326: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-19T04:30:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:50 smithi107 ceph-mon[30659]: pgmap v326: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-19T04:30:51.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 352379 2.81 MB/sec execute 429 sec latency 572.115 ms 2024-09-19T04:30:52.259 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:30:52.627 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:52.718 INFO:tasks.workunit.client.0.smithi107.stdout: 1 352850 2.81 MB/sec execute 430 sec latency 737.424 ms 2024-09-19T04:30:53.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:52 smithi107 ceph-mon[30659]: pgmap v327: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-09-19T04:30:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:52 smithi115 ceph-mon[35477]: pgmap v327: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-09-19T04:30:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.214+0000 7f99c1200640 1 -- 172.21.15.107:0/1149976703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99bc077040 msgr2=0x7f99bc112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.214+0000 7f99c1200640 1 --2- 172.21.15.107:0/1149976703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99bc077040 0x7f99bc112330 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009900 tx=0x7f99a802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.215+0000 7f99c1200640 1 -- 172.21.15.107:0/1149976703 shutdown_connections 2024-09-19T04:30:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.215+0000 7f99c1200640 1 --2- 172.21.15.107:0/1149976703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99bc112870 0x7f99bc114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.215+0000 7f99c1200640 1 --2- 172.21.15.107:0/1149976703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99bc077040 0x7f99bc112330 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.215+0000 7f99c1200640 1 --2- 172.21.15.107:0/1149976703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99bc076720 0x7f99bc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.215+0000 7f99c1200640 1 -- 172.21.15.107:0/1149976703 >> 172.21.15.107:0/1149976703 conn(0x7f99bc0ff8d0 msgr2=0x7f99bc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.215+0000 7f99c1200640 1 -- 172.21.15.107:0/1149976703 shutdown_connections 2024-09-19T04:30:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.215+0000 7f99c1200640 1 -- 172.21.15.107:0/1149976703 wait complete. 2024-09-19T04:30:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99c1200640 1 Processor -- start 2024-09-19T04:30:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99c1200640 1 -- start start 2024-09-19T04:30:53.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99c1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99bc076720 0x7f99bc1a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99c1200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99bc077040 0x7f99bc1a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99c1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99bc112870 0x7f99bc19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99c1200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc118430 con 0x7f99bc112870 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99c1200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc1182b0 con 0x7f99bc077040 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99c1200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc1185b0 con 0x7f99bc076720 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99bc112870 0x7f99bc19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99baa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99bc077040 0x7f99bc1a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99bc112870 0x7f99bc19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36234/0 (socket says 172.21.15.107:36234) 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99baa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99bc077040 0x7f99bc1a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44116/0 (socket says 172.21.15.107:44116) 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99bbe00640 1 -- 172.21.15.107:0/3025955700 learned_addr learned my addr 172.21.15.107:0/3025955700 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.216+0000 7f99bb400640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99bc076720 0x7f99bc1a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.217+0000 7f99baa00640 1 -- 172.21.15.107:0/3025955700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99bc076720 msgr2=0x7f99bc1a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.217+0000 7f99baa00640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99bc076720 0x7f99bc1a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.217+0000 7f99baa00640 1 -- 172.21.15.107:0/3025955700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99bc112870 msgr2=0x7f99bc19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.217+0000 7f99baa00640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99bc112870 0x7f99bc19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.217+0000 7f99baa00640 1 -- 172.21.15.107:0/3025955700 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc19ef90 con 0x7f99bc077040 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.217+0000 7f99bbe00640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99bc112870 0x7f99bc19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.217+0000 7f99baa00640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99bc077040 0x7f99bc1a4b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f99a802a910 tx=0x7f99a8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.217+0000 7f99b3e00640 1 -- 172.21.15.107:0/3025955700 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a803f070 con 0x7f99bc077040 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.217+0000 7f99c1200640 1 -- 172.21.15.107:0/3025955700 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99bc19f220 con 0x7f99bc077040 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.218+0000 7f99b3e00640 1 -- 172.21.15.107:0/3025955700 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a8031c90 con 0x7f99bc077040 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.218+0000 7f99b3e00640 1 -- 172.21.15.107:0/3025955700 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8035030 con 0x7f99bc077040 2024-09-19T04:30:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.218+0000 7f99c1200640 1 -- 172.21.15.107:0/3025955700 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99bc1ab400 con 0x7f99bc077040 2024-09-19T04:30:53.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.221+0000 7f99b3e00640 1 -- 172.21.15.107:0/3025955700 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f99a8042070 con 0x7f99bc077040 2024-09-19T04:30:53.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.221+0000 7f99c53f3740 1 Processor -- start 2024-09-19T04:30:53.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.221+0000 7f99c53f3740 1 -- start start 2024-09-19T04:30:53.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.221+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 0x556af0a28130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.221+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a28670 0x556af0a329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.221+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 0x556af0a35320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.221+0000 7f99c53f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556af098e3e0 con 0x556af0a32f30 2024-09-19T04:30:53.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.221+0000 7f99c53f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556af0935d80 con 0x556af0a27d50 2024-09-19T04:30:53.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99c53f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556af098a3a0 con 0x556af0a28670 2024-09-19T04:30:53.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99bbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 0x556af0a28130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 0x556af0a35320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99bb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a28670 0x556af0a329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99bb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a28670 0x556af0a329f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35520/0 (socket says 172.21.15.107:35520) 2024-09-19T04:30:53.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99bbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 0x556af0a28130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44118/0 (socket says 172.21.15.107:44118) 2024-09-19T04:30:53.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99bb400640 1 -- 172.21.15.107:0/2598591530 learned_addr learned my addr 172.21.15.107:0/2598591530 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:53.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:52 smithi145 ceph-mon[35096]: pgmap v327: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-09-19T04:30:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99bbe00640 1 -- 172.21.15.107:0/2598591530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a28670 msgr2=0x556af0a329f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99bbe00640 1 --2- 172.21.15.107:0/2598591530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a28670 0x556af0a329f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99bbe00640 1 -- 172.21.15.107:0/2598591530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 msgr2=0x556af0a35320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99bbe00640 1 --2- 172.21.15.107:0/2598591530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 0x556af0a35320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.222+0000 7f99bbe00640 1 -- 172.21.15.107:0/2598591530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556af0954ae0 con 0x556af0a27d50 2024-09-19T04:30:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.223+0000 7f99bbe00640 1 --2- 172.21.15.107:0/2598591530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 0x556af0a28130 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f99b4011760 tx=0x7f99b4011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8824d20c322332a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.223+0000 7f99b3e00640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f99a00a33b0 0x7f99a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.223+0000 7f99b0c00640 1 -- 172.21.15.107:0/2598591530 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b4004d20 con 0x556af0a27d50 2024-09-19T04:30:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.223+0000 7f99b3e00640 1 -- 172.21.15.107:0/3025955700 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f99a80efff0 con 0x7f99bc077040 2024-09-19T04:30:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.223+0000 7f99b0c00640 1 -- 172.21.15.107:0/2598591530 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b4005020 con 0x556af0a27d50 2024-09-19T04:30:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.223+0000 7f99bb400640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f99a00a33b0 0x7f99a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.223+0000 7f99b0c00640 1 -- 172.21.15.107:0/2598591530 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b4019690 con 0x556af0a27d50 2024-09-19T04:30:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99c53f3740 1 -- 172.21.15.107:0/2598591530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 msgr2=0x556af0a28130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99bb400640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f99a00a33b0 0x7f99a00a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x556af0a25140 tx=0x7f99ac00b410 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99c53f3740 1 --2- 172.21.15.107:0/2598591530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 0x556af0a28130 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f99b4011760 tx=0x7f99b4011c20 comp rx=0 tx=0).stop 2024-09-19T04:30:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99c53f3740 1 -- 172.21.15.107:0/2598591530 shutdown_connections 2024-09-19T04:30:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99c53f3740 1 --2- 172.21.15.107:0/2598591530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 0x556af0a35320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99c53f3740 1 --2- 172.21.15.107:0/2598591530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a28670 0x556af0a329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99c53f3740 1 --2- 172.21.15.107:0/2598591530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 0x556af0a28130 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99c53f3740 1 -- 172.21.15.107:0/2598591530 >> 172.21.15.107:0/2598591530 conn(0x556af0a242e0 msgr2=0x556af0a246c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99c53f3740 1 -- 172.21.15.107:0/2598591530 shutdown_connections 2024-09-19T04:30:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99c53f3740 1 -- 172.21.15.107:0/2598591530 wait complete. 2024-09-19T04:30:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.224+0000 7f99c53f3740 1 -- start start 2024-09-19T04:30:53.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.225+0000 7f99c53f3740 1 Processor -- start 2024-09-19T04:30:53.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.225+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 0x556af0a2fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.225+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a28670 0x556af0a302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.225+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a32f30 0x556af0a307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.225+0000 7f99c53f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556af0a35ba0 con 0x556af0a28670 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.225+0000 7f99c53f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556af09490f0 con 0x556af0a27d50 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.225+0000 7f99c53f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556af0a35d20 con 0x556af0a32f30 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.225+0000 7f99c53f3740 10 client.0 _ll_register_callbacks cb 0x556af093a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.225+0000 7f99baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a28670 0x556af0a302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a28670 0x556af0a302a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36260/0 (socket says 172.21.15.107:36260) 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99baa00640 1 -- 172.21.15.107:0/3699859060 learned_addr learned my addr 172.21.15.107:0/3699859060 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99bb400640 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 0x556af0a2fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99bbe00640 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a32f30 0x556af0a307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99baa00640 1 -- 172.21.15.107:0/3699859060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a32f30 msgr2=0x556af0a307e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99baa00640 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a32f30 0x556af0a307e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99baa00640 1 -- 172.21.15.107:0/3699859060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 msgr2=0x556af0a2fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99baa00640 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 0x556af0a2fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99baa00640 1 -- 172.21.15.107:0/3699859060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af06ff3c0 con 0x556af0a28670 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99baa00640 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a28670 0x556af0a302a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f99a8004650 tx=0x7f99a8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944fb3bf280715e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99a5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99a5600640 1 -- 172.21.15.107:0/3699859060 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8049070 con 0x556af0a28670 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99a5600640 1 -- 172.21.15.107:0/3699859060 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f99a80ed9e0 con 0x556af0a28670 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.226+0000 7f99c53f3740 1 -- 172.21.15.107:0/3699859060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b4005020 con 0x556af0a28670 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99a5600640 1 -- 172.21.15.107:0/3699859060 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f99a80b8640 con 0x556af0a28670 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99c53f3740 10 client.17043 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99c53f3740 1 -- 172.21.15.107:0/3699859060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556af0a417a0 con 0x556af0a28670 2024-09-19T04:30:53.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99a5600640 1 -- 172.21.15.107:0/3699859060 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b4005020 con 0x556af0a28670 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99a5600640 1 -- 172.21.15.107:0/3699859060 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f99a80b8a10 con 0x556af0a28670 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99c53f3740 10 client.17043 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99c53f3740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99c53f3740 20 client.17043 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99c53f3740 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556af0a42b90 0x556af0a44fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99c53f3740 4 client.17043 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.227+0000 7f99c53f3740 1 -- 172.21.15.107:0/3699859060 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556af099b950 con 0x556af0a42b90 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.228+0000 7f99bb400640 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556af0a42b90 0x556af0a44fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.228+0000 7f99bb400640 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556af0a42b90 0x556af0a44fe0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0dc797c31e8f892 server_cookie=2c73c7b718ab10cb in_seq=0 out_seq=0 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.228+0000 7f99a5600640 10 client.17043 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.229+0000 7f99a5600640 1 -- 172.21.15.107:0/3699859060 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556af099b950 con 0x556af0a42b90 2024-09-19T04:30:53.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.229+0000 7f99a5600640 10 client.17043 handle_command_reply: tid=0 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.229+0000 7f99c53f3740 1 client.17043 shutdown 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.229+0000 7f99c53f3740 1 -- 172.21.15.107:0/3699859060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a28670 msgr2=0x556af0a302a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.229+0000 7f99c53f3740 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a28670 0x556af0a302a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f99a8004650 tx=0x7f99a8003f90 comp rx=0 tx=0).stop 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.230+0000 7f99c53f3740 1 -- 172.21.15.107:0/3699859060 shutdown_connections 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.230+0000 7f99c53f3740 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556af0a42b90 0x556af0a44fe0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.230+0000 7f99c53f3740 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a32f30 0x556af0a307e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.230+0000 7f99c53f3740 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a28670 0x556af0a302a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.230+0000 7f99c53f3740 1 --2- 172.21.15.107:0/3699859060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a27d50 0x556af0a2fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.230+0000 7f99c53f3740 1 -- 172.21.15.107:0/3699859060 >> 172.21.15.107:0/3699859060 conn(0x556af0a242e0 msgr2=0x556af0a29fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.230+0000 7f99a5600640 0 client.17043 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.230+0000 7f99c53f3740 1 -- 172.21.15.107:0/3699859060 shutdown_connections 2024-09-19T04:30:53.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.230+0000 7f99c53f3740 1 -- 172.21.15.107:0/3699859060 wait complete. 2024-09-19T04:30:53.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.230+0000 7f99c53f3740 20 client.17043 trim_cache size 0 max 16384 2024-09-19T04:30:53.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.247+0000 7f99c53f3740 1 Processor -- start 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.247+0000 7f99c53f3740 1 -- start start 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a27d50 0x556af0a2a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a2e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a32f30 0x556af0a30430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99c53f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc1b3c80 con 0x556af0a27d50 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99c53f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc1b57c0 con 0x556af0a28670 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99c53f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc1b4350 con 0x556af0a32f30 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a27d50 0x556af0a2a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a27d50 0x556af0a2a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36274/0 (socket says 172.21.15.107:36274) 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99bb400640 1 -- 172.21.15.107:0/1757233302 learned_addr learned my addr 172.21.15.107:0/1757233302 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99bbe00640 1 --2- 172.21.15.107:0/1757233302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a32f30 0x556af0a30430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99baa00640 1 --2- 172.21.15.107:0/1757233302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a2e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99baa00640 1 -- 172.21.15.107:0/1757233302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a32f30 msgr2=0x556af0a30430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99baa00640 1 --2- 172.21.15.107:0/1757233302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a32f30 0x556af0a30430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99baa00640 1 -- 172.21.15.107:0/1757233302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a27d50 msgr2=0x556af0a2a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99baa00640 1 --2- 172.21.15.107:0/1757233302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a27d50 0x556af0a2a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99baa00640 1 -- 172.21.15.107:0/1757233302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556af0a45660 con 0x556af0a28670 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.248+0000 7f99bb400640 1 --2- 172.21.15.107:0/1757233302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a27d50 0x556af0a2a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.249+0000 7f99baa00640 1 --2- 172.21.15.107:0/1757233302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a2e650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f99a80ed810 tx=0x7f99a8005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da60717bf9f6b255 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.249+0000 7f99a7e00640 1 -- 172.21.15.107:0/1757233302 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8049070 con 0x556af0a28670 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.249+0000 7f99a7e00640 1 -- 172.21.15.107:0/1757233302 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a80bc3a0 con 0x556af0a28670 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.249+0000 7f99a7e00640 1 -- 172.21.15.107:0/1757233302 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a80bbd00 con 0x556af0a28670 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 -- 172.21.15.107:0/1757233302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 msgr2=0x556af0a2e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 --2- 172.21.15.107:0/1757233302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a2e650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f99a80ed810 tx=0x7f99a8005e50 comp rx=0 tx=0).stop 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 -- 172.21.15.107:0/1757233302 shutdown_connections 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 --2- 172.21.15.107:0/1757233302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a32f30 0x556af0a30430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 --2- 172.21.15.107:0/1757233302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a2e650 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 --2- 172.21.15.107:0/1757233302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a27d50 0x556af0a2a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 -- 172.21.15.107:0/1757233302 >> 172.21.15.107:0/1757233302 conn(0x556af0a242e0 msgr2=0x556af0a2fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 -- 172.21.15.107:0/1757233302 shutdown_connections 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 -- 172.21.15.107:0/1757233302 wait complete. 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 -- start start 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.250+0000 7f99c53f3740 1 Processor -- start 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a27d50 0x556af0a3af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a3b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99c53f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 0x556af0a37f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99c53f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc1b50f0 con 0x556af0a32f30 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99c53f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc1b4a20 con 0x556af0a28670 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99c53f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc1b1c80 con 0x556af0a27d50 2024-09-19T04:30:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99c53f3740 10 client.0 _ll_register_callbacks cb 0x7f99a80f3830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:53.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 0x556af0a37f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99baa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a3b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99bb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a27d50 0x556af0a3af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99baa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a3b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44164/0 (socket says 172.21.15.107:44164) 2024-09-19T04:30:53.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.251+0000 7f99baa00640 1 -- 172.21.15.107:0/1661076262 learned_addr learned my addr 172.21.15.107:0/1661076262 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:53.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99bb400640 1 -- 172.21.15.107:0/1661076262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 msgr2=0x556af0a3b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99bb400640 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a3b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99bb400640 1 -- 172.21.15.107:0/1661076262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 msgr2=0x556af0a37f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:53.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99bb400640 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 0x556af0a37f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99bb400640 1 -- 172.21.15.107:0/1661076262 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af0a25390 con 0x556af0a27d50 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99baa00640 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a3b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99bbe00640 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 0x556af0a37f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99bb400640 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a27d50 0x556af0a3af70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f99ac01e5d0 tx=0x7f99ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d566d10b4cca041 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99a5600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99a5600640 1 -- 172.21.15.107:0/1661076262 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac01ed80 con 0x556af0a27d50 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99a5600640 1 -- 172.21.15.107:0/1661076262 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f99ac004020 con 0x556af0a27d50 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.252+0000 7f99c53f3740 1 -- 172.21.15.107:0/1661076262 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a80bc3a0 con 0x556af0a27d50 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99a5600640 1 -- 172.21.15.107:0/1661076262 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f99ac004220 con 0x556af0a27d50 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99c53f3740 10 client.27061 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99c53f3740 1 -- 172.21.15.107:0/1661076262 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556af0aadce0 con 0x556af0a27d50 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99a5600640 1 -- 172.21.15.107:0/1661076262 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a80bc3a0 con 0x556af0a27d50 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99a5600640 1 -- 172.21.15.107:0/1661076262 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f99ac039070 con 0x556af0a27d50 2024-09-19T04:30:53.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99c53f3740 10 client.27061 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99c53f3740 10 client.27061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99c53f3740 20 client.27061 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99c53f3740 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556af0aae460 0x556af0ab0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99c53f3740 4 client.27061 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.253+0000 7f99c53f3740 1 -- 172.21.15.107:0/1661076262 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556af0995e60 con 0x556af0aae460 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.254+0000 7f99baa00640 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556af0aae460 0x556af0ab0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.254+0000 7f99baa00640 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556af0aae460 0x556af0ab0880 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a89a43b3ad458fc server_cookie=5e728a4de9e34c06 in_seq=0 out_seq=0 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.254+0000 7f99a5600640 10 client.27061 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.254+0000 7f99a5600640 1 -- 172.21.15.107:0/1661076262 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556af0995e60 con 0x556af0aae460 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.254+0000 7f99a5600640 10 client.27061 handle_command_reply: tid=0 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.254+0000 7f99c53f3740 1 client.27061 shutdown 2024-09-19T04:30:53.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.254+0000 7f99c53f3740 1 -- 172.21.15.107:0/1661076262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a27d50 msgr2=0x556af0a3af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.254+0000 7f99c53f3740 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a27d50 0x556af0a3af70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f99ac01e5d0 tx=0x7f99ac0046c0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c53f3740 1 -- 172.21.15.107:0/1661076262 shutdown_connections 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c53f3740 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556af0aae460 0x556af0ab0880 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c53f3740 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556af0a32f30 0x556af0a37f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99a5600640 0 client.27061 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c53f3740 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556af0a28670 0x556af0a3b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c53f3740 1 --2- 172.21.15.107:0/1661076262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556af0a27d50 0x556af0a3af70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c53f3740 1 -- 172.21.15.107:0/1661076262 >> 172.21.15.107:0/1661076262 conn(0x556af0a242e0 msgr2=0x556af0a23380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c53f3740 1 -- 172.21.15.107:0/1661076262 shutdown_connections 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c53f3740 1 -- 172.21.15.107:0/1661076262 wait complete. 2024-09-19T04:30:53.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c53f3740 20 client.27061 trim_cache size 0 max 16384 2024-09-19T04:30:53.261 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:30:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:30:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:30:53.261 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c1200640 1 -- 172.21.15.107:0/3025955700 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f99a00a33b0 msgr2=0x7f99a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c1200640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f99a00a33b0 0x7f99a00a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x556af0a25140 tx=0x7f99ac00b410 comp rx=0 tx=0).stop 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c1200640 1 -- 172.21.15.107:0/3025955700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99bc077040 msgr2=0x7f99bc1a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.255+0000 7f99c1200640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99bc077040 0x7f99bc1a4b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f99a802a910 tx=0x7f99a8003b90 comp rx=0 tx=0).stop 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.256+0000 7f99c1200640 1 -- 172.21.15.107:0/3025955700 shutdown_connections 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.256+0000 7f99c1200640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f99a00a33b0 0x7f99a00a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.256+0000 7f99c1200640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99bc112870 0x7f99bc19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.256+0000 7f99c1200640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99bc077040 0x7f99bc1a4b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.256+0000 7f99c1200640 1 --2- 172.21.15.107:0/3025955700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99bc076720 0x7f99bc1a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.256+0000 7f99c1200640 1 -- 172.21.15.107:0/3025955700 >> 172.21.15.107:0/3025955700 conn(0x7f99bc0ff8d0 msgr2=0x7f99bc101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:53.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.256+0000 7f99c1200640 1 -- 172.21.15.107:0/3025955700 shutdown_connections 2024-09-19T04:30:53.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:53.256+0000 7f99c1200640 1 -- 172.21.15.107:0/3025955700 wait complete. 2024-09-19T04:30:53.719 INFO:tasks.workunit.client.0.smithi107.stdout: 1 352850 2.81 MB/sec execute 431 sec latency 1737.554 ms 2024-09-19T04:30:54.254 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:30:54.254 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:30:54.659 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:54.719 INFO:tasks.workunit.client.0.smithi107.stdout: 1 354506 2.82 MB/sec execute 432 sec latency 1980.173 ms 2024-09-19T04:30:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:54 smithi115 ceph-mon[35477]: pgmap v328: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-09-19T04:30:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:54 smithi145 ceph-mon[35096]: pgmap v328: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-09-19T04:30:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:54 smithi107 ceph-mon[30659]: pgmap v328: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-09-19T04:30:55.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.364+0000 7f0d91e00640 1 -- 172.21.15.107:0/1821848112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d8c1079c0 msgr2=0x7f0d8c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.364+0000 7f0d91e00640 1 --2- 172.21.15.107:0/1821848112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d8c1079c0 0x7f0d8c107da0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009900 tx=0x7f0d8002a410 comp rx=0 tx=0).stop 2024-09-19T04:30:55.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.364+0000 7f0d91e00640 1 -- 172.21.15.107:0/1821848112 shutdown_connections 2024-09-19T04:30:55.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.364+0000 7f0d91e00640 1 --2- 172.21.15.107:0/1821848112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d8c102360 0x7f0d8c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.364+0000 7f0d91e00640 1 --2- 172.21.15.107:0/1821848112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d8c1019c0 0x7f0d8c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.364+0000 7f0d91e00640 1 --2- 172.21.15.107:0/1821848112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d8c1079c0 0x7f0d8c107da0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.364+0000 7f0d91e00640 1 -- 172.21.15.107:0/1821848112 >> 172.21.15.107:0/1821848112 conn(0x7f0d8c0fd740 msgr2=0x7f0d8c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:55.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.365+0000 7f0d91e00640 1 -- 172.21.15.107:0/1821848112 shutdown_connections 2024-09-19T04:30:55.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.365+0000 7f0d91e00640 1 -- 172.21.15.107:0/1821848112 wait complete. 2024-09-19T04:30:55.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.365+0000 7f0d91e00640 1 Processor -- start 2024-09-19T04:30:55.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d91e00640 1 -- start start 2024-09-19T04:30:55.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d8c1019c0 0x7f0d8c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d8c102360 0x7f0d8c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d8c1079c0 0x7f0d8c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d91e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c111e10 con 0x7f0d8c102360 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d91e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c111c90 con 0x7f0d8c1019c0 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d91e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c111f90 con 0x7f0d8c1079c0 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d90a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d8c1019c0 0x7f0d8c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d90a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d8c1019c0 0x7f0d8c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44184/0 (socket says 172.21.15.107:44184) 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d90a00640 1 -- 172.21.15.107:0/2843468384 learned_addr learned my addr 172.21.15.107:0/2843468384 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d91400640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d8c1079c0 0x7f0d8c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.366+0000 7f0d8be00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d8c102360 0x7f0d8c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.367+0000 7f0d90a00640 1 -- 172.21.15.107:0/2843468384 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d8c1079c0 msgr2=0x7f0d8c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.367+0000 7f0d90a00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d8c1079c0 0x7f0d8c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.367+0000 7f0d90a00640 1 -- 172.21.15.107:0/2843468384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d8c102360 msgr2=0x7f0d8c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.367+0000 7f0d90a00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d8c102360 0x7f0d8c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.367+0000 7f0d90a00640 1 -- 172.21.15.107:0/2843468384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c1a6790 con 0x7f0d8c1019c0 2024-09-19T04:30:55.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.367+0000 7f0d90a00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d8c1019c0 0x7f0d8c0744f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0d8002aa00 tx=0x7f0d80003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:55.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.367+0000 7f0d89600640 1 -- 172.21.15.107:0/2843468384 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8003f070 con 0x7f0d8c1019c0 2024-09-19T04:30:55.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.367+0000 7f0d89600640 1 -- 172.21.15.107:0/2843468384 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d80031c30 con 0x7f0d8c1019c0 2024-09-19T04:30:55.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.367+0000 7f0d89600640 1 -- 172.21.15.107:0/2843468384 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80035030 con 0x7f0d8c1019c0 2024-09-19T04:30:55.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.367+0000 7f0d91e00640 1 -- 172.21.15.107:0/2843468384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8c1a6a20 con 0x7f0d8c1019c0 2024-09-19T04:30:55.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.368+0000 7f0d91e00640 1 -- 172.21.15.107:0/2843468384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8c1a6eb0 con 0x7f0d8c1019c0 2024-09-19T04:30:55.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.369+0000 7f0d91e00640 1 -- 172.21.15.107:0/2843468384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8c101e20 con 0x7f0d8c1019c0 2024-09-19T04:30:55.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.370+0000 7f0d89600640 1 -- 172.21.15.107:0/2843468384 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f0d80042070 con 0x7f0d8c1019c0 2024-09-19T04:30:55.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.372+0000 7f0d89600640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d740a3430 0x7f0d740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:55.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.372+0000 7f0d89600640 1 -- 172.21.15.107:0/2843468384 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f0d800effe0 con 0x7f0d8c1019c0 2024-09-19T04:30:55.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.372+0000 7f0d8be00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d740a3430 0x7f0d740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:55.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.372+0000 7f0d8be00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d740a3430 0x7f0d740a58f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c075f80 tx=0x7f0d78005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:55.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.376+0000 7f0d89600640 1 -- 172.21.15.107:0/2843468384 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0d800bc9a0 con 0x7f0d8c1019c0 2024-09-19T04:30:55.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.626+0000 7f0d91e00640 1 -- 172.21.15.107:0/2843468384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d8c075a20 con 0x7f0d8c1019c0 2024-09-19T04:30:55.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.634+0000 7f0d89600640 1 -- 172.21.15.107:0/2843468384 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11992 (secure 0 0 0) 0x7f0d8c075a20 con 0x7f0d8c1019c0 2024-09-19T04:30:55.643 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:30:55.644 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":48,"btime":"2024-09-19T04:29:19:894497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:29:18.944591+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24490,24496,24500,24539,24517]},"id":1}]} 2024-09-19T04:30:55.644 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 48 2024-09-19T04:30:55.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 -- 172.21.15.107:0/2843468384 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d740a3430 msgr2=0x7f0d740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d740a3430 0x7f0d740a58f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c075f80 tx=0x7f0d78005f90 comp rx=0 tx=0).stop 2024-09-19T04:30:55.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 -- 172.21.15.107:0/2843468384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d8c1019c0 msgr2=0x7f0d8c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:55.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d8c1019c0 0x7f0d8c0744f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0d8002aa00 tx=0x7f0d80003c10 comp rx=0 tx=0).stop 2024-09-19T04:30:55.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 -- 172.21.15.107:0/2843468384 shutdown_connections 2024-09-19T04:30:55.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0d740a3430 0x7f0d740a58f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d8c1079c0 0x7f0d8c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d8c102360 0x7f0d8c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 --2- 172.21.15.107:0/2843468384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d8c1019c0 0x7f0d8c0744f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:55.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 -- 172.21.15.107:0/2843468384 >> 172.21.15.107:0/2843468384 conn(0x7f0d8c0fd740 msgr2=0x7f0d8c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:55.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 -- 172.21.15.107:0/2843468384 shutdown_connections 2024-09-19T04:30:55.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:55.638+0000 7f0d91e00640 1 -- 172.21.15.107:0/2843468384 wait complete. 2024-09-19T04:30:55.719 INFO:tasks.workunit.client.0.smithi107.stdout: 1 355435 2.83 MB/sec execute 433 sec latency 117.314 ms 2024-09-19T04:30:56.133 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:55 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2843468384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:30:56.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:55 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2843468384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:30:56.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:55 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2843468384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:30:56.719 INFO:tasks.workunit.client.0.smithi107.stdout: 1 356392 2.83 MB/sec execute 434 sec latency 294.541 ms 2024-09-19T04:30:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:56 smithi115 ceph-mon[35477]: pgmap v329: 97 pgs: 97 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-09-19T04:30:57.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:56 smithi107 ceph-mon[30659]: pgmap v329: 97 pgs: 97 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-09-19T04:30:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:56 smithi145 ceph-mon[35096]: pgmap v329: 97 pgs: 97 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-09-19T04:30:57.250 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:30:57.617 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:57.719 INFO:tasks.workunit.client.0.smithi107.stdout: 1 356392 2.82 MB/sec execute 435 sec latency 1054.370 ms 2024-09-19T04:30:58.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.171+0000 7f8adbe00640 1 -- 172.21.15.107:0/462493516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8adc108f60 msgr2=0x7f8adc110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:58.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.171+0000 7f8adbe00640 1 --2- 172.21.15.107:0/462493516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8adc108f60 0x7f8adc110ae0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009ca0 tx=0x7f8ad402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.171+0000 7f8adbe00640 1 -- 172.21.15.107:0/462493516 shutdown_connections 2024-09-19T04:30:58.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.171+0000 7f8adbe00640 1 --2- 172.21.15.107:0/462493516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8adc108f60 0x7f8adc110ae0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.171+0000 7f8adbe00640 1 --2- 172.21.15.107:0/462493516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8adc1047e0 0x7f8adc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.171+0000 7f8adbe00640 1 --2- 172.21.15.107:0/462493516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8adc103e30 0x7f8adc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.171+0000 7f8adbe00640 1 -- 172.21.15.107:0/462493516 >> 172.21.15.107:0/462493516 conn(0x7f8adc0ff8d0 msgr2=0x7f8adc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:58.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.171+0000 7f8adbe00640 1 -- 172.21.15.107:0/462493516 shutdown_connections 2024-09-19T04:30:58.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.171+0000 7f8adbe00640 1 -- 172.21.15.107:0/462493516 wait complete. 2024-09-19T04:30:58.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.172+0000 7f8adbe00640 1 Processor -- start 2024-09-19T04:30:58.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.172+0000 7f8adbe00640 1 -- start start 2024-09-19T04:30:58.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.172+0000 7f8adbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8adc103e30 0x7f8adc076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.172+0000 7f8adbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8adc1047e0 0x7f8adc076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.172+0000 7f8adbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8adc108f60 0x7f8adc077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.172+0000 7f8adbe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc113fc0 con 0x7f8adc108f60 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.172+0000 7f8adbe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc113e40 con 0x7f8adc103e30 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.172+0000 7f8adbe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc114140 con 0x7f8adc1047e0 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8adc108f60 0x7f8adc077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8ada000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8adc1047e0 0x7f8adc076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8adc103e30 0x7f8adc076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8adc108f60 0x7f8adc077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36318/0 (socket says 172.21.15.107:36318) 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adb400640 1 -- 172.21.15.107:0/2989870935 learned_addr learned my addr 172.21.15.107:0/2989870935 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adaa00640 1 -- 172.21.15.107:0/2989870935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8adc1047e0 msgr2=0x7f8adc076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adaa00640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8adc1047e0 0x7f8adc076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adaa00640 1 -- 172.21.15.107:0/2989870935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8adc108f60 msgr2=0x7f8adc077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:58.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adaa00640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8adc108f60 0x7f8adc077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adaa00640 1 -- 172.21.15.107:0/2989870935 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc1aaac0 con 0x7f8adc103e30 2024-09-19T04:30:58.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adb400640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8adc108f60 0x7f8adc077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:58.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.173+0000 7f8adaa00640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8adc103e30 0x7f8adc076700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00ea50 tx=0x7f8acc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:58.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.174+0000 7f8ad3400640 1 -- 172.21.15.107:0/2989870935 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00cd00 con 0x7f8adc103e30 2024-09-19T04:30:58.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.174+0000 7f8adbe00640 1 -- 172.21.15.107:0/2989870935 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8adc1aadb0 con 0x7f8adc103e30 2024-09-19T04:30:58.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.174+0000 7f8ad3400640 1 -- 172.21.15.107:0/2989870935 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8acc010910 con 0x7f8adc103e30 2024-09-19T04:30:58.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.174+0000 7f8ad3400640 1 -- 172.21.15.107:0/2989870935 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc018920 con 0x7f8adc103e30 2024-09-19T04:30:58.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.174+0000 7f8adbe00640 1 -- 172.21.15.107:0/2989870935 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8adc1ab2c0 con 0x7f8adc103e30 2024-09-19T04:30:58.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.177+0000 7f8ae4274740 1 Processor -- start 2024-09-19T04:30:58.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.177+0000 7f8ae4274740 1 -- start start 2024-09-19T04:30:58.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.177+0000 7f8ad3400640 1 -- 172.21.15.107:0/2989870935 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f8acc010ac0 con 0x7f8adc103e30 2024-09-19T04:30:58.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.177+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa699220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.177+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa699760 0x5559aa6a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.177+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 0x5559aa6a6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.177+0000 7f8ae4274740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5559aa5ff3e0 con 0x5559aa699760 2024-09-19T04:30:58.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.177+0000 7f8ae4274740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5559aa5a6d80 con 0x5559aa698e40 2024-09-19T04:30:58.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.177+0000 7f8ae4274740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5559aa5fb3a0 con 0x5559aa6a4020 2024-09-19T04:30:58.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8adaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa699760 0x5559aa6a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8adb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa699220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8adaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa699760 0x5559aa6a3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36324/0 (socket says 172.21.15.107:36324) 2024-09-19T04:30:58.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8adaa00640 1 -- 172.21.15.107:0/4261164988 learned_addr learned my addr 172.21.15.107:0/4261164988 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:58.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8ada000640 1 --2- 172.21.15.107:0/4261164988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 0x5559aa6a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8ada000640 1 -- 172.21.15.107:0/4261164988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 msgr2=0x5559aa699220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:58.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8ada000640 1 --2- 172.21.15.107:0/4261164988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa699220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8ada000640 1 -- 172.21.15.107:0/4261164988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa699760 msgr2=0x5559aa6a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:58.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8ada000640 1 --2- 172.21.15.107:0/4261164988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa699760 0x5559aa6a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8ada000640 1 -- 172.21.15.107:0/4261164988 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559aa5c5ae0 con 0x5559aa6a4020 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8adaa00640 1 --2- 172.21.15.107:0/4261164988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa699760 0x5559aa6a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.178+0000 7f8ada000640 1 --2- 172.21.15.107:0/4261164988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 0x5559aa6a6410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8adc077f50 tx=0x7f8ac8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be3c0e5e38b95324 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.179+0000 7f8ac7400640 1 -- 172.21.15.107:0/4261164988 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8004050 con 0x5559aa6a4020 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.179+0000 7f8ac7400640 1 -- 172.21.15.107:0/4261164988 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8005500 con 0x5559aa6a4020 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.179+0000 7f8ac7400640 1 -- 172.21.15.107:0/4261164988 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8004d90 con 0x5559aa6a4020 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.179+0000 7f8ad3400640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ac00a3430 0x7f8ac00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.179+0000 7f8ad3400640 1 -- 172.21.15.107:0/2989870935 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f8acc0cfff0 con 0x7f8adc103e30 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.179+0000 7f8adb400640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ac00a3430 0x7f8ac00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.179+0000 7f8ae4274740 1 -- 172.21.15.107:0/4261164988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 msgr2=0x5559aa6a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.179+0000 7f8ae4274740 1 --2- 172.21.15.107:0/4261164988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 0x5559aa6a6410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8adc077f50 tx=0x7f8ac8011810 comp rx=0 tx=0).stop 2024-09-19T04:30:58.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.180+0000 7f8ae4274740 1 -- 172.21.15.107:0/4261164988 shutdown_connections 2024-09-19T04:30:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.180+0000 7f8ae4274740 1 --2- 172.21.15.107:0/4261164988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 0x5559aa6a6410 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.180+0000 7f8ae4274740 1 --2- 172.21.15.107:0/4261164988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa699760 0x5559aa6a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.180+0000 7f8ae4274740 1 --2- 172.21.15.107:0/4261164988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa699220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.180+0000 7f8ae4274740 1 -- 172.21.15.107:0/4261164988 >> 172.21.15.107:0/4261164988 conn(0x5559aa6953d0 msgr2=0x5559aa6957b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.180+0000 7f8ae4274740 1 -- 172.21.15.107:0/4261164988 shutdown_connections 2024-09-19T04:30:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.180+0000 7f8ae4274740 1 -- 172.21.15.107:0/4261164988 wait complete. 2024-09-19T04:30:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.180+0000 7f8adb400640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ac00a3430 0x7f8ac00a58f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4031f80 tx=0x7f8ad4004a60 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.180+0000 7f8ae4274740 1 -- start start 2024-09-19T04:30:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.181+0000 7f8ae4274740 1 Processor -- start 2024-09-19T04:30:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.181+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa6a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.181+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 0x5559aa6a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.181+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 0x5559aa6a1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.181+0000 7f8ae4274740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5559aa6a6bf0 con 0x5559aa6a4020 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.181+0000 7f8ae4274740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5559aa5ba0f0 con 0x5559aa698e40 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.181+0000 7f8ae4274740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5559aa6a6d70 con 0x5559aa699760 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.181+0000 7f8ae4274740 10 client.0 _ll_register_callbacks cb 0x5559aa5a1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.182+0000 7f8adb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 0x5559aa6a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.182+0000 7f8adaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa6a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.182+0000 7f8ada000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 0x5559aa6a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.182+0000 7f8adaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa6a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44204/0 (socket says 172.21.15.107:44204) 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.182+0000 7f8adaa00640 1 -- 172.21.15.107:0/799457390 learned_addr learned my addr 172.21.15.107:0/799457390 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.182+0000 7f8adb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 0x5559aa6a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36328/0 (socket says 172.21.15.107:36328) 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.183+0000 7f8adaa00640 1 -- 172.21.15.107:0/799457390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 msgr2=0x5559aa6a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.183+0000 7f8adaa00640 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 0x5559aa6a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.183+0000 7f8adaa00640 1 -- 172.21.15.107:0/799457390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 msgr2=0x5559aa6a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:58.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.183+0000 7f8adaa00640 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 0x5559aa6a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.183+0000 7f8adaa00640 1 -- 172.21.15.107:0/799457390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559aa3703c0 con 0x5559aa698e40 2024-09-19T04:30:58.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.183+0000 7f8ada000640 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 0x5559aa6a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:58.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.184+0000 7f8adaa00640 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa6a0e80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8acc098d50 tx=0x7f8acc010e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1bca390287025b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:58.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.184+0000 7f8abbe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:58.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.184+0000 7f8abbe00640 1 -- 172.21.15.107:0/799457390 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc09bec0 con 0x5559aa698e40 2024-09-19T04:30:58.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.184+0000 7f8abbe00640 1 -- 172.21.15.107:0/799457390 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f8acc0cd9c0 con 0x5559aa698e40 2024-09-19T04:30:58.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.184+0000 7f8ae4274740 1 -- 172.21.15.107:0/799457390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac8005500 con 0x5559aa698e40 2024-09-19T04:30:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.184+0000 7f8abbe00640 1 -- 172.21.15.107:0/799457390 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8acc09bce0 con 0x5559aa698e40 2024-09-19T04:30:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.184+0000 7f8ae4274740 10 client.27116 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.184+0000 7f8ae4274740 1 -- 172.21.15.107:0/799457390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5559aa6b2bb0 con 0x5559aa698e40 2024-09-19T04:30:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.185+0000 7f8abbe00640 1 -- 172.21.15.107:0/799457390 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8005500 con 0x5559aa698e40 2024-09-19T04:30:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.185+0000 7f8abbe00640 1 -- 172.21.15.107:0/799457390 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f8acc0bc0b0 con 0x5559aa698e40 2024-09-19T04:30:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.185+0000 7f8ae4274740 10 client.27116 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.185+0000 7f8ae4274740 10 client.27116 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.185+0000 7f8ae4274740 20 client.27116 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.185+0000 7f8ae4274740 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5559aa6b4010 0x5559aa6b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.185+0000 7f8ae4274740 4 client.27116 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.185+0000 7f8ae4274740 1 -- 172.21.15.107:0/799457390 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559aa60c950 con 0x5559aa6b4010 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.186+0000 7f8ada000640 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5559aa6b4010 0x5559aa6b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.186+0000 7f8ada000640 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5559aa6b4010 0x5559aa6b6400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d258fc2ad6c64daf server_cookie=90dd18ee03ad5eef in_seq=0 out_seq=0 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.186+0000 7f8abbe00640 10 client.27116 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8abbe00640 1 -- 172.21.15.107:0/799457390 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559aa60c950 con 0x5559aa6b4010 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8abbe00640 10 client.27116 handle_command_reply: tid=0 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8ae4274740 1 client.27116 shutdown 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8ae4274740 1 -- 172.21.15.107:0/799457390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 msgr2=0x5559aa6a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8ae4274740 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa6a0e80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8acc098d50 tx=0x7f8acc010e30 comp rx=0 tx=0).stop 2024-09-19T04:30:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8ae4274740 1 -- 172.21.15.107:0/799457390 shutdown_connections 2024-09-19T04:30:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8ae4274740 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5559aa6b4010 0x5559aa6b6400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8ae4274740 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 0x5559aa6a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8abbe00640 0 client.27116 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:30:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8ae4274740 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 0x5559aa6a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8ae4274740 1 --2- 172.21.15.107:0/799457390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa6a0e80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.187+0000 7f8ae4274740 1 -- 172.21.15.107:0/799457390 >> 172.21.15.107:0/799457390 conn(0x5559aa6953d0 msgr2=0x5559aa69b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.188+0000 7f8ae4274740 1 -- 172.21.15.107:0/799457390 shutdown_connections 2024-09-19T04:30:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.188+0000 7f8ae4274740 1 -- 172.21.15.107:0/799457390 wait complete. 2024-09-19T04:30:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.188+0000 7f8ae4274740 20 client.27116 trim_cache size 0 max 16384 2024-09-19T04:30:58.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.201+0000 7f8ae4274740 1 Processor -- start 2024-09-19T04:30:58.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8ae4274740 1 -- start start 2024-09-19T04:30:58.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa698e40 0x5559aa69afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa699760 0x5559aa6bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 0x5559aa6bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8ae4274740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b3c60 con 0x5559aa698e40 2024-09-19T04:30:58.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8ae4274740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b57a0 con 0x5559aa699760 2024-09-19T04:30:58.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8ae4274740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b4330 con 0x5559aa6a4020 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8adaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa698e40 0x5559aa69afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8adaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa698e40 0x5559aa69afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36332/0 (socket says 172.21.15.107:36332) 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8adaa00640 1 -- 172.21.15.107:0/2520449653 learned_addr learned my addr 172.21.15.107:0/2520449653 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8ada000640 1 --2- 172.21.15.107:0/2520449653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa699760 0x5559aa6bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.202+0000 7f8adb400640 1 --2- 172.21.15.107:0/2520449653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 0x5559aa6bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.203+0000 7f8ada000640 1 -- 172.21.15.107:0/2520449653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 msgr2=0x5559aa6bc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.203+0000 7f8ada000640 1 --2- 172.21.15.107:0/2520449653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 0x5559aa6bc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.203+0000 7f8ada000640 1 -- 172.21.15.107:0/2520449653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa698e40 msgr2=0x5559aa69afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.203+0000 7f8ada000640 1 --2- 172.21.15.107:0/2520449653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa698e40 0x5559aa69afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.203+0000 7f8ada000640 1 -- 172.21.15.107:0/2520449653 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559aa699590 con 0x5559aa699760 2024-09-19T04:30:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.203+0000 7f8adaa00640 1 --2- 172.21.15.107:0/2520449653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa698e40 0x5559aa69afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:58.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.203+0000 7f8ada000640 1 --2- 172.21.15.107:0/2520449653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa699760 0x5559aa6bbc80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8011d10 tx=0x7f8ac8004a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b374a0598dc6eba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:58.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.203+0000 7f8ac6a00640 1 -- 172.21.15.107:0/2520449653 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8033070 con 0x5559aa699760 2024-09-19T04:30:58.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.203+0000 7f8ac6a00640 1 -- 172.21.15.107:0/2520449653 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8009d50 con 0x5559aa699760 2024-09-19T04:30:58.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.203+0000 7f8ac6a00640 1 -- 172.21.15.107:0/2520449653 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac802e640 con 0x5559aa699760 2024-09-19T04:30:58.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.204+0000 7f8ae4274740 1 -- 172.21.15.107:0/2520449653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa699760 msgr2=0x5559aa6bbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.204+0000 7f8ae4274740 1 --2- 172.21.15.107:0/2520449653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa699760 0x5559aa6bbc80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8011d10 tx=0x7f8ac8004a10 comp rx=0 tx=0).stop 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.204+0000 7f8ae4274740 1 -- 172.21.15.107:0/2520449653 shutdown_connections 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.204+0000 7f8ae4274740 1 --2- 172.21.15.107:0/2520449653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa6a4020 0x5559aa6bc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.204+0000 7f8ae4274740 1 --2- 172.21.15.107:0/2520449653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa699760 0x5559aa6bbc80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.204+0000 7f8ae4274740 1 --2- 172.21.15.107:0/2520449653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa698e40 0x5559aa69afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.204+0000 7f8ae4274740 1 -- 172.21.15.107:0/2520449653 >> 172.21.15.107:0/2520449653 conn(0x5559aa6953d0 msgr2=0x5559aa69a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.204+0000 7f8ae4274740 1 -- 172.21.15.107:0/2520449653 shutdown_connections 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.204+0000 7f8ae4274740 1 -- 172.21.15.107:0/2520449653 wait complete. 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.205+0000 7f8ae4274740 1 -- start start 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.205+0000 7f8ae4274740 1 Processor -- start 2024-09-19T04:30:58.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.205+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa716f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.205+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 0x5559aa717480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.205+0000 7f8ae4274740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 0x5559aa6a7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.205+0000 7f8ae4274740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b50d0 con 0x5559aa6a4020 2024-09-19T04:30:58.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.205+0000 7f8ae4274740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b4a00 con 0x5559aa698e40 2024-09-19T04:30:58.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.205+0000 7f8ae4274740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b1c60 con 0x5559aa699760 2024-09-19T04:30:58.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.205+0000 7f8ae4274740 10 client.0 _ll_register_callbacks cb 0x7f8acc0d37d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:58.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa716f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa716f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44222/0 (socket says 172.21.15.107:44222) 2024-09-19T04:30:58.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adaa00640 1 -- 172.21.15.107:0/1187451490 learned_addr learned my addr 172.21.15.107:0/1187451490 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:58.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8ada000640 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 0x5559aa717480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adb400640 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 0x5559aa6a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adaa00640 1 -- 172.21.15.107:0/1187451490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 msgr2=0x5559aa717480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adaa00640 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 0x5559aa717480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adaa00640 1 -- 172.21.15.107:0/1187451490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 msgr2=0x5559aa6a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adaa00640 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 0x5559aa6a7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adaa00640 1 -- 172.21.15.107:0/1187451490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559aa6a7c30 con 0x5559aa698e40 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adb400640 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 0x5559aa6a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8adaa00640 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa716f40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8acc09c080 tx=0x7f8acc0b5260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2095aa9901903ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.206+0000 7f8abbe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8abbe00640 1 -- 172.21.15.107:0/1187451490 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0cdbd0 con 0x5559aa698e40 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8abbe00640 1 -- 172.21.15.107:0/1187451490 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f8acc0d3380 con 0x5559aa698e40 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8ae4274740 1 -- 172.21.15.107:0/1187451490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac8009d50 con 0x5559aa698e40 2024-09-19T04:30:58.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8abbe00640 1 -- 172.21.15.107:0/1187451490 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8acc0a4f40 con 0x5559aa698e40 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8ae4274740 10 client.27122 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8ae4274740 1 -- 172.21.15.107:0/1187451490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5559aa71e710 con 0x5559aa698e40 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8abbe00640 1 -- 172.21.15.107:0/1187451490 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8009d50 con 0x5559aa698e40 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8abbe00640 1 -- 172.21.15.107:0/1187451490 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f8acc0980d0 con 0x5559aa698e40 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8ae4274740 10 client.27122 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8ae4274740 10 client.27122 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8ae4274740 20 client.27122 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8ae4274740 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5559aa71ee20 0x5559aa721210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8ae4274740 4 client.27122 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.207+0000 7f8ae4274740 1 -- 172.21.15.107:0/1187451490 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559aa606e60 con 0x5559aa71ee20 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.208+0000 7f8ada000640 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5559aa71ee20 0x5559aa721210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:58.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.208+0000 7f8ada000640 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5559aa71ee20 0x5559aa721210 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e729074c509bbf37 server_cookie=4aeb28397d906091 in_seq=0 out_seq=0 2024-09-19T04:30:58.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.208+0000 7f8abbe00640 10 client.27122 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:30:58.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.208+0000 7f8abbe00640 1 -- 172.21.15.107:0/1187451490 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559aa606e60 con 0x5559aa71ee20 2024-09-19T04:30:58.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.208+0000 7f8abbe00640 10 client.27122 handle_command_reply: tid=0 2024-09-19T04:30:58.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 client.27122 shutdown 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 -- 172.21.15.107:0/1187451490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 msgr2=0x5559aa716f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa716f40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8acc09c080 tx=0x7f8acc0b5260 comp rx=0 tx=0).stop 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 -- 172.21.15.107:0/1187451490 shutdown_connections 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x5559aa71ee20 0x5559aa721210 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559aa6a4020 0x5559aa6a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5559aa699760 0x5559aa717480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 --2- 172.21.15.107:0/1187451490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559aa698e40 0x5559aa716f40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 -- 172.21.15.107:0/1187451490 >> 172.21.15.107:0/1187451490 conn(0x5559aa6953d0 msgr2=0x5559aa69b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8abbe00640 0 client.27122 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 -- 172.21.15.107:0/1187451490 shutdown_connections 2024-09-19T04:30:58.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 1 -- 172.21.15.107:0/1187451490 wait complete. 2024-09-19T04:30:58.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.209+0000 7f8ae4274740 20 client.27122 trim_cache size 0 max 16384 2024-09-19T04:30:58.217 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:30:58.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 -- 172.21.15.107:0/2989870935 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ac00a3430 msgr2=0x7f8ac00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:58.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ac00a3430 0x7f8ac00a58f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4031f80 tx=0x7f8ad4004a60 comp rx=0 tx=0).stop 2024-09-19T04:30:58.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 -- 172.21.15.107:0/2989870935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8adc103e30 msgr2=0x7f8adc076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:58.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8adc103e30 0x7f8adc076700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00ea50 tx=0x7f8acc00ef10 comp rx=0 tx=0).stop 2024-09-19T04:30:58.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 -- 172.21.15.107:0/2989870935 shutdown_connections 2024-09-19T04:30:58.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8ac00a3430 0x7f8ac00a58f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8adc108f60 0x7f8adc077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8adc1047e0 0x7f8adc076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 --2- 172.21.15.107:0/2989870935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8adc103e30 0x7f8adc076700 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:58.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 -- 172.21.15.107:0/2989870935 >> 172.21.15.107:0/2989870935 conn(0x7f8adc0ff8d0 msgr2=0x7f8adc100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:58.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 -- 172.21.15.107:0/2989870935 shutdown_connections 2024-09-19T04:30:58.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:58.210+0000 7f8adbe00640 1 -- 172.21.15.107:0/2989870935 wait complete. 2024-09-19T04:30:58.719 INFO:tasks.workunit.client.0.smithi107.stdout: 1 356392 2.82 MB/sec execute 436 sec latency 2054.480 ms 2024-09-19T04:30:58.947 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:30:58 smithi107 ceph-mon[30659]: pgmap v330: 97 pgs: 97 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-09-19T04:30:58.985 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:30:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:30:58 smithi115 ceph-mon[35477]: pgmap v330: 97 pgs: 97 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-09-19T04:30:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:30:58 smithi145 ceph-mon[35096]: pgmap v330: 97 pgs: 97 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-09-19T04:30:59.371 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:30:59.719 INFO:tasks.workunit.client.0.smithi107.stdout: 1 357460 2.82 MB/sec execute 437 sec latency 2559.569 ms 2024-09-19T04:30:59.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.961+0000 7fe194a00640 1 -- 172.21.15.107:0/1648696583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1901079c0 msgr2=0x7fe190107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:59.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.961+0000 7fe194a00640 1 --2- 172.21.15.107:0/1648696583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1901079c0 0x7fe190107da0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe180009900 tx=0x7fe18002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.962+0000 7fe194a00640 1 -- 172.21.15.107:0/1648696583 shutdown_connections 2024-09-19T04:30:59.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.962+0000 7fe194a00640 1 --2- 172.21.15.107:0/1648696583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe190102360 0x7fe19010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.962+0000 7fe194a00640 1 --2- 172.21.15.107:0/1648696583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1901019c0 0x7fe190101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.962+0000 7fe194a00640 1 --2- 172.21.15.107:0/1648696583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1901079c0 0x7fe190107da0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.962+0000 7fe194a00640 1 -- 172.21.15.107:0/1648696583 >> 172.21.15.107:0/1648696583 conn(0x7fe1900fd740 msgr2=0x7fe1900ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:59.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.962+0000 7fe194a00640 1 -- 172.21.15.107:0/1648696583 shutdown_connections 2024-09-19T04:30:59.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.962+0000 7fe194a00640 1 -- 172.21.15.107:0/1648696583 wait complete. 2024-09-19T04:30:59.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.963+0000 7fe194a00640 1 Processor -- start 2024-09-19T04:30:59.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.963+0000 7fe194a00640 1 -- start start 2024-09-19T04:30:59.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.963+0000 7fe194a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1901019c0 0x7fe190198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.963+0000 7fe194a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe190102360 0x7fe1901987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.963+0000 7fe194a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1901079c0 0x7fe19019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.963+0000 7fe194a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe190111e10 con 0x7fe1901019c0 2024-09-19T04:30:59.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.963+0000 7fe194a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe190111c90 con 0x7fe190102360 2024-09-19T04:30:59.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.963+0000 7fe194a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe190111f90 con 0x7fe1901079c0 2024-09-19T04:30:59.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe190102360 0x7fe1901987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe190102360 0x7fe1901987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:41512/0 (socket says 172.21.15.107:41512) 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18de00640 1 -- 172.21.15.107:0/3195255523 learned_addr learned my addr 172.21.15.107:0/3195255523 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18e800640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1901019c0 0x7fe190198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18f200640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1901079c0 0x7fe19019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18de00640 1 -- 172.21.15.107:0/3195255523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1901079c0 msgr2=0x7fe19019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18de00640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1901079c0 0x7fe19019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18de00640 1 -- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1901019c0 msgr2=0x7fe190198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18de00640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1901019c0 0x7fe190198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18de00640 1 -- 172.21.15.107:0/3195255523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19019d240 con 0x7fe190102360 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.964+0000 7fe18e800640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1901019c0 0x7fe190198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.965+0000 7fe18de00640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe190102360 0x7fe1901987d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00d950 tx=0x7fe17c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.965+0000 7fe187400640 1 -- 172.21.15.107:0/3195255523 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c017070 con 0x7fe190102360 2024-09-19T04:30:59.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.965+0000 7fe187400640 1 -- 172.21.15.107:0/3195255523 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe17c009c30 con 0x7fe190102360 2024-09-19T04:30:59.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.965+0000 7fe187400640 1 -- 172.21.15.107:0/3195255523 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c00f6b0 con 0x7fe190102360 2024-09-19T04:30:59.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.965+0000 7fe194a00640 1 -- 172.21.15.107:0/3195255523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19019d530 con 0x7fe190102360 2024-09-19T04:30:59.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.965+0000 7fe194a00640 1 -- 172.21.15.107:0/3195255523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1901a4e10 con 0x7fe190102360 2024-09-19T04:30:59.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.969+0000 7fe197ed8740 1 Processor -- start 2024-09-19T04:30:59.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.969+0000 7fe197ed8740 1 -- start start 2024-09-19T04:30:59.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.969+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.969+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5577780f7600 0x557778105ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.969+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557778106430 0x557778108820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.969+0000 7fe197ed8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55777805d3e0 con 0x5577780f7600 2024-09-19T04:30:59.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.969+0000 7fe197ed8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557778004d80 con 0x557778106430 2024-09-19T04:30:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.969+0000 7fe197ed8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5577780593a0 con 0x5577780f6ce0 2024-09-19T04:30:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.969+0000 7fe18f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe18f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53240/0 (socket says 172.21.15.107:53240) 2024-09-19T04:30:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe18f200640 1 -- 172.21.15.107:0/1140369302 learned_addr learned my addr 172.21.15.107:0/1140369302 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe187400640 1 -- 172.21.15.107:0/3195255523 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fe17c0109d0 con 0x7fe190102360 2024-09-19T04:30:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe18de00640 1 --2- 172.21.15.107:0/1140369302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557778106430 0x557778108820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe18e800640 1 --2- 172.21.15.107:0/1140369302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5577780f7600 0x557778105ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe18f200640 1 -- 172.21.15.107:0/1140369302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557778106430 msgr2=0x557778108820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:59.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe18f200640 1 --2- 172.21.15.107:0/1140369302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557778106430 0x557778108820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe18f200640 1 -- 172.21.15.107:0/1140369302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5577780f7600 msgr2=0x557778105ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:59.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe18f200640 1 --2- 172.21.15.107:0/1140369302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5577780f7600 0x557778105ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe18f200640 1 -- 172.21.15.107:0/1140369302 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557778023ae0 con 0x5577780f6ce0 2024-09-19T04:30:59.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.970+0000 7fe18de00640 1 --2- 172.21.15.107:0/1140369302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557778106430 0x557778108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:59.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.971+0000 7fe18f200640 1 --2- 172.21.15.107:0/1140369302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f70c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe188009f50 tx=0x7fe1880118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa943cfb2a11511 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:59.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.971+0000 7fe17b400640 1 -- 172.21.15.107:0/1140369302 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188004b20 con 0x5577780f6ce0 2024-09-19T04:30:59.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.971+0000 7fe17b400640 1 -- 172.21.15.107:0/1140369302 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe188004cc0 con 0x5577780f6ce0 2024-09-19T04:30:59.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.971+0000 7fe17b400640 1 -- 172.21.15.107:0/1140369302 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188005020 con 0x5577780f6ce0 2024-09-19T04:30:59.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.971+0000 7fe187400640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe1740a34b0 0x7fe1740a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.972+0000 7fe187400640 1 -- 172.21.15.107:0/3195255523 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fe17c0cf950 con 0x7fe190102360 2024-09-19T04:30:59.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.972+0000 7fe197ed8740 1 -- 172.21.15.107:0/1140369302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 msgr2=0x5577780f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:59.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.972+0000 7fe18e800640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe1740a34b0 0x7fe1740a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.972+0000 7fe197ed8740 1 --2- 172.21.15.107:0/1140369302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f70c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe188009f50 tx=0x7fe1880118d0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.972+0000 7fe197ed8740 1 -- 172.21.15.107:0/1140369302 shutdown_connections 2024-09-19T04:30:59.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.972+0000 7fe197ed8740 1 --2- 172.21.15.107:0/1140369302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557778106430 0x557778108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.972+0000 7fe197ed8740 1 --2- 172.21.15.107:0/1140369302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5577780f7600 0x557778105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.972+0000 7fe197ed8740 1 --2- 172.21.15.107:0/1140369302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f70c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.972+0000 7fe197ed8740 1 -- 172.21.15.107:0/1140369302 >> 172.21.15.107:0/1140369302 conn(0x5577780f33d0 msgr2=0x5577780f37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:59.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.972+0000 7fe197ed8740 1 -- 172.21.15.107:0/1140369302 shutdown_connections 2024-09-19T04:30:59.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.973+0000 7fe197ed8740 1 -- 172.21.15.107:0/1140369302 wait complete. 2024-09-19T04:30:59.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.973+0000 7fe18e800640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe1740a34b0 0x7fe1740a5970 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe180000c00 tx=0x7fe180031440 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:59.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.973+0000 7fe197ed8740 1 -- start start 2024-09-19T04:30:59.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.974+0000 7fe197ed8740 1 Processor -- start 2024-09-19T04:30:59.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.974+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f6ce0 0x557778105c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.974+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5577780f7600 0x5577780fef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.974+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557778106430 0x5577780ff460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.974+0000 7fe197ed8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557778109000 con 0x5577780f7600 2024-09-19T04:30:59.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.974+0000 7fe197ed8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5577780180f0 con 0x5577780f6ce0 2024-09-19T04:30:59.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.974+0000 7fe197ed8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557778109180 con 0x557778106430 2024-09-19T04:30:59.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.974+0000 7fe197ed8740 10 client.0 _ll_register_callbacks cb 0x557777fffbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.974+0000 7fe18f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557778106430 0x5577780ff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557778106430 0x5577780ff460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53252/0 (socket says 172.21.15.107:53252) 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18f200640 1 -- 172.21.15.107:0/265034845 learned_addr learned my addr 172.21.15.107:0/265034845 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18de00640 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5577780f7600 0x5577780fef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18e800640 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f6ce0 0x557778105c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18e800640 1 -- 172.21.15.107:0/265034845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557778106430 msgr2=0x5577780ff460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18e800640 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557778106430 0x5577780ff460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18e800640 1 -- 172.21.15.107:0/265034845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5577780f7600 msgr2=0x5577780fef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18e800640 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5577780f7600 0x5577780fef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18e800640 1 -- 172.21.15.107:0/265034845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557777dce3c0 con 0x5577780f6ce0 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18f200640 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557778106430 0x5577780ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe18e800640 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f6ce0 0x557778105c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe18002a9e0 tx=0x7fe180009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ed51ea7d88182f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.975+0000 7fe16fe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:30:59.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.976+0000 7fe16fe00640 1 -- 172.21.15.107:0/265034845 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18004d070 con 0x5577780f6ce0 2024-09-19T04:30:59.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.976+0000 7fe16fe00640 1 -- 172.21.15.107:0/265034845 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fe180032750 con 0x5577780f6ce0 2024-09-19T04:30:59.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.976+0000 7fe197ed8740 1 -- 172.21.15.107:0/265034845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe188004cc0 con 0x5577780f6ce0 2024-09-19T04:30:59.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.976+0000 7fe16fe00640 1 -- 172.21.15.107:0/265034845 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe180059050 con 0x5577780f6ce0 2024-09-19T04:30:59.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.976+0000 7fe197ed8740 10 client.27134 fetch_fsmap learned FSMap version 48 2024-09-19T04:30:59.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.976+0000 7fe197ed8740 1 -- 172.21.15.107:0/265034845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557778119490 con 0x5577780f6ce0 2024-09-19T04:30:59.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.977+0000 7fe16fe00640 1 -- 172.21.15.107:0/265034845 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe188004cc0 con 0x5577780f6ce0 2024-09-19T04:30:59.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.977+0000 7fe16fe00640 1 -- 172.21.15.107:0/265034845 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fe1800498b0 con 0x5577780f6ce0 2024-09-19T04:30:59.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.977+0000 7fe197ed8740 10 client.27134 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:30:59.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.977+0000 7fe197ed8740 10 client.27134 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:30:59.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.977+0000 7fe197ed8740 20 client.27134 populate_metadata read hostname 'smithi107' 2024-09-19T04:30:59.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.977+0000 7fe197ed8740 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55777811a8a0 0x55777811ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.977+0000 7fe197ed8740 4 client.27134 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:30:59.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.977+0000 7fe197ed8740 1 -- 172.21.15.107:0/265034845 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55777806a950 con 0x55777811a8a0 2024-09-19T04:30:59.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.977+0000 7fe18de00640 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55777811a8a0 0x55777811ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.978+0000 7fe18de00640 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55777811a8a0 0x55777811ccd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0bcc8fab16d0a63 server_cookie=c56947e3703b8d3f in_seq=0 out_seq=0 2024-09-19T04:30:59.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.978+0000 7fe16fe00640 10 client.27134 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:30:59.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe16fe00640 1 -- 172.21.15.107:0/265034845 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55777806a950 con 0x55777811a8a0 2024-09-19T04:30:59.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe16fe00640 10 client.27134 handle_command_reply: tid=0 2024-09-19T04:30:59.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 client.27134 shutdown 2024-09-19T04:30:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 -- 172.21.15.107:0/265034845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f6ce0 msgr2=0x557778105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:30:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f6ce0 0x557778105c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe18002a9e0 tx=0x7fe180009260 comp rx=0 tx=0).stop 2024-09-19T04:30:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 -- 172.21.15.107:0/265034845 shutdown_connections 2024-09-19T04:30:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x55777811a8a0 0x55777811ccd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557778106430 0x5577780ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5577780f7600 0x5577780fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe16fe00640 0 client.27134 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:30:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 --2- 172.21.15.107:0/265034845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f6ce0 0x557778105c90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 -- 172.21.15.107:0/265034845 >> 172.21.15.107:0/265034845 conn(0x5577780f33d0 msgr2=0x5577780f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:30:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 -- 172.21.15.107:0/265034845 shutdown_connections 2024-09-19T04:30:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.979+0000 7fe197ed8740 1 -- 172.21.15.107:0/265034845 wait complete. 2024-09-19T04:30:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.980+0000 7fe197ed8740 20 client.27134 trim_cache size 0 max 16384 2024-09-19T04:30:59.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.994+0000 7fe197ed8740 1 Processor -- start 2024-09-19T04:30:59.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.994+0000 7fe197ed8740 1 -- start start 2024-09-19T04:30:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 0x5577781225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557778106430 0x557778122b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:30:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe197ed8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901ab520 con 0x557778106430 2024-09-19T04:30:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe197ed8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901ad060 con 0x5577780f7600 2024-09-19T04:30:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe197ed8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901abbf0 con 0x5577780f6ce0 2024-09-19T04:30:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe18e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe18e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53258/0 (socket says 172.21.15.107:53258) 2024-09-19T04:30:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe18e800640 1 -- 172.21.15.107:0/1393046430 learned_addr learned my addr 172.21.15.107:0/1393046430 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:30:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe18f200640 1 --2- 172.21.15.107:0/1393046430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557778106430 0x557778122b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.995+0000 7fe18de00640 1 --2- 172.21.15.107:0/1393046430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 0x5577781225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:30:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.996+0000 7fe18e800640 1 -- 172.21.15.107:0/1393046430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 msgr2=0x5577781225f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.996+0000 7fe18e800640 1 --2- 172.21.15.107:0/1393046430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 0x5577781225f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.996+0000 7fe18e800640 1 -- 172.21.15.107:0/1393046430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557778106430 msgr2=0x557778122b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:30:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.996+0000 7fe18e800640 1 --2- 172.21.15.107:0/1393046430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557778106430 0x557778122b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:30:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.996+0000 7fe18e800640 1 -- 172.21.15.107:0/1393046430 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577780f9d40 con 0x5577780f6ce0 2024-09-19T04:30:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.996+0000 7fe18e800640 1 --2- 172.21.15.107:0/1393046430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f9800 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe180049d00 tx=0x7fe180009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d6a4717a31a973c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:30:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.996+0000 7fe17aa00640 1 -- 172.21.15.107:0/1393046430 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180061070 con 0x5577780f6ce0 2024-09-19T04:30:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.996+0000 7fe17aa00640 1 -- 172.21.15.107:0/1393046430 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe180049930 con 0x5577780f6ce0 2024-09-19T04:30:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.996+0000 7fe17aa00640 1 -- 172.21.15.107:0/1393046430 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18004d050 con 0x5577780f6ce0 2024-09-19T04:31:00.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.997+0000 7fe197ed8740 1 -- 172.21.15.107:0/1393046430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 msgr2=0x5577780f9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.997+0000 7fe197ed8740 1 --2- 172.21.15.107:0/1393046430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f9800 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe180049d00 tx=0x7fe180009610 comp rx=0 tx=0).stop 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.997+0000 7fe197ed8740 1 -- 172.21.15.107:0/1393046430 shutdown_connections 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.997+0000 7fe197ed8740 1 --2- 172.21.15.107:0/1393046430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557778106430 0x557778122b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.997+0000 7fe197ed8740 1 --2- 172.21.15.107:0/1393046430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 0x5577781225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.997+0000 7fe197ed8740 1 --2- 172.21.15.107:0/1393046430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x5577780f9800 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.997+0000 7fe197ed8740 1 -- 172.21.15.107:0/1393046430 >> 172.21.15.107:0/1393046430 conn(0x5577780f33d0 msgr2=0x5577780f24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.997+0000 7fe197ed8740 1 -- 172.21.15.107:0/1393046430 shutdown_connections 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.997+0000 7fe197ed8740 1 -- 172.21.15.107:0/1393046430 wait complete. 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.998+0000 7fe197ed8740 1 -- start start 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.998+0000 7fe197ed8740 1 Processor -- start 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x557778117390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 0x5577781126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:00.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe197ed8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557778106430 0x557778112c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:00.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe197ed8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901ac990 con 0x557778106430 2024-09-19T04:31:00.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe197ed8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901ac2c0 con 0x5577780f7600 2024-09-19T04:31:00.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe197ed8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901a9520 con 0x5577780f6ce0 2024-09-19T04:31:00.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe197ed8740 10 client.0 _ll_register_callbacks cb 0x7fe180009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:00.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 0x5577781126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:00.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x557778117390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:00.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x557778117390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53266/0 (socket says 172.21.15.107:53266) 2024-09-19T04:31:00.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18e800640 1 -- 172.21.15.107:0/4083185360 learned_addr learned my addr 172.21.15.107:0/4083185360 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:00.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18f200640 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557778106430 0x557778112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:00.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18de00640 1 -- 172.21.15.107:0/4083185360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 msgr2=0x557778117390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:00.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18de00640 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x557778117390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18de00640 1 -- 172.21.15.107:0/4083185360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557778106430 msgr2=0x557778112c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:00.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18de00640 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557778106430 0x557778112c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18de00640 1 -- 172.21.15.107:0/4083185360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557778113240 con 0x5577780f7600 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:30:59.999+0000 7fe18e800640 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x557778117390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.000+0000 7fe18de00640 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 0x5577781126c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0cd060 tx=0x7fe17c0d54d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af172cd778b0740 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.000+0000 7fe16fe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.000+0000 7fe16fe00640 1 -- 172.21.15.107:0/4083185360 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c020040 con 0x5577780f7600 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.000+0000 7fe16fe00640 1 -- 172.21.15.107:0/4083185360 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fe17c098260 con 0x5577780f7600 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.000+0000 7fe197ed8740 1 -- 172.21.15.107:0/4083185360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe180049930 con 0x5577780f7600 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.000+0000 7fe16fe00640 1 -- 172.21.15.107:0/4083185360 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe17c09c2f0 con 0x5577780f7600 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.000+0000 7fe197ed8740 10 client.27140 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.000+0000 7fe197ed8740 1 -- 172.21.15.107:0/4083185360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5577781853d0 con 0x5577780f7600 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.000+0000 7fe16fe00640 1 -- 172.21.15.107:0/4083185360 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe180049930 con 0x5577780f7600 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.000+0000 7fe16fe00640 1 -- 172.21.15.107:0/4083185360 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fe17c0860b0 con 0x5577780f7600 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.001+0000 7fe197ed8740 10 client.27140 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:00.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.001+0000 7fe197ed8740 10 client.27140 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.001+0000 7fe197ed8740 20 client.27140 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.001+0000 7fe197ed8740 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557778185b00 0x557778187f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.001+0000 7fe197ed8740 4 client.27140 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.001+0000 7fe197ed8740 1 -- 172.21.15.107:0/4083185360 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557778064e60 con 0x557778185b00 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.001+0000 7fe18e800640 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557778185b00 0x557778187f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.001+0000 7fe18e800640 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557778185b00 0x557778187f20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2f9b5f08448b8617 server_cookie=feebe7156774a456 in_seq=0 out_seq=0 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.001+0000 7fe16fe00640 10 client.27140 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe16fe00640 1 -- 172.21.15.107:0/4083185360 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557778064e60 con 0x557778185b00 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe16fe00640 10 client.27140 handle_command_reply: tid=0 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 client.27140 shutdown 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 -- 172.21.15.107:0/4083185360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 msgr2=0x5577781126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:00.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 0x5577781126c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0cd060 tx=0x7fe17c0d54d0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 -- 172.21.15.107:0/4083185360 shutdown_connections 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557778185b00 0x557778187f20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557778106430 0x557778112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe16fe00640 0 client.27140 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5577780f7600 0x5577781126c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 --2- 172.21.15.107:0/4083185360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5577780f6ce0 0x557778117390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 -- 172.21.15.107:0/4083185360 >> 172.21.15.107:0/4083185360 conn(0x5577780f33d0 msgr2=0x557778102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 -- 172.21.15.107:0/4083185360 shutdown_connections 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 1 -- 172.21.15.107:0/4083185360 wait complete. 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.002+0000 7fe197ed8740 20 client.27140 trim_cache size 0 max 16384 2024-09-19T04:31:00.008 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:31:00.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 -- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe1740a34b0 msgr2=0x7fe1740a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:00.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe1740a34b0 0x7fe1740a5970 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe180000c00 tx=0x7fe180031440 comp rx=0 tx=0).stop 2024-09-19T04:31:00.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 -- 172.21.15.107:0/3195255523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe190102360 msgr2=0x7fe1901987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:00.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe190102360 0x7fe1901987d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00d950 tx=0x7fe17c00de10 comp rx=0 tx=0).stop 2024-09-19T04:31:00.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 -- 172.21.15.107:0/3195255523 shutdown_connections 2024-09-19T04:31:00.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe1740a34b0 0x7fe1740a5970 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1901079c0 0x7fe19019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe190102360 0x7fe1901987d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 --2- 172.21.15.107:0/3195255523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1901019c0 0x7fe190198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:00.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 -- 172.21.15.107:0/3195255523 >> 172.21.15.107:0/3195255523 conn(0x7fe1900fd740 msgr2=0x7fe1900fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:00.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.003+0000 7fe194a00640 1 -- 172.21.15.107:0/3195255523 shutdown_connections 2024-09-19T04:31:00.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:00.004+0000 7fe194a00640 1 -- 172.21.15.107:0/3195255523 wait complete. 2024-09-19T04:31:00.719 INFO:tasks.workunit.client.0.smithi107.stdout: 1 358702 2.84 MB/sec execute 438 sec latency 67.555 ms 2024-09-19T04:31:01.044 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:31:01.130 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:00 smithi107 ceph-mon[30659]: pgmap v331: 97 pgs: 97 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-09-19T04:31:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:00 smithi115 ceph-mon[35477]: pgmap v331: 97 pgs: 97 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-09-19T04:31:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:00 smithi145 ceph-mon[35096]: pgmap v331: 97 pgs: 97 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-09-19T04:31:01.434 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:01.720 INFO:tasks.workunit.client.0.smithi107.stdout: 1 358702 2.83 MB/sec execute 439 sec latency 1067.611 ms 2024-09-19T04:31:01.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.988+0000 7f8a41e00640 1 -- 172.21.15.107:0/58159643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a3c102370 msgr2=0x7f8a3c10e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:01.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.988+0000 7f8a41e00640 1 --2- 172.21.15.107:0/58159643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a3c102370 0x7f8a3c10e8a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009c10 tx=0x7f8a3402a6e0 comp rx=0 tx=0).stop 2024-09-19T04:31:01.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.989+0000 7f8a41e00640 1 -- 172.21.15.107:0/58159643 shutdown_connections 2024-09-19T04:31:01.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.989+0000 7f8a41e00640 1 --2- 172.21.15.107:0/58159643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a3c102370 0x7f8a3c10e8a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:01.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.989+0000 7f8a41e00640 1 --2- 172.21.15.107:0/58159643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a3c1019d0 0x7f8a3c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:01.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.989+0000 7f8a41e00640 1 --2- 172.21.15.107:0/58159643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a3c1079d0 0x7f8a3c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:01.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.989+0000 7f8a41e00640 1 -- 172.21.15.107:0/58159643 >> 172.21.15.107:0/58159643 conn(0x7f8a3c0fd710 msgr2=0x7f8a3c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:01.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.989+0000 7f8a41e00640 1 -- 172.21.15.107:0/58159643 shutdown_connections 2024-09-19T04:31:01.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.989+0000 7f8a41e00640 1 -- 172.21.15.107:0/58159643 wait complete. 2024-09-19T04:31:01.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.990+0000 7f8a41e00640 1 Processor -- start 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.990+0000 7f8a41e00640 1 -- start start 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a41e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a3c1019d0 0x7f8a3c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a41e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a3c102370 0x7f8a3c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a41e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a3c1079d0 0x7f8a3c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a41e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c111dd0 con 0x7f8a3c102370 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a41e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c111c50 con 0x7f8a3c1019d0 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a41e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c111f50 con 0x7f8a3c1079d0 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a3c102370 0x7f8a3c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a40a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a3c1019d0 0x7f8a3c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a3c102370 0x7f8a3c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60914/0 (socket says 172.21.15.107:60914) 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a3be00640 1 -- 172.21.15.107:0/2741375548 learned_addr learned my addr 172.21.15.107:0/2741375548 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:01.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a41400640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a3c1079d0 0x7f8a3c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:01.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a40a00640 1 -- 172.21.15.107:0/2741375548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a3c1079d0 msgr2=0x7f8a3c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:01.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a40a00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a3c1079d0 0x7f8a3c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:01.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a40a00640 1 -- 172.21.15.107:0/2741375548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a3c102370 msgr2=0x7f8a3c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a40a00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a3c102370 0x7f8a3c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a40a00640 1 -- 172.21.15.107:0/2741375548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c19f460 con 0x7f8a3c1019d0 2024-09-19T04:31:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.991+0000 7f8a3be00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a3c102370 0x7f8a3c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.992+0000 7f8a40a00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a3c1019d0 0x7f8a3c19a4b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8a30005fd0 tx=0x7f8a3000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.992+0000 7f8a39600640 1 -- 172.21.15.107:0/2741375548 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3000cd00 con 0x7f8a3c1019d0 2024-09-19T04:31:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.992+0000 7f8a39600640 1 -- 172.21.15.107:0/2741375548 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a300035f0 con 0x7f8a3c1019d0 2024-09-19T04:31:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.992+0000 7f8a41e00640 1 -- 172.21.15.107:0/2741375548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c19f750 con 0x7f8a3c1019d0 2024-09-19T04:31:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.992+0000 7f8a39600640 1 -- 172.21.15.107:0/2741375548 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30010670 con 0x7f8a3c1019d0 2024-09-19T04:31:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.992+0000 7f8a41e00640 1 -- 172.21.15.107:0/2741375548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c1a7030 con 0x7f8a3c1019d0 2024-09-19T04:31:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.995+0000 7f8a45fd7740 1 Processor -- start 2024-09-19T04:31:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.995+0000 7f8a39600640 1 -- 172.21.15.107:0/2741375548 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f8a300037a0 con 0x7f8a3c1019d0 2024-09-19T04:31:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.995+0000 7f8a45fd7740 1 -- start start 2024-09-19T04:31:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a0e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0e760 0x560a15a18ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a19020 0x560a15a1b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560a159743e0 con 0x560a15a19020 2024-09-19T04:31:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560a1591bd80 con 0x560a15a0e760 2024-09-19T04:31:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560a159703a0 con 0x560a15a0de40 2024-09-19T04:31:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a19020 0x560a15a1b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a41400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a0e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:01.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a19020 0x560a15a1b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60928/0 (socket says 172.21.15.107:60928) 2024-09-19T04:31:01.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a3be00640 1 -- 172.21.15.107:0/1163284342 learned_addr learned my addr 172.21.15.107:0/1163284342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:01.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a41400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a0e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53298/0 (socket says 172.21.15.107:53298) 2024-09-19T04:31:01.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.996+0000 7f8a40a00640 1 --2- 172.21.15.107:0/1163284342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0e760 0x560a15a18ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.997+0000 7f8a41400640 1 -- 172.21.15.107:0/1163284342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0e760 msgr2=0x560a15a18ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:02.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.997+0000 7f8a41400640 1 --2- 172.21.15.107:0/1163284342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0e760 0x560a15a18ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.997+0000 7f8a41400640 1 -- 172.21.15.107:0/1163284342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a19020 msgr2=0x560a15a1b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.997+0000 7f8a41400640 1 --2- 172.21.15.107:0/1163284342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a19020 0x560a15a1b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.997+0000 7f8a41400640 1 -- 172.21.15.107:0/1163284342 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a1593aae0 con 0x560a15a0de40 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.997+0000 7f8a41400640 1 --2- 172.21.15.107:0/1163284342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a0e220 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8a34034040 tx=0x7f8a34005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6661710975043a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.997+0000 7f8a2dc00640 1 -- 172.21.15.107:0/1163284342 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34041030 con 0x560a15a0de40 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.997+0000 7f8a39600640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a240a33b0 0x7f8a240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.997+0000 7f8a2dc00640 1 -- 172.21.15.107:0/1163284342 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a34030bf0 con 0x560a15a0de40 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a2dc00640 1 -- 172.21.15.107:0/1163284342 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34036040 con 0x560a15a0de40 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a39600640 1 -- 172.21.15.107:0/2741375548 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f8a300cfe50 con 0x7f8a3c1019d0 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a40a00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a240a33b0 0x7f8a240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1163284342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 msgr2=0x560a15a0e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:02.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1163284342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a0e220 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8a34034040 tx=0x7f8a34005ec0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1163284342 shutdown_connections 2024-09-19T04:31:02.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1163284342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a19020 0x560a15a1b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1163284342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0e760 0x560a15a18ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1163284342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a0e220 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1163284342 >> 172.21.15.107:0/1163284342 conn(0x560a15a0a3d0 msgr2=0x560a15a0a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:02.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a40a00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a240a33b0 0x7f8a240a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a300cd670 tx=0x7f8a300d39a0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:02.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.998+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1163284342 shutdown_connections 2024-09-19T04:31:02.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.999+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1163284342 wait complete. 2024-09-19T04:31:02.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.999+0000 7f8a45fd7740 1 -- start start 2024-09-19T04:31:02.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:01.999+0000 7f8a45fd7740 1 Processor -- start 2024-09-19T04:31:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a15ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a16420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 0x560a15a16960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560a15a1bbf0 con 0x560a15a0e760 2024-09-19T04:31:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560a1592f0f0 con 0x560a15a19020 2024-09-19T04:31:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560a15a1bd70 con 0x560a15a0de40 2024-09-19T04:31:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a45fd7740 10 client.0 _ll_register_callbacks cb 0x560a15916bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a41400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a15ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a41400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a15ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53304/0 (socket says 172.21.15.107:53304) 2024-09-19T04:31:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a41400640 1 -- 172.21.15.107:0/1518845170 learned_addr learned my addr 172.21.15.107:0/1518845170 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a41400640 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 0x560a15a16960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a3be00640 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a16420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a41400640 1 -- 172.21.15.107:0/1518845170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 msgr2=0x560a15a16960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a41400640 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 0x560a15a16960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a41400640 1 -- 172.21.15.107:0/1518845170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 msgr2=0x560a15a16420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a41400640 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a16420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a41400640 1 -- 172.21.15.107:0/1518845170 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a156e53c0 con 0x560a15a0de40 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.000+0000 7f8a3be00640 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a16420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.001+0000 7f8a41400640 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a15ee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009560 tx=0x7f8a34009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb31c4dc3422dc82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.001+0000 7f8a22000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.001+0000 7f8a22000640 1 -- 172.21.15.107:0/1518845170 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34036050 con 0x560a15a0de40 2024-09-19T04:31:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.001+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1518845170 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a34030bf0 con 0x560a15a0de40 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.001+0000 7f8a22000640 1 -- 172.21.15.107:0/1518845170 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f8a34003b20 con 0x560a15a0de40 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.001+0000 7f8a22000640 1 -- 172.21.15.107:0/1518845170 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8a34058050 con 0x560a15a0de40 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.001+0000 7f8a45fd7740 10 client.27115 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.001+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1518845170 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560a15a1bc40 con 0x560a15a0de40 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.002+0000 7f8a22000640 1 -- 172.21.15.107:0/1518845170 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a34030bf0 con 0x560a15a0de40 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.002+0000 7f8a22000640 1 -- 172.21.15.107:0/1518845170 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f8a3403d460 con 0x560a15a0de40 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.002+0000 7f8a45fd7740 10 client.27115 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.002+0000 7f8a45fd7740 10 client.27115 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.002+0000 7f8a45fd7740 20 client.27115 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.002+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x560a15a28d80 0x560a15a2b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.002+0000 7f8a45fd7740 4 client.27115 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.002+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1518845170 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a15981950 con 0x560a15a28d80 2024-09-19T04:31:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.003+0000 7f8a3be00640 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x560a15a28d80 0x560a15a2b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.003+0000 7f8a3be00640 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x560a15a28d80 0x560a15a2b170 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=571e3c4c2799f393 server_cookie=531abeef35711b93 in_seq=0 out_seq=0 2024-09-19T04:31:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.003+0000 7f8a22000640 10 client.27115 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:31:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a22000640 1 -- 172.21.15.107:0/1518845170 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a15981950 con 0x560a15a28d80 2024-09-19T04:31:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a22000640 10 client.27115 handle_command_reply: tid=0 2024-09-19T04:31:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 client.27115 shutdown 2024-09-19T04:31:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1518845170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 msgr2=0x560a15a15ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a15ee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009560 tx=0x7f8a34009300 comp rx=0 tx=0).stop 2024-09-19T04:31:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1518845170 shutdown_connections 2024-09-19T04:31:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x560a15a28d80 0x560a15a2b170 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 0x560a15a16960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a16420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a22000640 0 client.27115 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:31:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1518845170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a15ee0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1518845170 >> 172.21.15.107:0/1518845170 conn(0x560a15a0a3d0 msgr2=0x560a15a100b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1518845170 shutdown_connections 2024-09-19T04:31:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.004+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1518845170 wait complete. 2024-09-19T04:31:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.005+0000 7f8a45fd7740 20 client.27115 trim_cache size 0 max 16384 2024-09-19T04:31:02.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.018+0000 7f8a45fd7740 1 Processor -- start 2024-09-19T04:31:02.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.018+0000 7f8a45fd7740 1 -- start start 2024-09-19T04:31:02.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.018+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.018+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a0ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.018+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 0x560a15a30890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.018+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1af8c0 con 0x560a15a0e760 2024-09-19T04:31:02.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.018+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1b1400 con 0x560a15a19020 2024-09-19T04:31:02.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.018+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1aff90 con 0x560a15a0de40 2024-09-19T04:31:02.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.018+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a0ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a41400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a41400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 0x560a15a30890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a41400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 0x560a15a30890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:41592/0 (socket says 172.21.15.107:41592) 2024-09-19T04:31:02.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a41400640 1 -- 172.21.15.107:0/1520719468 learned_addr learned my addr 172.21.15.107:0/1520719468 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:02.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a41400640 1 -- 172.21.15.107:0/1520719468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 msgr2=0x560a15a157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:02.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a41400640 1 --2- 172.21.15.107:0/1520719468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a157a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a41400640 1 -- 172.21.15.107:0/1520719468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 msgr2=0x560a15a0ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:02.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a41400640 1 --2- 172.21.15.107:0/1520719468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a0ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a41400640 1 -- 172.21.15.107:0/1520719468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a15a0dc50 con 0x560a15a19020 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a3be00640 1 --2- 172.21.15.107:0/1520719468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a41400640 1 --2- 172.21.15.107:0/1520719468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 0x560a15a30890 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a340309d0 tx=0x7f8a34000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c651aae91f77ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a2d200640 1 -- 172.21.15.107:0/1520719468 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34036070 con 0x560a15a19020 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a2d200640 1 -- 172.21.15.107:0/1520719468 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a34030bf0 con 0x560a15a19020 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.019+0000 7f8a2d200640 1 -- 172.21.15.107:0/1520719468 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34041040 con 0x560a15a19020 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.020+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1520719468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 msgr2=0x560a15a30890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.020+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1520719468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 0x560a15a30890 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a340309d0 tx=0x7f8a34000f40 comp rx=0 tx=0).stop 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.020+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1520719468 shutdown_connections 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.020+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1520719468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a19020 0x560a15a30890 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.020+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1520719468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.020+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/1520719468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a0de40 0x560a15a157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.020+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1520719468 >> 172.21.15.107:0/1520719468 conn(0x560a15a0a3d0 msgr2=0x560a15a0f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.020+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1520719468 shutdown_connections 2024-09-19T04:31:02.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.020+0000 7f8a45fd7740 1 -- 172.21.15.107:0/1520719468 wait complete. 2024-09-19T04:31:02.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.021+0000 7f8a45fd7740 1 -- start start 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.021+0000 7f8a45fd7740 1 Processor -- start 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.021+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0de40 0x560a15a21710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.021+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a12ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.021+0000 7f8a45fd7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a19020 0x560a15a131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.021+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1b0d30 con 0x560a15a0e760 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.021+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1b0660 con 0x560a15a0de40 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.021+0000 7f8a45fd7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1ad8c0 con 0x560a15a19020 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.021+0000 7f8a45fd7740 10 client.0 _ll_register_callbacks cb 0x7f8a28009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a41400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0de40 0x560a15a21710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a12ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a41400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0de40 0x560a15a21710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:41596/0 (socket says 172.21.15.107:41596) 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a41400640 1 -- 172.21.15.107:0/638151121 learned_addr learned my addr 172.21.15.107:0/638151121 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a41400640 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a19020 0x560a15a131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a41400640 1 -- 172.21.15.107:0/638151121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a19020 msgr2=0x560a15a131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a41400640 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a19020 0x560a15a131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a41400640 1 -- 172.21.15.107:0/638151121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 msgr2=0x560a15a12ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a41400640 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a12ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a41400640 1 -- 172.21.15.107:0/638151121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a15a13a10 con 0x560a15a0de40 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a3be00640 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a12ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a41400640 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0de40 0x560a15a21710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009b30 tx=0x7f8a3402ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c556ba20ec540b73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.022+0000 7f8a22000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a22000640 1 -- 172.21.15.107:0/638151121 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34041070 con 0x560a15a0de40 2024-09-19T04:31:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a22000640 1 -- 172.21.15.107:0/638151121 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f8a3403d460 con 0x560a15a0de40 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a45fd7740 1 -- 172.21.15.107:0/638151121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a34030bf0 con 0x560a15a0de40 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a22000640 1 -- 172.21.15.107:0/638151121 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8a34036040 con 0x560a15a0de40 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a45fd7740 10 client.27158 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a45fd7740 1 -- 172.21.15.107:0/638151121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560a15a938c0 con 0x560a15a0de40 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a22000640 1 -- 172.21.15.107:0/638151121 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a34030bf0 con 0x560a15a0de40 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a22000640 1 -- 172.21.15.107:0/638151121 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f8a340043f0 con 0x560a15a0de40 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a45fd7740 10 client.27158 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a45fd7740 10 client.27158 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a45fd7740 20 client.27158 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x560a15a93ca0 0x560a15a96090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a45fd7740 4 client.27158 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.023+0000 7f8a45fd7740 1 -- 172.21.15.107:0/638151121 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a1597be60 con 0x560a15a93ca0 2024-09-19T04:31:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.024+0000 7f8a3be00640 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x560a15a93ca0 0x560a15a96090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.024+0000 7f8a3be00640 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x560a15a93ca0 0x560a15a96090 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c7b4bcab36ddd15 server_cookie=3763ec5f24d7ee39 in_seq=0 out_seq=0 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.024+0000 7f8a22000640 10 client.27158 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.024+0000 7f8a22000640 1 -- 172.21.15.107:0/638151121 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a1597be60 con 0x560a15a93ca0 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.024+0000 7f8a22000640 10 client.27158 handle_command_reply: tid=0 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.024+0000 7f8a45fd7740 1 client.27158 shutdown 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 1 -- 172.21.15.107:0/638151121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0de40 msgr2=0x560a15a21710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0de40 0x560a15a21710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009b30 tx=0x7f8a3402ae20 comp rx=0 tx=0).stop 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 1 -- 172.21.15.107:0/638151121 shutdown_connections 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x560a15a93ca0 0x560a15a96090 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560a15a19020 0x560a15a131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a15a0e760 0x560a15a12ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 1 --2- 172.21.15.107:0/638151121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a15a0de40 0x560a15a21710 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a22000640 0 client.27158 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:31:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 1 -- 172.21.15.107:0/638151121 >> 172.21.15.107:0/638151121 conn(0x560a15a0a3d0 msgr2=0x560a15a100b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 1 -- 172.21.15.107:0/638151121 shutdown_connections 2024-09-19T04:31:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 1 -- 172.21.15.107:0/638151121 wait complete. 2024-09-19T04:31:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.025+0000 7f8a45fd7740 20 client.27158 trim_cache size 0 max 16384 2024-09-19T04:31:02.031 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:31:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 -- 172.21.15.107:0/2741375548 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a240a33b0 msgr2=0x7f8a240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a240a33b0 0x7f8a240a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a300cd670 tx=0x7f8a300d39a0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 -- 172.21.15.107:0/2741375548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a3c1019d0 msgr2=0x7f8a3c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a3c1019d0 0x7f8a3c19a4b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8a30005fd0 tx=0x7f8a3000ee50 comp rx=0 tx=0).stop 2024-09-19T04:31:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 -- 172.21.15.107:0/2741375548 shutdown_connections 2024-09-19T04:31:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8a240a33b0 0x7f8a240a5870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a3c1079d0 0x7f8a3c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a3c102370 0x7f8a3c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 --2- 172.21.15.107:0/2741375548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a3c1019d0 0x7f8a3c19a4b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:02.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 -- 172.21.15.107:0/2741375548 >> 172.21.15.107:0/2741375548 conn(0x7f8a3c0fd710 msgr2=0x7f8a3c10d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:02.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 -- 172.21.15.107:0/2741375548 shutdown_connections 2024-09-19T04:31:02.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:02.026+0000 7f8a41e00640 1 -- 172.21.15.107:0/2741375548 wait complete. 2024-09-19T04:31:02.720 INFO:tasks.workunit.client.0.smithi107.stdout: 1 358789 2.82 MB/sec execute 440 sec latency 1219.663 ms 2024-09-19T04:31:02.899 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:31:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:02 smithi115 ceph-mon[35477]: pgmap v332: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 176 op/s 2024-09-19T04:31:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:02 smithi145 ceph-mon[35096]: pgmap v332: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 176 op/s 2024-09-19T04:31:03.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:02 smithi107 ceph-mon[30659]: pgmap v332: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 176 op/s 2024-09-19T04:31:03.265 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:03.720 INFO:tasks.workunit.client.0.smithi107.stdout: 1 358789 2.82 MB/sec execute 441 sec latency 1769.527 ms 2024-09-19T04:31:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.824+0000 7f6522e00640 1 -- 172.21.15.107:0/2791672979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c102790 msgr2=0x7f651c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.824+0000 7f6522e00640 1 --2- 172.21.15.107:0/2791672979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c102790 0x7f651c1067e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6508009900 tx=0x7f650802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.824+0000 7f6522e00640 1 -- 172.21.15.107:0/2791672979 shutdown_connections 2024-09-19T04:31:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.824+0000 7f6522e00640 1 --2- 172.21.15.107:0/2791672979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f651c106f10 0x7f651c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.824+0000 7f6522e00640 1 --2- 172.21.15.107:0/2791672979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c102790 0x7f651c1067e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.824+0000 7f6522e00640 1 --2- 172.21.15.107:0/2791672979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f651c101de0 0x7f651c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.824+0000 7f6522e00640 1 -- 172.21.15.107:0/2791672979 >> 172.21.15.107:0/2791672979 conn(0x7f651c0fd8c0 msgr2=0x7f651c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.824+0000 7f6522e00640 1 -- 172.21.15.107:0/2791672979 shutdown_connections 2024-09-19T04:31:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.824+0000 7f6522e00640 1 -- 172.21.15.107:0/2791672979 wait complete. 2024-09-19T04:31:03.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.825+0000 7f6522e00640 1 Processor -- start 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.825+0000 7f6522e00640 1 -- start start 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f6522e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c101de0 0x7f651c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f6522e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f651c102790 0x7f651c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f6522e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f651c106f10 0x7f651c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f6522e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f651c111fc0 con 0x7f651c101de0 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f6522e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f651c111e40 con 0x7f651c106f10 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f6522e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f651c112140 con 0x7f651c102790 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f651be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c101de0 0x7f651c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f651be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c101de0 0x7f651c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60964/0 (socket says 172.21.15.107:60964) 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f651be00640 1 -- 172.21.15.107:0/1011950077 learned_addr learned my addr 172.21.15.107:0/1011950077 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:03.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f651b400640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f651c102790 0x7f651c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f6520e00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f651c106f10 0x7f651c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f651be00640 1 -- 172.21.15.107:0/1011950077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f651c102790 msgr2=0x7f651c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f651be00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f651c102790 0x7f651c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f651be00640 1 -- 172.21.15.107:0/1011950077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f651c106f10 msgr2=0x7f651c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f651be00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f651c106f10 0x7f651c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.826+0000 7f651be00640 1 -- 172.21.15.107:0/1011950077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651c19f5e0 con 0x7f651c101de0 2024-09-19T04:31:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.827+0000 7f651be00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c101de0 0x7f651c19a630 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f650c00e9f0 tx=0x7f650c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.827+0000 7f6518c00640 1 -- 172.21.15.107:0/1011950077 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c00cd80 con 0x7f651c101de0 2024-09-19T04:31:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.827+0000 7f6522e00640 1 -- 172.21.15.107:0/1011950077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651c19f8d0 con 0x7f651c101de0 2024-09-19T04:31:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.827+0000 7f6518c00640 1 -- 172.21.15.107:0/1011950077 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f650c010910 con 0x7f651c101de0 2024-09-19T04:31:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.827+0000 7f6518c00640 1 -- 172.21.15.107:0/1011950077 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c0188f0 con 0x7f651c101de0 2024-09-19T04:31:03.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.827+0000 7f6522e00640 1 -- 172.21.15.107:0/1011950077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651c1a71b0 con 0x7f651c101de0 2024-09-19T04:31:03.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.830+0000 7f65257cf740 1 Processor -- start 2024-09-19T04:31:03.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.830+0000 7f6518c00640 1 -- 172.21.15.107:0/1011950077 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f650c010ac0 con 0x7f651c101de0 2024-09-19T04:31:03.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.830+0000 7f65257cf740 1 -- start start 2024-09-19T04:31:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f65257cf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 0x559c84b1d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f65257cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b27910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f65257cf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b27e50 0x559c84b2a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f65257cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559c84a833e0 con 0x559c84b1d590 2024-09-19T04:31:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f65257cf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c84a2ad80 con 0x559c84b1cc70 2024-09-19T04:31:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f65257cf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559c84a7f3a0 con 0x559c84b27e50 2024-09-19T04:31:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f651be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b27910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f651be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b27910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60978/0 (socket says 172.21.15.107:60978) 2024-09-19T04:31:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f651be00640 1 -- 172.21.15.107:0/500911004 learned_addr learned my addr 172.21.15.107:0/500911004 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f6520e00640 1 --2- 172.21.15.107:0/500911004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 0x559c84b1d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.831+0000 7f651b400640 1 --2- 172.21.15.107:0/500911004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b27e50 0x559c84b2a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f651be00640 1 -- 172.21.15.107:0/500911004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b27e50 msgr2=0x559c84b2a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f651be00640 1 --2- 172.21.15.107:0/500911004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b27e50 0x559c84b2a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f651be00640 1 -- 172.21.15.107:0/500911004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 msgr2=0x559c84b1d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f651be00640 1 --2- 172.21.15.107:0/500911004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 0x559c84b1d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f651be00640 1 -- 172.21.15.107:0/500911004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c84a49ae0 con 0x559c84b1d590 2024-09-19T04:31:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f6518c00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f65000a33b0 0x7f65000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f651be00640 1 --2- 172.21.15.107:0/500911004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b27910 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f650c0cd740 tx=0x7f650c0d8780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d64762d0784e876 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f6520e00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f65000a33b0 0x7f65000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f6518c00640 1 -- 172.21.15.107:0/1011950077 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f650c0d00f0 con 0x7f651c101de0 2024-09-19T04:31:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f6511600640 1 -- 172.21.15.107:0/500911004 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c0986e0 con 0x559c84b1d590 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f6511600640 1 -- 172.21.15.107:0/500911004 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f650c09c440 con 0x559c84b1d590 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.832+0000 7f6511600640 1 -- 172.21.15.107:0/500911004 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c0d3c90 con 0x559c84b1d590 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.833+0000 7f65257cf740 1 -- 172.21.15.107:0/500911004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 msgr2=0x559c84b27910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.833+0000 7f6520e00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f65000a33b0 0x7f65000a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6514005fd0 tx=0x7f6514009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.833+0000 7f65257cf740 1 --2- 172.21.15.107:0/500911004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b27910 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f650c0cd740 tx=0x7f650c0d8780 comp rx=0 tx=0).stop 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.833+0000 7f65257cf740 1 -- 172.21.15.107:0/500911004 shutdown_connections 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.833+0000 7f65257cf740 1 --2- 172.21.15.107:0/500911004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b27e50 0x559c84b2a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.833+0000 7f65257cf740 1 --2- 172.21.15.107:0/500911004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b27910 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.833+0000 7f65257cf740 1 --2- 172.21.15.107:0/500911004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 0x559c84b1d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.833+0000 7f65257cf740 1 -- 172.21.15.107:0/500911004 >> 172.21.15.107:0/500911004 conn(0x559c84b19200 msgr2=0x559c84b195e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.833+0000 7f65257cf740 1 -- 172.21.15.107:0/500911004 shutdown_connections 2024-09-19T04:31:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.833+0000 7f65257cf740 1 -- 172.21.15.107:0/500911004 wait complete. 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.834+0000 7f65257cf740 1 -- start start 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.834+0000 7f65257cf740 1 Processor -- start 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f65257cf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b1cc70 0x559c84b24d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f65257cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f65257cf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 0x559c84b25800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f65257cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559c84b2a9c0 con 0x559c84b1d590 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f65257cf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c84a3e0f0 con 0x559c84b27e50 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f65257cf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559c84b2ab40 con 0x559c84b1cc70 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f65257cf740 10 client.0 _ll_register_callbacks cb 0x559c84a25bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f651b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f651b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60984/0 (socket says 172.21.15.107:60984) 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f651b400640 1 -- 172.21.15.107:0/599621086 learned_addr learned my addr 172.21.15.107:0/599621086 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f651be00640 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b1cc70 0x559c84b24d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f6520e00640 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 0x559c84b25800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f6520e00640 1 -- 172.21.15.107:0/599621086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b1cc70 msgr2=0x559c84b24d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f6520e00640 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b1cc70 0x559c84b24d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f6520e00640 1 -- 172.21.15.107:0/599621086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 msgr2=0x559c84b252c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f6520e00640 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b252c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.835+0000 7f6520e00640 1 -- 172.21.15.107:0/599621086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c847f43c0 con 0x559c84b27e50 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.836+0000 7f6520e00640 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 0x559c84b25800 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6514011dd0 tx=0x7f6514004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ad7f2a8e39a55b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.836+0000 7f6506000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.836+0000 7f6506000640 1 -- 172.21.15.107:0/599621086 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514004b20 con 0x559c84b27e50 2024-09-19T04:31:03.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.836+0000 7f65257cf740 1 -- 172.21.15.107:0/599621086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f650c09c440 con 0x559c84b27e50 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.836+0000 7f6506000640 1 -- 172.21.15.107:0/599621086 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f6514004cc0 con 0x559c84b27e50 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.836+0000 7f6506000640 1 -- 172.21.15.107:0/599621086 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f651400f810 con 0x559c84b27e50 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.836+0000 7f65257cf740 10 client.27170 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.836+0000 7f65257cf740 1 -- 172.21.15.107:0/599621086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x559c84b36b00 con 0x559c84b27e50 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.837+0000 7f6506000640 1 -- 172.21.15.107:0/599621086 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f650c09c440 con 0x559c84b27e50 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.837+0000 7f6506000640 1 -- 172.21.15.107:0/599621086 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f651402e070 con 0x559c84b27e50 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.837+0000 7f65257cf740 10 client.27170 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.837+0000 7f65257cf740 10 client.27170 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.837+0000 7f65257cf740 20 client.27170 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.837+0000 7f65257cf740 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x559c84b381f0 0x559c84b3a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.837+0000 7f65257cf740 4 client.27170 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.837+0000 7f65257cf740 1 -- 172.21.15.107:0/599621086 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c84a90950 con 0x559c84b381f0 2024-09-19T04:31:03.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.837+0000 7f651be00640 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x559c84b381f0 0x559c84b3a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.838+0000 7f651be00640 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x559c84b381f0 0x559c84b3a630 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52319da3a60396ab server_cookie=abdc3db5e140b616 in_seq=0 out_seq=0 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.838+0000 7f6506000640 10 client.27170 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f6506000640 1 -- 172.21.15.107:0/599621086 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c84a90950 con 0x559c84b381f0 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f6506000640 10 client.27170 handle_command_reply: tid=0 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 client.27170 shutdown 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 -- 172.21.15.107:0/599621086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 msgr2=0x559c84b25800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 0x559c84b25800 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6514011dd0 tx=0x7f6514004ab0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 -- 172.21.15.107:0/599621086 shutdown_connections 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x559c84b381f0 0x559c84b3a630 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 0x559c84b25800 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b252c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 --2- 172.21.15.107:0/599621086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b1cc70 0x559c84b24d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f6506000640 0 client.27170 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:31:03.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 -- 172.21.15.107:0/599621086 >> 172.21.15.107:0/599621086 conn(0x559c84b19200 msgr2=0x559c84b1ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:03.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 -- 172.21.15.107:0/599621086 shutdown_connections 2024-09-19T04:31:03.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 1 -- 172.21.15.107:0/599621086 wait complete. 2024-09-19T04:31:03.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.839+0000 7f65257cf740 20 client.27170 trim_cache size 0 max 16384 2024-09-19T04:31:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f65257cf740 1 Processor -- start 2024-09-19T04:31:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f65257cf740 1 -- start start 2024-09-19T04:31:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f65257cf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 0x559c84b26cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f65257cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b3ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f65257cf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b27e50 0x559c84b40480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f65257cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1af9e0 con 0x559c84b1d590 2024-09-19T04:31:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f65257cf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1b1520 con 0x559c84b1cc70 2024-09-19T04:31:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f65257cf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1b00b0 con 0x559c84b27e50 2024-09-19T04:31:03.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f651b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b3ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f651be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 0x559c84b26cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f651b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b3ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60992/0 (socket says 172.21.15.107:60992) 2024-09-19T04:31:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f651b400640 1 -- 172.21.15.107:0/2847375438 learned_addr learned my addr 172.21.15.107:0/2847375438 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:03.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.853+0000 7f6520e00640 1 --2- 172.21.15.107:0/2847375438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b27e50 0x559c84b40480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.854+0000 7f651be00640 1 -- 172.21.15.107:0/2847375438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b27e50 msgr2=0x559c84b40480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:03.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.854+0000 7f651be00640 1 --2- 172.21.15.107:0/2847375438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b27e50 0x559c84b40480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.854+0000 7f651be00640 1 -- 172.21.15.107:0/2847375438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 msgr2=0x559c84b3ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:03.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.854+0000 7f651be00640 1 --2- 172.21.15.107:0/2847375438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b3ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.854+0000 7f651be00640 1 -- 172.21.15.107:0/2847375438 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c84b19030 con 0x559c84b1cc70 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.854+0000 7f651be00640 1 --2- 172.21.15.107:0/2847375438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 0x559c84b26cf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f650c0cd740 tx=0x7f650c0980d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14e25997dd470503 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.854+0000 7f6510c00640 1 -- 172.21.15.107:0/2847375438 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c09bda0 con 0x559c84b1cc70 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.854+0000 7f6510c00640 1 -- 172.21.15.107:0/2847375438 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f650c0a0740 con 0x559c84b1cc70 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.854+0000 7f6510c00640 1 -- 172.21.15.107:0/2847375438 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c09bf80 con 0x559c84b1cc70 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.855+0000 7f65257cf740 1 -- 172.21.15.107:0/2847375438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 msgr2=0x559c84b26cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.855+0000 7f65257cf740 1 --2- 172.21.15.107:0/2847375438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 0x559c84b26cf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f650c0cd740 tx=0x7f650c0980d0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.855+0000 7f65257cf740 1 -- 172.21.15.107:0/2847375438 shutdown_connections 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.855+0000 7f65257cf740 1 --2- 172.21.15.107:0/2847375438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b27e50 0x559c84b40480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.855+0000 7f65257cf740 1 --2- 172.21.15.107:0/2847375438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1d590 0x559c84b3ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.855+0000 7f65257cf740 1 --2- 172.21.15.107:0/2847375438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b1cc70 0x559c84b26cf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.855+0000 7f65257cf740 1 -- 172.21.15.107:0/2847375438 >> 172.21.15.107:0/2847375438 conn(0x559c84b19200 msgr2=0x559c84b265e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.855+0000 7f65257cf740 1 -- 172.21.15.107:0/2847375438 shutdown_connections 2024-09-19T04:31:03.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.855+0000 7f65257cf740 1 -- 172.21.15.107:0/2847375438 wait complete. 2024-09-19T04:31:03.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.855+0000 7f65257cf740 1 -- start start 2024-09-19T04:31:03.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f65257cf740 1 Processor -- start 2024-09-19T04:31:03.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f65257cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1cc70 0x559c84b2bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f65257cf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b1d590 0x559c84b2c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f65257cf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 0x559c84b2c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f65257cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1b0e50 con 0x559c84b1cc70 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f65257cf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1b0780 con 0x559c84b27e50 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f65257cf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1ad9e0 con 0x559c84b1d590 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f65257cf740 10 client.0 _ll_register_callbacks cb 0x7f6514009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f651be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1cc70 0x559c84b2bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f651be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1cc70 0x559c84b2bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32772/0 (socket says 172.21.15.107:32772) 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f651be00640 1 -- 172.21.15.107:0/4267086014 learned_addr learned my addr 172.21.15.107:0/4267086014 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.856+0000 7f651b400640 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b1d590 0x559c84b2c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f6520e00640 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 0x559c84b2c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f651be00640 1 -- 172.21.15.107:0/4267086014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b1d590 msgr2=0x559c84b2c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f651be00640 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b1d590 0x559c84b2c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f651be00640 1 -- 172.21.15.107:0/4267086014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 msgr2=0x559c84b2c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f651be00640 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 0x559c84b2c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f651be00640 1 -- 172.21.15.107:0/4267086014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c84b19e90 con 0x559c84b1cc70 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f651be00640 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1cc70 0x559c84b2bda0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f650c0a0900 tx=0x7f650c0cd800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d477e068c2a6fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f6506000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f6506000640 1 -- 172.21.15.107:0/4267086014 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c09c210 con 0x559c84b1cc70 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f6506000640 1 -- 172.21.15.107:0/4267086014 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f650c0b51f0 con 0x559c84b1cc70 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.857+0000 7f65257cf740 1 -- 172.21.15.107:0/4267086014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c84b209a0 con 0x559c84b1cc70 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f6506000640 1 -- 172.21.15.107:0/4267086014 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f650c09bf80 con 0x559c84b1cc70 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f65257cf740 10 client.17127 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f65257cf740 1 -- 172.21.15.107:0/4267086014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x559c84b14770 con 0x559c84b1cc70 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f6506000640 1 -- 172.21.15.107:0/4267086014 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559c84b209a0 con 0x559c84b1cc70 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f6506000640 1 -- 172.21.15.107:0/4267086014 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f650c0a4c00 con 0x559c84b1cc70 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f65257cf740 10 client.17127 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f65257cf740 10 client.17127 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f65257cf740 20 client.17127 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:03.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f65257cf740 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x559c84ba32e0 0x559c84ba56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f65257cf740 4 client.17127 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.858+0000 7f65257cf740 1 -- 172.21.15.107:0/4267086014 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c84a8ae60 con 0x559c84ba32e0 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.859+0000 7f651b400640 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x559c84ba32e0 0x559c84ba56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.859+0000 7f651b400640 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x559c84ba32e0 0x559c84ba56d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9edd89bbc98772d8 server_cookie=8fb15e994cb1cdc in_seq=0 out_seq=0 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.859+0000 7f6506000640 10 client.17127 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.859+0000 7f6506000640 1 -- 172.21.15.107:0/4267086014 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c84a8ae60 con 0x559c84ba32e0 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.859+0000 7f6506000640 10 client.17127 handle_command_reply: tid=0 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.859+0000 7f65257cf740 1 client.17127 shutdown 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.859+0000 7f65257cf740 1 -- 172.21.15.107:0/4267086014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1cc70 msgr2=0x559c84b2bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.859+0000 7f65257cf740 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1cc70 0x559c84b2bda0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f650c0a0900 tx=0x7f650c0cd800 comp rx=0 tx=0).stop 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f65257cf740 1 -- 172.21.15.107:0/4267086014 shutdown_connections 2024-09-19T04:31:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f65257cf740 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x559c84ba32e0 0x559c84ba56d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f65257cf740 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c84b27e50 0x559c84b2c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f65257cf740 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c84b1d590 0x559c84b2c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f65257cf740 1 --2- 172.21.15.107:0/4267086014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c84b1cc70 0x559c84b2bda0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f6506000640 0 client.17127 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:31:03.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f65257cf740 1 -- 172.21.15.107:0/4267086014 >> 172.21.15.107:0/4267086014 conn(0x559c84b19200 msgr2=0x559c84b25560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:03.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f65257cf740 1 -- 172.21.15.107:0/4267086014 shutdown_connections 2024-09-19T04:31:03.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f65257cf740 1 -- 172.21.15.107:0/4267086014 wait complete. 2024-09-19T04:31:03.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f65257cf740 20 client.17127 trim_cache size 0 max 16384 2024-09-19T04:31:03.865 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:31:03.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f6522e00640 1 -- 172.21.15.107:0/1011950077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f65000a33b0 msgr2=0x7f65000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:03.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f6522e00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f65000a33b0 0x7f65000a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6514005fd0 tx=0x7f6514009500 comp rx=0 tx=0).stop 2024-09-19T04:31:03.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f6522e00640 1 -- 172.21.15.107:0/1011950077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c101de0 msgr2=0x7f651c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:03.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.860+0000 7f6522e00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c101de0 0x7f651c19a630 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f650c00e9f0 tx=0x7f650c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.861+0000 7f6522e00640 1 -- 172.21.15.107:0/1011950077 shutdown_connections 2024-09-19T04:31:03.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.861+0000 7f6522e00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f65000a33b0 0x7f65000a5870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.861+0000 7f6522e00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f651c106f10 0x7f651c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.861+0000 7f6522e00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f651c102790 0x7f651c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.861+0000 7f6522e00640 1 --2- 172.21.15.107:0/1011950077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f651c101de0 0x7f651c19a630 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:03.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.861+0000 7f6522e00640 1 -- 172.21.15.107:0/1011950077 >> 172.21.15.107:0/1011950077 conn(0x7f651c0fd8c0 msgr2=0x7f651c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:03.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.861+0000 7f6522e00640 1 -- 172.21.15.107:0/1011950077 shutdown_connections 2024-09-19T04:31:03.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:03.861+0000 7f6522e00640 1 -- 172.21.15.107:0/1011950077 wait complete. 2024-09-19T04:31:04.720 INFO:tasks.workunit.client.0.smithi107.stdout: 1 358789 2.81 MB/sec execute 442 sec latency 2769.627 ms 2024-09-19T04:31:04.860 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:31:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:04 smithi115 ceph-mon[35477]: pgmap v333: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-09-19T04:31:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:04 smithi145 ceph-mon[35096]: pgmap v333: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-09-19T04:31:05.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:04 smithi107 ceph-mon[30659]: pgmap v333: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-09-19T04:31:05.231 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:05.720 INFO:tasks.workunit.client.0.smithi107.stdout: 1 358789 2.80 MB/sec execute 443 sec latency 3769.691 ms 2024-09-19T04:31:05.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.811+0000 7f2416600640 1 -- 172.21.15.107:0/3487341284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410076720 msgr2=0x7f2410076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:05.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.811+0000 7f2416600640 1 --2- 172.21.15.107:0/3487341284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410076720 0x7f2410076b00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2404009900 tx=0x7f240402a410 comp rx=0 tx=0).stop 2024-09-19T04:31:05.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.811+0000 7f2416600640 1 -- 172.21.15.107:0/3487341284 shutdown_connections 2024-09-19T04:31:05.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.811+0000 7f2416600640 1 --2- 172.21.15.107:0/3487341284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2410112870 0x7f2410114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.811+0000 7f2416600640 1 --2- 172.21.15.107:0/3487341284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2410077040 0x7f2410112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.811+0000 7f2416600640 1 --2- 172.21.15.107:0/3487341284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410076720 0x7f2410076b00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.811+0000 7f2416600640 1 -- 172.21.15.107:0/3487341284 >> 172.21.15.107:0/3487341284 conn(0x7f24100ff910 msgr2=0x7f2410101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:05.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.811+0000 7f2416600640 1 -- 172.21.15.107:0/3487341284 shutdown_connections 2024-09-19T04:31:05.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.811+0000 7f2416600640 1 -- 172.21.15.107:0/3487341284 wait complete. 2024-09-19T04:31:05.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.812+0000 7f2416600640 1 Processor -- start 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.812+0000 7f2416600640 1 -- start start 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.812+0000 7f2416600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2410076720 0x7f24101a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.812+0000 7f2416600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410077040 0x7f24101a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.812+0000 7f2416600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2410112870 0x7f241019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.812+0000 7f2416600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24101183e0 con 0x7f2410077040 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2416600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2410118260 con 0x7f2410076720 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2416600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2410118560 con 0x7f2410112870 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2415200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2410076720 0x7f24101a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2415200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2410076720 0x7f24101a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:41648/0 (socket says 172.21.15.107:41648) 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2415200640 1 -- 172.21.15.107:0/358159562 learned_addr learned my addr 172.21.15.107:0/358159562 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:05.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f240fe00640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410077040 0x7f24101a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2415c00640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2410112870 0x7f241019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2415200640 1 -- 172.21.15.107:0/358159562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2410112870 msgr2=0x7f241019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2415200640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2410112870 0x7f241019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2415200640 1 -- 172.21.15.107:0/358159562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410077040 msgr2=0x7f24101a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2415200640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410077040 0x7f24101a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.813+0000 7f2415200640 1 -- 172.21.15.107:0/358159562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241019ef20 con 0x7f2410076720 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.814+0000 7f2415200640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2410076720 0x7f24101a4600 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2404006010 tx=0x7f2404004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.814+0000 7f240d600640 1 -- 172.21.15.107:0/358159562 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240403f030 con 0x7f2410076720 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.814+0000 7f2416600640 1 -- 172.21.15.107:0/358159562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241019f1b0 con 0x7f2410076720 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.814+0000 7f240d600640 1 -- 172.21.15.107:0/358159562 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2404004440 con 0x7f2410076720 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.814+0000 7f240d600640 1 -- 172.21.15.107:0/358159562 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404035070 con 0x7f2410076720 2024-09-19T04:31:05.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.814+0000 7f2416600640 1 -- 172.21.15.107:0/358159562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101ab460 con 0x7f2410076720 2024-09-19T04:31:05.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.817+0000 7f241a662740 1 Processor -- start 2024-09-19T04:31:05.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.817+0000 7f241a662740 1 -- start start 2024-09-19T04:31:05.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.817+0000 7f240d600640 1 -- 172.21.15.107:0/358159562 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f2404042030 con 0x7f2410076720 2024-09-19T04:31:05.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.817+0000 7f241a662740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae523220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.817+0000 7f241a662740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f241a662740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 0x5633ae530410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f241a662740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5633ae4893e0 con 0x5633ae52e020 2024-09-19T04:31:05.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f241a662740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5633ae430d80 con 0x5633ae523760 2024-09-19T04:31:05.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f241a662740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5633ae4853a0 con 0x5633ae522e40 2024-09-19T04:31:05.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f240fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 0x5633ae530410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f2415c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae523220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f2415200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f240fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 0x5633ae530410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32806/0 (socket says 172.21.15.107:32806) 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f240fe00640 1 -- 172.21.15.107:0/822794723 learned_addr learned my addr 172.21.15.107:0/822794723 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f2415c00640 1 -- 172.21.15.107:0/822794723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 msgr2=0x5633ae52dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f2415c00640 1 --2- 172.21.15.107:0/822794723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f2415c00640 1 -- 172.21.15.107:0/822794723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 msgr2=0x5633ae530410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f2415c00640 1 --2- 172.21.15.107:0/822794723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 0x5633ae530410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f2415c00640 1 -- 172.21.15.107:0/822794723 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633ae44fae0 con 0x5633ae522e40 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.818+0000 7f240fe00640 1 --2- 172.21.15.107:0/822794723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 0x5633ae530410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.819+0000 7f2415c00640 1 --2- 172.21.15.107:0/822794723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae523220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f241007aa10 tx=0x7f2408011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1edcbedee17baac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.819+0000 7f240d600640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f23f80a33b0 0x7f23f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.819+0000 7f2402000640 1 -- 172.21.15.107:0/822794723 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408004d20 con 0x5633ae522e40 2024-09-19T04:31:05.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.819+0000 7f2402000640 1 -- 172.21.15.107:0/822794723 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2408004050 con 0x5633ae522e40 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.819+0000 7f2402000640 1 -- 172.21.15.107:0/822794723 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408005280 con 0x5633ae522e40 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.819+0000 7f240d600640 1 -- 172.21.15.107:0/358159562 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f24040f0040 con 0x7f2410076720 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.820+0000 7f2415200640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f23f80a33b0 0x7f23f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.820+0000 7f241a662740 1 -- 172.21.15.107:0/822794723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 msgr2=0x5633ae523220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.820+0000 7f241a662740 1 --2- 172.21.15.107:0/822794723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae523220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f241007aa10 tx=0x7f2408011810 comp rx=0 tx=0).stop 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.820+0000 7f241a662740 1 -- 172.21.15.107:0/822794723 shutdown_connections 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.820+0000 7f241a662740 1 --2- 172.21.15.107:0/822794723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 0x5633ae530410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.820+0000 7f241a662740 1 --2- 172.21.15.107:0/822794723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.820+0000 7f241a662740 1 --2- 172.21.15.107:0/822794723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae523220 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.820+0000 7f241a662740 1 -- 172.21.15.107:0/822794723 >> 172.21.15.107:0/822794723 conn(0x5633ae51f3d0 msgr2=0x5633ae51f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.820+0000 7f241a662740 1 -- 172.21.15.107:0/822794723 shutdown_connections 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.821+0000 7f241a662740 1 -- 172.21.15.107:0/822794723 wait complete. 2024-09-19T04:31:05.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.821+0000 7f2415200640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f23f80a33b0 0x7f23f80a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f24040ed860 tx=0x7f24040f3b80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:05.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.821+0000 7f241a662740 1 -- start start 2024-09-19T04:31:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.821+0000 7f241a662740 1 Processor -- start 2024-09-19T04:31:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.822+0000 7f241a662740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae52aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.822+0000 7f241a662740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.822+0000 7f241a662740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 0x5633ae52b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.822+0000 7f241a662740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5633ae530bf0 con 0x5633ae52e020 2024-09-19T04:31:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.822+0000 7f241a662740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5633ae4440f0 con 0x5633ae523760 2024-09-19T04:31:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.822+0000 7f241a662740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5633ae530d70 con 0x5633ae522e40 2024-09-19T04:31:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.822+0000 7f241a662740 10 client.0 _ll_register_callbacks cb 0x5633ae42bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.823+0000 7f2415c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae52aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.823+0000 7f2415c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae52aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53422/0 (socket says 172.21.15.107:53422) 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.823+0000 7f2415c00640 1 -- 172.21.15.107:0/990268230 learned_addr learned my addr 172.21.15.107:0/990268230 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.823+0000 7f2415c00640 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 0x5633ae52b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.823+0000 7f2415c00640 1 -- 172.21.15.107:0/990268230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 msgr2=0x5633ae52b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.823+0000 7f240fe00640 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.824+0000 7f2415c00640 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.824+0000 7f2415c00640 1 -- 172.21.15.107:0/990268230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 msgr2=0x5633ae52b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.824+0000 7f2415c00640 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 0x5633ae52b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.824+0000 7f2415c00640 1 -- 172.21.15.107:0/990268230 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633ae1fa3c0 con 0x5633ae522e40 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.824+0000 7f240fe00640 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.824+0000 7f2415c00640 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae52aef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f24080053e0 tx=0x7f2408004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ee6fb2bc159a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:05.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.825+0000 7f23f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:31:05.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.826+0000 7f23f6a00640 1 -- 172.21.15.107:0/990268230 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240802b020 con 0x5633ae522e40 2024-09-19T04:31:05.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.826+0000 7f23f6a00640 1 -- 172.21.15.107:0/990268230 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f2408004400 con 0x5633ae522e40 2024-09-19T04:31:05.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.826+0000 7f241a662740 1 -- 172.21.15.107:0/990268230 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2408004050 con 0x5633ae522e40 2024-09-19T04:31:05.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.827+0000 7f23f6a00640 1 -- 172.21.15.107:0/990268230 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f240801d040 con 0x5633ae522e40 2024-09-19T04:31:05.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.827+0000 7f241a662740 10 client.27151 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:05.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.827+0000 7f241a662740 1 -- 172.21.15.107:0/990268230 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5633ae53cc10 con 0x5633ae522e40 2024-09-19T04:31:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.828+0000 7f23f6a00640 1 -- 172.21.15.107:0/990268230 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2408004050 con 0x5633ae522e40 2024-09-19T04:31:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.828+0000 7f23f6a00640 1 -- 172.21.15.107:0/990268230 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f2408023070 con 0x5633ae522e40 2024-09-19T04:31:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.829+0000 7f241a662740 10 client.27151 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.829+0000 7f241a662740 10 client.27151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.829+0000 7f241a662740 20 client.27151 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.829+0000 7f241a662740 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5633ae53dfc0 0x5633ae5403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.829+0000 7f241a662740 4 client.27151 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:05.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.829+0000 7f241a662740 1 -- 172.21.15.107:0/990268230 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633ae496950 con 0x5633ae53dfc0 2024-09-19T04:31:05.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.830+0000 7f240fe00640 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5633ae53dfc0 0x5633ae5403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.831+0000 7f240fe00640 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5633ae53dfc0 0x5633ae5403b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d19c382487e64f8c server_cookie=2a926cd73c062a1d in_seq=0 out_seq=0 2024-09-19T04:31:05.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.831+0000 7f23f6a00640 10 client.27151 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:05.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.832+0000 7f23f6a00640 1 -- 172.21.15.107:0/990268230 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633ae496950 con 0x5633ae53dfc0 2024-09-19T04:31:05.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.832+0000 7f23f6a00640 10 client.27151 handle_command_reply: tid=0 2024-09-19T04:31:05.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.832+0000 7f241a662740 1 client.27151 shutdown 2024-09-19T04:31:05.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.834+0000 7f241a662740 1 -- 172.21.15.107:0/990268230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 msgr2=0x5633ae52aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:05.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.834+0000 7f241a662740 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae52aef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f24080053e0 tx=0x7f2408004860 comp rx=0 tx=0).stop 2024-09-19T04:31:05.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.835+0000 7f241a662740 1 -- 172.21.15.107:0/990268230 shutdown_connections 2024-09-19T04:31:05.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.835+0000 7f241a662740 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5633ae53dfc0 0x5633ae5403b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.835+0000 7f241a662740 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae52e020 0x5633ae52b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.836+0000 7f241a662740 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.836+0000 7f241a662740 1 --2- 172.21.15.107:0/990268230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae522e40 0x5633ae52aef0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.836+0000 7f241a662740 1 -- 172.21.15.107:0/990268230 >> 172.21.15.107:0/990268230 conn(0x5633ae51f3d0 msgr2=0x5633ae5250b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:05.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.836+0000 7f23f6a00640 0 client.27151 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:05.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.841+0000 7f241a662740 1 -- 172.21.15.107:0/990268230 shutdown_connections 2024-09-19T04:31:05.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.842+0000 7f241a662740 1 -- 172.21.15.107:0/990268230 wait complete. 2024-09-19T04:31:05.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.842+0000 7f241a662740 20 client.27151 trim_cache size 0 max 16384 2024-09-19T04:31:05.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.865+0000 7f241a662740 1 Processor -- start 2024-09-19T04:31:05.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.866+0000 7f241a662740 1 -- start start 2024-09-19T04:31:05.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.866+0000 7f241a662740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae525810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.866+0000 7f241a662740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.866+0000 7f241a662740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae52e020 0x5633ae52ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.866+0000 7f241a662740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b3c90 con 0x5633ae522e40 2024-09-19T04:31:05.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.866+0000 7f241a662740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b57d0 con 0x5633ae523760 2024-09-19T04:31:05.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.866+0000 7f241a662740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b4360 con 0x5633ae52e020 2024-09-19T04:31:05.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.866+0000 7f2415c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae525810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.866+0000 7f2415c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae525810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32830/0 (socket says 172.21.15.107:32830) 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.866+0000 7f2415c00640 1 -- 172.21.15.107:0/3553173697 learned_addr learned my addr 172.21.15.107:0/3553173697 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.867+0000 7f240fe00640 1 --2- 172.21.15.107:0/3553173697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.867+0000 7f2415c00640 1 --2- 172.21.15.107:0/3553173697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae52e020 0x5633ae52ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.867+0000 7f2415c00640 1 -- 172.21.15.107:0/3553173697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae52e020 msgr2=0x5633ae52ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.867+0000 7f2415c00640 1 --2- 172.21.15.107:0/3553173697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae52e020 0x5633ae52ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.867+0000 7f2415c00640 1 -- 172.21.15.107:0/3553173697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 msgr2=0x5633ae52b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.867+0000 7f2415c00640 1 --2- 172.21.15.107:0/3553173697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.867+0000 7f2415c00640 1 -- 172.21.15.107:0/3553173697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633ae52d860 con 0x5633ae522e40 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.867+0000 7f2415c00640 1 --2- 172.21.15.107:0/3553173697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae525810 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2408004a90 tx=0x7f2408005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec56f450d053968 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.868+0000 7f2401600640 1 -- 172.21.15.107:0/3553173697 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240803f070 con 0x5633ae522e40 2024-09-19T04:31:05.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.868+0000 7f2401600640 1 -- 172.21.15.107:0/3553173697 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24080058a0 con 0x5633ae522e40 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.868+0000 7f241a662740 1 -- 172.21.15.107:0/3553173697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 msgr2=0x5633ae525810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.868+0000 7f241a662740 1 --2- 172.21.15.107:0/3553173697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae525810 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2408004a90 tx=0x7f2408005350 comp rx=0 tx=0).stop 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.869+0000 7f241a662740 1 -- 172.21.15.107:0/3553173697 shutdown_connections 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.869+0000 7f241a662740 1 --2- 172.21.15.107:0/3553173697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae52e020 0x5633ae52ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.869+0000 7f241a662740 1 --2- 172.21.15.107:0/3553173697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae52b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.869+0000 7f241a662740 1 --2- 172.21.15.107:0/3553173697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae525810 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.869+0000 7f241a662740 1 -- 172.21.15.107:0/3553173697 >> 172.21.15.107:0/3553173697 conn(0x5633ae51f3d0 msgr2=0x5633ae52c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.869+0000 7f241a662740 1 -- 172.21.15.107:0/3553173697 shutdown_connections 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.869+0000 7f241a662740 1 -- 172.21.15.107:0/3553173697 wait complete. 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.870+0000 7f241a662740 1 -- start start 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.870+0000 7f241a662740 1 Processor -- start 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.870+0000 7f241a662740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae530f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.870+0000 7f241a662740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae5314c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.870+0000 7f241a662740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae52e020 0x5633ae531a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.870+0000 7f241a662740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b5100 con 0x5633ae522e40 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.870+0000 7f241a662740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b4a30 con 0x5633ae523760 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.870+0000 7f241a662740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b1c90 con 0x5633ae52e020 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.870+0000 7f241a662740 10 client.0 _ll_register_callbacks cb 0x7f23fc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.871+0000 7f2415c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae530f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.871+0000 7f2415c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae530f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32840/0 (socket says 172.21.15.107:32840) 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.871+0000 7f2415c00640 1 -- 172.21.15.107:0/3442352803 learned_addr learned my addr 172.21.15.107:0/3442352803 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.871+0000 7f240fe00640 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae5314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.871+0000 7f2415c00640 1 -- 172.21.15.107:0/3442352803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae52e020 msgr2=0x5633ae531a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.871+0000 7f2415c00640 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae52e020 0x5633ae531a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.871+0000 7f2415c00640 1 -- 172.21.15.107:0/3442352803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 msgr2=0x5633ae5314c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.872+0000 7f2415c00640 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae5314c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.872+0000 7f2415c00640 1 -- 172.21.15.107:0/3442352803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633ae5a6ce0 con 0x5633ae522e40 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.872+0000 7f240fe00640 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae5314c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.872+0000 7f2415c00640 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae530f80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f24080199c0 tx=0x7f2408009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fda753a5f5d9c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.872+0000 7f23f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.872+0000 7f23f6a00640 1 -- 172.21.15.107:0/3442352803 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240803f040 con 0x5633ae522e40 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.872+0000 7f23f6a00640 1 -- 172.21.15.107:0/3442352803 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f2408019bf0 con 0x5633ae522e40 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.872+0000 7f241a662740 1 -- 172.21.15.107:0/3442352803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24080058a0 con 0x5633ae522e40 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.873+0000 7f23f6a00640 1 -- 172.21.15.107:0/3442352803 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2408004ac0 con 0x5633ae522e40 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.873+0000 7f241a662740 10 client.17145 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:05.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.873+0000 7f241a662740 1 -- 172.21.15.107:0/3442352803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5633ae5a7fe0 con 0x5633ae522e40 2024-09-19T04:31:05.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.873+0000 7f23f6a00640 1 -- 172.21.15.107:0/3442352803 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24080058a0 con 0x5633ae522e40 2024-09-19T04:31:05.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.874+0000 7f23f6a00640 1 -- 172.21.15.107:0/3442352803 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f240804b070 con 0x5633ae522e40 2024-09-19T04:31:05.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.874+0000 7f241a662740 10 client.17145 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:05.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.874+0000 7f241a662740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:05.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.874+0000 7f241a662740 20 client.17145 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:05.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.874+0000 7f241a662740 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5633ae5a8500 0x5633ae5aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:05.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.874+0000 7f241a662740 4 client.17145 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:05.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.874+0000 7f241a662740 1 -- 172.21.15.107:0/3442352803 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633ae490e60 con 0x5633ae5a8500 2024-09-19T04:31:05.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.875+0000 7f240fe00640 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5633ae5a8500 0x5633ae5aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.875+0000 7f240fe00640 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5633ae5a8500 0x5633ae5aa8f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52bd9d4af24e3426 server_cookie=25d24b8d8b73e97f in_seq=0 out_seq=0 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.875+0000 7f23f6a00640 10 client.17145 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.875+0000 7f23f6a00640 1 -- 172.21.15.107:0/3442352803 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633ae490e60 con 0x5633ae5a8500 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.875+0000 7f23f6a00640 10 client.17145 handle_command_reply: tid=0 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.875+0000 7f241a662740 1 client.17145 shutdown 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f241a662740 1 -- 172.21.15.107:0/3442352803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 msgr2=0x5633ae530f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f241a662740 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae530f80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f24080199c0 tx=0x7f2408009590 comp rx=0 tx=0).stop 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f241a662740 1 -- 172.21.15.107:0/3442352803 shutdown_connections 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f241a662740 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5633ae5a8500 0x5633ae5aa8f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f241a662740 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633ae52e020 0x5633ae531a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f241a662740 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633ae523760 0x5633ae5314c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f241a662740 1 --2- 172.21.15.107:0/3442352803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633ae522e40 0x5633ae530f80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f241a662740 1 -- 172.21.15.107:0/3442352803 >> 172.21.15.107:0/3442352803 conn(0x5633ae51f3d0 msgr2=0x5633ae52c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:05.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f23f6a00640 0 client.17145 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:05.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f241a662740 1 -- 172.21.15.107:0/3442352803 shutdown_connections 2024-09-19T04:31:05.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.876+0000 7f241a662740 1 -- 172.21.15.107:0/3442352803 wait complete. 2024-09-19T04:31:05.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.877+0000 7f241a662740 20 client.17145 trim_cache size 0 max 16384 2024-09-19T04:31:05.890 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:31:05.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.877+0000 7f2416600640 1 -- 172.21.15.107:0/358159562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f23f80a33b0 msgr2=0x7f23f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:05.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.877+0000 7f2416600640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f23f80a33b0 0x7f23f80a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f24040ed860 tx=0x7f24040f3b80 comp rx=0 tx=0).stop 2024-09-19T04:31:05.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.878+0000 7f2416600640 1 -- 172.21.15.107:0/358159562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2410076720 msgr2=0x7f24101a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:05.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.878+0000 7f2416600640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2410076720 0x7f24101a4600 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2404006010 tx=0x7f2404004040 comp rx=0 tx=0).stop 2024-09-19T04:31:05.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.878+0000 7f2416600640 1 -- 172.21.15.107:0/358159562 shutdown_connections 2024-09-19T04:31:05.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.878+0000 7f2416600640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f23f80a33b0 0x7f23f80a5870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.878+0000 7f2416600640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2410112870 0x7f241019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.878+0000 7f2416600640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410077040 0x7f24101a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.878+0000 7f2416600640 1 --2- 172.21.15.107:0/358159562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2410076720 0x7f24101a4600 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:05.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.878+0000 7f2416600640 1 -- 172.21.15.107:0/358159562 >> 172.21.15.107:0/358159562 conn(0x7f24100ff910 msgr2=0x7f2410101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:05.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.878+0000 7f2416600640 1 -- 172.21.15.107:0/358159562 shutdown_connections 2024-09-19T04:31:05.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:05.878+0000 7f2416600640 1 -- 172.21.15.107:0/358159562 wait complete. 2024-09-19T04:31:06.720 INFO:tasks.workunit.client.0.smithi107.stdout: 1 358789 2.80 MB/sec execute 444 sec latency 4769.815 ms 2024-09-19T04:31:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:06 smithi115 ceph-mon[35477]: pgmap v334: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-19T04:31:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:06 smithi145 ceph-mon[35096]: pgmap v334: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-19T04:31:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:06 smithi107 ceph-mon[30659]: pgmap v334: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-19T04:31:07.720 INFO:tasks.workunit.client.0.smithi107.stdout: 1 359500 2.80 MB/sec execute 445 sec latency 4923.151 ms 2024-09-19T04:31:08.301 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:31:08.302 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:31:08.673 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:08.720 INFO:tasks.workunit.client.0.smithi107.stdout: 1 359941 2.80 MB/sec execute 446 sec latency 543.716 ms 2024-09-19T04:31:09.137 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:08 smithi107 ceph-mon[30659]: pgmap v335: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-09-19T04:31:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:08 smithi115 ceph-mon[35477]: pgmap v335: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-09-19T04:31:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:08 smithi145 ceph-mon[35096]: pgmap v335: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-09-19T04:31:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.252+0000 7fa08d600640 1 -- 172.21.15.107:0/550448597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa088109140 msgr2=0x7fa088110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.252+0000 7fa08d600640 1 --2- 172.21.15.107:0/550448597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa088109140 0x7fa088110cc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa080009ca0 tx=0x7fa08002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.252+0000 7fa08d600640 1 -- 172.21.15.107:0/550448597 shutdown_connections 2024-09-19T04:31:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.252+0000 7fa08d600640 1 --2- 172.21.15.107:0/550448597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa088109140 0x7fa088110cc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.252+0000 7fa08d600640 1 --2- 172.21.15.107:0/550448597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0881049c0 0x7fa088108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.252+0000 7fa08d600640 1 --2- 172.21.15.107:0/550448597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa088104010 0x7fa0881043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.252+0000 7fa08d600640 1 -- 172.21.15.107:0/550448597 >> 172.21.15.107:0/550448597 conn(0x7fa0880ffa50 msgr2=0x7fa088101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.252+0000 7fa08d600640 1 -- 172.21.15.107:0/550448597 shutdown_connections 2024-09-19T04:31:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.252+0000 7fa08d600640 1 -- 172.21.15.107:0/550448597 wait complete. 2024-09-19T04:31:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.253+0000 7fa08d600640 1 Processor -- start 2024-09-19T04:31:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.253+0000 7fa08d600640 1 -- start start 2024-09-19T04:31:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.253+0000 7fa08d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa088104010 0x7fa08819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.253+0000 7fa08d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0881049c0 0x7fa08819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.253+0000 7fa08d600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa088109140 0x7fa08819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.253+0000 7fa08d600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa088114150 con 0x7fa088104010 2024-09-19T04:31:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.253+0000 7fa08d600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa088113fd0 con 0x7fa088109140 2024-09-19T04:31:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.253+0000 7fa08d600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881142d0 con 0x7fa0881049c0 2024-09-19T04:31:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa086000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0881049c0 0x7fa08819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa086a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa088104010 0x7fa08819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa086000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0881049c0 0x7fa08819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:52922/0 (socket says 172.21.15.107:52922) 2024-09-19T04:31:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa086000640 1 -- 172.21.15.107:0/1524632636 learned_addr learned my addr 172.21.15.107:0/1524632636 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa087400640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa088109140 0x7fa08819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa086000640 1 -- 172.21.15.107:0/1524632636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa088109140 msgr2=0x7fa08819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa086000640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa088109140 0x7fa08819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa086000640 1 -- 172.21.15.107:0/1524632636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa088104010 msgr2=0x7fa08819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa086000640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa088104010 0x7fa08819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa086000640 1 -- 172.21.15.107:0/1524632636 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08819f640 con 0x7fa0881049c0 2024-09-19T04:31:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa086000640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0881049c0 0x7fa08819abd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa07400e9f0 tx=0x7fa07400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.254+0000 7fa07f400640 1 -- 172.21.15.107:0/1524632636 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07400cd80 con 0x7fa0881049c0 2024-09-19T04:31:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.255+0000 7fa07f400640 1 -- 172.21.15.107:0/1524632636 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa074010910 con 0x7fa0881049c0 2024-09-19T04:31:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.255+0000 7fa07f400640 1 -- 172.21.15.107:0/1524632636 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074018990 con 0x7fa0881049c0 2024-09-19T04:31:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.255+0000 7fa08d600640 1 -- 172.21.15.107:0/1524632636 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08819f930 con 0x7fa0881049c0 2024-09-19T04:31:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.255+0000 7fa08d600640 1 -- 172.21.15.107:0/1524632636 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0881a7210 con 0x7fa0881049c0 2024-09-19T04:31:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.258+0000 7fa08ffca740 1 Processor -- start 2024-09-19T04:31:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.258+0000 7fa07f400640 1 -- 172.21.15.107:0/1524632636 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fa074010ac0 con 0x7fa0881049c0 2024-09-19T04:31:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.258+0000 7fa08ffca740 1 -- start start 2024-09-19T04:31:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.258+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464beac0 0x5562464beea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.258+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.258+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464c9ca0 0x5562464cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.258+0000 7fa08ffca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556246425290 con 0x5562464beac0 2024-09-19T04:31:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.258+0000 7fa08ffca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5562463ccb40 con 0x5562464bf3e0 2024-09-19T04:31:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.258+0000 7fa08ffca740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556246421250 con 0x5562464c9ca0 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38600/0 (socket says 172.21.15.107:38600) 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086a00640 1 -- 172.21.15.107:0/675451689 learned_addr learned my addr 172.21.15.107:0/675451689 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa087400640 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464beac0 0x5562464beea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086000640 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464c9ca0 0x5562464cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086a00640 1 -- 172.21.15.107:0/675451689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464c9ca0 msgr2=0x5562464cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086a00640 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464c9ca0 0x5562464cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086a00640 1 -- 172.21.15.107:0/675451689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464beac0 msgr2=0x5562464beea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086a00640 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464beac0 0x5562464beea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086a00640 1 -- 172.21.15.107:0/675451689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562461966a0 con 0x5562464bf3e0 2024-09-19T04:31:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086000640 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464c9ca0 0x5562464cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa087400640 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464beac0 0x5562464beea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.259+0000 7fa086a00640 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464c9760 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa078009cf0 tx=0x7fa078011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=765f91089ed9079f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.260+0000 7fa07f400640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa06c0a3430 0x7fa06c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.260+0000 7fa07f400640 1 -- 172.21.15.107:0/1524632636 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fa0740cff30 con 0x7fa0881049c0 2024-09-19T04:31:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.260+0000 7fa073400640 1 -- 172.21.15.107:0/675451689 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078004230 con 0x5562464bf3e0 2024-09-19T04:31:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.260+0000 7fa087400640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa06c0a3430 0x7fa06c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.260+0000 7fa073400640 1 -- 172.21.15.107:0/675451689 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0780043d0 con 0x5562464bf3e0 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.260+0000 7fa073400640 1 -- 172.21.15.107:0/675451689 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0780198c0 con 0x5562464bf3e0 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa08ffca740 1 -- 172.21.15.107:0/675451689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 msgr2=0x5562464c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa08ffca740 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464c9760 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa078009cf0 tx=0x7fa078011810 comp rx=0 tx=0).stop 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa087400640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa06c0a3430 0x7fa06c0a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa080004af0 tx=0x7fa080004a80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa08ffca740 1 -- 172.21.15.107:0/675451689 shutdown_connections 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa08ffca740 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464c9ca0 0x5562464cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa08ffca740 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464c9760 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa08ffca740 1 --2- 172.21.15.107:0/675451689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464beac0 0x5562464beea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa08ffca740 1 -- 172.21.15.107:0/675451689 >> 172.21.15.107:0/675451689 conn(0x5562464bb050 msgr2=0x5562464bb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa08ffca740 1 -- 172.21.15.107:0/675451689 shutdown_connections 2024-09-19T04:31:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa08ffca740 1 -- 172.21.15.107:0/675451689 wait complete. 2024-09-19T04:31:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.261+0000 7fa08ffca740 1 -- start start 2024-09-19T04:31:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa08ffca740 1 Processor -- start 2024-09-19T04:31:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 0x5562464c6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 0x5562464c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa08ffca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5562464cc910 con 0x5562464bf3e0 2024-09-19T04:31:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa08ffca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5562463dffa0 con 0x5562464c9ca0 2024-09-19T04:31:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa08ffca740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5562464cca90 con 0x5562464beac0 2024-09-19T04:31:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa08ffca740 10 client.0 _ll_register_callbacks cb 0x556246133ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa086000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 0x5562464c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa086a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa087400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 0x5562464c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa086a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:52944/0 (socket says 172.21.15.107:52944) 2024-09-19T04:31:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa086000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 0x5562464c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39068/0 (socket says 172.21.15.107:39068) 2024-09-19T04:31:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa086a00640 1 -- 172.21.15.107:0/1107134924 learned_addr learned my addr 172.21.15.107:0/1107134924 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.262+0000 7fa087400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 0x5562464c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38608/0 (socket says 172.21.15.107:38608) 2024-09-19T04:31:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.263+0000 7fa087400640 1 -- 172.21.15.107:0/1107134924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 msgr2=0x5562464c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.263+0000 7fa087400640 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.263+0000 7fa087400640 1 -- 172.21.15.107:0/1107134924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 msgr2=0x5562464c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.263+0000 7fa087400640 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 0x5562464c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.263+0000 7fa087400640 1 -- 172.21.15.107:0/1107134924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562464c7600 con 0x5562464c9ca0 2024-09-19T04:31:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.263+0000 7fa086a00640 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.263+0000 7fa086000640 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 0x5562464c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.263+0000 7fa087400640 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 0x5562464c6dd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa080006010 tx=0x7fa08002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f4570277277f9d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.263+0000 7fa067e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.264+0000 7fa067e00640 1 -- 172.21.15.107:0/1107134924 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080003e00 con 0x5562464c9ca0 2024-09-19T04:31:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.264+0000 7fa067e00640 1 -- 172.21.15.107:0/1107134924 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fa080004250 con 0x5562464c9ca0 2024-09-19T04:31:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.264+0000 7fa08ffca740 1 -- 172.21.15.107:0/1107134924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0780043d0 con 0x5562464c9ca0 2024-09-19T04:31:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.264+0000 7fa067e00640 1 -- 172.21.15.107:0/1107134924 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa080004450 con 0x5562464c9ca0 2024-09-19T04:31:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.264+0000 7fa08ffca740 10 client.27203 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.264+0000 7fa08ffca740 1 -- 172.21.15.107:0/1107134924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5562464d8820 con 0x5562464c9ca0 2024-09-19T04:31:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.264+0000 7fa067e00640 1 -- 172.21.15.107:0/1107134924 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0780043d0 con 0x5562464c9ca0 2024-09-19T04:31:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.264+0000 7fa067e00640 1 -- 172.21.15.107:0/1107134924 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fa08005a020 con 0x5562464c9ca0 2024-09-19T04:31:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.264+0000 7fa08ffca740 10 client.27203 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.265+0000 7fa08ffca740 10 client.27203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.265+0000 7fa08ffca740 20 client.27203 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:09.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.265+0000 7fa08ffca740 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5562464d9ce0 0x5562464dc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.265+0000 7fa08ffca740 4 client.27203 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:09.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.265+0000 7fa08ffca740 1 -- 172.21.15.107:0/1107134924 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556246432800 con 0x5562464d9ce0 2024-09-19T04:31:09.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.265+0000 7fa086a00640 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5562464d9ce0 0x5562464dc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.265+0000 7fa086a00640 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5562464d9ce0 0x5562464dc0f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9059ed1cbe567f59 server_cookie=6977482d066b986f in_seq=0 out_seq=0 2024-09-19T04:31:09.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.265+0000 7fa067e00640 10 client.27203 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa067e00640 1 -- 172.21.15.107:0/1107134924 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556246432800 con 0x5562464d9ce0 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa067e00640 10 client.27203 handle_command_reply: tid=0 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa08ffca740 1 client.27203 shutdown 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa08ffca740 1 -- 172.21.15.107:0/1107134924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 msgr2=0x5562464c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa08ffca740 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 0x5562464c6dd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa080006010 tx=0x7fa08002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa08ffca740 1 -- 172.21.15.107:0/1107134924 shutdown_connections 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa08ffca740 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5562464d9ce0 0x5562464dc0f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa08ffca740 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 0x5562464c6dd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa08ffca740 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 0x5562464c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa08ffca740 1 --2- 172.21.15.107:0/1107134924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa08ffca740 1 -- 172.21.15.107:0/1107134924 >> 172.21.15.107:0/1107134924 conn(0x5562464bb050 msgr2=0x5562464c0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:09.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.267+0000 7fa067e00640 0 client.27203 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:09.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.268+0000 7fa08ffca740 1 -- 172.21.15.107:0/1107134924 shutdown_connections 2024-09-19T04:31:09.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.268+0000 7fa08ffca740 1 -- 172.21.15.107:0/1107134924 wait complete. 2024-09-19T04:31:09.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.268+0000 7fa08ffca740 20 client.27203 trim_cache size 0 max 16384 2024-09-19T04:31:09.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.287+0000 7fa08ffca740 1 Processor -- start 2024-09-19T04:31:09.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.287+0000 7fa08ffca740 1 -- start start 2024-09-19T04:31:09.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.287+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.287+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 0x5562464e1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.287+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 0x5562464e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.287+0000 7fa08ffca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881afa90 con 0x5562464bf3e0 2024-09-19T04:31:09.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.287+0000 7fa08ffca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881b15d0 con 0x5562464c9ca0 2024-09-19T04:31:09.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.287+0000 7fa08ffca740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881b0160 con 0x5562464beac0 2024-09-19T04:31:09.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa086a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa086a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:52948/0 (socket says 172.21.15.107:52948) 2024-09-19T04:31:09.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa086a00640 1 -- 172.21.15.107:0/2495259455 learned_addr learned my addr 172.21.15.107:0/2495259455 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:09.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa086000640 1 --2- 172.21.15.107:0/2495259455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 0x5562464e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa087400640 1 --2- 172.21.15.107:0/2495259455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 0x5562464e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa086a00640 1 -- 172.21.15.107:0/2495259455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 msgr2=0x5562464e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:09.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa086a00640 1 --2- 172.21.15.107:0/2495259455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 0x5562464e1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa086a00640 1 -- 172.21.15.107:0/2495259455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 msgr2=0x5562464e1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:09.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa086a00640 1 --2- 172.21.15.107:0/2495259455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 0x5562464e1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa086a00640 1 -- 172.21.15.107:0/2495259455 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562464c4de0 con 0x5562464beac0 2024-09-19T04:31:09.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.288+0000 7fa086a00640 1 --2- 172.21.15.107:0/2495259455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c1dc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa078005b90 tx=0x7fa078011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=889e7d0897cfe2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:09.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.289+0000 7fa072a00640 1 -- 172.21.15.107:0/2495259455 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0780037f0 con 0x5562464beac0 2024-09-19T04:31:09.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.289+0000 7fa072a00640 1 -- 172.21.15.107:0/2495259455 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa078003990 con 0x5562464beac0 2024-09-19T04:31:09.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.289+0000 7fa072a00640 1 -- 172.21.15.107:0/2495259455 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078019e10 con 0x5562464beac0 2024-09-19T04:31:09.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.289+0000 7fa08ffca740 1 -- 172.21.15.107:0/2495259455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 msgr2=0x5562464c1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:09.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.289+0000 7fa08ffca740 1 --2- 172.21.15.107:0/2495259455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c1dc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa078005b90 tx=0x7fa078011810 comp rx=0 tx=0).stop 2024-09-19T04:31:09.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.290+0000 7fa08ffca740 1 -- 172.21.15.107:0/2495259455 shutdown_connections 2024-09-19T04:31:09.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.290+0000 7fa08ffca740 1 --2- 172.21.15.107:0/2495259455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464c9ca0 0x5562464e1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.290+0000 7fa08ffca740 1 --2- 172.21.15.107:0/2495259455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464bf3e0 0x5562464e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.290+0000 7fa08ffca740 1 --2- 172.21.15.107:0/2495259455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464c1dc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.290+0000 7fa08ffca740 1 -- 172.21.15.107:0/2495259455 >> 172.21.15.107:0/2495259455 conn(0x5562464bb050 msgr2=0x5562464c16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:09.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.290+0000 7fa08ffca740 1 -- 172.21.15.107:0/2495259455 shutdown_connections 2024-09-19T04:31:09.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.290+0000 7fa08ffca740 1 -- 172.21.15.107:0/2495259455 wait complete. 2024-09-19T04:31:09.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.290+0000 7fa08ffca740 1 -- start start 2024-09-19T04:31:09.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.290+0000 7fa08ffca740 1 Processor -- start 2024-09-19T04:31:09.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa08ffca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464c9ca0 0x5562464c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa08ffca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881b0f00 con 0x5562464c9ca0 2024-09-19T04:31:09.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa08ffca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881b0830 con 0x5562464bf3e0 2024-09-19T04:31:09.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa08ffca740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881ada90 con 0x5562464beac0 2024-09-19T04:31:09.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa08ffca740 10 client.0 _ll_register_callbacks cb 0x7fa074005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:09.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa086000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa087400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464c9ca0 0x5562464c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa086000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38618/0 (socket says 172.21.15.107:38618) 2024-09-19T04:31:09.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa086000640 1 -- 172.21.15.107:0/308849966 learned_addr learned my addr 172.21.15.107:0/308849966 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:09.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa086a00640 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa086000640 1 -- 172.21.15.107:0/308849966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 msgr2=0x5562464d20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:09.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa086000640 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464d20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa086000640 1 -- 172.21.15.107:0/308849966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464c9ca0 msgr2=0x5562464c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:09.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa086000640 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464c9ca0 0x5562464c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa086000640 1 -- 172.21.15.107:0/308849966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562464c3d20 con 0x5562464bf3e0 2024-09-19T04:31:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.291+0000 7fa086a00640 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464d20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.292+0000 7fa086000640 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464d2610 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa07409bf80 tx=0x7fa07409bbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57ac96f329239d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.292+0000 7fa067e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.292+0000 7fa067e00640 1 -- 172.21.15.107:0/308849966 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0740bc2b0 con 0x5562464bf3e0 2024-09-19T04:31:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.292+0000 7fa08ffca740 1 -- 172.21.15.107:0/308849966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa078003990 con 0x5562464bf3e0 2024-09-19T04:31:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.292+0000 7fa067e00640 1 -- 172.21.15.107:0/308849966 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fa0740a0120 con 0x5562464bf3e0 2024-09-19T04:31:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.292+0000 7fa067e00640 1 -- 172.21.15.107:0/308849966 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa0740a4e40 con 0x5562464bf3e0 2024-09-19T04:31:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.292+0000 7fa08ffca740 10 client.27209 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.292+0000 7fa08ffca740 1 -- 172.21.15.107:0/308849966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556246544780 con 0x5562464bf3e0 2024-09-19T04:31:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.292+0000 7fa067e00640 1 -- 172.21.15.107:0/308849966 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa078003990 con 0x5562464bf3e0 2024-09-19T04:31:09.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.293+0000 7fa067e00640 1 -- 172.21.15.107:0/308849966 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fa07409b6d0 con 0x5562464bf3e0 2024-09-19T04:31:09.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.293+0000 7fa08ffca740 10 client.27209 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:09.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.293+0000 7fa08ffca740 10 client.27209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:09.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.293+0000 7fa08ffca740 20 client.27209 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:09.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.293+0000 7fa08ffca740 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556246544e50 0x556246547240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:09.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.293+0000 7fa08ffca740 4 client.27209 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:31:09.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.293+0000 7fa08ffca740 1 -- 172.21.15.107:0/308849966 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55624642cd10 con 0x556246544e50 2024-09-19T04:31:09.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.293+0000 7fa086a00640 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556246544e50 0x556246547240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:09.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.294+0000 7fa086a00640 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556246544e50 0x556246547240 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81809ab2be585f7a server_cookie=c4416755b2cbf903 in_seq=0 out_seq=0 2024-09-19T04:31:09.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.294+0000 7fa067e00640 10 client.27209 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa067e00640 1 -- 172.21.15.107:0/308849966 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55624642cd10 con 0x556246544e50 2024-09-19T04:31:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa067e00640 10 client.27209 handle_command_reply: tid=0 2024-09-19T04:31:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 client.27209 shutdown 2024-09-19T04:31:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 -- 172.21.15.107:0/308849966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 msgr2=0x5562464d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464d2610 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa07409bf80 tx=0x7fa07409bbe0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 -- 172.21.15.107:0/308849966 shutdown_connections 2024-09-19T04:31:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556246544e50 0x556246547240 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562464c9ca0 0x5562464c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562464bf3e0 0x5562464d2610 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 --2- 172.21.15.107:0/308849966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562464beac0 0x5562464d20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 -- 172.21.15.107:0/308849966 >> 172.21.15.107:0/308849966 conn(0x5562464bb050 msgr2=0x5562464c1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:09.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa067e00640 0 client.27209 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:09.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 -- 172.21.15.107:0/308849966 shutdown_connections 2024-09-19T04:31:09.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 1 -- 172.21.15.107:0/308849966 wait complete. 2024-09-19T04:31:09.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.295+0000 7fa08ffca740 20 client.27209 trim_cache size 0 max 16384 2024-09-19T04:31:09.303 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:31:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:31:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "a4b9e14b-137e-4fe7-87fd-49cf52ede497", 2024-09-19T04:31:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:31:09.304 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:31:09.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.296+0000 7fa08d600640 1 -- 172.21.15.107:0/1524632636 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa06c0a3430 msgr2=0x7fa06c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.296+0000 7fa08d600640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa06c0a3430 0x7fa06c0a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa080004af0 tx=0x7fa080004a80 comp rx=0 tx=0).stop 2024-09-19T04:31:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.296+0000 7fa08d600640 1 -- 172.21.15.107:0/1524632636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0881049c0 msgr2=0x7fa08819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.296+0000 7fa08d600640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0881049c0 0x7fa08819abd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa07400e9f0 tx=0x7fa07400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.297+0000 7fa08d600640 1 -- 172.21.15.107:0/1524632636 shutdown_connections 2024-09-19T04:31:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.297+0000 7fa08d600640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa06c0a3430 0x7fa06c0a58f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.297+0000 7fa08d600640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa088109140 0x7fa08819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.297+0000 7fa08d600640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0881049c0 0x7fa08819abd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.297+0000 7fa08d600640 1 --2- 172.21.15.107:0/1524632636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa088104010 0x7fa08819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.297+0000 7fa08d600640 1 -- 172.21.15.107:0/1524632636 >> 172.21.15.107:0/1524632636 conn(0x7fa0880ffa50 msgr2=0x7fa088100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:09.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.297+0000 7fa08d600640 1 -- 172.21.15.107:0/1524632636 shutdown_connections 2024-09-19T04:31:09.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:09.297+0000 7fa08d600640 1 -- 172.21.15.107:0/1524632636 wait complete. 2024-09-19T04:31:09.720 INFO:tasks.workunit.client.0.smithi107.stdout: 1 360831 2.80 MB/sec execute 447 sec latency 1046.152 ms 2024-09-19T04:31:09.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:09 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:31:09.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:09 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:09.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:09 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:31:09.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:09 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:31:09.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:09 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:09.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:09 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:09 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:31:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:09 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:09 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:31:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:09 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:31:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:09 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:09 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:10.218 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:31:10.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:09 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:31:10.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:09 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:10.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:09 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:31:10.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:09 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:31:10.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:09 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:10.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:09 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:10.617 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:10.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 362277 2.82 MB/sec execute 448 sec latency 11.802 ms 2024-09-19T04:31:11.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:10 smithi107 ceph-mon[30659]: pgmap v336: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-09-19T04:31:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:31:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:10 smithi115 ceph-mon[35477]: pgmap v336: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-09-19T04:31:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:31:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:31:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:31:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:31:11.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:31:11.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:31:11.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:11.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:10 smithi145 ceph-mon[35096]: pgmap v336: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-09-19T04:31:11.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.235+0000 7ff28fe00640 1 -- 172.21.15.107:0/76798776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2900745c0 msgr2=0x7ff2900749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:11.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.235+0000 7ff28fe00640 1 --2- 172.21.15.107:0/76798776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2900745c0 0x7ff2900749a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff280009860 tx=0x7ff28002a3d0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.235+0000 7ff28fe00640 1 -- 172.21.15.107:0/76798776 shutdown_connections 2024-09-19T04:31:11.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.235+0000 7ff28fe00640 1 --2- 172.21.15.107:0/76798776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff29010e260 0x7ff290110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.235+0000 7ff28fe00640 1 --2- 172.21.15.107:0/76798776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff290074ee0 0x7ff29010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.235+0000 7ff28fe00640 1 --2- 172.21.15.107:0/76798776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2900745c0 0x7ff2900749a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.235+0000 7ff28fe00640 1 -- 172.21.15.107:0/76798776 >> 172.21.15.107:0/76798776 conn(0x7ff2900fd740 msgr2=0x7ff2900ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:11.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.236+0000 7ff28fe00640 1 -- 172.21.15.107:0/76798776 shutdown_connections 2024-09-19T04:31:11.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.236+0000 7ff28fe00640 1 -- 172.21.15.107:0/76798776 wait complete. 2024-09-19T04:31:11.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.236+0000 7ff28fe00640 1 Processor -- start 2024-09-19T04:31:11.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff28fe00640 1 -- start start 2024-09-19T04:31:11.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff28fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2900745c0 0x7ff29019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff28fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff290074ee0 0x7ff29019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff28fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff29010e260 0x7ff29019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff28fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff290114000 con 0x7ff2900745c0 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff28fe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff290113e80 con 0x7ff290074ee0 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff28fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff290114180 con 0x7ff29010e260 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff286000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff290074ee0 0x7ff29019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff28ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2900745c0 0x7ff29019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff286000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff290074ee0 0x7ff29019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38630/0 (socket says 172.21.15.107:38630) 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.237+0000 7ff286000640 1 -- 172.21.15.107:0/1999846019 learned_addr learned my addr 172.21.15.107:0/1999846019 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff286000640 1 -- 172.21.15.107:0/1999846019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff29010e260 msgr2=0x7ff29019ed90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff286000640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff29010e260 0x7ff29019ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff286000640 1 -- 172.21.15.107:0/1999846019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2900745c0 msgr2=0x7ff29019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff286000640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2900745c0 0x7ff29019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff286000640 1 -- 172.21.15.107:0/1999846019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29019f470 con 0x7ff290074ee0 2024-09-19T04:31:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff286000640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff290074ee0 0x7ff29019aa00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff27800d950 tx=0x7ff27800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff287e00640 1 -- 172.21.15.107:0/1999846019 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278017070 con 0x7ff290074ee0 2024-09-19T04:31:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff28fe00640 1 -- 172.21.15.107:0/1999846019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29019f760 con 0x7ff290074ee0 2024-09-19T04:31:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff287e00640 1 -- 172.21.15.107:0/1999846019 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff278009c30 con 0x7ff290074ee0 2024-09-19T04:31:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff28fe00640 1 -- 172.21.15.107:0/1999846019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2901a7040 con 0x7ff290074ee0 2024-09-19T04:31:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.238+0000 7ff287e00640 1 -- 172.21.15.107:0/1999846019 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278020060 con 0x7ff290074ee0 2024-09-19T04:31:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.241+0000 7ff29822d740 1 Processor -- start 2024-09-19T04:31:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff29822d740 1 -- start start 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff29822d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebbd50 0x558979ebc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff29822d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebc670 0x558979ec69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff287e00640 1 -- 172.21.15.107:0/1999846019 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7ff2780109d0 con 0x7ff290074ee0 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff29822d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ec6f30 0x558979ec9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff29822d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558979e223e0 con 0x558979ec6f30 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff29822d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558979dc9d80 con 0x558979ebc670 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff29822d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558979e1e3a0 con 0x558979ebbd50 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff28ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebc670 0x558979ec69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff286000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ec6f30 0x558979ec9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff28ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebc670 0x558979ec69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38634/0 (socket says 172.21.15.107:38634) 2024-09-19T04:31:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.242+0000 7ff28ea00640 1 -- 172.21.15.107:0/3579123473 learned_addr learned my addr 172.21.15.107:0/3579123473 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff28ea00640 1 -- 172.21.15.107:0/3579123473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebbd50 msgr2=0x558979ebc130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff28ea00640 1 --2- 172.21.15.107:0/3579123473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebbd50 0x558979ebc130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff28ea00640 1 -- 172.21.15.107:0/3579123473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ec6f30 msgr2=0x558979ec9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff28ea00640 1 --2- 172.21.15.107:0/3579123473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ec6f30 0x558979ec9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff28ea00640 1 -- 172.21.15.107:0/3579123473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558979de8ae0 con 0x558979ebc670 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff286000640 1 --2- 172.21.15.107:0/3579123473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ec6f30 0x558979ec9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff28ea00640 1 --2- 172.21.15.107:0/3579123473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebc670 0x558979ec69f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff280034000 tx=0x7ff280030b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fdf1ac3a130cd79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff27fe00640 1 -- 172.21.15.107:0/3579123473 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280004030 con 0x558979ebc670 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff27fe00640 1 -- 172.21.15.107:0/3579123473 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff280041030 con 0x558979ebc670 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff27fe00640 1 -- 172.21.15.107:0/3579123473 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280005430 con 0x558979ebc670 2024-09-19T04:31:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.243+0000 7ff287e00640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2740a3430 0x7ff2740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff287e00640 1 -- 172.21.15.107:0/1999846019 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7ff2780cf510 con 0x7ff290074ee0 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff28f400640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2740a3430 0x7ff2740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff29822d740 1 -- 172.21.15.107:0/3579123473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebc670 msgr2=0x558979ec69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff29822d740 1 --2- 172.21.15.107:0/3579123473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebc670 0x558979ec69f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff280034000 tx=0x7ff280030b30 comp rx=0 tx=0).stop 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff29822d740 1 -- 172.21.15.107:0/3579123473 shutdown_connections 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff29822d740 1 --2- 172.21.15.107:0/3579123473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ec6f30 0x558979ec9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff29822d740 1 --2- 172.21.15.107:0/3579123473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebc670 0x558979ec69f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff29822d740 1 --2- 172.21.15.107:0/3579123473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebbd50 0x558979ebc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff29822d740 1 -- 172.21.15.107:0/3579123473 >> 172.21.15.107:0/3579123473 conn(0x558979eb82e0 msgr2=0x558979eb86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff29822d740 1 -- 172.21.15.107:0/3579123473 shutdown_connections 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.244+0000 7ff29822d740 1 -- 172.21.15.107:0/3579123473 wait complete. 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.245+0000 7ff28f400640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2740a3430 0x7ff2740a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff29006a6f0 tx=0x7ff2880095c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:11.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.245+0000 7ff29822d740 1 -- start start 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.245+0000 7ff29822d740 1 Processor -- start 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff29822d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ec3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff29822d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 0x558979ec4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff29822d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ec6f30 0x558979ec4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff29822d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558979ec9ba0 con 0x558979ebc670 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff29822d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558979ddd0f0 con 0x558979ebbd50 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff29822d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558979ec9d20 con 0x558979ec6f30 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff29822d740 10 client.0 _ll_register_callbacks cb 0x558979dce9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff286000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 0x558979ec4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff28ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ec3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff286000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 0x558979ec4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39126/0 (socket says 172.21.15.107:39126) 2024-09-19T04:31:11.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff286000640 1 -- 172.21.15.107:0/162672047 learned_addr learned my addr 172.21.15.107:0/162672047 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff286000640 1 -- 172.21.15.107:0/162672047 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ec6f30 msgr2=0x558979ec4840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff286000640 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ec6f30 0x558979ec4840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff286000640 1 -- 172.21.15.107:0/162672047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 msgr2=0x558979ec3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.246+0000 7ff286000640 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ec3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.247+0000 7ff286000640 1 -- 172.21.15.107:0/162672047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558979b933c0 con 0x558979ebc670 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.247+0000 7ff28ea00640 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ec3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.247+0000 7ff286000640 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 0x558979ec4300 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff278000c00 tx=0x7ff27809b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3338b5a7d35a3267 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.247+0000 7ff27cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.247+0000 7ff27cc00640 1 -- 172.21.15.107:0/162672047 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27809b7b0 con 0x558979ebc670 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.247+0000 7ff27cc00640 1 -- 172.21.15.107:0/162672047 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7ff2780ccea0 con 0x558979ebc670 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.247+0000 7ff29822d740 1 -- 172.21.15.107:0/162672047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff280041030 con 0x558979ebc670 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.247+0000 7ff27cc00640 1 -- 172.21.15.107:0/162672047 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff2780cd0a0 con 0x558979ebc670 2024-09-19T04:31:11.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.247+0000 7ff29822d740 10 client.17175 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.247+0000 7ff29822d740 1 -- 172.21.15.107:0/162672047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558979ed57e0 con 0x558979ebc670 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.248+0000 7ff27cc00640 1 -- 172.21.15.107:0/162672047 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff280041030 con 0x558979ebc670 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.248+0000 7ff27cc00640 1 -- 172.21.15.107:0/162672047 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7ff2780cf140 con 0x558979ebc670 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.248+0000 7ff29822d740 10 client.17175 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.248+0000 7ff29822d740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.248+0000 7ff29822d740 20 client.17175 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.248+0000 7ff29822d740 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558979ed6ca0 0x558979ed9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.248+0000 7ff29822d740 4 client.17175 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.248+0000 7ff29822d740 1 -- 172.21.15.107:0/162672047 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558979e2f950 con 0x558979ed6ca0 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.249+0000 7ff28ea00640 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558979ed6ca0 0x558979ed9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.249+0000 7ff28ea00640 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558979ed6ca0 0x558979ed9090 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db2e56321189961 server_cookie=e4f9a6bcf080a6a4 in_seq=0 out_seq=0 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.249+0000 7ff27cc00640 10 client.17175 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:11.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff27cc00640 1 -- 172.21.15.107:0/162672047 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558979e2f950 con 0x558979ed6ca0 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff27cc00640 10 client.17175 handle_command_reply: tid=0 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 client.17175 shutdown 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 -- 172.21.15.107:0/162672047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 msgr2=0x558979ec4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 0x558979ec4300 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff278000c00 tx=0x7ff27809b3b0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 -- 172.21.15.107:0/162672047 shutdown_connections 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558979ed6ca0 0x558979ed9090 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ec6f30 0x558979ec4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 0x558979ec4300 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 --2- 172.21.15.107:0/162672047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ec3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 -- 172.21.15.107:0/162672047 >> 172.21.15.107:0/162672047 conn(0x558979eb82e0 msgr2=0x558979ebdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:11.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff27cc00640 0 client.17175 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:11.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 -- 172.21.15.107:0/162672047 shutdown_connections 2024-09-19T04:31:11.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.250+0000 7ff29822d740 1 -- 172.21.15.107:0/162672047 wait complete. 2024-09-19T04:31:11.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.251+0000 7ff29822d740 20 client.17175 trim_cache size 0 max 16384 2024-09-19T04:31:11.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.268+0000 7ff29822d740 1 Processor -- start 2024-09-19T04:31:11.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.268+0000 7ff29822d740 1 -- start start 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.268+0000 7ff29822d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ede840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.268+0000 7ff29822d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 0x558979eded80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.268+0000 7ff29822d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ec6f30 0x558979ec1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.268+0000 7ff29822d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901af8a0 con 0x558979ebc670 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.268+0000 7ff29822d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901b13e0 con 0x558979ebbd50 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.268+0000 7ff29822d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901aff70 con 0x558979ec6f30 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff286000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 0x558979eded80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff28ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ede840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff28ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ede840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38648/0 (socket says 172.21.15.107:38648) 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff28ea00640 1 -- 172.21.15.107:0/4191734883 learned_addr learned my addr 172.21.15.107:0/4191734883 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:11.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff28f400640 1 --2- 172.21.15.107:0/4191734883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ec6f30 0x558979ec1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff28ea00640 1 -- 172.21.15.107:0/4191734883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ec6f30 msgr2=0x558979ec1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:11.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff28ea00640 1 --2- 172.21.15.107:0/4191734883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ec6f30 0x558979ec1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff28ea00640 1 -- 172.21.15.107:0/4191734883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 msgr2=0x558979eded80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:11.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff28ea00640 1 --2- 172.21.15.107:0/4191734883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 0x558979eded80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff28ea00640 1 -- 172.21.15.107:0/4191734883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558979ebc4a0 con 0x558979ebbd50 2024-09-19T04:31:11.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.269+0000 7ff28ea00640 1 --2- 172.21.15.107:0/4191734883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ede840 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff280004430 tx=0x7ff280005650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a02d73b20f5a38f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:11.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.270+0000 7ff27f400640 1 -- 172.21.15.107:0/4191734883 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2800303a0 con 0x558979ebbd50 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.270+0000 7ff27f400640 1 -- 172.21.15.107:0/4191734883 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28005b070 con 0x558979ebbd50 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.270+0000 7ff27f400640 1 -- 172.21.15.107:0/4191734883 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280004ef0 con 0x558979ebbd50 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.270+0000 7ff29822d740 1 -- 172.21.15.107:0/4191734883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 msgr2=0x558979ede840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.270+0000 7ff29822d740 1 --2- 172.21.15.107:0/4191734883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ede840 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff280004430 tx=0x7ff280005650 comp rx=0 tx=0).stop 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.270+0000 7ff29822d740 1 -- 172.21.15.107:0/4191734883 shutdown_connections 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.270+0000 7ff29822d740 1 --2- 172.21.15.107:0/4191734883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ec6f30 0x558979ec1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.270+0000 7ff29822d740 1 --2- 172.21.15.107:0/4191734883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebc670 0x558979eded80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.270+0000 7ff29822d740 1 --2- 172.21.15.107:0/4191734883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ebbd50 0x558979ede840 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.270+0000 7ff29822d740 1 -- 172.21.15.107:0/4191734883 >> 172.21.15.107:0/4191734883 conn(0x558979eb82e0 msgr2=0x558979ec5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.271+0000 7ff29822d740 1 -- 172.21.15.107:0/4191734883 shutdown_connections 2024-09-19T04:31:11.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.271+0000 7ff29822d740 1 -- 172.21.15.107:0/4191734883 wait complete. 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.271+0000 7ff29822d740 1 -- start start 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.271+0000 7ff29822d740 1 Processor -- start 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff29822d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebbd50 0x558979ecf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff29822d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebc670 0x558979ecab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff29822d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ec6f30 0x558979ecb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff29822d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901b0d10 con 0x558979ebbd50 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff29822d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901b0640 con 0x558979ec6f30 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff29822d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff2901ad8a0 con 0x558979ebc670 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff29822d740 10 client.0 _ll_register_callbacks cb 0x7ff27800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff286000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebc670 0x558979ecab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff28ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebbd50 0x558979ecf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff286000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebc670 0x558979ecab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53022/0 (socket says 172.21.15.107:53022) 2024-09-19T04:31:11.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff28f400640 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ec6f30 0x558979ecb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff286000640 1 -- 172.21.15.107:0/3727365375 learned_addr learned my addr 172.21.15.107:0/3727365375 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:11.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff28f400640 1 -- 172.21.15.107:0/3727365375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebc670 msgr2=0x558979ecab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:11.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff28f400640 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebc670 0x558979ecab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff28f400640 1 -- 172.21.15.107:0/3727365375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebbd50 msgr2=0x558979ecf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:11.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff28f400640 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebbd50 0x558979ecf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.272+0000 7ff28f400640 1 -- 172.21.15.107:0/3727365375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558979f49020 con 0x558979ec6f30 2024-09-19T04:31:11.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.273+0000 7ff286000640 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebc670 0x558979ecab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.273+0000 7ff28ea00640 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebbd50 0x558979ecf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.273+0000 7ff28f400640 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ec6f30 0x558979ecb300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x558979dd9110 tx=0x7ff288004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9299d74699a3afcc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.273+0000 7ff27cc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.273+0000 7ff27cc00640 1 -- 172.21.15.107:0/3727365375 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288019a60 con 0x558979ec6f30 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.273+0000 7ff27cc00640 1 -- 172.21.15.107:0/3727365375 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7ff288019c00 con 0x558979ec6f30 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.273+0000 7ff29822d740 1 -- 172.21.15.107:0/3727365375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff28005b070 con 0x558979ec6f30 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.273+0000 7ff27cc00640 1 -- 172.21.15.107:0/3727365375 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff288019e00 con 0x558979ec6f30 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.273+0000 7ff29822d740 10 client.27227 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.273+0000 7ff29822d740 1 -- 172.21.15.107:0/3727365375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558979f4a1c0 con 0x558979ec6f30 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.274+0000 7ff27cc00640 1 -- 172.21.15.107:0/3727365375 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28005b070 con 0x558979ec6f30 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.274+0000 7ff27cc00640 1 -- 172.21.15.107:0/3727365375 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7ff28802a020 con 0x558979ec6f30 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.274+0000 7ff29822d740 10 client.27227 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:11.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.274+0000 7ff29822d740 10 client.27227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:11.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.274+0000 7ff29822d740 20 client.27227 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:11.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.274+0000 7ff29822d740 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558979f4a960 0x558979f4cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:11.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.274+0000 7ff29822d740 4 client.27227 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:31:11.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.274+0000 7ff29822d740 1 -- 172.21.15.107:0/3727365375 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558979e29e60 con 0x558979f4a960 2024-09-19T04:31:11.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.274+0000 7ff28ea00640 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558979f4a960 0x558979f4cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:11.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff28ea00640 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558979f4a960 0x558979f4cda0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d961e7626fb7b971 server_cookie=25b65810b5c69040 in_seq=0 out_seq=0 2024-09-19T04:31:11.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff27cc00640 10 client.27227 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:11.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff27cc00640 1 -- 172.21.15.107:0/3727365375 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558979e29e60 con 0x558979f4a960 2024-09-19T04:31:11.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff27cc00640 10 client.27227 handle_command_reply: tid=0 2024-09-19T04:31:11.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 client.27227 shutdown 2024-09-19T04:31:11.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 -- 172.21.15.107:0/3727365375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ec6f30 msgr2=0x558979ecb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:11.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ec6f30 0x558979ecb300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x558979dd9110 tx=0x7ff288004a40 comp rx=0 tx=0).stop 2024-09-19T04:31:11.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 -- 172.21.15.107:0/3727365375 shutdown_connections 2024-09-19T04:31:11.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x558979f4a960 0x558979f4cda0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558979ec6f30 0x558979ecb300 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558979ebc670 0x558979ecab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff27cc00640 0 client.27227 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:11.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 --2- 172.21.15.107:0/3727365375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558979ebbd50 0x558979ecf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 -- 172.21.15.107:0/3727365375 >> 172.21.15.107:0/3727365375 conn(0x558979eb82e0 msgr2=0x558979ec3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:11.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 -- 172.21.15.107:0/3727365375 shutdown_connections 2024-09-19T04:31:11.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 1 -- 172.21.15.107:0/3727365375 wait complete. 2024-09-19T04:31:11.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.275+0000 7ff29822d740 20 client.27227 trim_cache size 0 max 16384 2024-09-19T04:31:11.281 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:31:11.282 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:31:11.282 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:31:11.282 INFO:teuthology.orchestra.run.smithi107.stdout: "a4b9e14b-137e-4fe7-87fd-49cf52ede497": { 2024-09-19T04:31:11.282 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:31:11.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "a4b9e14b-137e-4fe7-87fd-49cf52ede497", 2024-09-19T04:31:11.282 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:31:11.282 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:31:11.282 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:31:11.282 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:31:11.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 -- 172.21.15.107:0/1999846019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2740a3430 msgr2=0x7ff2740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:11.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2740a3430 0x7ff2740a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff29006a6f0 tx=0x7ff2880095c0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 -- 172.21.15.107:0/1999846019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff290074ee0 msgr2=0x7ff29019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:11.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff290074ee0 0x7ff29019aa00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff27800d950 tx=0x7ff27800de10 comp rx=0 tx=0).stop 2024-09-19T04:31:11.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 -- 172.21.15.107:0/1999846019 shutdown_connections 2024-09-19T04:31:11.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff2740a3430 0x7ff2740a58f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff29010e260 0x7ff29019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff290074ee0 0x7ff29019aa00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 --2- 172.21.15.107:0/1999846019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2900745c0 0x7ff29019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:11.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 -- 172.21.15.107:0/1999846019 >> 172.21.15.107:0/1999846019 conn(0x7ff2900fd740 msgr2=0x7ff2900fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:11.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 -- 172.21.15.107:0/1999846019 shutdown_connections 2024-09-19T04:31:11.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:11.276+0000 7ff28fe00640 1 -- 172.21.15.107:0/1999846019 wait complete. 2024-09-19T04:31:11.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 362933 2.81 MB/sec execute 449 sec latency 533.220 ms 2024-09-19T04:31:12.353 INFO:tasks.cephfs.filesystem:scrub status for tag:a4b9e14b-137e-4fe7-87fd-49cf52ede497 - {'path': '/', 'tag': 'a4b9e14b-137e-4fe7-87fd-49cf52ede497', 'options': 'recursive,force'} 2024-09-19T04:31:12.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 363483 2.82 MB/sec execute 450 sec latency 621.017 ms 2024-09-19T04:31:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:12 smithi115 ceph-mon[35477]: pgmap v337: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-09-19T04:31:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:12 smithi115 ceph-mon[35477]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:31:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:12 smithi145 ceph-mon[35096]: pgmap v337: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-09-19T04:31:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:12 smithi145 ceph-mon[35096]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:31:13.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:12 smithi107 ceph-mon[30659]: pgmap v337: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-09-19T04:31:13.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:12 smithi107 ceph-mon[30659]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:31:13.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 363483 2.81 MB/sec execute 451 sec latency 1621.162 ms 2024-09-19T04:31:14.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 364395 2.81 MB/sec execute 452 sec latency 2126.777 ms 2024-09-19T04:31:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:14 smithi115 ceph-mon[35477]: pgmap v339: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-19T04:31:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:14 smithi145 ceph-mon[35096]: pgmap v339: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-19T04:31:15.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:14 smithi107 ceph-mon[30659]: pgmap v339: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-19T04:31:15.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 365808 2.83 MB/sec execute 453 sec latency 11.377 ms 2024-09-19T04:31:16.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 366725 2.83 MB/sec execute 454 sec latency 369.241 ms 2024-09-19T04:31:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:16 smithi115 ceph-mon[35477]: pgmap v340: 97 pgs: 97 active+clean; 289 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-19T04:31:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:16 smithi115 ceph-mon[35477]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:31:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:16 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:31:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:16 smithi145 ceph-mon[35096]: pgmap v340: 97 pgs: 97 active+clean; 289 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-19T04:31:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:16 smithi145 ceph-mon[35096]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:31:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:16 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:31:17.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:16 smithi107 ceph-mon[30659]: pgmap v340: 97 pgs: 97 active+clean; 289 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-19T04:31:17.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:16 smithi107 ceph-mon[30659]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:31:17.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:16 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:31:17.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 367032 2.83 MB/sec execute 455 sec latency 638.043 ms 2024-09-19T04:31:18.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:17 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:31:18.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:17 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:31:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:17 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:31:18.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:17 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:31:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:17 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:31:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:17 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:31:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:17 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:31:18.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:17 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:31:18.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:17 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:31:18.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:17 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:31:18.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:17 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:31:18.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:17 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:31:18.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 367032 2.82 MB/sec execute 456 sec latency 1638.165 ms 2024-09-19T04:31:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:18 smithi115 ceph-mon[35477]: pgmap v342: 97 pgs: 97 active+clean; 289 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-19T04:31:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:18 smithi145 ceph-mon[35096]: pgmap v342: 97 pgs: 97 active+clean; 289 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-19T04:31:19.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:18 smithi107 ceph-mon[30659]: pgmap v342: 97 pgs: 97 active+clean; 289 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-19T04:31:19.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 367769 2.82 MB/sec execute 457 sec latency 2154.866 ms 2024-09-19T04:31:20.721 INFO:tasks.workunit.client.0.smithi107.stdout: 1 369293 2.84 MB/sec execute 458 sec latency 13.613 ms 2024-09-19T04:31:21.195 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:20 smithi107 ceph-mon[30659]: pgmap v343: 97 pgs: 97 active+clean; 289 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 990 KiB/s wr, 85 op/s 2024-09-19T04:31:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:20 smithi115 ceph-mon[35477]: pgmap v343: 97 pgs: 97 active+clean; 289 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 990 KiB/s wr, 85 op/s 2024-09-19T04:31:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:20 smithi145 ceph-mon[35096]: pgmap v343: 97 pgs: 97 active+clean; 289 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 990 KiB/s wr, 85 op/s 2024-09-19T04:31:21.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 369862 2.83 MB/sec execute 459 sec latency 540.968 ms 2024-09-19T04:31:22.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 370575 2.84 MB/sec execute 460 sec latency 549.743 ms 2024-09-19T04:31:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:22 smithi115 ceph-mon[35477]: pgmap v344: 97 pgs: 97 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-09-19T04:31:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:22 smithi145 ceph-mon[35096]: pgmap v344: 97 pgs: 97 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-09-19T04:31:23.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:22 smithi107 ceph-mon[30659]: pgmap v344: 97 pgs: 97 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-09-19T04:31:23.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 370575 2.83 MB/sec execute 461 sec latency 1467.976 ms 2024-09-19T04:31:24.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 372090 2.84 MB/sec execute 462 sec latency 1707.556 ms 2024-09-19T04:31:25.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:24 smithi115 ceph-mon[35477]: pgmap v345: 97 pgs: 97 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-19T04:31:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:24 smithi145 ceph-mon[35096]: pgmap v345: 97 pgs: 97 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-19T04:31:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:24 smithi107 ceph-mon[30659]: pgmap v345: 97 pgs: 97 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-19T04:31:25.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 373334 2.85 MB/sec execute 463 sec latency 14.763 ms 2024-09-19T04:31:26.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 374044 2.85 MB/sec execute 464 sec latency 410.823 ms 2024-09-19T04:31:27.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:26 smithi107 ceph-mon[30659]: pgmap v346: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 151 op/s 2024-09-19T04:31:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:26 smithi115 ceph-mon[35477]: pgmap v346: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 151 op/s 2024-09-19T04:31:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:26 smithi145 ceph-mon[35096]: pgmap v346: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 151 op/s 2024-09-19T04:31:27.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 374123 2.85 MB/sec execute 465 sec latency 619.236 ms 2024-09-19T04:31:28.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 374123 2.84 MB/sec execute 466 sec latency 1619.374 ms 2024-09-19T04:31:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:28 smithi115 ceph-mon[35477]: pgmap v347: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-19T04:31:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:28 smithi145 ceph-mon[35096]: pgmap v347: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-19T04:31:29.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:28 smithi107 ceph-mon[30659]: pgmap v347: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-19T04:31:29.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 375039 2.84 MB/sec execute 467 sec latency 2124.403 ms 2024-09-19T04:31:30.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 376481 2.86 MB/sec execute 468 sec latency 11.897 ms 2024-09-19T04:31:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:30 smithi115 ceph-mon[35477]: pgmap v348: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-09-19T04:31:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:30 smithi145 ceph-mon[35096]: pgmap v348: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-09-19T04:31:31.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:30 smithi107 ceph-mon[30659]: pgmap v348: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-09-19T04:31:31.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 377480 2.86 MB/sec execute 469 sec latency 243.620 ms 2024-09-19T04:31:32.722 INFO:tasks.workunit.client.0.smithi107.stdout: 1 377666 2.86 MB/sec execute 470 sec latency 869.265 ms 2024-09-19T04:31:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:32 smithi115 ceph-mon[35477]: pgmap v349: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-19T04:31:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:32 smithi145 ceph-mon[35096]: pgmap v349: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-19T04:31:33.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:32 smithi107 ceph-mon[30659]: pgmap v349: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-19T04:31:33.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 377666 2.85 MB/sec execute 471 sec latency 1869.380 ms 2024-09-19T04:31:34.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 378773 2.86 MB/sec execute 472 sec latency 2372.146 ms 2024-09-19T04:31:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:34 smithi115 ceph-mon[35477]: pgmap v350: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-09-19T04:31:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:34 smithi145 ceph-mon[35096]: pgmap v350: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-09-19T04:31:35.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:34 smithi107 ceph-mon[30659]: pgmap v350: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-09-19T04:31:35.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 380126 2.87 MB/sec execute 473 sec latency 11.673 ms 2024-09-19T04:31:36.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 381255 2.87 MB/sec execute 474 sec latency 86.353 ms 2024-09-19T04:31:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:36 smithi115 ceph-mon[35477]: pgmap v351: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-19T04:31:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:36 smithi145 ceph-mon[35096]: pgmap v351: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-19T04:31:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:36 smithi107 ceph-mon[30659]: pgmap v351: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-19T04:31:37.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 381255 2.87 MB/sec execute 475 sec latency 1086.438 ms 2024-09-19T04:31:38.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 381255 2.86 MB/sec execute 476 sec latency 2086.545 ms 2024-09-19T04:31:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:38 smithi115 ceph-mon[35477]: pgmap v352: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:38 smithi145 ceph-mon[35096]: pgmap v352: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:39.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:38 smithi107 ceph-mon[30659]: pgmap v352: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:39.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 382322 2.87 MB/sec execute 477 sec latency 2586.582 ms 2024-09-19T04:31:40.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 383651 2.88 MB/sec execute 478 sec latency 12.684 ms 2024-09-19T04:31:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:40 smithi115 ceph-mon[35477]: pgmap v353: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:40 smithi145 ceph-mon[35096]: pgmap v353: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:41.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:40 smithi107 ceph-mon[30659]: pgmap v353: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:41.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 384772 2.88 MB/sec execute 479 sec latency 55.651 ms 2024-09-19T04:31:42.354 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:31:42.721 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:42.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 384796 2.88 MB/sec execute 480 sec latency 653.395 ms 2024-09-19T04:31:43.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:42 smithi107 ceph-mon[30659]: pgmap v354: 97 pgs: 97 active+clean; 314 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 153 op/s 2024-09-19T04:31:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:42 smithi115 ceph-mon[35477]: pgmap v354: 97 pgs: 97 active+clean; 314 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 153 op/s 2024-09-19T04:31:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:42 smithi145 ceph-mon[35096]: pgmap v354: 97 pgs: 97 active+clean; 314 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 153 op/s 2024-09-19T04:31:43.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.309+0000 7f4099e00640 1 -- 172.21.15.107:0/2828180263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4094109120 msgr2=0x7f4094110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:43.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.309+0000 7f4099e00640 1 --2- 172.21.15.107:0/2828180263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4094109120 0x7f4094110ca0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f408c009ca0 tx=0x7f408c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.309+0000 7f4099e00640 1 -- 172.21.15.107:0/2828180263 shutdown_connections 2024-09-19T04:31:43.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.309+0000 7f4099e00640 1 --2- 172.21.15.107:0/2828180263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4094109120 0x7f4094110ca0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.309+0000 7f4099e00640 1 --2- 172.21.15.107:0/2828180263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40941049a0 0x7f40941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.309+0000 7f4099e00640 1 --2- 172.21.15.107:0/2828180263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4094103ff0 0x7f40941043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.309+0000 7f4099e00640 1 -- 172.21.15.107:0/2828180263 >> 172.21.15.107:0/2828180263 conn(0x7f40940ffab0 msgr2=0x7f4094101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.310+0000 7f4099e00640 1 -- 172.21.15.107:0/2828180263 shutdown_connections 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.310+0000 7f4099e00640 1 -- 172.21.15.107:0/2828180263 wait complete. 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4099e00640 1 Processor -- start 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4099e00640 1 -- start start 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4099e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4094103ff0 0x7f409419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4099e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40941049a0 0x7f409419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4099e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4094109120 0x7f409419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4099e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4094114130 con 0x7f40941049a0 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4099e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4094113fb0 con 0x7f4094109120 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4099e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40941142b0 con 0x7f4094103ff0 2024-09-19T04:31:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4092a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4094103ff0 0x7f409419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4093400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4094109120 0x7f409419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4092a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4094103ff0 0x7f409419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53274/0 (socket says 172.21.15.107:53274) 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4092a00640 1 -- 172.21.15.107:0/3303023697 learned_addr learned my addr 172.21.15.107:0/3303023697 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.311+0000 7f4092000640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40941049a0 0x7f409419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4092a00640 1 -- 172.21.15.107:0/3303023697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4094109120 msgr2=0x7f409419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4092a00640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4094109120 0x7f409419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4092a00640 1 -- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40941049a0 msgr2=0x7f409419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4092a00640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40941049a0 0x7f409419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4092a00640 1 -- 172.21.15.107:0/3303023697 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409419f660 con 0x7f4094103ff0 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4092000640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40941049a0 0x7f409419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4093400640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4094109120 0x7f409419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4092a00640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4094103ff0 0x7f409419a6b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f408400e9f0 tx=0x7f408400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4098a00640 1 -- 172.21.15.107:0/3303023697 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408400cd80 con 0x7f4094103ff0 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4099e00640 1 -- 172.21.15.107:0/3303023697 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409419f950 con 0x7f4094103ff0 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4098a00640 1 -- 172.21.15.107:0/3303023697 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4084010910 con 0x7f4094103ff0 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.312+0000 7f4098a00640 1 -- 172.21.15.107:0/3303023697 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840188f0 con 0x7f4094103ff0 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.313+0000 7f4099e00640 1 -- 172.21.15.107:0/3303023697 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40941a7230 con 0x7f4094103ff0 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.316+0000 7f409c875740 1 Processor -- start 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.316+0000 7f4098a00640 1 -- 172.21.15.107:0/3303023697 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f4084010ac0 con 0x7f4094103ff0 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.316+0000 7f409c875740 1 -- start start 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.316+0000 7f409c875740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face53f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.316+0000 7f409c875740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.316+0000 7f409c875740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face61160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.316+0000 7f409c875740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555facdba3e0 con 0x555face53b90 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.316+0000 7f409c875740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555facd61d80 con 0x555face5ed70 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.316+0000 7f409c875740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555facdb63a0 con 0x555face544b0 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4092a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4093400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face53f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4092a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53278/0 (socket says 172.21.15.107:53278) 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4092a00640 1 -- 172.21.15.107:0/1008009681 learned_addr learned my addr 172.21.15.107:0/1008009681 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4092000640 1 --2- 172.21.15.107:0/1008009681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face61160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4092a00640 1 -- 172.21.15.107:0/1008009681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 msgr2=0x555face61160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4092a00640 1 --2- 172.21.15.107:0/1008009681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face61160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4092a00640 1 -- 172.21.15.107:0/1008009681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 msgr2=0x555face53f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4092a00640 1 --2- 172.21.15.107:0/1008009681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face53f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4092a00640 1 -- 172.21.15.107:0/1008009681 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555facd80ae0 con 0x555face544b0 2024-09-19T04:31:43.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.317+0000 7f4092000640 1 --2- 172.21.15.107:0/1008009681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face61160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:43.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.318+0000 7f4092a00640 1 --2- 172.21.15.107:0/1008009681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5e830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f40840cd790 tx=0x7f40840d8910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a382ff17cc697834 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:43.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.318+0000 7f4098a00640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40780a3430 0x7f40780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.318+0000 7f407fe00640 1 -- 172.21.15.107:0/1008009681 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840e3070 con 0x555face544b0 2024-09-19T04:31:43.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.318+0000 7f407fe00640 1 -- 172.21.15.107:0/1008009681 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408409cae0 con 0x555face544b0 2024-09-19T04:31:43.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.318+0000 7f407fe00640 1 -- 172.21.15.107:0/1008009681 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840953b0 con 0x555face544b0 2024-09-19T04:31:43.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.318+0000 7f4093400640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40780a3430 0x7f40780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f4098a00640 1 -- 172.21.15.107:0/3303023697 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f40840d0140 con 0x7f4094103ff0 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f409c875740 1 -- 172.21.15.107:0/1008009681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 msgr2=0x555face5e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f409c875740 1 --2- 172.21.15.107:0/1008009681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5e830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f40840cd790 tx=0x7f40840d8910 comp rx=0 tx=0).stop 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f4093400640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40780a3430 0x7f40780a58f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f408c031f80 tx=0x7f408c004a60 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f409c875740 1 -- 172.21.15.107:0/1008009681 shutdown_connections 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f409c875740 1 --2- 172.21.15.107:0/1008009681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face61160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f409c875740 1 --2- 172.21.15.107:0/1008009681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5e830 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f409c875740 1 --2- 172.21.15.107:0/1008009681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face53f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f409c875740 1 -- 172.21.15.107:0/1008009681 >> 172.21.15.107:0/1008009681 conn(0x555face50120 msgr2=0x555face50500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f409c875740 1 -- 172.21.15.107:0/1008009681 shutdown_connections 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.319+0000 7f409c875740 1 -- 172.21.15.107:0/1008009681 wait complete. 2024-09-19T04:31:43.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.320+0000 7f409c875740 1 -- start start 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.320+0000 7f409c875740 1 Processor -- start 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f409c875740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face53b90 0x555face5bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f409c875740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face544b0 0x555face5c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f409c875740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face5ed70 0x555face5c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f409c875740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555face619e0 con 0x555face544b0 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f409c875740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555facd750f0 con 0x555face53b90 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f409c875740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555face61b60 con 0x555face5ed70 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f409c875740 10 client.0 _ll_register_callbacks cb 0x555facd669e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f4092a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face53b90 0x555face5bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f4092a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face53b90 0x555face5bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:49332/0 (socket says 172.21.15.107:49332) 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f4092a00640 1 -- 172.21.15.107:0/1118627882 learned_addr learned my addr 172.21.15.107:0/1118627882 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f4092000640 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face544b0 0x555face5c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f4093400640 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face5ed70 0x555face5c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f4092a00640 1 -- 172.21.15.107:0/1118627882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face5ed70 msgr2=0x555face5c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f4092a00640 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face5ed70 0x555face5c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f4092a00640 1 -- 172.21.15.107:0/1118627882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face544b0 msgr2=0x555face5c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f4092a00640 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face544b0 0x555face5c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.321+0000 7f4092a00640 1 -- 172.21.15.107:0/1118627882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555facb2b3c0 con 0x555face53b90 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.322+0000 7f4092a00640 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face53b90 0x555face5bc10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f408400e9c0 tx=0x7f4084009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0721e36069f3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.322+0000 7f407cc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.322+0000 7f407cc00640 1 -- 172.21.15.107:0/1118627882 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084016330 con 0x555face53b90 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.322+0000 7f407cc00640 1 -- 172.21.15.107:0/1118627882 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f40840205a0 con 0x555face53b90 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.322+0000 7f409c875740 1 -- 172.21.15.107:0/1118627882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f408409cae0 con 0x555face53b90 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.322+0000 7f407cc00640 1 -- 172.21.15.107:0/1118627882 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4084016510 con 0x555face53b90 2024-09-19T04:31:43.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.322+0000 7f409c875740 10 client.27239 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.322+0000 7f409c875740 1 -- 172.21.15.107:0/1118627882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555face619e0 con 0x555face53b90 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.323+0000 7f407cc00640 1 -- 172.21.15.107:0/1118627882 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408409cae0 con 0x555face53b90 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.323+0000 7f407cc00640 1 -- 172.21.15.107:0/1118627882 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f40840d3380 con 0x555face53b90 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.323+0000 7f409c875740 10 client.27239 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.323+0000 7f409c875740 10 client.27239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.323+0000 7f409c875740 20 client.27239 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.323+0000 7f409c875740 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555face6e7c0 0x555face70bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.323+0000 7f409c875740 4 client.27239 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.323+0000 7f409c875740 1 -- 172.21.15.107:0/1118627882 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555facdc7950 con 0x555face6e7c0 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.323+0000 7f4092000640 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555face6e7c0 0x555face70bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.324+0000 7f4092000640 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555face6e7c0 0x555face70bb0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9a8e8bf79f107ab server_cookie=1bbdc880a2765666 in_seq=0 out_seq=0 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.324+0000 7f407cc00640 10 client.27239 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.324+0000 7f407cc00640 1 -- 172.21.15.107:0/1118627882 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555facdc7950 con 0x555face6e7c0 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f407cc00640 10 client.27239 handle_command_reply: tid=0 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 client.27239 shutdown 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 -- 172.21.15.107:0/1118627882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face53b90 msgr2=0x555face5bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face53b90 0x555face5bc10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f408400e9c0 tx=0x7f4084009700 comp rx=0 tx=0).stop 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 -- 172.21.15.107:0/1118627882 shutdown_connections 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555face6e7c0 0x555face70bb0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face5ed70 0x555face5c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face544b0 0x555face5c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 --2- 172.21.15.107:0/1118627882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face53b90 0x555face5bc10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 -- 172.21.15.107:0/1118627882 >> 172.21.15.107:0/1118627882 conn(0x555face50120 msgr2=0x555face55e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f407cc00640 0 client.27239 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 -- 172.21.15.107:0/1118627882 shutdown_connections 2024-09-19T04:31:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 1 -- 172.21.15.107:0/1118627882 wait complete. 2024-09-19T04:31:43.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.325+0000 7f409c875740 20 client.27239 trim_cache size 0 max 16384 2024-09-19T04:31:43.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.345+0000 7f409c875740 1 Processor -- start 2024-09-19T04:31:43.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.345+0000 7f409c875740 1 -- start start 2024-09-19T04:31:43.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f409c875740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f409c875740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f409c875740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face762a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f409c875740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40941afac0 con 0x555face53b90 2024-09-19T04:31:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f409c875740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40941b1600 con 0x555face5ed70 2024-09-19T04:31:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f409c875740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40941b0190 con 0x555face544b0 2024-09-19T04:31:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f4092000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f4092000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53302/0 (socket says 172.21.15.107:53302) 2024-09-19T04:31:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f4092000640 1 -- 172.21.15.107:0/4092503309 learned_addr learned my addr 172.21.15.107:0/4092503309 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f4092a00640 1 --2- 172.21.15.107:0/4092503309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f4093400640 1 --2- 172.21.15.107:0/4092503309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f4092000640 1 -- 172.21.15.107:0/4092503309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 msgr2=0x555face762a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:43.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f4092000640 1 --2- 172.21.15.107:0/4092503309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face762a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f4092000640 1 -- 172.21.15.107:0/4092503309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 msgr2=0x555face565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:43.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f4092000640 1 --2- 172.21.15.107:0/4092503309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.346+0000 7f4092000640 1 -- 172.21.15.107:0/4092503309 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555face53910 con 0x555face544b0 2024-09-19T04:31:43.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.347+0000 7f4092000640 1 --2- 172.21.15.107:0/4092503309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5d470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x555face5b4a0 tx=0x7f40800138f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f5121e411d307e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:43.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.347+0000 7f407f400640 1 -- 172.21.15.107:0/4092503309 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080005220 con 0x555face544b0 2024-09-19T04:31:43.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.347+0000 7f407f400640 1 -- 172.21.15.107:0/4092503309 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4080004c80 con 0x555face544b0 2024-09-19T04:31:43.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.347+0000 7f407f400640 1 -- 172.21.15.107:0/4092503309 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408001e8f0 con 0x555face544b0 2024-09-19T04:31:43.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.348+0000 7f409c875740 1 -- 172.21.15.107:0/4092503309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 msgr2=0x555face5d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.348+0000 7f409c875740 1 --2- 172.21.15.107:0/4092503309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5d470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x555face5b4a0 tx=0x7f40800138f0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.348+0000 7f409c875740 1 -- 172.21.15.107:0/4092503309 shutdown_connections 2024-09-19T04:31:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.348+0000 7f409c875740 1 --2- 172.21.15.107:0/4092503309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.348+0000 7f409c875740 1 --2- 172.21.15.107:0/4092503309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face5d470 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.348+0000 7f409c875740 1 --2- 172.21.15.107:0/4092503309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.348+0000 7f409c875740 1 -- 172.21.15.107:0/4092503309 >> 172.21.15.107:0/4092503309 conn(0x555face50120 msgr2=0x555face5abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.348+0000 7f409c875740 1 -- 172.21.15.107:0/4092503309 shutdown_connections 2024-09-19T04:31:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.348+0000 7f409c875740 1 -- 172.21.15.107:0/4092503309 wait complete. 2024-09-19T04:31:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.349+0000 7f409c875740 1 -- start start 2024-09-19T04:31:43.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.349+0000 7f409c875740 1 Processor -- start 2024-09-19T04:31:43.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.349+0000 7f409c875740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face62d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.349+0000 7f409c875740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face63250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.349+0000 7f409c875740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face63790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f409c875740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40941b0f30 con 0x555face53b90 2024-09-19T04:31:43.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f409c875740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40941b0860 con 0x555face5ed70 2024-09-19T04:31:43.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f409c875740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40941adac0 con 0x555face544b0 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f409c875740 10 client.0 _ll_register_callbacks cb 0x7f40840d8300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4092a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face62d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4092000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face63250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4092a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face62d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37226/0 (socket says 172.21.15.107:37226) 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4092000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face63250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53308/0 (socket says 172.21.15.107:53308) 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4092a00640 1 -- 172.21.15.107:0/2821623358 learned_addr learned my addr 172.21.15.107:0/2821623358 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4093400640 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face63790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4093400640 1 -- 172.21.15.107:0/2821623358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 msgr2=0x555face63250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4093400640 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face63250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4093400640 1 -- 172.21.15.107:0/2821623358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 msgr2=0x555face62d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4093400640 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face62d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.350+0000 7f4093400640 1 -- 172.21.15.107:0/2821623358 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555faced8470 con 0x555face5ed70 2024-09-19T04:31:43.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.351+0000 7f4093400640 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face63790 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f408c006010 tx=0x7f408c02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed555b8148a9f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.351+0000 7f407cc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.351+0000 7f407cc00640 1 -- 172.21.15.107:0/2821623358 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c059060 con 0x555face5ed70 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.351+0000 7f407cc00640 1 -- 172.21.15.107:0/2821623358 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f408c0046a0 con 0x555face5ed70 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.351+0000 7f409c875740 1 -- 172.21.15.107:0/2821623358 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4080004c80 con 0x555face5ed70 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.351+0000 7f407cc00640 1 -- 172.21.15.107:0/2821623358 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f408c02f610 con 0x555face5ed70 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.351+0000 7f409c875740 10 client.27245 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.351+0000 7f409c875740 1 -- 172.21.15.107:0/2821623358 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555faced9760 con 0x555face5ed70 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.351+0000 7f407cc00640 1 -- 172.21.15.107:0/2821623358 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4080004c80 con 0x555face5ed70 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.351+0000 7f407cc00640 1 -- 172.21.15.107:0/2821623358 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f408c053070 con 0x555face5ed70 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.352+0000 7f409c875740 10 client.27245 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.352+0000 7f409c875740 10 client.27245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.352+0000 7f409c875740 20 client.27245 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.352+0000 7f409c875740 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555faced9e80 0x555facedc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:43.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.352+0000 7f409c875740 4 client.27245 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:31:43.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.352+0000 7f409c875740 1 -- 172.21.15.107:0/2821623358 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555facdc1e60 con 0x555faced9e80 2024-09-19T04:31:43.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.352+0000 7f4092a00640 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555faced9e80 0x555facedc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:43.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.352+0000 7f4092a00640 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555faced9e80 0x555facedc2a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0af5245a9cb1132 server_cookie=401e9cef3328f274 in_seq=0 out_seq=0 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.352+0000 7f407cc00640 10 client.27245 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f407cc00640 1 -- 172.21.15.107:0/2821623358 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555facdc1e60 con 0x555faced9e80 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f407cc00640 10 client.27245 handle_command_reply: tid=0 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 client.27245 shutdown 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 -- 172.21.15.107:0/2821623358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 msgr2=0x555face63790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face63790 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f408c006010 tx=0x7f408c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 -- 172.21.15.107:0/2821623358 shutdown_connections 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x555faced9e80 0x555facedc2a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555face5ed70 0x555face63790 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555face544b0 0x555face63250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 --2- 172.21.15.107:0/2821623358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555face53b90 0x555face62d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 -- 172.21.15.107:0/2821623358 >> 172.21.15.107:0/2821623358 conn(0x555face50120 msgr2=0x555face5a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:43.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f407cc00640 0 client.27245 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:43.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 -- 172.21.15.107:0/2821623358 shutdown_connections 2024-09-19T04:31:43.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 1 -- 172.21.15.107:0/2821623358 wait complete. 2024-09-19T04:31:43.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.353+0000 7f409c875740 20 client.27245 trim_cache size 0 max 16384 2024-09-19T04:31:43.358 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:31:43.358 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:31:43.358 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:31:43.358 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:31:43.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 -- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40780a3430 msgr2=0x7f40780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:43.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40780a3430 0x7f40780a58f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f408c031f80 tx=0x7f408c004a60 comp rx=0 tx=0).stop 2024-09-19T04:31:43.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 -- 172.21.15.107:0/3303023697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4094103ff0 msgr2=0x7f409419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:43.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4094103ff0 0x7f409419a6b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f408400e9f0 tx=0x7f408400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 -- 172.21.15.107:0/3303023697 shutdown_connections 2024-09-19T04:31:43.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f40780a3430 0x7f40780a58f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4094109120 0x7f409419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40941049a0 0x7f409419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 --2- 172.21.15.107:0/3303023697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4094103ff0 0x7f409419a6b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:43.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 -- 172.21.15.107:0/3303023697 >> 172.21.15.107:0/3303023697 conn(0x7f40940ffab0 msgr2=0x7f4094100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:43.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 -- 172.21.15.107:0/3303023697 shutdown_connections 2024-09-19T04:31:43.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:43.354+0000 7f4099e00640 1 -- 172.21.15.107:0/3303023697 wait complete. 2024-09-19T04:31:43.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 384796 2.87 MB/sec execute 481 sec latency 1398.441 ms 2024-09-19T04:31:44.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:31:44.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:43 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:31:44.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:43 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:31:44.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:43 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:44.232 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:31:44.232 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:31:44.644 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:44.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 386122 2.88 MB/sec execute 482 sec latency 1637.221 ms 2024-09-19T04:31:45.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:44 smithi107 ceph-mon[30659]: pgmap v355: 97 pgs: 97 active+clean; 314 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 889 KiB/s wr, 97 op/s 2024-09-19T04:31:45.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:44 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:31:45.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:44 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:31:45.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:44 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:44 smithi115 ceph-mon[35477]: pgmap v355: 97 pgs: 97 active+clean; 314 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 889 KiB/s wr, 97 op/s 2024-09-19T04:31:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:44 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:31:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:44 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:31:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:44 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:44 smithi145 ceph-mon[35096]: pgmap v355: 97 pgs: 97 active+clean; 314 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 889 KiB/s wr, 97 op/s 2024-09-19T04:31:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:44 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:31:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:44 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:31:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:44 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:31:45.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.336+0000 7f282a200640 1 -- 172.21.15.107:0/3512214444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f28241079c0 msgr2=0x7f2824107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:45.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.336+0000 7f282a200640 1 --2- 172.21.15.107:0/3512214444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f28241079c0 0x7f2824107da0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2814009890 tx=0x7f281402a370 comp rx=0 tx=0).stop 2024-09-19T04:31:45.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.336+0000 7f282a200640 1 -- 172.21.15.107:0/3512214444 shutdown_connections 2024-09-19T04:31:45.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.336+0000 7f282a200640 1 --2- 172.21.15.107:0/3512214444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2824102360 0x7f282410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:45.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.336+0000 7f282a200640 1 --2- 172.21.15.107:0/3512214444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28241019c0 0x7f2824101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:45.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.336+0000 7f282a200640 1 --2- 172.21.15.107:0/3512214444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f28241079c0 0x7f2824107da0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:45.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.336+0000 7f282a200640 1 -- 172.21.15.107:0/3512214444 >> 172.21.15.107:0/3512214444 conn(0x7f28240fd740 msgr2=0x7f28240ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:45.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.337+0000 7f282a200640 1 -- 172.21.15.107:0/3512214444 shutdown_connections 2024-09-19T04:31:45.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.337+0000 7f282a200640 1 -- 172.21.15.107:0/3512214444 wait complete. 2024-09-19T04:31:45.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.337+0000 7f282a200640 1 Processor -- start 2024-09-19T04:31:45.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f282a200640 1 -- start start 2024-09-19T04:31:45.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f282a200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28241019c0 0x7f28241a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:45.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f282a200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2824102360 0x7f28241a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:45.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f282a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28241079c0 0x7f282410daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:45.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f282a200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2824111d90 con 0x7f28241079c0 2024-09-19T04:31:45.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f282a200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2824111c10 con 0x7f28241019c0 2024-09-19T04:31:45.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f282a200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2824111f10 con 0x7f2824102360 2024-09-19T04:31:45.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f2828e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28241019c0 0x7f28241a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:45.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f281be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2824102360 0x7f28241a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f2828e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28241019c0 0x7f28241a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:49370/0 (socket says 172.21.15.107:49370) 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f2828e00640 1 -- 172.21.15.107:0/140571802 learned_addr learned my addr 172.21.15.107:0/140571802 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.338+0000 7f2829800640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28241079c0 0x7f282410daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f2828e00640 1 -- 172.21.15.107:0/140571802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2824102360 msgr2=0x7f28241a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f2828e00640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2824102360 0x7f28241a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f2828e00640 1 -- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28241079c0 msgr2=0x7f282410daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f2828e00640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28241079c0 0x7f282410daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f2828e00640 1 -- 172.21.15.107:0/140571802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282410e360 con 0x7f28241019c0 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f2829800640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28241079c0 0x7f282410daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f281be00640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2824102360 0x7f28241a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f2828e00640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28241019c0 0x7f28241a46c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2814035040 tx=0x7f2814033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:45.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f2822000640 1 -- 172.21.15.107:0/140571802 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814037040 con 0x7f28241019c0 2024-09-19T04:31:45.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f282a200640 1 -- 172.21.15.107:0/140571802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282410e5f0 con 0x7f28241019c0 2024-09-19T04:31:45.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f2822000640 1 -- 172.21.15.107:0/140571802 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2814040070 con 0x7f28241019c0 2024-09-19T04:31:45.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f2822000640 1 -- 172.21.15.107:0/140571802 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814003a10 con 0x7f28241019c0 2024-09-19T04:31:45.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.339+0000 7f282a200640 1 -- 172.21.15.107:0/140571802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28241ab5d0 con 0x7f28241019c0 2024-09-19T04:31:45.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.340+0000 7f282a200640 1 -- 172.21.15.107:0/140571802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2824103100 con 0x7f28241019c0 2024-09-19T04:31:45.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.343+0000 7f2822000640 1 -- 172.21.15.107:0/140571802 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f2814004170 con 0x7f28241019c0 2024-09-19T04:31:45.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.344+0000 7f2822000640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28080a33b0 0x7f28080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:45.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.344+0000 7f2822000640 1 -- 172.21.15.107:0/140571802 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f28140efc80 con 0x7f28241019c0 2024-09-19T04:31:45.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.344+0000 7f281be00640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28080a33b0 0x7f28080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:45.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.345+0000 7f281be00640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28080a33b0 0x7f28080a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f280c0096c0 tx=0x7f280c009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:45.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.348+0000 7f2822000640 1 -- 172.21.15.107:0/140571802 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f28140bc730 con 0x7f28241019c0 2024-09-19T04:31:45.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.595+0000 7f282a200640 1 -- 172.21.15.107:0/140571802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28241ab7a0 con 0x7f28241019c0 2024-09-19T04:31:45.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.596+0000 7f2822000640 1 -- 172.21.15.107:0/140571802 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11992 (secure 0 0 0) 0x7f28140bc030 con 0x7f28241019c0 2024-09-19T04:31:45.598 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:31:45.598 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":48,"btime":"2024-09-19T04:29:19:894497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:29:18.944591+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24490,24496,24500,24539,24517]},"id":1}]} 2024-09-19T04:31:45.598 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 48 2024-09-19T04:31:45.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 -- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28080a33b0 msgr2=0x7f28080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:45.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28080a33b0 0x7f28080a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f280c0096c0 tx=0x7f280c009260 comp rx=0 tx=0).stop 2024-09-19T04:31:45.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 -- 172.21.15.107:0/140571802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28241019c0 msgr2=0x7f28241a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:45.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28241019c0 0x7f28241a46c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2814035040 tx=0x7f2814033b60 comp rx=0 tx=0).stop 2024-09-19T04:31:45.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 -- 172.21.15.107:0/140571802 shutdown_connections 2024-09-19T04:31:45.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f28080a33b0 0x7f28080a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:45.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28241079c0 0x7f282410daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:45.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2824102360 0x7f28241a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:45.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 --2- 172.21.15.107:0/140571802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28241019c0 0x7f28241a46c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:45.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 -- 172.21.15.107:0/140571802 >> 172.21.15.107:0/140571802 conn(0x7f28240fd740 msgr2=0x7f28240fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:45.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.599+0000 7f282a200640 1 -- 172.21.15.107:0/140571802 shutdown_connections 2024-09-19T04:31:45.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:45.600+0000 7f282a200640 1 -- 172.21.15.107:0/140571802 wait complete. 2024-09-19T04:31:45.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 387353 2.89 MB/sec execute 483 sec latency 11.464 ms 2024-09-19T04:31:46.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:45 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/140571802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:31:46.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:45 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/140571802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:31:46.317 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:31:46.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:45 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/140571802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:31:46.692 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:46.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 388368 2.89 MB/sec execute 484 sec latency 331.639 ms 2024-09-19T04:31:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:46 smithi145 ceph-mon[35096]: pgmap v356: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-09-19T04:31:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:46 smithi107 ceph-mon[30659]: pgmap v356: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-09-19T04:31:47.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:46 smithi115 ceph-mon[35477]: pgmap v356: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-09-19T04:31:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.675+0000 7efe1da00640 1 -- 172.21.15.107:0/2748352416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1810e430 msgr2=0x7efe18110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.675+0000 7efe1da00640 1 --2- 172.21.15.107:0/2748352416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1810e430 0x7efe18110820 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efe10009ca0 tx=0x7efe1002a950 comp rx=0 tx=0).stop 2024-09-19T04:31:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.675+0000 7efe1da00640 1 -- 172.21.15.107:0/2748352416 shutdown_connections 2024-09-19T04:31:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.675+0000 7efe1da00640 1 --2- 172.21.15.107:0/2748352416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1810e430 0x7efe18110820 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.675+0000 7efe1da00640 1 --2- 172.21.15.107:0/2748352416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe18074ee0 0x7efe1810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.675+0000 7efe1da00640 1 --2- 172.21.15.107:0/2748352416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe180745c0 0x7efe180749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.675+0000 7efe1da00640 1 -- 172.21.15.107:0/2748352416 >> 172.21.15.107:0/2748352416 conn(0x7efe180fd8b0 msgr2=0x7efe180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.675+0000 7efe1da00640 1 -- 172.21.15.107:0/2748352416 shutdown_connections 2024-09-19T04:31:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.675+0000 7efe1da00640 1 -- 172.21.15.107:0/2748352416 wait complete. 2024-09-19T04:31:47.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.676+0000 7efe1da00640 1 Processor -- start 2024-09-19T04:31:47.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.676+0000 7efe1da00640 1 -- start start 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.676+0000 7efe1da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe180745c0 0x7efe1819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.676+0000 7efe1da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe18074ee0 0x7efe1819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.676+0000 7efe1da00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe1810e430 0x7efe1819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.676+0000 7efe1da00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe181141c0 con 0x7efe180745c0 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.676+0000 7efe1da00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe18114040 con 0x7efe18074ee0 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.676+0000 7efe1da00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efe18114340 con 0x7efe1810e430 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe180745c0 0x7efe1819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe180745c0 0x7efe1819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37268/0 (socket says 172.21.15.107:37268) 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16a00640 1 -- 172.21.15.107:0/2713444508 learned_addr learned my addr 172.21.15.107:0/2713444508 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16000640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe18074ee0 0x7efe1819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe17400640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe1810e430 0x7efe1819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16a00640 1 -- 172.21.15.107:0/2713444508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe1810e430 msgr2=0x7efe1819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16a00640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe1810e430 0x7efe1819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16a00640 1 -- 172.21.15.107:0/2713444508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe18074ee0 msgr2=0x7efe1819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:47.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16a00640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe18074ee0 0x7efe1819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16a00640 1 -- 172.21.15.107:0/2713444508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1819f610 con 0x7efe180745c0 2024-09-19T04:31:47.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16000640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe18074ee0 0x7efe1819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:47.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.677+0000 7efe16a00640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe180745c0 0x7efe1819a660 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efe0800e9f0 tx=0x7efe0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:47.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.678+0000 7efe0f400640 1 -- 172.21.15.107:0/2713444508 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0800cd80 con 0x7efe180745c0 2024-09-19T04:31:47.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.678+0000 7efe0f400640 1 -- 172.21.15.107:0/2713444508 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe08005500 con 0x7efe180745c0 2024-09-19T04:31:47.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.678+0000 7efe0f400640 1 -- 172.21.15.107:0/2713444508 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08010690 con 0x7efe180745c0 2024-09-19T04:31:47.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.678+0000 7efe1da00640 1 -- 172.21.15.107:0/2713444508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1819f900 con 0x7efe180745c0 2024-09-19T04:31:47.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.678+0000 7efe1da00640 1 -- 172.21.15.107:0/2713444508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe181a71e0 con 0x7efe180745c0 2024-09-19T04:31:47.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.681+0000 7efe203c0740 1 Processor -- start 2024-09-19T04:31:47.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.681+0000 7efe203c0740 1 -- start start 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.681+0000 7efe0f400640 1 -- 172.21.15.107:0/2713444508 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7efe080046c0 con 0x7efe180745c0 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.681+0000 7efe203c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be7c70 0x55b250be8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.681+0000 7efe203c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be8590 0x55b250bf2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe203c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250bf2e50 0x55b250bf5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe203c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b250b4e3e0 con 0x55b250be8590 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe203c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b250af5d80 con 0x55b250bf2e50 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe203c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b250b4a3a0 con 0x55b250be7c70 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe17400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be7c70 0x55b250be8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe17400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be7c70 0x55b250be8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53366/0 (socket says 172.21.15.107:53366) 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe17400640 1 -- 172.21.15.107:0/3974940356 learned_addr learned my addr 172.21.15.107:0/3974940356 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe16000640 1 --2- 172.21.15.107:0/3974940356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250bf2e50 0x55b250bf5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe16a00640 1 --2- 172.21.15.107:0/3974940356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be8590 0x55b250bf2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe16000640 1 -- 172.21.15.107:0/3974940356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be7c70 msgr2=0x55b250be8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:47.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe16000640 1 --2- 172.21.15.107:0/3974940356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be7c70 0x55b250be8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe16000640 1 -- 172.21.15.107:0/3974940356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be8590 msgr2=0x55b250bf2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:47.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe16000640 1 --2- 172.21.15.107:0/3974940356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be8590 0x55b250bf2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe16000640 1 -- 172.21.15.107:0/3974940356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b250b14ae0 con 0x55b250bf2e50 2024-09-19T04:31:47.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.682+0000 7efe16a00640 1 --2- 172.21.15.107:0/3974940356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be8590 0x55b250bf2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:47.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.683+0000 7efe16000640 1 --2- 172.21.15.107:0/3974940356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250bf2e50 0x55b250bf5240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efe04009f20 tx=0x7efe040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fabbeb4b23bd79a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.683+0000 7efe03e00640 1 -- 172.21.15.107:0/3974940356 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04004030 con 0x55b250bf2e50 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.683+0000 7efe0f400640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efdfc0a3430 0x7efdfc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.683+0000 7efe03e00640 1 -- 172.21.15.107:0/3974940356 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe04005500 con 0x55b250bf2e50 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.683+0000 7efe03e00640 1 -- 172.21.15.107:0/3974940356 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04004e20 con 0x55b250bf2e50 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.683+0000 7efe0f400640 1 -- 172.21.15.107:0/2713444508 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7efe080d00e0 con 0x7efe180745c0 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.683+0000 7efe17400640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efdfc0a3430 0x7efdfc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.684+0000 7efe203c0740 1 -- 172.21.15.107:0/3974940356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250bf2e50 msgr2=0x55b250bf5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.684+0000 7efe203c0740 1 --2- 172.21.15.107:0/3974940356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250bf2e50 0x55b250bf5240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efe04009f20 tx=0x7efe040118d0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.684+0000 7efe17400640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efdfc0a3430 0x7efdfc0a58f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe10009a60 tx=0x7efe10005d80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.684+0000 7efe203c0740 1 -- 172.21.15.107:0/3974940356 shutdown_connections 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.684+0000 7efe203c0740 1 --2- 172.21.15.107:0/3974940356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250bf2e50 0x55b250bf5240 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.684+0000 7efe203c0740 1 --2- 172.21.15.107:0/3974940356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be8590 0x55b250bf2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.684+0000 7efe203c0740 1 --2- 172.21.15.107:0/3974940356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be7c70 0x55b250be8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.684+0000 7efe203c0740 1 -- 172.21.15.107:0/3974940356 >> 172.21.15.107:0/3974940356 conn(0x55b250be4200 msgr2=0x55b250be45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:47.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.684+0000 7efe203c0740 1 -- 172.21.15.107:0/3974940356 shutdown_connections 2024-09-19T04:31:47.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.684+0000 7efe203c0740 1 -- 172.21.15.107:0/3974940356 wait complete. 2024-09-19T04:31:47.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.685+0000 7efe203c0740 1 -- start start 2024-09-19T04:31:47.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.685+0000 7efe203c0740 1 Processor -- start 2024-09-19T04:31:47.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.685+0000 7efe203c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 0x55b250befcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.685+0000 7efe203c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 0x55b250bf01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.685+0000 7efe203c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 0x55b250bf0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.685+0000 7efe203c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b250bf5a20 con 0x55b250bf2e50 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.685+0000 7efe203c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b250b090f0 con 0x55b250be7c70 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.685+0000 7efe203c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b250bf5ba0 con 0x55b250be8590 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe203c0740 10 client.0 _ll_register_callbacks cb 0x55b250af0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe16a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 0x55b250befcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe16a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 0x55b250befcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:49398/0 (socket says 172.21.15.107:49398) 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe16a00640 1 -- 172.21.15.107:0/1911710202 learned_addr learned my addr 172.21.15.107:0/1911710202 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe17400640 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 0x55b250bf0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe16000640 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 0x55b250bf01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe16a00640 1 -- 172.21.15.107:0/1911710202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 msgr2=0x55b250bf01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe16a00640 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 0x55b250bf01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe16a00640 1 -- 172.21.15.107:0/1911710202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 msgr2=0x55b250bf0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe16a00640 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 0x55b250bf0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe16a00640 1 -- 172.21.15.107:0/1911710202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2508bf3c0 con 0x55b250be7c70 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe16a00640 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 0x55b250befcb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efe0800e9c0 tx=0x7efe0809c150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7111e344d679bd2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.686+0000 7efe00c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe00c00640 1 -- 172.21.15.107:0/1911710202 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe080cdc80 con 0x55b250be7c70 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe00c00640 1 -- 172.21.15.107:0/1911710202 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7efe080a4b60 con 0x55b250be7c70 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe203c0740 1 -- 172.21.15.107:0/1911710202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe04005500 con 0x55b250be7c70 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe00c00640 1 -- 172.21.15.107:0/1911710202 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7efe0809ca80 con 0x55b250be7c70 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe203c0740 10 client.27263 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe203c0740 1 -- 172.21.15.107:0/1911710202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b250c01a60 con 0x55b250be7c70 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe00c00640 1 -- 172.21.15.107:0/1911710202 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe04005500 con 0x55b250be7c70 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe00c00640 1 -- 172.21.15.107:0/1911710202 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7efe0809b7d0 con 0x55b250be7c70 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe203c0740 10 client.27263 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe203c0740 10 client.27263 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe203c0740 20 client.27263 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe203c0740 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b250c02e60 0x55b250c052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe203c0740 4 client.27263 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.687+0000 7efe203c0740 1 -- 172.21.15.107:0/1911710202 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b250b5b950 con 0x55b250c02e60 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.688+0000 7efe16000640 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b250c02e60 0x55b250c052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.688+0000 7efe16000640 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b250c02e60 0x55b250c052d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=41653d3d359b344f server_cookie=cb8173f491c50369 in_seq=0 out_seq=0 2024-09-19T04:31:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.688+0000 7efe00c00640 10 client.27263 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:31:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.689+0000 7efe00c00640 1 -- 172.21.15.107:0/1911710202 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b250b5b950 con 0x55b250c02e60 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.689+0000 7efe00c00640 10 client.27263 handle_command_reply: tid=0 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.689+0000 7efe203c0740 1 client.27263 shutdown 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.689+0000 7efe203c0740 1 -- 172.21.15.107:0/1911710202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 msgr2=0x55b250befcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.689+0000 7efe203c0740 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 0x55b250befcb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efe0800e9c0 tx=0x7efe0809c150 comp rx=0 tx=0).stop 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.690+0000 7efe203c0740 1 -- 172.21.15.107:0/1911710202 shutdown_connections 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.690+0000 7efe203c0740 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b250c02e60 0x55b250c052d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.690+0000 7efe203c0740 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 0x55b250bf0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.690+0000 7efe203c0740 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 0x55b250bf01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.690+0000 7efe203c0740 1 --2- 172.21.15.107:0/1911710202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 0x55b250befcb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.690+0000 7efe203c0740 1 -- 172.21.15.107:0/1911710202 >> 172.21.15.107:0/1911710202 conn(0x55b250be4200 msgr2=0x55b250be9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.690+0000 7efe00c00640 0 client.27263 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:31:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.690+0000 7efe203c0740 1 -- 172.21.15.107:0/1911710202 shutdown_connections 2024-09-19T04:31:47.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.690+0000 7efe203c0740 1 -- 172.21.15.107:0/1911710202 wait complete. 2024-09-19T04:31:47.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.690+0000 7efe203c0740 20 client.27263 trim_cache size 0 max 16384 2024-09-19T04:31:47.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe203c0740 1 Processor -- start 2024-09-19T04:31:47.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe203c0740 1 -- start start 2024-09-19T04:31:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe203c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be7c70 0x55b250bea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe203c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be8590 0x55b250c0ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe203c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250bf2e50 0x55b250c0b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe203c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe181afa30 con 0x55b250be7c70 2024-09-19T04:31:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe203c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe181b1570 con 0x55b250be8590 2024-09-19T04:31:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe203c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efe181b0100 con 0x55b250bf2e50 2024-09-19T04:31:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe16a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be7c70 0x55b250bea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe16a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be7c70 0x55b250bea6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37278/0 (socket says 172.21.15.107:37278) 2024-09-19T04:31:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe16a00640 1 -- 172.21.15.107:0/4266331683 learned_addr learned my addr 172.21.15.107:0/4266331683 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe17400640 1 --2- 172.21.15.107:0/4266331683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250bf2e50 0x55b250c0b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.704+0000 7efe16000640 1 --2- 172.21.15.107:0/4266331683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be8590 0x55b250c0ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.705+0000 7efe16a00640 1 -- 172.21.15.107:0/4266331683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250bf2e50 msgr2=0x55b250c0b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.705+0000 7efe16a00640 1 --2- 172.21.15.107:0/4266331683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250bf2e50 0x55b250c0b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.705+0000 7efe16a00640 1 -- 172.21.15.107:0/4266331683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be8590 msgr2=0x55b250c0ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.705+0000 7efe16a00640 1 --2- 172.21.15.107:0/4266331683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be8590 0x55b250c0ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.705+0000 7efe16a00640 1 -- 172.21.15.107:0/4266331683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b250be7aa0 con 0x55b250be7c70 2024-09-19T04:31:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.705+0000 7efe16a00640 1 --2- 172.21.15.107:0/4266331683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be7c70 0x55b250bea6d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efe0809a630 tx=0x7efe08004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff36aa72bf3af11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.705+0000 7efe03400640 1 -- 172.21.15.107:0/4266331683 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe080cdc80 con 0x55b250be7c70 2024-09-19T04:31:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.705+0000 7efe03400640 1 -- 172.21.15.107:0/4266331683 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0809c380 con 0x55b250be7c70 2024-09-19T04:31:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.705+0000 7efe03400640 1 -- 172.21.15.107:0/4266331683 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe080a4b60 con 0x55b250be7c70 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.706+0000 7efe203c0740 1 -- 172.21.15.107:0/4266331683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be7c70 msgr2=0x55b250bea6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.706+0000 7efe203c0740 1 --2- 172.21.15.107:0/4266331683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be7c70 0x55b250bea6d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efe0809a630 tx=0x7efe08004ae0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.706+0000 7efe203c0740 1 -- 172.21.15.107:0/4266331683 shutdown_connections 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.706+0000 7efe203c0740 1 --2- 172.21.15.107:0/4266331683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250bf2e50 0x55b250c0b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.706+0000 7efe203c0740 1 --2- 172.21.15.107:0/4266331683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be8590 0x55b250c0ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.706+0000 7efe203c0740 1 --2- 172.21.15.107:0/4266331683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250be7c70 0x55b250bea6d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.706+0000 7efe203c0740 1 -- 172.21.15.107:0/4266331683 >> 172.21.15.107:0/4266331683 conn(0x55b250be4200 msgr2=0x55b250bf1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.706+0000 7efe203c0740 1 -- 172.21.15.107:0/4266331683 shutdown_connections 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.706+0000 7efe203c0740 1 -- 172.21.15.107:0/4266331683 wait complete. 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.706+0000 7efe203c0740 1 -- start start 2024-09-19T04:31:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.707+0000 7efe203c0740 1 Processor -- start 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.707+0000 7efe203c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 0x55b250bf6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.707+0000 7efe203c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 0x55b250bf7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.707+0000 7efe203c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 0x55b250bf7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.707+0000 7efe203c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe181b0ea0 con 0x55b250bf2e50 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.707+0000 7efe203c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe181b07d0 con 0x55b250be7c70 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.707+0000 7efe203c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efe181ada30 con 0x55b250be8590 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.707+0000 7efe203c0740 10 client.0 _ll_register_callbacks cb 0x7efe0800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.707+0000 7efe17400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 0x55b250bf7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe16000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 0x55b250bf7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe17400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 0x55b250bf7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37284/0 (socket says 172.21.15.107:37284) 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe17400640 1 -- 172.21.15.107:0/3558976120 learned_addr learned my addr 172.21.15.107:0/3558976120 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe16a00640 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 0x55b250bf6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe16000640 1 -- 172.21.15.107:0/3558976120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 msgr2=0x55b250bf6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe16000640 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 0x55b250bf6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe16000640 1 -- 172.21.15.107:0/3558976120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 msgr2=0x55b250bf7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe16000640 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 0x55b250bf7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe16000640 1 -- 172.21.15.107:0/3558976120 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b250bdf770 con 0x55b250be8590 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe17400640 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 0x55b250bf7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe16000640 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 0x55b250bf7310 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe0401b040 tx=0x7efe04004890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf6dbf7d1ed35b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe00c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.708+0000 7efe00c00640 1 -- 172.21.15.107:0/3558976120 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04004250 con 0x55b250be8590 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe203c0740 1 -- 172.21.15.107:0/3558976120 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe0809c380 con 0x55b250be8590 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe00c00640 1 -- 172.21.15.107:0/3558976120 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7efe040043f0 con 0x55b250be8590 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe00c00640 1 -- 172.21.15.107:0/3558976120 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7efe040035f0 con 0x55b250be8590 2024-09-19T04:31:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe203c0740 10 client.27220 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe203c0740 1 -- 172.21.15.107:0/3558976120 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b250c6dea0 con 0x55b250be8590 2024-09-19T04:31:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe00c00640 1 -- 172.21.15.107:0/3558976120 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0809c380 con 0x55b250be8590 2024-09-19T04:31:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe00c00640 1 -- 172.21.15.107:0/3558976120 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7efe0403a020 con 0x55b250be8590 2024-09-19T04:31:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe203c0740 10 client.27220 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe203c0740 10 client.27220 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:31:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe203c0740 20 client.27220 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe203c0740 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b250c6e460 0x55b250c70880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe203c0740 4 client.27220 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.709+0000 7efe203c0740 1 -- 172.21.15.107:0/3558976120 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b250b55e60 con 0x55b250c6e460 2024-09-19T04:31:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.710+0000 7efe16a00640 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b250c6e460 0x55b250c70880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.710+0000 7efe16a00640 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b250c6e460 0x55b250c70880 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc923088d11af138 server_cookie=4bc68d5a01bb3e57 in_seq=0 out_seq=0 2024-09-19T04:31:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.710+0000 7efe00c00640 10 client.27220 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:31:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.710+0000 7efe00c00640 1 -- 172.21.15.107:0/3558976120 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b250b55e60 con 0x55b250c6e460 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.710+0000 7efe00c00640 10 client.27220 handle_command_reply: tid=0 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.710+0000 7efe203c0740 1 client.27220 shutdown 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.710+0000 7efe203c0740 1 -- 172.21.15.107:0/3558976120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 msgr2=0x55b250bf7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.710+0000 7efe203c0740 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 0x55b250bf7310 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe0401b040 tx=0x7efe04004890 comp rx=0 tx=0).stop 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe203c0740 1 -- 172.21.15.107:0/3558976120 shutdown_connections 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe203c0740 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b250c6e460 0x55b250c70880 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe203c0740 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b250bf2e50 0x55b250bf7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe203c0740 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b250be8590 0x55b250bf7310 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe00c00640 0 client.27220 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe203c0740 1 --2- 172.21.15.107:0/3558976120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b250be7c70 0x55b250bf6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe203c0740 1 -- 172.21.15.107:0/3558976120 >> 172.21.15.107:0/3558976120 conn(0x55b250be4200 msgr2=0x55b250be2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe203c0740 1 -- 172.21.15.107:0/3558976120 shutdown_connections 2024-09-19T04:31:47.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe203c0740 1 -- 172.21.15.107:0/3558976120 wait complete. 2024-09-19T04:31:47.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe203c0740 20 client.27220 trim_cache size 0 max 16384 2024-09-19T04:31:47.717 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:31:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe1da00640 1 -- 172.21.15.107:0/2713444508 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efdfc0a3430 msgr2=0x7efdfc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe1da00640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efdfc0a3430 0x7efdfc0a58f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe10009a60 tx=0x7efe10005d80 comp rx=0 tx=0).stop 2024-09-19T04:31:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe1da00640 1 -- 172.21.15.107:0/2713444508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe180745c0 msgr2=0x7efe1819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.711+0000 7efe1da00640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe180745c0 0x7efe1819a660 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efe0800e9f0 tx=0x7efe0800edf0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.712+0000 7efe1da00640 1 -- 172.21.15.107:0/2713444508 shutdown_connections 2024-09-19T04:31:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.712+0000 7efe1da00640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7efdfc0a3430 0x7efdfc0a58f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.712+0000 7efe1da00640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe1810e430 0x7efe1819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.712+0000 7efe1da00640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe18074ee0 0x7efe1819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.712+0000 7efe1da00640 1 --2- 172.21.15.107:0/2713444508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe180745c0 0x7efe1819a660 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.712+0000 7efe1da00640 1 -- 172.21.15.107:0/2713444508 >> 172.21.15.107:0/2713444508 conn(0x7efe180fd8b0 msgr2=0x7efe180fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.712+0000 7efe1da00640 1 -- 172.21.15.107:0/2713444508 shutdown_connections 2024-09-19T04:31:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:47.712+0000 7efe1da00640 1 -- 172.21.15.107:0/2713444508 wait complete. 2024-09-19T04:31:47.723 INFO:tasks.workunit.client.0.smithi107.stdout: 1 388368 2.89 MB/sec execute 485 sec latency 1331.682 ms 2024-09-19T04:31:48.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 388368 2.88 MB/sec execute 486 sec latency 2331.776 ms 2024-09-19T04:31:48.788 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:31:49.164 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:48 smithi145 ceph-mon[35096]: pgmap v357: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:48 smithi107 ceph-mon[30659]: pgmap v357: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:49.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:48 smithi115 ceph-mon[35477]: pgmap v357: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:49.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 389435 2.89 MB/sec execute 487 sec latency 2834.540 ms 2024-09-19T04:31:49.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.724+0000 7fb639400640 1 -- 172.21.15.107:0/3363150259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb634109120 msgr2=0x7fb634110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:49.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.724+0000 7fb639400640 1 --2- 172.21.15.107:0/3363150259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb634109120 0x7fb634110ca0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009ca0 tx=0x7fb62c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.724+0000 7fb639400640 1 -- 172.21.15.107:0/3363150259 shutdown_connections 2024-09-19T04:31:49.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.724+0000 7fb639400640 1 --2- 172.21.15.107:0/3363150259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb634109120 0x7fb634110ca0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.724+0000 7fb639400640 1 --2- 172.21.15.107:0/3363150259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6341049a0 0x7fb6341089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.724+0000 7fb639400640 1 --2- 172.21.15.107:0/3363150259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb634103ff0 0x7fb6341043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.724+0000 7fb639400640 1 -- 172.21.15.107:0/3363150259 >> 172.21.15.107:0/3363150259 conn(0x7fb6340ffa90 msgr2=0x7fb634101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:49.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.724+0000 7fb639400640 1 -- 172.21.15.107:0/3363150259 shutdown_connections 2024-09-19T04:31:49.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.724+0000 7fb639400640 1 -- 172.21.15.107:0/3363150259 wait complete. 2024-09-19T04:31:49.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.725+0000 7fb639400640 1 Processor -- start 2024-09-19T04:31:49.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.725+0000 7fb639400640 1 -- start start 2024-09-19T04:31:49.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb639400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb634103ff0 0x7fb63419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb639400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6341049a0 0x7fb63419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb639400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb634109120 0x7fb63419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb639400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb634114150 con 0x7fb634109120 2024-09-19T04:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb639400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb634113fd0 con 0x7fb6341049a0 2024-09-19T04:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb639400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341142d0 con 0x7fb634103ff0 2024-09-19T04:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb632800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb634103ff0 0x7fb63419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb632800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb634103ff0 0x7fb63419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:50962/0 (socket says 172.21.15.107:50962) 2024-09-19T04:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb632800640 1 -- 172.21.15.107:0/995422967 learned_addr learned my addr 172.21.15.107:0/995422967 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb62be00640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6341049a0 0x7fb63419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb633200640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb634109120 0x7fb63419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.726+0000 7fb632800640 1 -- 172.21.15.107:0/995422967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6341049a0 msgr2=0x7fb63419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.727+0000 7fb632800640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6341049a0 0x7fb63419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.727+0000 7fb632800640 1 -- 172.21.15.107:0/995422967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb634109120 msgr2=0x7fb63419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.727+0000 7fb632800640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb634109120 0x7fb63419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.727+0000 7fb632800640 1 -- 172.21.15.107:0/995422967 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63419f560 con 0x7fb634103ff0 2024-09-19T04:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.727+0000 7fb632800640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb634103ff0 0x7fb63419a5b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb62400e9f0 tx=0x7fb62400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.727+0000 7fb62b400640 1 -- 172.21.15.107:0/995422967 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62400cd80 con 0x7fb634103ff0 2024-09-19T04:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.727+0000 7fb62b400640 1 -- 172.21.15.107:0/995422967 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb624010910 con 0x7fb634103ff0 2024-09-19T04:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.727+0000 7fb62b400640 1 -- 172.21.15.107:0/995422967 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624018990 con 0x7fb634103ff0 2024-09-19T04:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.727+0000 7fb639400640 1 -- 172.21.15.107:0/995422967 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63419f850 con 0x7fb634103ff0 2024-09-19T04:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.727+0000 7fb639400640 1 -- 172.21.15.107:0/995422967 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6341a7130 con 0x7fb634103ff0 2024-09-19T04:31:49.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.730+0000 7fb62b400640 1 -- 172.21.15.107:0/995422967 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fb624010ac0 con 0x7fb634103ff0 2024-09-19T04:31:49.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.730+0000 7fb63be45740 1 Processor -- start 2024-09-19T04:31:49.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.731+0000 7fb63be45740 1 -- start start 2024-09-19T04:31:49.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.731+0000 7fb63be45740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 0x5629379c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.731+0000 7fb63be45740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379c0590 0x5629379ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.731+0000 7fb63be45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379cae50 0x5629379cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.731+0000 7fb63be45740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5629379263e0 con 0x5629379cae50 2024-09-19T04:31:49.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.731+0000 7fb63be45740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629378cdd80 con 0x5629379c0590 2024-09-19T04:31:49.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.731+0000 7fb63be45740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5629379223a0 con 0x5629379bfc70 2024-09-19T04:31:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.731+0000 7fb632800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379c0590 0x5629379ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.731+0000 7fb632800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379c0590 0x5629379ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44984/0 (socket says 172.21.15.107:44984) 2024-09-19T04:31:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.732+0000 7fb632800640 1 -- 172.21.15.107:0/1383770513 learned_addr learned my addr 172.21.15.107:0/1383770513 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.732+0000 7fb633200640 1 --2- 172.21.15.107:0/1383770513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 0x5629379c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.732+0000 7fb62be00640 1 --2- 172.21.15.107:0/1383770513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379cae50 0x5629379cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.732+0000 7fb632800640 1 -- 172.21.15.107:0/1383770513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 msgr2=0x5629379c0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.732+0000 7fb632800640 1 --2- 172.21.15.107:0/1383770513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 0x5629379c0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.732+0000 7fb632800640 1 -- 172.21.15.107:0/1383770513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379cae50 msgr2=0x5629379cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.732+0000 7fb632800640 1 --2- 172.21.15.107:0/1383770513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379cae50 0x5629379cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.732+0000 7fb632800640 1 -- 172.21.15.107:0/1383770513 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629378ecae0 con 0x5629379c0590 2024-09-19T04:31:49.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.732+0000 7fb62b400640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb6180a33b0 0x7fb6180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.732+0000 7fb632800640 1 --2- 172.21.15.107:0/1383770513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379c0590 0x5629379ca910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb6240cd770 tx=0x7fb6240d8780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b16e2452a676671 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.733+0000 7fb62b400640 1 -- 172.21.15.107:0/995422967 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fb6240d0120 con 0x7fb634103ff0 2024-09-19T04:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.733+0000 7fb633200640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb6180a33b0 0x7fb6180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.733+0000 7fb623400640 1 -- 172.21.15.107:0/1383770513 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624098660 con 0x5629379c0590 2024-09-19T04:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.733+0000 7fb623400640 1 -- 172.21.15.107:0/1383770513 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62409cac0 con 0x5629379c0590 2024-09-19T04:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.733+0000 7fb623400640 1 -- 172.21.15.107:0/1383770513 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6240d3cc0 con 0x5629379c0590 2024-09-19T04:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.734+0000 7fb63be45740 1 -- 172.21.15.107:0/1383770513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379c0590 msgr2=0x5629379ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.734+0000 7fb633200640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb6180a33b0 0x7fb6180a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb62c005e30 tx=0x7fb62c005c30 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.734+0000 7fb63be45740 1 --2- 172.21.15.107:0/1383770513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379c0590 0x5629379ca910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb6240cd770 tx=0x7fb6240d8780 comp rx=0 tx=0).stop 2024-09-19T04:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.734+0000 7fb63be45740 1 -- 172.21.15.107:0/1383770513 shutdown_connections 2024-09-19T04:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.734+0000 7fb63be45740 1 --2- 172.21.15.107:0/1383770513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379cae50 0x5629379cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.734+0000 7fb63be45740 1 --2- 172.21.15.107:0/1383770513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379c0590 0x5629379ca910 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.734+0000 7fb63be45740 1 --2- 172.21.15.107:0/1383770513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 0x5629379c0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.734+0000 7fb63be45740 1 -- 172.21.15.107:0/1383770513 >> 172.21.15.107:0/1383770513 conn(0x5629379bc200 msgr2=0x5629379bc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.734+0000 7fb63be45740 1 -- 172.21.15.107:0/1383770513 shutdown_connections 2024-09-19T04:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.734+0000 7fb63be45740 1 -- 172.21.15.107:0/1383770513 wait complete. 2024-09-19T04:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb63be45740 1 -- start start 2024-09-19T04:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb63be45740 1 Processor -- start 2024-09-19T04:31:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb63be45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379bfc70 0x5629379c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb63be45740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 0x5629379c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb63be45740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb63be45740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5629379cd9c0 con 0x5629379bfc70 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb63be45740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629378e10f0 con 0x5629379cae50 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb63be45740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5629379cdb40 con 0x5629379c0590 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb63be45740 10 client.0 _ll_register_callbacks cb 0x5629378c8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb633200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb62be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 0x5629379c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb632800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379bfc70 0x5629379c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.735+0000 7fb633200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379c8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44988/0 (socket says 172.21.15.107:44988) 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb632800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379bfc70 0x5629379c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44480/0 (socket says 172.21.15.107:44480) 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb633200640 1 -- 172.21.15.107:0/3213671123 learned_addr learned my addr 172.21.15.107:0/3213671123 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb62be00640 1 -- 172.21.15.107:0/3213671123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 msgr2=0x5629379c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb62be00640 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb62be00640 1 -- 172.21.15.107:0/3213671123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379bfc70 msgr2=0x5629379c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb62be00640 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379bfc70 0x5629379c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb62be00640 1 -- 172.21.15.107:0/3213671123 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629376973c0 con 0x5629379c0590 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb62be00640 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 0x5629379c8340 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6340ffeb0 tx=0x7fb61c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ec18142a91670a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb617e00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb617e00640 1 -- 172.21.15.107:0/3213671123 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c004050 con 0x5629379c0590 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb63be45740 1 -- 172.21.15.107:0/3213671123 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb62409cac0 con 0x5629379c0590 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.736+0000 7fb617e00640 1 -- 172.21.15.107:0/3213671123 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fb61c0041f0 con 0x5629379c0590 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb617e00640 1 -- 172.21.15.107:0/3213671123 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb61c0043f0 con 0x5629379c0590 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb63be45740 10 client.27232 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb63be45740 1 -- 172.21.15.107:0/3213671123 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5629379d9b70 con 0x5629379c0590 2024-09-19T04:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb617e00640 1 -- 172.21.15.107:0/3213671123 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62409cac0 con 0x5629379c0590 2024-09-19T04:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb617e00640 1 -- 172.21.15.107:0/3213671123 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fb61c003b80 con 0x5629379c0590 2024-09-19T04:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb63be45740 10 client.27232 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb63be45740 10 client.27232 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb63be45740 20 client.27232 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb63be45740 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5629379db080 0x5629379dd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb63be45740 4 client.27232 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.737+0000 7fb63be45740 1 -- 172.21.15.107:0/3213671123 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562937933950 con 0x5629379db080 2024-09-19T04:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.738+0000 7fb632800640 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5629379db080 0x5629379dd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.738+0000 7fb632800640 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5629379db080 0x5629379dd4a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=988771c46660d84d server_cookie=ea28c4b30c743711 in_seq=0 out_seq=0 2024-09-19T04:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.738+0000 7fb617e00640 10 client.27232 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb617e00640 1 -- 172.21.15.107:0/3213671123 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562937933950 con 0x5629379db080 2024-09-19T04:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb617e00640 10 client.27232 handle_command_reply: tid=0 2024-09-19T04:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb63be45740 1 client.27232 shutdown 2024-09-19T04:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb63be45740 1 -- 172.21.15.107:0/3213671123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 msgr2=0x5629379c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb63be45740 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 0x5629379c8340 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6340ffeb0 tx=0x7fb61c012b60 comp rx=0 tx=0).stop 2024-09-19T04:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb63be45740 1 -- 172.21.15.107:0/3213671123 shutdown_connections 2024-09-19T04:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb63be45740 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x5629379db080 0x5629379dd4a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb63be45740 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb63be45740 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 0x5629379c8340 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb63be45740 1 --2- 172.21.15.107:0/3213671123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379bfc70 0x5629379c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb63be45740 1 -- 172.21.15.107:0/3213671123 >> 172.21.15.107:0/3213671123 conn(0x5629379bc200 msgr2=0x5629379c1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.739+0000 7fb617e00640 0 client.27232 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.740+0000 7fb63be45740 1 -- 172.21.15.107:0/3213671123 shutdown_connections 2024-09-19T04:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.740+0000 7fb63be45740 1 -- 172.21.15.107:0/3213671123 wait complete. 2024-09-19T04:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.740+0000 7fb63be45740 20 client.27232 trim_cache size 0 max 16384 2024-09-19T04:31:49.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.754+0000 7fb63be45740 1 Processor -- start 2024-09-19T04:31:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.754+0000 7fb63be45740 1 -- start start 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.754+0000 7fb63be45740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379bfc70 0x5629379c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.754+0000 7fb63be45740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 0x5629379c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.754+0000 7fb63be45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379cae50 0x5629379e2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.754+0000 7fb63be45740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341af9e0 con 0x5629379cae50 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.754+0000 7fb63be45740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341b1520 con 0x5629379bfc70 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.754+0000 7fb63be45740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341b00b0 con 0x5629379c0590 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb632800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379bfc70 0x5629379c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb632800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379bfc70 0x5629379c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44992/0 (socket says 172.21.15.107:44992) 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb632800640 1 -- 172.21.15.107:0/1599477488 learned_addr learned my addr 172.21.15.107:0/1599477488 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb62be00640 1 --2- 172.21.15.107:0/1599477488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 0x5629379c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb633200640 1 --2- 172.21.15.107:0/1599477488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379cae50 0x5629379e2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb632800640 1 -- 172.21.15.107:0/1599477488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 msgr2=0x5629379c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:49.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb632800640 1 --2- 172.21.15.107:0/1599477488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 0x5629379c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb632800640 1 -- 172.21.15.107:0/1599477488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379cae50 msgr2=0x5629379e2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:49.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb632800640 1 --2- 172.21.15.107:0/1599477488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379cae50 0x5629379e2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb632800640 1 -- 172.21.15.107:0/1599477488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629379b7770 con 0x5629379bfc70 2024-09-19T04:31:49.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb62be00640 1 --2- 172.21.15.107:0/1599477488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 0x5629379c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:49.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb632800640 1 --2- 172.21.15.107:0/1599477488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379bfc70 0x5629379c66d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb62409a670 tx=0x7fb6240d8cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d6db1b603b260d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:49.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.755+0000 7fb622a00640 1 -- 172.21.15.107:0/1599477488 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62409bd20 con 0x5629379bfc70 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.756+0000 7fb622a00640 1 -- 172.21.15.107:0/1599477488 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62409cac0 con 0x5629379bfc70 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.756+0000 7fb622a00640 1 -- 172.21.15.107:0/1599477488 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62409bf00 con 0x5629379bfc70 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.756+0000 7fb63be45740 1 -- 172.21.15.107:0/1599477488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379bfc70 msgr2=0x5629379c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.756+0000 7fb63be45740 1 --2- 172.21.15.107:0/1599477488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379bfc70 0x5629379c66d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb62409a670 tx=0x7fb6240d8cb0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.757+0000 7fb63be45740 1 -- 172.21.15.107:0/1599477488 shutdown_connections 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.757+0000 7fb63be45740 1 --2- 172.21.15.107:0/1599477488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379cae50 0x5629379e2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.757+0000 7fb63be45740 1 --2- 172.21.15.107:0/1599477488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379c0590 0x5629379c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.757+0000 7fb63be45740 1 --2- 172.21.15.107:0/1599477488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379bfc70 0x5629379c66d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.757+0000 7fb63be45740 1 -- 172.21.15.107:0/1599477488 >> 172.21.15.107:0/1599477488 conn(0x5629379bc200 msgr2=0x5629379c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.757+0000 7fb63be45740 1 -- 172.21.15.107:0/1599477488 shutdown_connections 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.757+0000 7fb63be45740 1 -- 172.21.15.107:0/1599477488 wait complete. 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.757+0000 7fb63be45740 1 -- start start 2024-09-19T04:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.757+0000 7fb63be45740 1 Processor -- start 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb63be45740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 0x5629379cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb63be45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379c0590 0x5629379cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb63be45740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb63be45740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341b0e50 con 0x5629379c0590 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb63be45740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341b0780 con 0x5629379cae50 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb63be45740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341ad9e0 con 0x5629379bfc70 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb63be45740 10 client.0 _ll_register_callbacks cb 0x7fb61c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb632800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 0x5629379cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb633200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb633200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:44998/0 (socket says 172.21.15.107:44998) 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb633200640 1 -- 172.21.15.107:0/3262417595 learned_addr learned my addr 172.21.15.107:0/3262417595 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.758+0000 7fb62be00640 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379c0590 0x5629379cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb633200640 1 -- 172.21.15.107:0/3262417595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 msgr2=0x5629379cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb633200640 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 0x5629379cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb633200640 1 -- 172.21.15.107:0/3262417595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379c0590 msgr2=0x5629379cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb633200640 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379c0590 0x5629379cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb633200640 1 -- 172.21.15.107:0/3262417595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629379bce90 con 0x5629379cae50 2024-09-19T04:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb632800640 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 0x5629379cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb633200640 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379cf4f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb62c035000 tx=0x7fb62c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e01fce178fab5153 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb617e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb617e00640 1 -- 172.21.15.107:0/3262417595 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c004250 con 0x5629379cae50 2024-09-19T04:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb617e00640 1 -- 172.21.15.107:0/3262417595 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fb62c0043f0 con 0x5629379cae50 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb63be45740 1 -- 172.21.15.107:0/3262417595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb62409cac0 con 0x5629379cae50 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb617e00640 1 -- 172.21.15.107:0/3262417595 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb62c0045f0 con 0x5629379cae50 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb63be45740 10 client.27287 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.759+0000 7fb63be45740 1 -- 172.21.15.107:0/3262417595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562937a4e5e0 con 0x5629379cae50 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.760+0000 7fb617e00640 1 -- 172.21.15.107:0/3262417595 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62409cac0 con 0x5629379cae50 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.760+0000 7fb617e00640 1 -- 172.21.15.107:0/3262417595 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fb62c05a070 con 0x5629379cae50 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.760+0000 7fb63be45740 10 client.27287 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.760+0000 7fb63be45740 10 client.27287 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.760+0000 7fb63be45740 20 client.27287 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.760+0000 7fb63be45740 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x562937a4ed80 0x562937a511a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.760+0000 7fb63be45740 4 client.27287 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.760+0000 7fb63be45740 1 -- 172.21.15.107:0/3262417595 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56293792de60 con 0x562937a4ed80 2024-09-19T04:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.760+0000 7fb632800640 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x562937a4ed80 0x562937a511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb632800640 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x562937a4ed80 0x562937a511a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b8e7e6801e178a2 server_cookie=218ccc6c3f4f00ee in_seq=0 out_seq=0 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb617e00640 10 client.27287 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb617e00640 1 -- 172.21.15.107:0/3262417595 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56293792de60 con 0x562937a4ed80 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb617e00640 10 client.27287 handle_command_reply: tid=0 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb63be45740 1 client.27287 shutdown 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb63be45740 1 -- 172.21.15.107:0/3262417595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 msgr2=0x5629379cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb63be45740 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379cf4f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb62c035000 tx=0x7fb62c0095c0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb63be45740 1 -- 172.21.15.107:0/3262417595 shutdown_connections 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb63be45740 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x562937a4ed80 0x562937a511a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb63be45740 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629379cae50 0x5629379cf4f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb617e00640 0 client.27287 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb63be45740 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629379c0590 0x5629379cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb63be45740 1 --2- 172.21.15.107:0/3262417595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629379bfc70 0x5629379cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.761+0000 7fb63be45740 1 -- 172.21.15.107:0/3262417595 >> 172.21.15.107:0/3262417595 conn(0x5629379bc200 msgr2=0x5629379bab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.762+0000 7fb63be45740 1 -- 172.21.15.107:0/3262417595 shutdown_connections 2024-09-19T04:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.762+0000 7fb63be45740 1 -- 172.21.15.107:0/3262417595 wait complete. 2024-09-19T04:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.762+0000 7fb63be45740 20 client.27287 trim_cache size 0 max 16384 2024-09-19T04:31:49.767 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.762+0000 7fb639400640 1 -- 172.21.15.107:0/995422967 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb6180a33b0 msgr2=0x7fb6180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.762+0000 7fb639400640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb6180a33b0 0x7fb6180a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb62c005e30 tx=0x7fb62c005c30 comp rx=0 tx=0).stop 2024-09-19T04:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.762+0000 7fb639400640 1 -- 172.21.15.107:0/995422967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb634103ff0 msgr2=0x7fb63419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.762+0000 7fb639400640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb634103ff0 0x7fb63419a5b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb62400e9f0 tx=0x7fb62400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.763+0000 7fb639400640 1 -- 172.21.15.107:0/995422967 shutdown_connections 2024-09-19T04:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.763+0000 7fb639400640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb6180a33b0 0x7fb6180a5870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.763+0000 7fb639400640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb634109120 0x7fb63419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.763+0000 7fb639400640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6341049a0 0x7fb63419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.763+0000 7fb639400640 1 --2- 172.21.15.107:0/995422967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb634103ff0 0x7fb63419a5b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.763+0000 7fb639400640 1 -- 172.21.15.107:0/995422967 >> 172.21.15.107:0/995422967 conn(0x7fb6340ffa90 msgr2=0x7fb634101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.763+0000 7fb639400640 1 -- 172.21.15.107:0/995422967 shutdown_connections 2024-09-19T04:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:49.763+0000 7fb639400640 1 -- 172.21.15.107:0/995422967 wait complete. 2024-09-19T04:31:50.621 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:31:50.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 390690 2.90 MB/sec execute 488 sec latency 33.167 ms 2024-09-19T04:31:51.022 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:50 smithi145 ceph-mon[35096]: pgmap v358: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:50 smithi107 ceph-mon[30659]: pgmap v358: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:51.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:50 smithi115 ceph-mon[35477]: pgmap v358: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-19T04:31:51.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.701+0000 7f5d9fe00640 1 -- 172.21.15.107:0/368733171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5da0103e30 msgr2=0x7f5da0104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:51.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.701+0000 7f5d9fe00640 1 --2- 172.21.15.107:0/368733171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5da0103e30 0x7f5da0104210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009900 tx=0x7f5d9002a410 comp rx=0 tx=0).stop 2024-09-19T04:31:51.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.701+0000 7f5d9fe00640 1 -- 172.21.15.107:0/368733171 shutdown_connections 2024-09-19T04:31:51.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.701+0000 7f5d9fe00640 1 --2- 172.21.15.107:0/368733171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da0108f60 0x7f5da0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.701+0000 7f5d9fe00640 1 --2- 172.21.15.107:0/368733171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5da01047e0 0x7f5da0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.701+0000 7f5d9fe00640 1 --2- 172.21.15.107:0/368733171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5da0103e30 0x7f5da0104210 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.701+0000 7f5d9fe00640 1 -- 172.21.15.107:0/368733171 >> 172.21.15.107:0/368733171 conn(0x7f5da00ff910 msgr2=0x7f5da0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:51.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.701+0000 7f5d9fe00640 1 -- 172.21.15.107:0/368733171 shutdown_connections 2024-09-19T04:31:51.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.701+0000 7f5d9fe00640 1 -- 172.21.15.107:0/368733171 wait complete. 2024-09-19T04:31:51.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.702+0000 7f5d9fe00640 1 Processor -- start 2024-09-19T04:31:51.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.702+0000 7f5d9fe00640 1 -- start start 2024-09-19T04:31:51.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5da0103e30 0x7f5da019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da01047e0 0x7f5da019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5da0108f60 0x7f5da019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0113fd0 con 0x7f5da0108f60 2024-09-19T04:31:51.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9fe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0113e50 con 0x7f5da01047e0 2024-09-19T04:31:51.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0114150 con 0x7f5da0103e30 2024-09-19T04:31:51.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5da0103e30 0x7f5da019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da01047e0 0x7f5da019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5da0103e30 0x7f5da019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51048/0 (socket says 172.21.15.107:51048) 2024-09-19T04:31:51.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9ea00640 1 -- 172.21.15.107:0/1502003051 learned_addr learned my addr 172.21.15.107:0/1502003051 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:51.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9f400640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5da0108f60 0x7f5da019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9ea00640 1 -- 172.21.15.107:0/1502003051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da01047e0 msgr2=0x7f5da019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:51.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9ea00640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da01047e0 0x7f5da019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9ea00640 1 -- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5da0108f60 msgr2=0x7f5da019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:51.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9ea00640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5da0108f60 0x7f5da019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9ea00640 1 -- 172.21.15.107:0/1502003051 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da019f420 con 0x7f5da0103e30 2024-09-19T04:31:51.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9e000640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da01047e0 0x7f5da019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:51.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.703+0000 7f5d9f400640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5da0108f60 0x7f5da019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:51.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.704+0000 7f5d9ea00640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5da0103e30 0x7f5da019a470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5d90004770 tx=0x7f5d90003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:51.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.704+0000 7f5d97400640 1 -- 172.21.15.107:0/1502003051 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9003f070 con 0x7f5da0103e30 2024-09-19T04:31:51.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.704+0000 7f5d9fe00640 1 -- 172.21.15.107:0/1502003051 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da019f6b0 con 0x7f5da0103e30 2024-09-19T04:31:51.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.704+0000 7f5d97400640 1 -- 172.21.15.107:0/1502003051 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d90031ca0 con 0x7f5da0103e30 2024-09-19T04:31:51.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.704+0000 7f5d97400640 1 -- 172.21.15.107:0/1502003051 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90035030 con 0x7f5da0103e30 2024-09-19T04:31:51.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.704+0000 7f5d9fe00640 1 -- 172.21.15.107:0/1502003051 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da01a6ff0 con 0x7f5da0103e30 2024-09-19T04:31:51.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.707+0000 7f5d97400640 1 -- 172.21.15.107:0/1502003051 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f5d90042070 con 0x7f5da0103e30 2024-09-19T04:31:51.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.707+0000 7f5da8624740 1 Processor -- start 2024-09-19T04:31:51.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.707+0000 7f5da8624740 1 -- start start 2024-09-19T04:31:51.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5da8624740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad20e40 0x55e9fad21220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5da8624740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad21760 0x55e9fad2bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5da8624740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad2c020 0x55e9fad2e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5da8624740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e9fac873e0 con 0x55e9fad20e40 2024-09-19T04:31:51.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5da8624740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e9fac2ed80 con 0x55e9fad2c020 2024-09-19T04:31:51.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5da8624740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e9fac833a0 con 0x55e9fad21760 2024-09-19T04:31:51.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad21760 0x55e9fad2bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad2c020 0x55e9fad2e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad2c020 0x55e9fad2e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45024/0 (socket says 172.21.15.107:45024) 2024-09-19T04:31:51.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5d9e000640 1 -- 172.21.15.107:0/2786631040 learned_addr learned my addr 172.21.15.107:0/2786631040 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:51.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5d9f400640 1 --2- 172.21.15.107:0/2786631040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad20e40 0x55e9fad21220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5d9e000640 1 -- 172.21.15.107:0/2786631040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad21760 msgr2=0x55e9fad2bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:51.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5d9e000640 1 --2- 172.21.15.107:0/2786631040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad21760 0x55e9fad2bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5d9e000640 1 -- 172.21.15.107:0/2786631040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad20e40 msgr2=0x55e9fad21220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:51.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5d9e000640 1 --2- 172.21.15.107:0/2786631040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad20e40 0x55e9fad21220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.708+0000 7f5d9e000640 1 -- 172.21.15.107:0/2786631040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9fac4dae0 con 0x55e9fad2c020 2024-09-19T04:31:51.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.709+0000 7f5d9e000640 1 --2- 172.21.15.107:0/2786631040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad2c020 0x55e9fad2e410 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0114f0 tx=0x7f5d8c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf119489ba1a5f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:51.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.709+0000 7f5d97400640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d840a33b0 0x7f5d840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.709+0000 7f5d8be00640 1 -- 172.21.15.107:0/2786631040 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c005020 con 0x55e9fad2c020 2024-09-19T04:31:51.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.709+0000 7f5d8be00640 1 -- 172.21.15.107:0/2786631040 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d8c0047a0 con 0x55e9fad2c020 2024-09-19T04:31:51.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.709+0000 7f5d8be00640 1 -- 172.21.15.107:0/2786631040 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00f9c0 con 0x55e9fad2c020 2024-09-19T04:31:51.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.709+0000 7f5d97400640 1 -- 172.21.15.107:0/1502003051 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f5d900f03f0 con 0x7f5da0103e30 2024-09-19T04:31:51.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.709+0000 7f5d9f400640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d840a33b0 0x7f5d840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.710+0000 7f5da8624740 1 -- 172.21.15.107:0/2786631040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad2c020 msgr2=0x55e9fad2e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:51.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.710+0000 7f5da8624740 1 --2- 172.21.15.107:0/2786631040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad2c020 0x55e9fad2e410 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0114f0 tx=0x7f5d8c0119b0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.710+0000 7f5d9f400640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d840a33b0 0x7f5d840a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5d98005fd0 tx=0x7f5d98009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:51.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.710+0000 7f5da8624740 1 -- 172.21.15.107:0/2786631040 shutdown_connections 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.710+0000 7f5da8624740 1 --2- 172.21.15.107:0/2786631040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad2c020 0x55e9fad2e410 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.710+0000 7f5da8624740 1 --2- 172.21.15.107:0/2786631040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad21760 0x55e9fad2bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.710+0000 7f5da8624740 1 --2- 172.21.15.107:0/2786631040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad20e40 0x55e9fad21220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.710+0000 7f5da8624740 1 -- 172.21.15.107:0/2786631040 >> 172.21.15.107:0/2786631040 conn(0x55e9fad1d3d0 msgr2=0x55e9fad1d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.710+0000 7f5da8624740 1 -- 172.21.15.107:0/2786631040 shutdown_connections 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.710+0000 7f5da8624740 1 -- 172.21.15.107:0/2786631040 wait complete. 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.711+0000 7f5da8624740 1 -- start start 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.711+0000 7f5da8624740 1 Processor -- start 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.711+0000 7f5da8624740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad20e40 0x55e9fad28e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.711+0000 7f5da8624740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad29370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.711+0000 7f5da8624740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad2c020 0x55e9fad298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.711+0000 7f5da8624740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e9fad2ebf0 con 0x55e9fad2c020 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.711+0000 7f5da8624740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e9fac420f0 con 0x55e9fad21760 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.711+0000 7f5da8624740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e9fad2ed70 con 0x55e9fad20e40 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.711+0000 7f5da8624740 10 client.0 _ll_register_callbacks cb 0x55e9fac29bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad29370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad20e40 0x55e9fad28e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad29370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45040/0 (socket says 172.21.15.107:45040) 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9e000640 1 -- 172.21.15.107:0/1441383728 learned_addr learned my addr 172.21.15.107:0/1441383728 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad20e40 0x55e9fad28e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51068/0 (socket says 172.21.15.107:51068) 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9ea00640 1 -- 172.21.15.107:0/1441383728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 msgr2=0x55e9fad29370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9ea00640 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad29370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9ea00640 1 -- 172.21.15.107:0/1441383728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad2c020 msgr2=0x55e9fad298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:51.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9ea00640 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad2c020 0x55e9fad298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9ea00640 1 -- 172.21.15.107:0/1441383728 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9fa9f83c0 con 0x55e9fad20e40 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.712+0000 7f5d9e000640 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad29370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.713+0000 7f5d9ea00640 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad20e40 0x55e9fad28e30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5d900edb00 tx=0x7f5d9003b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0bba18df9eacd76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.713+0000 7f5d88c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.713+0000 7f5d88c00640 1 -- 172.21.15.107:0/1441383728 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90049050 con 0x55e9fad20e40 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.713+0000 7f5d88c00640 1 -- 172.21.15.107:0/1441383728 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f5d9003b4f0 con 0x55e9fad20e40 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.713+0000 7f5da8624740 1 -- 172.21.15.107:0/1441383728 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8c0047a0 con 0x55e9fad20e40 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.713+0000 7f5d88c00640 1 -- 172.21.15.107:0/1441383728 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5d900bc290 con 0x55e9fad20e40 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.713+0000 7f5da8624740 10 client.27250 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.713+0000 7f5da8624740 1 -- 172.21.15.107:0/1441383728 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e9fad2ec50 con 0x55e9fad20e40 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.713+0000 7f5d88c00640 1 -- 172.21.15.107:0/1441383728 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d8c0047a0 con 0x55e9fad20e40 2024-09-19T04:31:51.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.714+0000 7f5d88c00640 1 -- 172.21.15.107:0/1441383728 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f5d900f0030 con 0x55e9fad20e40 2024-09-19T04:31:51.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.714+0000 7f5da8624740 10 client.27250 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:51.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.714+0000 7f5da8624740 10 client.27250 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:31:51.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.714+0000 7f5da8624740 20 client.27250 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:51.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.714+0000 7f5da8624740 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e9fad3c030 0x55e9fad3e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.714+0000 7f5da8624740 4 client.27250 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:51.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.714+0000 7f5da8624740 1 -- 172.21.15.107:0/1441383728 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9fac94950 con 0x55e9fad3c030 2024-09-19T04:31:51.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.714+0000 7f5d9e000640 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e9fad3c030 0x55e9fad3e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.715+0000 7f5d9e000640 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e9fad3c030 0x55e9fad3e4f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab670ed1c8623a10 server_cookie=e95a20b612ce581b in_seq=0 out_seq=0 2024-09-19T04:31:51.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.715+0000 7f5d88c00640 10 client.27250 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:31:51.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.715+0000 7f5d88c00640 1 -- 172.21.15.107:0/1441383728 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9fac94950 con 0x55e9fad3c030 2024-09-19T04:31:51.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.715+0000 7f5d88c00640 10 client.27250 handle_command_reply: tid=0 2024-09-19T04:31:51.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 client.27250 shutdown 2024-09-19T04:31:51.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 -- 172.21.15.107:0/1441383728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad20e40 msgr2=0x55e9fad28e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:51.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad20e40 0x55e9fad28e30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5d900edb00 tx=0x7f5d9003b460 comp rx=0 tx=0).stop 2024-09-19T04:31:51.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 -- 172.21.15.107:0/1441383728 shutdown_connections 2024-09-19T04:31:51.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e9fad3c030 0x55e9fad3e4f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad2c020 0x55e9fad298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad29370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 --2- 172.21.15.107:0/1441383728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad20e40 0x55e9fad28e30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5d88c00640 0 client.27250 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:31:51.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 -- 172.21.15.107:0/1441383728 >> 172.21.15.107:0/1441383728 conn(0x55e9fad1d3d0 msgr2=0x55e9fad23100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:51.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 -- 172.21.15.107:0/1441383728 shutdown_connections 2024-09-19T04:31:51.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 1 -- 172.21.15.107:0/1441383728 wait complete. 2024-09-19T04:31:51.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.716+0000 7f5da8624740 20 client.27250 trim_cache size 0 max 16384 2024-09-19T04:31:51.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 391911 2.90 MB/sec execute 489 sec latency 126.278 ms 2024-09-19T04:31:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.730+0000 7f5da8624740 1 Processor -- start 2024-09-19T04:31:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.730+0000 7f5da8624740 1 -- start start 2024-09-19T04:31:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5da8624740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad20e40 0x55e9fad43e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5da8624740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad44340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5da8624740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad28df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5da8624740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01af880 con 0x55e9fad20e40 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5da8624740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01b13c0 con 0x55e9fad21760 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5da8624740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01aff50 con 0x55e9fad2c020 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad44340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad44340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45044/0 (socket says 172.21.15.107:45044) 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9e000640 1 -- 172.21.15.107:0/189070662 learned_addr learned my addr 172.21.15.107:0/189070662 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9ea00640 1 --2- 172.21.15.107:0/189070662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad20e40 0x55e9fad43e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9f400640 1 --2- 172.21.15.107:0/189070662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad28df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9e000640 1 -- 172.21.15.107:0/189070662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 msgr2=0x55e9fad28df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9e000640 1 --2- 172.21.15.107:0/189070662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad28df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9e000640 1 -- 172.21.15.107:0/189070662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad20e40 msgr2=0x55e9fad43e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9e000640 1 --2- 172.21.15.107:0/189070662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad20e40 0x55e9fad43e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9e000640 1 -- 172.21.15.107:0/189070662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9fad1d200 con 0x55e9fad21760 2024-09-19T04:31:51.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.731+0000 7f5d9f400640 1 --2- 172.21.15.107:0/189070662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad28df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:51.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.732+0000 7f5d9e000640 1 --2- 172.21.15.107:0/189070662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad44340 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0058c0 tx=0x7f5d8c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ae1ed2aa1cf2f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:51.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.732+0000 7f5d8b400640 1 -- 172.21.15.107:0/189070662 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c02f070 con 0x55e9fad21760 2024-09-19T04:31:51.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.732+0000 7f5d8b400640 1 -- 172.21.15.107:0/189070662 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d8c004c60 con 0x55e9fad21760 2024-09-19T04:31:51.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.732+0000 7f5d8b400640 1 -- 172.21.15.107:0/189070662 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c029340 con 0x55e9fad21760 2024-09-19T04:31:51.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.733+0000 7f5da8624740 1 -- 172.21.15.107:0/189070662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 msgr2=0x55e9fad44340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.733+0000 7f5da8624740 1 --2- 172.21.15.107:0/189070662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad44340 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0058c0 tx=0x7f5d8c0119b0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.733+0000 7f5da8624740 1 -- 172.21.15.107:0/189070662 shutdown_connections 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.733+0000 7f5da8624740 1 --2- 172.21.15.107:0/189070662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad28df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.733+0000 7f5da8624740 1 --2- 172.21.15.107:0/189070662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad21760 0x55e9fad44340 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.733+0000 7f5da8624740 1 --2- 172.21.15.107:0/189070662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad20e40 0x55e9fad43e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.733+0000 7f5da8624740 1 -- 172.21.15.107:0/189070662 >> 172.21.15.107:0/189070662 conn(0x55e9fad1d3d0 msgr2=0x55e9fad27f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.733+0000 7f5da8624740 1 -- 172.21.15.107:0/189070662 shutdown_connections 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.733+0000 7f5da8624740 1 -- 172.21.15.107:0/189070662 wait complete. 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.733+0000 7f5da8624740 1 -- start start 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5da8624740 1 Processor -- start 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5da8624740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad20e40 0x55e9fad34800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5da8624740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad21760 0x55e9fad2fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5da8624740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5da8624740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01b0cf0 con 0x55e9fad21760 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5da8624740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01b0620 con 0x55e9fad20e40 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5da8624740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01ad880 con 0x55e9fad2c020 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5da8624740 10 client.0 _ll_register_callbacks cb 0x7f5d900f3b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad21760 0x55e9fad2fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad30090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51094/0 (socket says 172.21.15.107:51094) 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad20e40 0x55e9fad34800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.734+0000 7f5d9f400640 1 -- 172.21.15.107:0/1407179543 learned_addr learned my addr 172.21.15.107:0/1407179543 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.735+0000 7f5d9f400640 1 -- 172.21.15.107:0/1407179543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad20e40 msgr2=0x55e9fad34800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.735+0000 7f5d9f400640 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad20e40 0x55e9fad34800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.735+0000 7f5d9f400640 1 -- 172.21.15.107:0/1407179543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad21760 msgr2=0x55e9fad2fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.735+0000 7f5d9f400640 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad21760 0x55e9fad2fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.735+0000 7f5d9f400640 1 -- 172.21.15.107:0/1407179543 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9fad1cc80 con 0x55e9fad2c020 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.735+0000 7f5d9f400640 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad30090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5d98010aa0 tx=0x7f5d98004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b073763996e25b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.735+0000 7f5d88c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.735+0000 7f5d88c00640 1 -- 172.21.15.107:0/1407179543 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9801fdc0 con 0x55e9fad2c020 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.735+0000 7f5da8624740 1 -- 172.21.15.107:0/1407179543 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8c004c60 con 0x55e9fad2c020 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.735+0000 7f5d88c00640 1 -- 172.21.15.107:0/1407179543 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f5d98004b20 con 0x55e9fad2c020 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5d88c00640 1 -- 172.21.15.107:0/1407179543 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5d98004d20 con 0x55e9fad2c020 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5da8624740 10 client.27256 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5da8624740 1 -- 172.21.15.107:0/1407179543 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e9fada6d30 con 0x55e9fad2c020 2024-09-19T04:31:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5d88c00640 1 -- 172.21.15.107:0/1407179543 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d8c004c60 con 0x55e9fad2c020 2024-09-19T04:31:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5d88c00640 1 -- 172.21.15.107:0/1407179543 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f5d9802e030 con 0x55e9fad2c020 2024-09-19T04:31:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5da8624740 10 client.27256 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5da8624740 10 client.27256 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:31:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5da8624740 20 client.27256 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5da8624740 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e9fada7400 0x55e9fada97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5da8624740 4 client.27256 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5da8624740 1 -- 172.21.15.107:0/1407179543 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9fac8ee60 con 0x55e9fada7400 2024-09-19T04:31:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.736+0000 7f5d9ea00640 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e9fada7400 0x55e9fada97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5d9ea00640 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e9fada7400 0x55e9fada97f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef10efa9458ccadb server_cookie=1beab7492da709e in_seq=0 out_seq=0 2024-09-19T04:31:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5d88c00640 10 client.27256 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:31:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5d88c00640 1 -- 172.21.15.107:0/1407179543 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9fac8ee60 con 0x55e9fada7400 2024-09-19T04:31:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5d88c00640 10 client.27256 handle_command_reply: tid=0 2024-09-19T04:31:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5da8624740 1 client.27256 shutdown 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5da8624740 1 -- 172.21.15.107:0/1407179543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 msgr2=0x55e9fad30090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5da8624740 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad30090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5d98010aa0 tx=0x7f5d98004ab0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5da8624740 1 -- 172.21.15.107:0/1407179543 shutdown_connections 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5da8624740 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55e9fada7400 0x55e9fada97f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5da8624740 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9fad2c020 0x55e9fad30090 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5da8624740 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9fad21760 0x55e9fad2fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5d88c00640 0 client.27256 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5da8624740 1 --2- 172.21.15.107:0/1407179543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9fad20e40 0x55e9fad34800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.737+0000 7f5da8624740 1 -- 172.21.15.107:0/1407179543 >> 172.21.15.107:0/1407179543 conn(0x55e9fad1d3d0 msgr2=0x55e9fad1bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.738+0000 7f5da8624740 1 -- 172.21.15.107:0/1407179543 shutdown_connections 2024-09-19T04:31:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.738+0000 7f5da8624740 1 -- 172.21.15.107:0/1407179543 wait complete. 2024-09-19T04:31:51.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.738+0000 7f5da8624740 20 client.27256 trim_cache size 0 max 16384 2024-09-19T04:31:51.744 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:31:51.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.738+0000 7f5d9fe00640 1 -- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d840a33b0 msgr2=0x7f5d840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:51.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.738+0000 7f5d9fe00640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d840a33b0 0x7f5d840a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5d98005fd0 tx=0x7f5d98009500 comp rx=0 tx=0).stop 2024-09-19T04:31:51.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.738+0000 7f5d9fe00640 1 -- 172.21.15.107:0/1502003051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5da0103e30 msgr2=0x7f5da019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:51.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.738+0000 7f5d9fe00640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5da0103e30 0x7f5da019a470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5d90004770 tx=0x7f5d90003c10 comp rx=0 tx=0).stop 2024-09-19T04:31:51.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.739+0000 7f5d9fe00640 1 -- 172.21.15.107:0/1502003051 shutdown_connections 2024-09-19T04:31:51.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.739+0000 7f5d9fe00640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5d840a33b0 0x7f5d840a5870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.739+0000 7f5d9fe00640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5da0108f60 0x7f5da019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.739+0000 7f5d9fe00640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da01047e0 0x7f5da019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.739+0000 7f5d9fe00640 1 --2- 172.21.15.107:0/1502003051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5da0103e30 0x7f5da019a470 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:51.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.739+0000 7f5d9fe00640 1 -- 172.21.15.107:0/1502003051 >> 172.21.15.107:0/1502003051 conn(0x7f5da00ff910 msgr2=0x7f5da0100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:51.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.739+0000 7f5d9fe00640 1 -- 172.21.15.107:0/1502003051 shutdown_connections 2024-09-19T04:31:51.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:51.739+0000 7f5d9fe00640 1 -- 172.21.15.107:0/1502003051 wait complete. 2024-09-19T04:31:52.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 391911 2.90 MB/sec execute 490 sec latency 1126.372 ms 2024-09-19T04:31:52.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:31:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:52 smithi145 ceph-mon[35096]: pgmap v359: 97 pgs: 97 active+clean; 323 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-09-19T04:31:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:52 smithi107 ceph-mon[30659]: pgmap v359: 97 pgs: 97 active+clean; 323 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-09-19T04:31:53.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:52 smithi115 ceph-mon[35477]: pgmap v359: 97 pgs: 97 active+clean; 323 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-09-19T04:31:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:53.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 391911 2.89 MB/sec execute 491 sec latency 2126.499 ms 2024-09-19T04:31:53.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.907+0000 7f45aec00640 1 -- 172.21.15.107:0/1655313379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45a8103ff0 msgr2=0x7f45a81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:53.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.907+0000 7f45aec00640 1 --2- 172.21.15.107:0/1655313379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45a8103ff0 0x7f45a81043d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4598009900 tx=0x7f459802a410 comp rx=0 tx=0).stop 2024-09-19T04:31:53.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.907+0000 7f45aec00640 1 -- 172.21.15.107:0/1655313379 shutdown_connections 2024-09-19T04:31:53.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.907+0000 7f45aec00640 1 --2- 172.21.15.107:0/1655313379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a8109120 0x7f45a8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.907+0000 7f45aec00640 1 --2- 172.21.15.107:0/1655313379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a81049a0 0x7f45a81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.907+0000 7f45aec00640 1 --2- 172.21.15.107:0/1655313379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45a8103ff0 0x7f45a81043d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.907+0000 7f45aec00640 1 -- 172.21.15.107:0/1655313379 >> 172.21.15.107:0/1655313379 conn(0x7f45a80ffa90 msgr2=0x7f45a8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.907+0000 7f45aec00640 1 -- 172.21.15.107:0/1655313379 shutdown_connections 2024-09-19T04:31:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.907+0000 7f45aec00640 1 -- 172.21.15.107:0/1655313379 wait complete. 2024-09-19T04:31:53.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.908+0000 7f45aec00640 1 Processor -- start 2024-09-19T04:31:53.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.908+0000 7f45aec00640 1 -- start start 2024-09-19T04:31:53.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.908+0000 7f45aec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45a8103ff0 0x7f45a819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.908+0000 7f45aec00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a81049a0 0x7f45a819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.908+0000 7f45aec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8109120 0x7f45a819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.908+0000 7f45aec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8114190 con 0x7f45a8103ff0 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.908+0000 7f45aec00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8114010 con 0x7f45a81049a0 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.908+0000 7f45aec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8114310 con 0x7f45a8109120 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45a8103ff0 0x7f45a819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45a8103ff0 0x7f45a819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44580/0 (socket says 172.21.15.107:44580) 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f45a7e00640 1 -- 172.21.15.107:0/3734397441 learned_addr learned my addr 172.21.15.107:0/3734397441 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f459fe00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a81049a0 0x7f45a819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f45acc00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8109120 0x7f45a819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f45a7e00640 1 -- 172.21.15.107:0/3734397441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8109120 msgr2=0x7f45a819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:53.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f45a7e00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8109120 0x7f45a819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f45a7e00640 1 -- 172.21.15.107:0/3734397441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a81049a0 msgr2=0x7f45a819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:53.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f45a7e00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a81049a0 0x7f45a819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f45a7e00640 1 -- 172.21.15.107:0/3734397441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a819f5c0 con 0x7f45a8103ff0 2024-09-19T04:31:53.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.909+0000 7f45a7e00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45a8103ff0 0x7f45a819a610 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4598004770 tx=0x7f4598003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:53.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.910+0000 7f45a5600640 1 -- 172.21.15.107:0/3734397441 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459803f070 con 0x7f45a8103ff0 2024-09-19T04:31:53.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.910+0000 7f45aec00640 1 -- 172.21.15.107:0/3734397441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a819f850 con 0x7f45a8103ff0 2024-09-19T04:31:53.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.910+0000 7f45a5600640 1 -- 172.21.15.107:0/3734397441 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4598031ca0 con 0x7f45a8103ff0 2024-09-19T04:31:53.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.910+0000 7f45a5600640 1 -- 172.21.15.107:0/3734397441 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598035030 con 0x7f45a8103ff0 2024-09-19T04:31:53.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.910+0000 7f45aec00640 1 -- 172.21.15.107:0/3734397441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a81a7190 con 0x7f45a8103ff0 2024-09-19T04:31:53.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.913+0000 7f45a5600640 1 -- 172.21.15.107:0/3734397441 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f4598042070 con 0x7f45a8103ff0 2024-09-19T04:31:53.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.913+0000 7f45b1830740 1 Processor -- start 2024-09-19T04:31:53.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.913+0000 7f45b1830740 1 -- start start 2024-09-19T04:31:53.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.913+0000 7f45b1830740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.913+0000 7f45b1830740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.913+0000 7f45b1830740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.913+0000 7f45b1830740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5629c31273e0 con 0x5629c31c0c70 2024-09-19T04:31:53.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.913+0000 7f45b1830740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629c30ced80 con 0x5629c31cbe50 2024-09-19T04:31:53.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45b1830740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5629c31233a0 con 0x5629c31c1590 2024-09-19T04:31:53.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51128/0 (socket says 172.21.15.107:51128) 2024-09-19T04:31:53.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45a7e00640 1 -- 172.21.15.107:0/746703968 learned_addr learned my addr 172.21.15.107:0/746703968 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:53.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f459fe00640 1 --2- 172.21.15.107:0/746703968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45acc00640 1 --2- 172.21.15.107:0/746703968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45a7e00640 1 -- 172.21.15.107:0/746703968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 msgr2=0x5629c31ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45a7e00640 1 --2- 172.21.15.107:0/746703968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45a7e00640 1 -- 172.21.15.107:0/746703968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 msgr2=0x5629c31c1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45a7e00640 1 --2- 172.21.15.107:0/746703968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45a7e00640 1 -- 172.21.15.107:0/746703968 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c30edae0 con 0x5629c31c1590 2024-09-19T04:31:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.914+0000 7f45a5600640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f458c0a33b0 0x7f458c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.915+0000 7f45a7e00640 1 --2- 172.21.15.107:0/746703968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31cb910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f45980ed980 tx=0x7f45980f85c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aea6c4f2d3000795 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:53.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.915+0000 7f45a5600640 1 -- 172.21.15.107:0/3734397441 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f45980f0330 con 0x7f45a8103ff0 2024-09-19T04:31:53.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.915+0000 7f45acc00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f458c0a33b0 0x7f458c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.915+0000 7f459d200640 1 -- 172.21.15.107:0/746703968 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598049070 con 0x5629c31c1590 2024-09-19T04:31:53.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.915+0000 7f459d200640 1 -- 172.21.15.107:0/746703968 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4598031e50 con 0x5629c31c1590 2024-09-19T04:31:53.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.915+0000 7f459d200640 1 -- 172.21.15.107:0/746703968 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980b8870 con 0x5629c31c1590 2024-09-19T04:31:53.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.916+0000 7f45b1830740 1 -- 172.21.15.107:0/746703968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 msgr2=0x5629c31cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:53.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.916+0000 7f45b1830740 1 --2- 172.21.15.107:0/746703968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31cb910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f45980ed980 tx=0x7f45980f85c0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.916+0000 7f45acc00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f458c0a33b0 0x7f458c0a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005fd0 tx=0x7f45a0009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:53.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.916+0000 7f45b1830740 1 -- 172.21.15.107:0/746703968 shutdown_connections 2024-09-19T04:31:53.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.916+0000 7f45b1830740 1 --2- 172.21.15.107:0/746703968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.916+0000 7f45b1830740 1 --2- 172.21.15.107:0/746703968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31cb910 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.916+0000 7f45b1830740 1 --2- 172.21.15.107:0/746703968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.916+0000 7f45b1830740 1 -- 172.21.15.107:0/746703968 >> 172.21.15.107:0/746703968 conn(0x5629c31bd200 msgr2=0x5629c31bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:53.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.916+0000 7f45b1830740 1 -- 172.21.15.107:0/746703968 shutdown_connections 2024-09-19T04:31:53.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.916+0000 7f45b1830740 1 -- 172.21.15.107:0/746703968 wait complete. 2024-09-19T04:31:53.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.917+0000 7f45b1830740 1 -- start start 2024-09-19T04:31:53.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.917+0000 7f45b1830740 1 Processor -- start 2024-09-19T04:31:53.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.917+0000 7f45b1830740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31c0c70 0x5629c31c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.917+0000 7f45b1830740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.917+0000 7f45b1830740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31cbe50 0x5629c31c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.917+0000 7f45b1830740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5629c31ce7e0 con 0x5629c31cbe50 2024-09-19T04:31:53.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.917+0000 7f45b1830740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629c30e20f0 con 0x5629c31c0c70 2024-09-19T04:31:53.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.917+0000 7f45b1830740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5629c31ce960 con 0x5629c31c1590 2024-09-19T04:31:53.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.917+0000 7f45b1830740 10 client.0 _ll_register_callbacks cb 0x5629c30c9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:53.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.917+0000 7f45acc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31cbe50 0x5629c31c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31c0c70 0x5629c31c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45acc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31cbe50 0x5629c31c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44600/0 (socket says 172.21.15.107:44600) 2024-09-19T04:31:53.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45acc00640 1 -- 172.21.15.107:0/3894975147 learned_addr learned my addr 172.21.15.107:0/3894975147 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:53.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31c0c70 0x5629c31c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45096/0 (socket says 172.21.15.107:45096) 2024-09-19T04:31:53.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f459fe00640 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45a7e00640 1 -- 172.21.15.107:0/3894975147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 msgr2=0x5629c31c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45a7e00640 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45a7e00640 1 -- 172.21.15.107:0/3894975147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31cbe50 msgr2=0x5629c31c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45a7e00640 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31cbe50 0x5629c31c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45a7e00640 1 -- 172.21.15.107:0/3894975147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c2e983c0 con 0x5629c31c0c70 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45a7e00640 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31c0c70 0x5629c31c8d20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x5629c31be060 tx=0x7f4598005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0c0c6099d30094 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f4595600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f4595600640 1 -- 172.21.15.107:0/3894975147 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598049040 con 0x5629c31c0c70 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f45b1830740 1 -- 172.21.15.107:0/3894975147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4598031e50 con 0x5629c31c0c70 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.918+0000 7f4595600640 1 -- 172.21.15.107:0/3894975147 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f45980edcc0 con 0x5629c31c0c70 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f4595600640 1 -- 172.21.15.107:0/3894975147 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f45980bbf30 con 0x5629c31c0c70 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f45b1830740 10 client.27317 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f45b1830740 1 -- 172.21.15.107:0/3894975147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5629c31ce7e0 con 0x5629c31c0c70 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f4595600640 1 -- 172.21.15.107:0/3894975147 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4598031e50 con 0x5629c31c0c70 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f4595600640 1 -- 172.21.15.107:0/3894975147 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f45980bc830 con 0x5629c31c0c70 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f45b1830740 10 client.27317 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f45b1830740 10 client.27317 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f45b1830740 20 client.27317 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f45b1830740 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5629c31db990 0x5629c31dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f45b1830740 4 client.27317 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.919+0000 7f45b1830740 1 -- 172.21.15.107:0/3894975147 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c3134950 con 0x5629c31db990 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.920+0000 7f459fe00640 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5629c31db990 0x5629c31dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.920+0000 7f459fe00640 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5629c31db990 0x5629c31dddb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74fa855d02aae41e server_cookie=9dfee3071a18e7 in_seq=0 out_seq=0 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.920+0000 7f4595600640 10 client.27317 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.921+0000 7f4595600640 1 -- 172.21.15.107:0/3894975147 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629c3134950 con 0x5629c31db990 2024-09-19T04:31:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.921+0000 7f4595600640 10 client.27317 handle_command_reply: tid=0 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 client.27317 shutdown 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 -- 172.21.15.107:0/3894975147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31c0c70 msgr2=0x5629c31c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31c0c70 0x5629c31c8d20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x5629c31be060 tx=0x7f4598005a20 comp rx=0 tx=0).stop 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 -- 172.21.15.107:0/3894975147 shutdown_connections 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5629c31db990 0x5629c31dddb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31cbe50 0x5629c31c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f4595600640 0 client.27317 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 --2- 172.21.15.107:0/3894975147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31c0c70 0x5629c31c8d20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 -- 172.21.15.107:0/3894975147 >> 172.21.15.107:0/3894975147 conn(0x5629c31bd200 msgr2=0x5629c31c2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 -- 172.21.15.107:0/3894975147 shutdown_connections 2024-09-19T04:31:53.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 1 -- 172.21.15.107:0/3894975147 wait complete. 2024-09-19T04:31:53.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.922+0000 7f45b1830740 20 client.27317 trim_cache size 0 max 16384 2024-09-19T04:31:53.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45b1830740 1 Processor -- start 2024-09-19T04:31:53.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45b1830740 1 -- start start 2024-09-19T04:31:53.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45b1830740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45b1830740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31e3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45b1830740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31e3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45b1830740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45a81af970 con 0x5629c31c0c70 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45b1830740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f45a81b14b0 con 0x5629c31cbe50 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45b1830740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f45a81b0040 con 0x5629c31c1590 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44614/0 (socket says 172.21.15.107:44614) 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.936+0000 7f45a7e00640 1 -- 172.21.15.107:0/3915545678 learned_addr learned my addr 172.21.15.107:0/3915545678 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f45acc00640 1 --2- 172.21.15.107:0/3915545678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31e3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f459fe00640 1 --2- 172.21.15.107:0/3915545678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f45a7e00640 1 -- 172.21.15.107:0/3915545678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 msgr2=0x5629c31e3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f45a7e00640 1 --2- 172.21.15.107:0/3915545678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31e3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f45a7e00640 1 -- 172.21.15.107:0/3915545678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 msgr2=0x5629c31e3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f45a7e00640 1 --2- 172.21.15.107:0/3915545678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31e3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f45a7e00640 1 -- 172.21.15.107:0/3915545678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c31bd030 con 0x5629c31c0c70 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f459fe00640 1 --2- 172.21.15.107:0/3915545678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31e3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f45a7e00640 1 --2- 172.21.15.107:0/3915545678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c36d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4598009320 tx=0x7f459803b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e4cb6db43f2f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f4597e00640 1 -- 172.21.15.107:0/3915545678 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598049040 con 0x5629c31c0c70 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f4597e00640 1 -- 172.21.15.107:0/3915545678 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4598031e50 con 0x5629c31c0c70 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.937+0000 7f4597e00640 1 -- 172.21.15.107:0/3915545678 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980bc110 con 0x5629c31c0c70 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.938+0000 7f45b1830740 1 -- 172.21.15.107:0/3915545678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 msgr2=0x5629c31c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.938+0000 7f45b1830740 1 --2- 172.21.15.107:0/3915545678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c36d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4598009320 tx=0x7f459803b720 comp rx=0 tx=0).stop 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.938+0000 7f45b1830740 1 -- 172.21.15.107:0/3915545678 shutdown_connections 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.938+0000 7f45b1830740 1 --2- 172.21.15.107:0/3915545678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31e3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.938+0000 7f45b1830740 1 --2- 172.21.15.107:0/3915545678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31e3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.938+0000 7f45b1830740 1 --2- 172.21.15.107:0/3915545678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c36d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.938+0000 7f45b1830740 1 -- 172.21.15.107:0/3915545678 >> 172.21.15.107:0/3915545678 conn(0x5629c31bd200 msgr2=0x5629c31cada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:53.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.938+0000 7f45b1830740 1 -- 172.21.15.107:0/3915545678 shutdown_connections 2024-09-19T04:31:53.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.938+0000 7f45b1830740 1 -- 172.21.15.107:0/3915545678 wait complete. 2024-09-19T04:31:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.938+0000 7f45b1830740 1 -- start start 2024-09-19T04:31:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45b1830740 1 Processor -- start 2024-09-19T04:31:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45b1830740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45b1830740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31c6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45b1830740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31c6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45b1830740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45a81b0de0 con 0x5629c31c0c70 2024-09-19T04:31:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45b1830740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f45a81b0710 con 0x5629c31cbe50 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45b1830740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f45a81ad970 con 0x5629c31c1590 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45b1830740 10 client.0 _ll_register_callbacks cb 0x7f45a0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44630/0 (socket says 172.21.15.107:44630) 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45a7e00640 1 -- 172.21.15.107:0/2851308240 learned_addr learned my addr 172.21.15.107:0/2851308240 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f459fe00640 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.939+0000 7f45acc00640 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f45a7e00640 1 -- 172.21.15.107:0/2851308240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 msgr2=0x5629c31c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f45a7e00640 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f45a7e00640 1 -- 172.21.15.107:0/2851308240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 msgr2=0x5629c31c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f45a7e00640 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f45a7e00640 1 -- 172.21.15.107:0/2851308240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c324de40 con 0x5629c31c0c70 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f459fe00640 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f45a7e00640 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c5e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f45980c0b40 tx=0x7f45980f85c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1591fc81f069db5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f4595600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f4595600640 1 -- 172.21.15.107:0/2851308240 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598049070 con 0x5629c31c0c70 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f4595600640 1 -- 172.21.15.107:0/2851308240 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f45980c4db0 con 0x5629c31c0c70 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.940+0000 7f45b1830740 1 -- 172.21.15.107:0/2851308240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4598031e50 con 0x5629c31c0c70 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f4595600640 1 -- 172.21.15.107:0/2851308240 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f45980bc3a0 con 0x5629c31c0c70 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f45b1830740 10 client.17274 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f45b1830740 1 -- 172.21.15.107:0/2851308240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5629c324de40 con 0x5629c31c0c70 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f4595600640 1 -- 172.21.15.107:0/2851308240 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4598031e50 con 0x5629c31c0c70 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f4595600640 1 -- 172.21.15.107:0/2851308240 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f45980bba20 con 0x5629c31c0c70 2024-09-19T04:31:53.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f45b1830740 10 client.17274 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f45b1830740 10 client.17274 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f45b1830740 20 client.17274 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f45b1830740 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5629c324f3c0 0x5629c32517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f45b1830740 4 client.17274 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.941+0000 7f45b1830740 1 -- 172.21.15.107:0/2851308240 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629c312ee60 con 0x5629c324f3c0 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f459fe00640 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5629c324f3c0 0x5629c32517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f459fe00640 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5629c324f3c0 0x5629c32517e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a525967378dd4e7a server_cookie=f245cab23ee9b9bf in_seq=0 out_seq=0 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f4595600640 10 client.17274 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f4595600640 1 -- 172.21.15.107:0/2851308240 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629c312ee60 con 0x5629c324f3c0 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f4595600640 10 client.17274 handle_command_reply: tid=0 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f45b1830740 1 client.17274 shutdown 2024-09-19T04:31:53.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f45b1830740 1 -- 172.21.15.107:0/2851308240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 msgr2=0x5629c31c5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f45b1830740 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c5e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f45980c0b40 tx=0x7f45980f85c0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f45b1830740 1 -- 172.21.15.107:0/2851308240 shutdown_connections 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f45b1830740 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x5629c324f3c0 0x5629c32517e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f45b1830740 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629c31cbe50 0x5629c31c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f45b1830740 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629c31c1590 0x5629c31c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.942+0000 7f45b1830740 1 --2- 172.21.15.107:0/2851308240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c31c0c70 0x5629c31c5e10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.943+0000 7f45b1830740 1 -- 172.21.15.107:0/2851308240 >> 172.21.15.107:0/2851308240 conn(0x5629c31bd200 msgr2=0x5629c31d69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.943+0000 7f4595600640 0 client.17274 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.943+0000 7f45b1830740 1 -- 172.21.15.107:0/2851308240 shutdown_connections 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.943+0000 7f45b1830740 1 -- 172.21.15.107:0/2851308240 wait complete. 2024-09-19T04:31:53.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.943+0000 7f45b1830740 20 client.17274 trim_cache size 0 max 16384 2024-09-19T04:31:53.950 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:31:53.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.943+0000 7f45aec00640 1 -- 172.21.15.107:0/3734397441 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f458c0a33b0 msgr2=0x7f458c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:53.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.943+0000 7f45aec00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f458c0a33b0 0x7f458c0a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005fd0 tx=0x7f45a0009500 comp rx=0 tx=0).stop 2024-09-19T04:31:53.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.943+0000 7f45aec00640 1 -- 172.21.15.107:0/3734397441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45a8103ff0 msgr2=0x7f45a819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:53.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.943+0000 7f45aec00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45a8103ff0 0x7f45a819a610 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4598004770 tx=0x7f4598003c10 comp rx=0 tx=0).stop 2024-09-19T04:31:53.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.944+0000 7f45aec00640 1 -- 172.21.15.107:0/3734397441 shutdown_connections 2024-09-19T04:31:53.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.944+0000 7f45aec00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f458c0a33b0 0x7f458c0a5870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.944+0000 7f45aec00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8109120 0x7f45a819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.944+0000 7f45aec00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a81049a0 0x7f45a819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.944+0000 7f45aec00640 1 --2- 172.21.15.107:0/3734397441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45a8103ff0 0x7f45a819a610 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:53.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.944+0000 7f45aec00640 1 -- 172.21.15.107:0/3734397441 >> 172.21.15.107:0/3734397441 conn(0x7f45a80ffa90 msgr2=0x7f45a8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:53.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.944+0000 7f45aec00640 1 -- 172.21.15.107:0/3734397441 shutdown_connections 2024-09-19T04:31:53.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:53.944+0000 7f45aec00640 1 -- 172.21.15.107:0/3734397441 wait complete. 2024-09-19T04:31:54.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 392821 2.89 MB/sec execute 492 sec latency 2633.142 ms 2024-09-19T04:31:54.911 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:31:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:54 smithi145 ceph-mon[35096]: pgmap v360: 97 pgs: 97 active+clean; 323 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-09-19T04:31:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:54 smithi107 ceph-mon[30659]: pgmap v360: 97 pgs: 97 active+clean; 323 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-09-19T04:31:55.315 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:55.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:54 smithi115 ceph-mon[35477]: pgmap v360: 97 pgs: 97 active+clean; 323 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-09-19T04:31:55.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 394300 2.91 MB/sec execute 493 sec latency 10.543 ms 2024-09-19T04:31:55.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.872+0000 7fa1cfe00640 1 -- 172.21.15.107:0/3836164938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d01047e0 msgr2=0x7fa1d0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:55.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.872+0000 7fa1cfe00640 1 --2- 172.21.15.107:0/3836164938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d01047e0 0x7fa1d0108830 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009900 tx=0x7fa1bc02a400 comp rx=0 tx=0).stop 2024-09-19T04:31:55.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.872+0000 7fa1cfe00640 1 -- 172.21.15.107:0/3836164938 shutdown_connections 2024-09-19T04:31:55.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.872+0000 7fa1cfe00640 1 --2- 172.21.15.107:0/3836164938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d0108f60 0x7fa1d0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.872+0000 7fa1cfe00640 1 --2- 172.21.15.107:0/3836164938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d01047e0 0x7fa1d0108830 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.872+0000 7fa1cfe00640 1 --2- 172.21.15.107:0/3836164938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d0103e30 0x7fa1d0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.872+0000 7fa1cfe00640 1 -- 172.21.15.107:0/3836164938 >> 172.21.15.107:0/3836164938 conn(0x7fa1d00ff910 msgr2=0x7fa1d0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:55.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.872+0000 7fa1cfe00640 1 -- 172.21.15.107:0/3836164938 shutdown_connections 2024-09-19T04:31:55.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.872+0000 7fa1cfe00640 1 -- 172.21.15.107:0/3836164938 wait complete. 2024-09-19T04:31:55.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.873+0000 7fa1cfe00640 1 Processor -- start 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.873+0000 7fa1cfe00640 1 -- start start 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d0103e30 0x7fa1d019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d01047e0 0x7fa1d019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d0108f60 0x7fa1d019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cfe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0114010 con 0x7fa1d0103e30 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cfe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0113e90 con 0x7fa1d01047e0 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cfe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0114190 con 0x7fa1d0108f60 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1ce000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d01047e0 0x7fa1d019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d0108f60 0x7fa1d019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1ce000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d01047e0 0x7fa1d019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45142/0 (socket says 172.21.15.107:45142) 2024-09-19T04:31:55.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1ce000640 1 -- 172.21.15.107:0/1892136063 learned_addr learned my addr 172.21.15.107:0/1892136063 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:55.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cea00640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d0103e30 0x7fa1d019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cf400640 1 -- 172.21.15.107:0/1892136063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d01047e0 msgr2=0x7fa1d019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:55.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cf400640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d01047e0 0x7fa1d019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cf400640 1 -- 172.21.15.107:0/1892136063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d0103e30 msgr2=0x7fa1d019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:55.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cf400640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d0103e30 0x7fa1d019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.874+0000 7fa1cf400640 1 -- 172.21.15.107:0/1892136063 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d019f440 con 0x7fa1d0108f60 2024-09-19T04:31:55.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.875+0000 7fa1cf400640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d0108f60 0x7fa1d019ed60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800ec50 tx=0x7fa1c800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:55.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.875+0000 7fa1d4a00640 1 -- 172.21.15.107:0/1892136063 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8003b70 con 0x7fa1d0108f60 2024-09-19T04:31:55.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.875+0000 7fa1d4a00640 1 -- 172.21.15.107:0/1892136063 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c8003d10 con 0x7fa1d0108f60 2024-09-19T04:31:55.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.875+0000 7fa1cfe00640 1 -- 172.21.15.107:0/1892136063 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d019f730 con 0x7fa1d0108f60 2024-09-19T04:31:55.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.875+0000 7fa1d4a00640 1 -- 172.21.15.107:0/1892136063 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8010890 con 0x7fa1d0108f60 2024-09-19T04:31:55.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.875+0000 7fa1cfe00640 1 -- 172.21.15.107:0/1892136063 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01a7010 con 0x7fa1d0108f60 2024-09-19T04:31:55.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.879+0000 7fa1d8a31740 1 Processor -- start 2024-09-19T04:31:55.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.879+0000 7fa1d4a00640 1 -- 172.21.15.107:0/1892136063 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fa1c8003fc0 con 0x7fa1d0108f60 2024-09-19T04:31:55.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1d8a31740 1 -- start start 2024-09-19T04:31:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed17760 0x55f71ed21ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 0x55f71ed24410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f71ec7d3e0 con 0x55f71ed17760 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f71ec24d80 con 0x55f71ed16e40 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f71ec793a0 con 0x55f71ed22020 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1cea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed17760 0x55f71ed21ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1ce000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1cea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed17760 0x55f71ed21ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44652/0 (socket says 172.21.15.107:44652) 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1cea00640 1 -- 172.21.15.107:0/2560056077 learned_addr learned my addr 172.21.15.107:0/2560056077 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.880+0000 7fa1ce000640 1 --2- 172.21.15.107:0/2560056077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 0x55f71ed24410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.881+0000 7fa1cea00640 1 -- 172.21.15.107:0/2560056077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 msgr2=0x55f71ed24410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.881+0000 7fa1cea00640 1 --2- 172.21.15.107:0/2560056077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 0x55f71ed24410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.881+0000 7fa1cea00640 1 -- 172.21.15.107:0/2560056077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 msgr2=0x55f71ed17220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.881+0000 7fa1cea00640 1 --2- 172.21.15.107:0/2560056077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed17220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.881+0000 7fa1cea00640 1 -- 172.21.15.107:0/2560056077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71ec43ae0 con 0x55f71ed17760 2024-09-19T04:31:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.881+0000 7fa1ce000640 1 --2- 172.21.15.107:0/2560056077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 0x55f71ed24410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.881+0000 7fa1d4a00640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa1b40a3430 0x7fa1b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.881+0000 7fa1cea00640 1 --2- 172.21.15.107:0/2560056077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed17760 0x55f71ed21ae0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009cf0 tx=0x7fa1c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3864283b8d8ad0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.881+0000 7fa1d4a00640 1 -- 172.21.15.107:0/1892136063 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fa1c80d0250 con 0x7fa1d0108f60 2024-09-19T04:31:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.882+0000 7fa1ce000640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa1b40a3430 0x7fa1b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.882+0000 7fa1bbe00640 1 -- 172.21.15.107:0/2560056077 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c001e070 con 0x55f71ed17760 2024-09-19T04:31:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.882+0000 7fa1bbe00640 1 -- 172.21.15.107:0/2560056077 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c0019b10 con 0x55f71ed17760 2024-09-19T04:31:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.882+0000 7fa1bbe00640 1 -- 172.21.15.107:0/2560056077 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000ea60 con 0x55f71ed17760 2024-09-19T04:31:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.882+0000 7fa1d8a31740 1 -- 172.21.15.107:0/2560056077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed17760 msgr2=0x55f71ed21ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.882+0000 7fa1ce000640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa1b40a3430 0x7fa1b40a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55f71ed14750 tx=0x7fa1bc041040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.882+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/2560056077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed17760 0x55f71ed21ae0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009cf0 tx=0x7fa1c0011810 comp rx=0 tx=0).stop 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.883+0000 7fa1d8a31740 1 -- 172.21.15.107:0/2560056077 shutdown_connections 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.883+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/2560056077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 0x55f71ed24410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.883+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/2560056077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed17760 0x55f71ed21ae0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.883+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/2560056077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed17220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.883+0000 7fa1d8a31740 1 -- 172.21.15.107:0/2560056077 >> 172.21.15.107:0/2560056077 conn(0x55f71ed133d0 msgr2=0x55f71ed137b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.883+0000 7fa1d8a31740 1 -- 172.21.15.107:0/2560056077 shutdown_connections 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.883+0000 7fa1d8a31740 1 -- 172.21.15.107:0/2560056077 wait complete. 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.883+0000 7fa1d8a31740 1 -- start start 2024-09-19T04:31:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.883+0000 7fa1d8a31740 1 Processor -- start 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed16e40 0x55f71ed1f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed17760 0x55f71ed1f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 0x55f71ed1fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f71ed24bf0 con 0x55f71ed16e40 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f71ec380f0 con 0x55f71ed17760 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f71ed24d70 con 0x55f71ed22020 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1d8a31740 10 client.0 _ll_register_callbacks cb 0x55f71ec1fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1ce000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed17760 0x55f71ed1f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1ce000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed17760 0x55f71ed1f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45164/0 (socket says 172.21.15.107:45164) 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1ce000640 1 -- 172.21.15.107:0/688189668 learned_addr learned my addr 172.21.15.107:0/688189668 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1cea00640 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed16e40 0x55f71ed1f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1cf400640 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 0x55f71ed1fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.884+0000 7fa1ce000640 1 -- 172.21.15.107:0/688189668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 msgr2=0x55f71ed1fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.885+0000 7fa1ce000640 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 0x55f71ed1fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.885+0000 7fa1ce000640 1 -- 172.21.15.107:0/688189668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed16e40 msgr2=0x55f71ed1f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.885+0000 7fa1ce000640 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed16e40 0x55f71ed1f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.885+0000 7fa1ce000640 1 -- 172.21.15.107:0/688189668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f71e9ee3c0 con 0x55f71ed17760 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.885+0000 7fa1cf400640 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 0x55f71ed1fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.885+0000 7fa1ce000640 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed17760 0x55f71ed1f540 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc006010 tx=0x7fa1bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ec712e7c43df2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.885+0000 7fa1b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.885+0000 7fa1b8c00640 1 -- 172.21.15.107:0/688189668 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc04e070 con 0x55f71ed17760 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.885+0000 7fa1b8c00640 1 -- 172.21.15.107:0/688189668 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fa1bc003a00 con 0x55f71ed17760 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.885+0000 7fa1d8a31740 1 -- 172.21.15.107:0/688189668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c0019b10 con 0x55f71ed17760 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1b8c00640 1 -- 172.21.15.107:0/688189668 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa1bc035040 con 0x55f71ed17760 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1d8a31740 10 client.27335 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1d8a31740 1 -- 172.21.15.107:0/688189668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f71ed30d10 con 0x55f71ed17760 2024-09-19T04:31:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1b8c00640 1 -- 172.21.15.107:0/688189668 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c0019b10 con 0x55f71ed17760 2024-09-19T04:31:55.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1b8c00640 1 -- 172.21.15.107:0/688189668 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fa1bc03bbb0 con 0x55f71ed17760 2024-09-19T04:31:55.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1d8a31740 10 client.27335 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:55.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1d8a31740 10 client.27335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:55.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1d8a31740 20 client.27335 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:55.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f71ed325e0 0x55f71ed34a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1d8a31740 4 client.27335 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:55.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.886+0000 7fa1d8a31740 1 -- 172.21.15.107:0/688189668 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f71ec8a950 con 0x55f71ed325e0 2024-09-19T04:31:55.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.887+0000 7fa1cea00640 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f71ed325e0 0x55f71ed34a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.887+0000 7fa1cea00640 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f71ed325e0 0x55f71ed34a40 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6647ea6e5dc1bef4 server_cookie=6c1eec5513e186b8 in_seq=0 out_seq=0 2024-09-19T04:31:55.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.887+0000 7fa1b8c00640 10 client.27335 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:55.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1b8c00640 1 -- 172.21.15.107:0/688189668 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f71ec8a950 con 0x55f71ed325e0 2024-09-19T04:31:55.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1b8c00640 10 client.27335 handle_command_reply: tid=0 2024-09-19T04:31:55.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1d8a31740 1 client.27335 shutdown 2024-09-19T04:31:55.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1d8a31740 1 -- 172.21.15.107:0/688189668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed17760 msgr2=0x55f71ed1f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:55.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed17760 0x55f71ed1f540 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc006010 tx=0x7fa1bc0046c0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1d8a31740 1 -- 172.21.15.107:0/688189668 shutdown_connections 2024-09-19T04:31:55.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f71ed325e0 0x55f71ed34a40 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed22020 0x55f71ed1fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed17760 0x55f71ed1f540 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/688189668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed16e40 0x55f71ed1f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1d8a31740 1 -- 172.21.15.107:0/688189668 >> 172.21.15.107:0/688189668 conn(0x55f71ed133d0 msgr2=0x55f71ed19100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:55.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1b8c00640 0 client.27335 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:55.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.888+0000 7fa1d8a31740 1 -- 172.21.15.107:0/688189668 shutdown_connections 2024-09-19T04:31:55.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.889+0000 7fa1d8a31740 1 -- 172.21.15.107:0/688189668 wait complete. 2024-09-19T04:31:55.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.889+0000 7fa1d8a31740 20 client.27335 trim_cache size 0 max 16384 2024-09-19T04:31:55.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.902+0000 7fa1d8a31740 1 Processor -- start 2024-09-19T04:31:55.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.902+0000 7fa1d8a31740 1 -- start start 2024-09-19T04:31:55.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.902+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed198f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.902+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 0x55f71ed3a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.902+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 0x55f71ed3a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01af830 con 0x55f71ed22020 2024-09-19T04:31:55.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b1370 con 0x55f71ed16e40 2024-09-19T04:31:55.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01aff00 con 0x55f71ed17760 2024-09-19T04:31:55.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1cea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1cea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed198f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45170/0 (socket says 172.21.15.107:45170) 2024-09-19T04:31:55.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1cea00640 1 -- 172.21.15.107:0/1092584717 learned_addr learned my addr 172.21.15.107:0/1092584717 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:55.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1ce000640 1 --2- 172.21.15.107:0/1092584717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 0x55f71ed3a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1cf400640 1 --2- 172.21.15.107:0/1092584717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 0x55f71ed3a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1cea00640 1 -- 172.21.15.107:0/1092584717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 msgr2=0x55f71ed3a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1cea00640 1 --2- 172.21.15.107:0/1092584717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 0x55f71ed3a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1cea00640 1 -- 172.21.15.107:0/1092584717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 msgr2=0x55f71ed3a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1cea00640 1 --2- 172.21.15.107:0/1092584717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 0x55f71ed3a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1cea00640 1 -- 172.21.15.107:0/1092584717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71ed24d00 con 0x55f71ed16e40 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.903+0000 7fa1ce000640 1 --2- 172.21.15.107:0/1092584717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 0x55f71ed3a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1cea00640 1 --2- 172.21.15.107:0/1092584717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed198f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0000c00 tx=0x7fa1c0009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48db2a5bb6086247 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1bb400640 1 -- 172.21.15.107:0/1092584717 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c001e040 con 0x55f71ed16e40 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1bb400640 1 -- 172.21.15.107:0/1092584717 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c001f040 con 0x55f71ed16e40 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1bb400640 1 -- 172.21.15.107:0/1092584717 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000ed20 con 0x55f71ed16e40 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1d8a31740 1 -- 172.21.15.107:0/1092584717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 msgr2=0x55f71ed198f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/1092584717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed198f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0000c00 tx=0x7fa1c0009c20 comp rx=0 tx=0).stop 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1d8a31740 1 -- 172.21.15.107:0/1092584717 shutdown_connections 2024-09-19T04:31:55.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/1092584717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 0x55f71ed3a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/1092584717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 0x55f71ed3a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/1092584717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed198f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.904+0000 7fa1d8a31740 1 -- 172.21.15.107:0/1092584717 >> 172.21.15.107:0/1092584717 conn(0x55f71ed133d0 msgr2=0x55f71ed1a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:55.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.905+0000 7fa1d8a31740 1 -- 172.21.15.107:0/1092584717 shutdown_connections 2024-09-19T04:31:55.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.905+0000 7fa1d8a31740 1 -- 172.21.15.107:0/1092584717 wait complete. 2024-09-19T04:31:55.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.905+0000 7fa1d8a31740 1 -- start start 2024-09-19T04:31:55.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.905+0000 7fa1d8a31740 1 Processor -- start 2024-09-19T04:31:55.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed25700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 0x55f71ed25c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1d8a31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 0x55f71ed9b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b0ca0 con 0x55f71ed22020 2024-09-19T04:31:55.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b05d0 con 0x55f71ed16e40 2024-09-19T04:31:55.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1d8a31740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01ad830 con 0x55f71ed17760 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1d8a31740 10 client.0 _ll_register_callbacks cb 0x7fa1bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1cea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed25700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1cf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 0x55f71ed9b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1cf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 0x55f71ed9b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44674/0 (socket says 172.21.15.107:44674) 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1ce000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 0x55f71ed25c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.906+0000 7fa1cf400640 1 -- 172.21.15.107:0/721828500 learned_addr learned my addr 172.21.15.107:0/721828500 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1cf400640 1 -- 172.21.15.107:0/721828500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 msgr2=0x55f71ed25c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1cf400640 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 0x55f71ed25c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1cf400640 1 -- 172.21.15.107:0/721828500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 msgr2=0x55f71ed25700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1cf400640 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed25700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1cf400640 1 -- 172.21.15.107:0/721828500 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f71ed9b8e0 con 0x55f71ed22020 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1cf400640 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 0x55f71ed9b2d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8005fd0 tx=0x7fa1c809c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72da2f3cd97c636 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:55.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1b8c00640 1 -- 172.21.15.107:0/721828500 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80b5490 con 0x55f71ed22020 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1b8c00640 1 -- 172.21.15.107:0/721828500 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fa1c809c750 con 0x55f71ed22020 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1d8a31740 1 -- 172.21.15.107:0/721828500 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c001f040 con 0x55f71ed22020 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1b8c00640 1 -- 172.21.15.107:0/721828500 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa1c8009820 con 0x55f71ed22020 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1d8a31740 10 client.17292 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.907+0000 7fa1d8a31740 1 -- 172.21.15.107:0/721828500 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f71ed9cc00 con 0x55f71ed22020 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.908+0000 7fa1b8c00640 1 -- 172.21.15.107:0/721828500 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c001f040 con 0x55f71ed22020 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.908+0000 7fa1b8c00640 1 -- 172.21.15.107:0/721828500 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fa1c80a4cd0 con 0x55f71ed22020 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.908+0000 7fa1d8a31740 10 client.17292 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.908+0000 7fa1d8a31740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.908+0000 7fa1d8a31740 20 client.17292 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.908+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f71ed9d370 0x55f71ed9f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:55.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.908+0000 7fa1d8a31740 4 client.17292 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:31:55.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.908+0000 7fa1d8a31740 1 -- 172.21.15.107:0/721828500 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f71ec84e60 con 0x55f71ed9d370 2024-09-19T04:31:55.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.908+0000 7fa1cea00640 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f71ed9d370 0x55f71ed9f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1cea00640 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f71ed9d370 0x55f71ed9f760 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1571901ccf92d708 server_cookie=813ff7a01836c45f in_seq=0 out_seq=0 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1b8c00640 10 client.17292 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1b8c00640 1 -- 172.21.15.107:0/721828500 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f71ec84e60 con 0x55f71ed9d370 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1b8c00640 10 client.17292 handle_command_reply: tid=0 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1d8a31740 1 client.17292 shutdown 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1d8a31740 1 -- 172.21.15.107:0/721828500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 msgr2=0x55f71ed9b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 0x55f71ed9b2d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8005fd0 tx=0x7fa1c809c380 comp rx=0 tx=0).stop 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1d8a31740 1 -- 172.21.15.107:0/721828500 shutdown_connections 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55f71ed9d370 0x55f71ed9f760 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71ed22020 0x55f71ed9b2d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f71ed17760 0x55f71ed25c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1b8c00640 0 client.17292 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:55.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1d8a31740 1 --2- 172.21.15.107:0/721828500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f71ed16e40 0x55f71ed25700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.909+0000 7fa1d8a31740 1 -- 172.21.15.107:0/721828500 >> 172.21.15.107:0/721828500 conn(0x55f71ed133d0 msgr2=0x55f71ed1efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:55.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.910+0000 7fa1d8a31740 1 -- 172.21.15.107:0/721828500 shutdown_connections 2024-09-19T04:31:55.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.910+0000 7fa1d8a31740 1 -- 172.21.15.107:0/721828500 wait complete. 2024-09-19T04:31:55.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.910+0000 7fa1d8a31740 20 client.17292 trim_cache size 0 max 16384 2024-09-19T04:31:55.915 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:31:55.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.910+0000 7fa1cfe00640 1 -- 172.21.15.107:0/1892136063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa1b40a3430 msgr2=0x7fa1b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:55.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.910+0000 7fa1cfe00640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa1b40a3430 0x7fa1b40a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55f71ed14750 tx=0x7fa1bc041040 comp rx=0 tx=0).stop 2024-09-19T04:31:55.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.910+0000 7fa1cfe00640 1 -- 172.21.15.107:0/1892136063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d0108f60 msgr2=0x7fa1d019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:55.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.910+0000 7fa1cfe00640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d0108f60 0x7fa1d019ed60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800ec50 tx=0x7fa1c800c6a0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.911+0000 7fa1cfe00640 1 -- 172.21.15.107:0/1892136063 shutdown_connections 2024-09-19T04:31:55.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.911+0000 7fa1cfe00640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa1b40a3430 0x7fa1b40a58f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.911+0000 7fa1cfe00640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d0108f60 0x7fa1d019ed60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.911+0000 7fa1cfe00640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d01047e0 0x7fa1d019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.911+0000 7fa1cfe00640 1 --2- 172.21.15.107:0/1892136063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d0103e30 0x7fa1d019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:55.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.911+0000 7fa1cfe00640 1 -- 172.21.15.107:0/1892136063 >> 172.21.15.107:0/1892136063 conn(0x7fa1d00ff910 msgr2=0x7fa1d0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:55.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.911+0000 7fa1cfe00640 1 -- 172.21.15.107:0/1892136063 shutdown_connections 2024-09-19T04:31:55.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:55.911+0000 7fa1cfe00640 1 -- 172.21.15.107:0/1892136063 wait complete. 2024-09-19T04:31:56.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 395459 2.91 MB/sec execute 494 sec latency 207.260 ms 2024-09-19T04:31:57.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:56 smithi107 ceph-mon[30659]: pgmap v361: 97 pgs: 97 active+clean; 327 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-09-19T04:31:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:56 smithi145 ceph-mon[35096]: pgmap v361: 97 pgs: 97 active+clean; 327 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-09-19T04:31:57.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:56 smithi115 ceph-mon[35477]: pgmap v361: 97 pgs: 97 active+clean; 327 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-09-19T04:31:57.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 395459 2.91 MB/sec execute 495 sec latency 1207.334 ms 2024-09-19T04:31:57.980 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:31:57.980 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:31:58.349 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:31:58.724 INFO:tasks.workunit.client.0.smithi107.stdout: 1 395459 2.90 MB/sec execute 496 sec latency 2207.375 ms 2024-09-19T04:31:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.889+0000 7f8e3f600640 1 -- 172.21.15.107:0/1738867585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e38106f10 msgr2=0x7f8e3810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.889+0000 7f8e3f600640 1 --2- 172.21.15.107:0/1738867585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e38106f10 0x7f8e3810ea90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009ca0 tx=0x7f8e3002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.889+0000 7f8e3f600640 1 -- 172.21.15.107:0/1738867585 shutdown_connections 2024-09-19T04:31:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.889+0000 7f8e3f600640 1 --2- 172.21.15.107:0/1738867585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e38106f10 0x7f8e3810ea90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.889+0000 7f8e3f600640 1 --2- 172.21.15.107:0/1738867585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e38102790 0x7f8e381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.889+0000 7f8e3f600640 1 --2- 172.21.15.107:0/1738867585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e38101de0 0x7f8e381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.889+0000 7f8e3f600640 1 -- 172.21.15.107:0/1738867585 >> 172.21.15.107:0/1738867585 conn(0x7f8e380fd8c0 msgr2=0x7f8e380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.889+0000 7f8e3f600640 1 -- 172.21.15.107:0/1738867585 shutdown_connections 2024-09-19T04:31:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.889+0000 7f8e3f600640 1 -- 172.21.15.107:0/1738867585 wait complete. 2024-09-19T04:31:58.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.890+0000 7f8e3f600640 1 Processor -- start 2024-09-19T04:31:58.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.890+0000 7f8e3f600640 1 -- start start 2024-09-19T04:31:58.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.890+0000 7f8e3f600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e38101de0 0x7f8e3819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.890+0000 7f8e3f600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e38102790 0x7f8e3819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3f600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e38106f10 0x7f8e3819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3f600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8e38111fc0 con 0x7f8e38101de0 2024-09-19T04:31:58.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3f600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8e38111e40 con 0x7f8e38102790 2024-09-19T04:31:58.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3f600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e38112140 con 0x7f8e38106f10 2024-09-19T04:31:58.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e38101de0 0x7f8e3819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e37e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e38102790 0x7f8e3819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e38101de0 0x7f8e3819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58468/0 (socket says 172.21.15.107:58468) 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3cc00640 1 -- 172.21.15.107:0/563437670 learned_addr learned my addr 172.21.15.107:0/563437670 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e37e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e38102790 0x7f8e3819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:43698/0 (socket says 172.21.15.107:43698) 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3d600640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e38106f10 0x7f8e3819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3cc00640 1 -- 172.21.15.107:0/563437670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e38106f10 msgr2=0x7f8e3819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3cc00640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e38106f10 0x7f8e3819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3cc00640 1 -- 172.21.15.107:0/563437670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e38102790 msgr2=0x7f8e3819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3cc00640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e38102790 0x7f8e3819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.891+0000 7f8e3cc00640 1 -- 172.21.15.107:0/563437670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3819f600 con 0x7f8e38101de0 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.892+0000 7f8e3cc00640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e38101de0 0x7f8e3819a650 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d950 tx=0x7f8e2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.892+0000 7f8e35600640 1 -- 172.21.15.107:0/563437670 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c017070 con 0x7f8e38101de0 2024-09-19T04:31:58.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.892+0000 7f8e35600640 1 -- 172.21.15.107:0/563437670 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c009c30 con 0x7f8e38101de0 2024-09-19T04:31:58.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.892+0000 7f8e3f600640 1 -- 172.21.15.107:0/563437670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3819f8f0 con 0x7f8e38101de0 2024-09-19T04:31:58.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.892+0000 7f8e35600640 1 -- 172.21.15.107:0/563437670 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00f6b0 con 0x7f8e38101de0 2024-09-19T04:31:58.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.892+0000 7f8e3f600640 1 -- 172.21.15.107:0/563437670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e381a71d0 con 0x7f8e38101de0 2024-09-19T04:31:58.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.895+0000 7f8e4224a740 1 Processor -- start 2024-09-19T04:31:58.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.895+0000 7f8e4224a740 1 -- start start 2024-09-19T04:31:58.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.895+0000 7f8e35600640 1 -- 172.21.15.107:0/563437670 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f8e2c0109d0 con 0x7f8e38101de0 2024-09-19T04:31:58.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.895+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e214ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.895+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 0x56450e222090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e4224a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56450e17b290 con 0x56450e214ac0 2024-09-19T04:31:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e4224a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56450e122b40 con 0x56450e21fca0 2024-09-19T04:31:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e4224a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56450e177250 con 0x56450e2153e0 2024-09-19T04:31:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e3d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e214ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e3d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e214ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58476/0 (socket says 172.21.15.107:58476) 2024-09-19T04:31:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e3d600640 1 -- 172.21.15.107:0/3386090070 learned_addr learned my addr 172.21.15.107:0/3386090070 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e3cc00640 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e37e00640 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 0x56450e222090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e3d600640 1 -- 172.21.15.107:0/3386090070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 msgr2=0x56450e21f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e3d600640 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e3d600640 1 -- 172.21.15.107:0/3386090070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 msgr2=0x56450e222090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e3d600640 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 0x56450e222090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e3d600640 1 -- 172.21.15.107:0/3386090070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56450deec6a0 con 0x56450e214ac0 2024-09-19T04:31:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.896+0000 7f8e37e00640 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 0x56450e222090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.897+0000 7f8e3cc00640 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.897+0000 7f8e3d600640 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e214ea0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009870 tx=0x7f8e300098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0d1d2091b41608 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.897+0000 7f8e35600640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e200a33b0 0x7f8e200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.897+0000 7f8e29c00640 1 -- 172.21.15.107:0/3386090070 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30031db0 con 0x56450e214ac0 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.897+0000 7f8e29c00640 1 -- 172.21.15.107:0/3386090070 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e30036040 con 0x56450e214ac0 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.897+0000 7f8e29c00640 1 -- 172.21.15.107:0/3386090070 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30004380 con 0x56450e214ac0 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.897+0000 7f8e3cc00640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e200a33b0 0x7f8e200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e35600640 1 -- 172.21.15.107:0/563437670 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f8e2c0cf9f0 con 0x7f8e38101de0 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e4224a740 1 -- 172.21.15.107:0/3386090070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 msgr2=0x56450e214ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e4224a740 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e214ea0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009870 tx=0x7f8e300098a0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e3cc00640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e200a33b0 0x7f8e200a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0d2fd0 tx=0x7f8e2c0d5530 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e4224a740 1 -- 172.21.15.107:0/3386090070 shutdown_connections 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e4224a740 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 0x56450e222090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e4224a740 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e4224a740 1 --2- 172.21.15.107:0/3386090070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e214ea0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e4224a740 1 -- 172.21.15.107:0/3386090070 >> 172.21.15.107:0/3386090070 conn(0x56450e211050 msgr2=0x56450e211430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:58.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e4224a740 1 -- 172.21.15.107:0/3386090070 shutdown_connections 2024-09-19T04:31:58.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.898+0000 7f8e4224a740 1 -- 172.21.15.107:0/3386090070 wait complete. 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.899+0000 7f8e4224a740 1 -- start start 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.899+0000 7f8e4224a740 1 Processor -- start 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e214ac0 0x56450e21f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e21fca0 0x56450e21ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e4224a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56450e222910 con 0x56450e21fca0 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e4224a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56450e135fa0 con 0x56450e214ac0 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e4224a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56450e222a90 con 0x56450e2153e0 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e4224a740 10 client.0 _ll_register_callbacks cb 0x56450de89ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e3d600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e214ac0 0x56450e21f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e37e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e3d600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e214ac0 0x56450e21f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:43728/0 (socket says 172.21.15.107:43728) 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e3d600640 1 -- 172.21.15.107:0/3747956197 learned_addr learned my addr 172.21.15.107:0/3747956197 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e37e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43860/0 (socket says 172.21.15.107:43860) 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e3d600640 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e21fca0 0x56450e21ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e37e00640 1 -- 172.21.15.107:0/3747956197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e214ac0 msgr2=0x56450e21f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e37e00640 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e214ac0 0x56450e21f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e37e00640 1 -- 172.21.15.107:0/3747956197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e21fca0 msgr2=0x56450e21ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e37e00640 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e21fca0 0x56450e21ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.900+0000 7f8e37e00640 1 -- 172.21.15.107:0/3747956197 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56450e21d640 con 0x56450e2153e0 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.901+0000 7f8e3d600640 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e21fca0 0x56450e21ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.901+0000 7f8e37e00640 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21c8b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8e3810ee50 tx=0x7f8e24012a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f108fb6cc5453de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.901+0000 7f8e1e000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:31:58.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.901+0000 7f8e1e000640 1 -- 172.21.15.107:0/3747956197 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24004050 con 0x56450e2153e0 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.901+0000 7f8e4224a740 1 -- 172.21.15.107:0/3747956197 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e30036040 con 0x56450e2153e0 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.901+0000 7f8e1e000640 1 -- 172.21.15.107:0/3747956197 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f8e240041f0 con 0x56450e2153e0 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e1e000640 1 -- 172.21.15.107:0/3747956197 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8e240043f0 con 0x56450e2153e0 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e4224a740 10 client.27301 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e4224a740 1 -- 172.21.15.107:0/3747956197 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56450e22e900 con 0x56450e2153e0 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e1e000640 1 -- 172.21.15.107:0/3747956197 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e30036040 con 0x56450e2153e0 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e1e000640 1 -- 172.21.15.107:0/3747956197 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f8e24023020 con 0x56450e2153e0 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e4224a740 10 client.27301 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e4224a740 10 client.27301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e4224a740 20 client.27301 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e4224a740 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56450e22fd10 0x56450e232100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e4224a740 4 client.27301 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:31:58.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.902+0000 7f8e4224a740 1 -- 172.21.15.107:0/3747956197 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56450e188800 con 0x56450e22fd10 2024-09-19T04:31:58.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.903+0000 7f8e3d600640 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56450e22fd10 0x56450e232100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.903+0000 7f8e3d600640 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56450e22fd10 0x56450e232100 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89bf230da032633d server_cookie=75a950dca753f62c in_seq=0 out_seq=0 2024-09-19T04:31:58.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.903+0000 7f8e1e000640 10 client.27301 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:58.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.904+0000 7f8e1e000640 1 -- 172.21.15.107:0/3747956197 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56450e188800 con 0x56450e22fd10 2024-09-19T04:31:58.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.904+0000 7f8e1e000640 10 client.27301 handle_command_reply: tid=0 2024-09-19T04:31:58.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.904+0000 7f8e4224a740 1 client.27301 shutdown 2024-09-19T04:31:58.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 1 -- 172.21.15.107:0/3747956197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 msgr2=0x56450e21c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:58.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21c8b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8e3810ee50 tx=0x7f8e24012a80 comp rx=0 tx=0).stop 2024-09-19T04:31:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 1 -- 172.21.15.107:0/3747956197 shutdown_connections 2024-09-19T04:31:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56450e22fd10 0x56450e232100 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e21fca0 0x56450e21ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e2153e0 0x56450e21c8b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 1 --2- 172.21.15.107:0/3747956197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e214ac0 0x56450e21f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e1e000640 0 client.27301 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 1 -- 172.21.15.107:0/3747956197 >> 172.21.15.107:0/3747956197 conn(0x56450e211050 msgr2=0x56450e216d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 1 -- 172.21.15.107:0/3747956197 shutdown_connections 2024-09-19T04:31:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 1 -- 172.21.15.107:0/3747956197 wait complete. 2024-09-19T04:31:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.905+0000 7f8e4224a740 20 client.27301 trim_cache size 0 max 16384 2024-09-19T04:31:58.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.922+0000 7f8e4224a740 1 Processor -- start 2024-09-19T04:31:58.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e4224a740 1 -- start start 2024-09-19T04:31:58.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e217e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e2153e0 0x56450e237960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e21fca0 0x56450e237ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e4224a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381afa90 con 0x56450e214ac0 2024-09-19T04:31:58.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e4224a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381b15d0 con 0x56450e2153e0 2024-09-19T04:31:58.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e4224a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381b0160 con 0x56450e21fca0 2024-09-19T04:31:58.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e37e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e2153e0 0x56450e237960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e3d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e217e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e37e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e2153e0 0x56450e237960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:43740/0 (socket says 172.21.15.107:43740) 2024-09-19T04:31:58.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e3d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e217e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58482/0 (socket says 172.21.15.107:58482) 2024-09-19T04:31:58.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.923+0000 7f8e37e00640 1 -- 172.21.15.107:0/1063858856 learned_addr learned my addr 172.21.15.107:0/1063858856 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:58.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e3d600640 1 --2- 172.21.15.107:0/1063858856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e21fca0 0x56450e237ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e37e00640 1 -- 172.21.15.107:0/1063858856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e21fca0 msgr2=0x56450e237ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:58.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e37e00640 1 --2- 172.21.15.107:0/1063858856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e21fca0 0x56450e237ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e37e00640 1 -- 172.21.15.107:0/1063858856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 msgr2=0x56450e217e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:58.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e37e00640 1 --2- 172.21.15.107:0/1063858856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e217e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e37e00640 1 -- 172.21.15.107:0/1063858856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56450e232730 con 0x56450e2153e0 2024-09-19T04:31:58.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e3d600640 1 --2- 172.21.15.107:0/1063858856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e21fca0 0x56450e237ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:31:58.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e37e00640 1 --2- 172.21.15.107:0/1063858856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e2153e0 0x56450e237960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8e24012420 tx=0x7f8e24005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9499d1d8aadad60d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:58.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e29200640 1 -- 172.21.15.107:0/1063858856 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e240035f0 con 0x56450e2153e0 2024-09-19T04:31:58.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e29200640 1 -- 172.21.15.107:0/1063858856 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2402a070 con 0x56450e2153e0 2024-09-19T04:31:58.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.924+0000 7f8e29200640 1 -- 172.21.15.107:0/1063858856 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24005530 con 0x56450e2153e0 2024-09-19T04:31:58.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.925+0000 7f8e4224a740 1 -- 172.21.15.107:0/1063858856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e2153e0 msgr2=0x56450e237960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:58.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.925+0000 7f8e4224a740 1 --2- 172.21.15.107:0/1063858856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e2153e0 0x56450e237960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8e24012420 tx=0x7f8e24005850 comp rx=0 tx=0).stop 2024-09-19T04:31:58.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.925+0000 7f8e4224a740 1 -- 172.21.15.107:0/1063858856 shutdown_connections 2024-09-19T04:31:58.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.925+0000 7f8e4224a740 1 --2- 172.21.15.107:0/1063858856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e21fca0 0x56450e237ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.925+0000 7f8e4224a740 1 --2- 172.21.15.107:0/1063858856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e2153e0 0x56450e237960 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.925+0000 7f8e4224a740 1 --2- 172.21.15.107:0/1063858856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e214ac0 0x56450e217e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.925+0000 7f8e4224a740 1 -- 172.21.15.107:0/1063858856 >> 172.21.15.107:0/1063858856 conn(0x56450e211050 msgr2=0x56450e217700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:58.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.925+0000 7f8e4224a740 1 -- 172.21.15.107:0/1063858856 shutdown_connections 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.925+0000 7f8e4224a740 1 -- 172.21.15.107:0/1063858856 wait complete. 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.925+0000 7f8e4224a740 1 -- start start 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.926+0000 7f8e4224a740 1 Processor -- start 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.926+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e214ac0 0x56450e2239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.926+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 0x56450e223ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.926+0000 7f8e4224a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e224430 0x56450e2a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.926+0000 7f8e4224a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381b0f00 con 0x56450e224430 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.926+0000 7f8e4224a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381b0830 con 0x56450e21fca0 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.926+0000 7f8e4224a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381ada90 con 0x56450e214ac0 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.926+0000 7f8e4224a740 10 client.0 _ll_register_callbacks cb 0x56450e21d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.926+0000 7f8e3d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e214ac0 0x56450e2239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e37e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 0x56450e223ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e3d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e214ac0 0x56450e2239b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43878/0 (socket says 172.21.15.107:43878) 2024-09-19T04:31:58.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e3d600640 1 -- 172.21.15.107:0/4037039150 learned_addr learned my addr 172.21.15.107:0/4037039150 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:31:58.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e3d600640 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e224430 0x56450e2a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e37e00640 1 -- 172.21.15.107:0/4037039150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e214ac0 msgr2=0x56450e2239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:58.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e37e00640 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e214ac0 0x56450e2239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e37e00640 1 -- 172.21.15.107:0/4037039150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e224430 msgr2=0x56450e2a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:31:58.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e37e00640 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e224430 0x56450e2a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e37e00640 1 -- 172.21.15.107:0/4037039150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56450e2a2680 con 0x56450e21fca0 2024-09-19T04:31:58.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e37e00640 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 0x56450e223ef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x56450e11db30 tx=0x7f8e240054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9960fa55f624c7bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:31:58.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e1e000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:31:58.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.927+0000 7f8e1e000640 1 -- 172.21.15.107:0/4037039150 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24003f90 con 0x56450e21fca0 2024-09-19T04:31:58.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.928+0000 7f8e4224a740 1 -- 172.21.15.107:0/4037039150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e2402a070 con 0x56450e21fca0 2024-09-19T04:31:58.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.928+0000 7f8e1e000640 1 -- 172.21.15.107:0/4037039150 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f8e24018030 con 0x56450e21fca0 2024-09-19T04:31:58.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.928+0000 7f8e1e000640 1 -- 172.21.15.107:0/4037039150 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8e2403e050 con 0x56450e21fca0 2024-09-19T04:31:58.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.928+0000 7f8e4224a740 10 client.27359 fetch_fsmap learned FSMap version 48 2024-09-19T04:31:58.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.928+0000 7f8e4224a740 1 -- 172.21.15.107:0/4037039150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56450e2a3770 con 0x56450e21fca0 2024-09-19T04:31:58.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.928+0000 7f8e1e000640 1 -- 172.21.15.107:0/4037039150 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2402a070 con 0x56450e21fca0 2024-09-19T04:31:58.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.928+0000 7f8e1e000640 1 -- 172.21.15.107:0/4037039150 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f8e24023040 con 0x56450e21fca0 2024-09-19T04:31:58.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.928+0000 7f8e4224a740 10 client.27359 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:31:58.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.928+0000 7f8e4224a740 10 client.27359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:31:58.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.929+0000 7f8e4224a740 20 client.27359 populate_metadata read hostname 'smithi107' 2024-09-19T04:31:58.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.929+0000 7f8e4224a740 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56450e2a3d70 0x56450e2a6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:31:58.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.929+0000 7f8e4224a740 4 client.27359 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:31:58.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.929+0000 7f8e4224a740 1 -- 172.21.15.107:0/4037039150 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56450e182d10 con 0x56450e2a3d70 2024-09-19T04:31:58.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.929+0000 7f8e3d600640 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56450e2a3d70 0x56450e2a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.929+0000 7f8e3d600640 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56450e2a3d70 0x56450e2a6160 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f85a6bb20f430856 server_cookie=fe2920ae03a05323 in_seq=0 out_seq=0 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.929+0000 7f8e1e000640 10 client.27359 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.930+0000 7f8e1e000640 1 -- 172.21.15.107:0/4037039150 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56450e182d10 con 0x56450e2a3d70 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.930+0000 7f8e1e000640 10 client.27359 handle_command_reply: tid=0 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.930+0000 7f8e4224a740 1 client.27359 shutdown 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 1 -- 172.21.15.107:0/4037039150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 msgr2=0x56450e223ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 0x56450e223ef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x56450e11db30 tx=0x7f8e240054e0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 1 -- 172.21.15.107:0/4037039150 shutdown_connections 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56450e2a3d70 0x56450e2a6160 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56450e224430 0x56450e2a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56450e21fca0 0x56450e223ef0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 1 --2- 172.21.15.107:0/4037039150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56450e214ac0 0x56450e2239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 1 -- 172.21.15.107:0/4037039150 >> 172.21.15.107:0/4037039150 conn(0x56450e211050 msgr2=0x56450e21ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:58.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e1e000640 0 client.27359 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:31:58.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 1 -- 172.21.15.107:0/4037039150 shutdown_connections 2024-09-19T04:31:58.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 1 -- 172.21.15.107:0/4037039150 wait complete. 2024-09-19T04:31:58.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.931+0000 7f8e4224a740 20 client.27359 trim_cache size 0 max 16384 2024-09-19T04:31:58.937 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:31:58.937 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:31:58.937 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "0f633d4d-8e30-48ea-a9b7-c0a272861a80", 2024-09-19T04:31:58.937 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:31:58.937 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:31:58.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 -- 172.21.15.107:0/563437670 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e200a33b0 msgr2=0x7f8e200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:58.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e200a33b0 0x7f8e200a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0d2fd0 tx=0x7f8e2c0d5530 comp rx=0 tx=0).stop 2024-09-19T04:31:58.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 -- 172.21.15.107:0/563437670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e38101de0 msgr2=0x7f8e3819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:31:58.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e38101de0 0x7f8e3819a650 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d950 tx=0x7f8e2c00de10 comp rx=0 tx=0).stop 2024-09-19T04:31:58.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 -- 172.21.15.107:0/563437670 shutdown_connections 2024-09-19T04:31:58.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e200a33b0 0x7f8e200a5870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e38106f10 0x7f8e3819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e38102790 0x7f8e3819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 --2- 172.21.15.107:0/563437670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e38101de0 0x7f8e3819a650 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:31:58.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 -- 172.21.15.107:0/563437670 >> 172.21.15.107:0/563437670 conn(0x7f8e380fd8c0 msgr2=0x7f8e380ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:31:58.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 -- 172.21.15.107:0/563437670 shutdown_connections 2024-09-19T04:31:58.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:31:58.932+0000 7f8e3f600640 1 -- 172.21.15.107:0/563437670 wait complete. 2024-09-19T04:31:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:58 smithi107 ceph-mon[30659]: pgmap v362: 97 pgs: 97 active+clean; 327 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-19T04:31:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:58 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:31:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:58 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:59.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:58 smithi115 ceph-mon[35477]: pgmap v362: 97 pgs: 97 active+clean; 327 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-19T04:31:59.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:58 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:31:59.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:58 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:59.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:58 smithi145 ceph-mon[35096]: pgmap v362: 97 pgs: 97 active+clean; 327 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-19T04:31:59.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:58 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:31:59.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:58 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:31:59.725 INFO:tasks.workunit.client.0.smithi107.stdout: 1 397130 2.91 MB/sec execute 497 sec latency 2439.650 ms 2024-09-19T04:31:59.798 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:32:00.195 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:00.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:00.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:00.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:31:59 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:32:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:31:59 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:32:00.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:00.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:00.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:31:59 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:00.725 INFO:tasks.workunit.client.0.smithi107.stdout: 1 398253 2.92 MB/sec execute 498 sec latency 10.672 ms 2024-09-19T04:32:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.772+0000 7fe50a000640 1 -- 172.21.15.107:0/3064166644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041019c0 msgr2=0x7fe504101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.772+0000 7fe50a000640 1 --2- 172.21.15.107:0/3064166644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041019c0 0x7fe504101e20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009930 tx=0x7fe4f002a440 comp rx=0 tx=0).stop 2024-09-19T04:32:00.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.772+0000 7fe50a000640 1 -- 172.21.15.107:0/3064166644 shutdown_connections 2024-09-19T04:32:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.772+0000 7fe50a000640 1 --2- 172.21.15.107:0/3064166644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe504102360 0x7fe50410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.772+0000 7fe50a000640 1 --2- 172.21.15.107:0/3064166644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041019c0 0x7fe504101e20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.772+0000 7fe50a000640 1 --2- 172.21.15.107:0/3064166644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5041079c0 0x7fe504107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.772+0000 7fe50a000640 1 -- 172.21.15.107:0/3064166644 >> 172.21.15.107:0/3064166644 conn(0x7fe5040fd740 msgr2=0x7fe5040ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:00.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.772+0000 7fe50a000640 1 -- 172.21.15.107:0/3064166644 shutdown_connections 2024-09-19T04:32:00.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.772+0000 7fe50a000640 1 -- 172.21.15.107:0/3064166644 wait complete. 2024-09-19T04:32:00.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.773+0000 7fe50a000640 1 Processor -- start 2024-09-19T04:32:00.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.773+0000 7fe50a000640 1 -- start start 2024-09-19T04:32:00.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe50a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041019c0 0x7fe5040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe50a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe504102360 0x7fe504074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe50a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5041079c0 0x7fe504074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe50a000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe504111e10 con 0x7fe5041019c0 2024-09-19T04:32:00.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe50a000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe504111c90 con 0x7fe5041079c0 2024-09-19T04:32:00.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe50a000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe504111f90 con 0x7fe504102360 2024-09-19T04:32:00.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe509600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5041079c0 0x7fe504074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe503e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe504102360 0x7fe504074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe508c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041019c0 0x7fe5040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe509600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5041079c0 0x7fe504074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:43778/0 (socket says 172.21.15.107:43778) 2024-09-19T04:32:00.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe509600640 1 -- 172.21.15.107:0/1643387087 learned_addr learned my addr 172.21.15.107:0/1643387087 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe509600640 1 -- 172.21.15.107:0/1643387087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe504102360 msgr2=0x7fe504074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe509600640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe504102360 0x7fe504074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe509600640 1 -- 172.21.15.107:0/1643387087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041019c0 msgr2=0x7fe5040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe509600640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041019c0 0x7fe5040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.774+0000 7fe509600640 1 -- 172.21.15.107:0/1643387087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5041a67f0 con 0x7fe5041079c0 2024-09-19T04:32:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.775+0000 7fe509600640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5041079c0 0x7fe504074f70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00eb10 tx=0x7fe4fc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.775+0000 7fe501600640 1 -- 172.21.15.107:0/1643387087 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc004b20 con 0x7fe5041079c0 2024-09-19T04:32:00.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.775+0000 7fe501600640 1 -- 172.21.15.107:0/1643387087 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc004cc0 con 0x7fe5041079c0 2024-09-19T04:32:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.775+0000 7fe50a000640 1 -- 172.21.15.107:0/1643387087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5041a6ae0 con 0x7fe5041079c0 2024-09-19T04:32:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.775+0000 7fe501600640 1 -- 172.21.15.107:0/1643387087 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc010760 con 0x7fe5041079c0 2024-09-19T04:32:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.775+0000 7fe50a000640 1 -- 172.21.15.107:0/1643387087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5041a6fa0 con 0x7fe5041079c0 2024-09-19T04:32:00.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.778+0000 7fe501600640 1 -- 172.21.15.107:0/1643387087 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fe4fc005020 con 0x7fe5041079c0 2024-09-19T04:32:00.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.779+0000 7fe50e1c8740 1 Processor -- start 2024-09-19T04:32:00.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.779+0000 7fe50e1c8740 1 -- start start 2024-09-19T04:32:00.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.779+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0866d50 0x5574c0867130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.779+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 0x5574c08719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.779+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0871f30 0x5574c0874320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.779+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5574c07cd3e0 con 0x5574c0867670 2024-09-19T04:32:00.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.779+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574c0774d80 con 0x5574c0871f30 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.779+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5574c07c93a0 con 0x5574c0866d50 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe503e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0866d50 0x5574c0867130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe503e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0866d50 0x5574c0867130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43904/0 (socket says 172.21.15.107:43904) 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe503e00640 1 -- 172.21.15.107:0/2810380974 learned_addr learned my addr 172.21.15.107:0/2810380974 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe508c00640 1 --2- 172.21.15.107:0/2810380974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 0x5574c08719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe503e00640 1 --2- 172.21.15.107:0/2810380974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0871f30 0x5574c0874320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe503e00640 1 -- 172.21.15.107:0/2810380974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0871f30 msgr2=0x5574c0874320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe503e00640 1 --2- 172.21.15.107:0/2810380974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0871f30 0x5574c0874320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe503e00640 1 -- 172.21.15.107:0/2810380974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 msgr2=0x5574c08719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe503e00640 1 --2- 172.21.15.107:0/2810380974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 0x5574c08719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe503e00640 1 -- 172.21.15.107:0/2810380974 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c0793ae0 con 0x5574c0866d50 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.780+0000 7fe508c00640 1 --2- 172.21.15.107:0/2810380974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 0x5574c08719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.781+0000 7fe503e00640 1 --2- 172.21.15.107:0/2810380974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0866d50 0x5574c0867130 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0031550 tx=0x7fe4f002a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9159f15f0aaf8fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.781+0000 7fe4f5c00640 1 -- 172.21.15.107:0/2810380974 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0035030 con 0x5574c0866d50 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.781+0000 7fe501600640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe4ec0a33b0 0x7fe4ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.781+0000 7fe4f5c00640 1 -- 172.21.15.107:0/2810380974 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f00046c0 con 0x5574c0866d50 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.781+0000 7fe4f5c00640 1 -- 172.21.15.107:0/2810380974 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0003850 con 0x5574c0866d50 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.781+0000 7fe501600640 1 -- 172.21.15.107:0/1643387087 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fe4fc0d0310 con 0x7fe5041079c0 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.781+0000 7fe508c00640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe4ec0a33b0 0x7fe4ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2810380974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0866d50 msgr2=0x5574c0867130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/2810380974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0866d50 0x5574c0867130 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0031550 tx=0x7fe4f002a440 comp rx=0 tx=0).stop 2024-09-19T04:32:00.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2810380974 shutdown_connections 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/2810380974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0871f30 0x5574c0874320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/2810380974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 0x5574c08719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/2810380974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0866d50 0x5574c0867130 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2810380974 >> 172.21.15.107:0/2810380974 conn(0x5574c08632e0 msgr2=0x5574c08636c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe508c00640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe4ec0a33b0 0x7fe4ec0a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5574c0864140 tx=0x7fe4f8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2810380974 shutdown_connections 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2810380974 wait complete. 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.782+0000 7fe50e1c8740 1 -- start start 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe50e1c8740 1 Processor -- start 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0866d50 0x5574c086ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 0x5574c086f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0871f30 0x5574c086f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5574c0874ba0 con 0x5574c0867670 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574c07880f0 con 0x5574c0866d50 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5574c0874d20 con 0x5574c0871f30 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe50e1c8740 10 client.0 _ll_register_callbacks cb 0x5574c07799e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe508c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0866d50 0x5574c086ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe509600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0871f30 0x5574c086f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe508c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0866d50 0x5574c086ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:43792/0 (socket says 172.21.15.107:43792) 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe508c00640 1 -- 172.21.15.107:0/1801418077 learned_addr learned my addr 172.21.15.107:0/1801418077 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.783+0000 7fe503e00640 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 0x5574c086f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe509600640 1 -- 172.21.15.107:0/1801418077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0866d50 msgr2=0x5574c086ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe509600640 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0866d50 0x5574c086ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe509600640 1 -- 172.21.15.107:0/1801418077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 msgr2=0x5574c086f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe509600640 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 0x5574c086f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe509600640 1 -- 172.21.15.107:0/1801418077 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c053e3c0 con 0x5574c0871f30 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe503e00640 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 0x5574c086f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe509600640 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0871f30 0x5574c086f9b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0cd9f0 tx=0x7fe4fc09c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=def52bec685c2d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe4ea000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe4ea000640 1 -- 172.21.15.107:0/1801418077 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0b4020 con 0x5574c0871f30 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe4ea000640 1 -- 172.21.15.107:0/1801418077 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fe4fc0a4d90 con 0x5574c0871f30 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe50e1c8740 1 -- 172.21.15.107:0/1801418077 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f00046c0 con 0x5574c0871f30 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe4ea000640 1 -- 172.21.15.107:0/1801418077 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe4fc0cf040 con 0x5574c0871f30 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe50e1c8740 10 client.27316 fetch_fsmap learned FSMap version 48 2024-09-19T04:32:00.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.784+0000 7fe50e1c8740 1 -- 172.21.15.107:0/1801418077 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5574c0880970 con 0x5574c0871f30 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.785+0000 7fe4ea000640 1 -- 172.21.15.107:0/1801418077 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f00046c0 con 0x5574c0871f30 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.785+0000 7fe4ea000640 1 -- 172.21.15.107:0/1801418077 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fe4fc005310 con 0x5574c0871f30 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.785+0000 7fe50e1c8740 10 client.27316 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.785+0000 7fe50e1c8740 10 client.27316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.785+0000 7fe50e1c8740 20 client.27316 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.785+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574c0881d80 0x5574c08841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.785+0000 7fe50e1c8740 4 client.27316 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.785+0000 7fe50e1c8740 1 -- 172.21.15.107:0/1801418077 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c07da950 con 0x5574c0881d80 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.785+0000 7fe508c00640 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574c0881d80 0x5574c08841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.786+0000 7fe508c00640 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574c0881d80 0x5574c08841b0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2282598fad82f8ad server_cookie=33160b230c14e61 in_seq=0 out_seq=0 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.786+0000 7fe4ea000640 10 client.27316 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:00.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe4ea000640 1 -- 172.21.15.107:0/1801418077 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574c07da950 con 0x5574c0881d80 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe4ea000640 10 client.27316 handle_command_reply: tid=0 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 client.27316 shutdown 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 -- 172.21.15.107:0/1801418077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0871f30 msgr2=0x5574c086f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0871f30 0x5574c086f9b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0cd9f0 tx=0x7fe4fc09c0f0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 -- 172.21.15.107:0/1801418077 shutdown_connections 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574c0881d80 0x5574c08841b0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0871f30 0x5574c086f9b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0867670 0x5574c086f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/1801418077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c0866d50 0x5574c086ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 -- 172.21.15.107:0/1801418077 >> 172.21.15.107:0/1801418077 conn(0x5574c08632e0 msgr2=0x5574c0869010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe4ea000640 0 client.27316 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 -- 172.21.15.107:0/1801418077 shutdown_connections 2024-09-19T04:32:00.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.787+0000 7fe50e1c8740 1 -- 172.21.15.107:0/1801418077 wait complete. 2024-09-19T04:32:00.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.788+0000 7fe50e1c8740 20 client.27316 trim_cache size 0 max 16384 2024-09-19T04:32:00.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.806+0000 7fe50e1c8740 1 Processor -- start 2024-09-19T04:32:00.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.806+0000 7fe50e1c8740 1 -- start start 2024-09-19T04:32:00.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.806+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0869800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.806+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0866d50 0x5574c0889a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.806+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 0x5574c0889fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.806+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041af8c0 con 0x5574c0866d50 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.806+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b1400 con 0x5574c08632e0 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.806+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041aff90 con 0x5574c0867670 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe508c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0869800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe509600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 0x5574c0889fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe508c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0869800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:43794/0 (socket says 172.21.15.107:43794) 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe508c00640 1 -- 172.21.15.107:0/3970463217 learned_addr learned my addr 172.21.15.107:0/3970463217 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe503e00640 1 --2- 172.21.15.107:0/3970463217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0866d50 0x5574c0889a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe509600640 1 -- 172.21.15.107:0/3970463217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 msgr2=0x5574c0869800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe509600640 1 --2- 172.21.15.107:0/3970463217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0869800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe509600640 1 -- 172.21.15.107:0/3970463217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0866d50 msgr2=0x5574c0889a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:00.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe509600640 1 --2- 172.21.15.107:0/3970463217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0866d50 0x5574c0889a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe509600640 1 -- 172.21.15.107:0/3970463217 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c0867460 con 0x5574c0867670 2024-09-19T04:32:00.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe508c00640 1 --2- 172.21.15.107:0/3970463217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0869800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:00.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe509600640 1 --2- 172.21.15.107:0/3970463217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 0x5574c0889fb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc09c2b0 tx=0x7fe4fc09c220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19708ddc732132fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:00.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.807+0000 7fe4f5200640 1 -- 172.21.15.107:0/3970463217 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0b4070 con 0x5574c0867670 2024-09-19T04:32:00.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.808+0000 7fe4f5200640 1 -- 172.21.15.107:0/3970463217 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc09c5b0 con 0x5574c0867670 2024-09-19T04:32:00.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.808+0000 7fe4f5200640 1 -- 172.21.15.107:0/3970463217 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc005310 con 0x5574c0867670 2024-09-19T04:32:00.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.808+0000 7fe50e1c8740 1 -- 172.21.15.107:0/3970463217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 msgr2=0x5574c0889fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.808+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/3970463217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 0x5574c0889fb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc09c2b0 tx=0x7fe4fc09c220 comp rx=0 tx=0).stop 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.808+0000 7fe50e1c8740 1 -- 172.21.15.107:0/3970463217 shutdown_connections 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.808+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/3970463217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 0x5574c0889fb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.808+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/3970463217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0866d50 0x5574c0889a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.808+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/3970463217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0869800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.808+0000 7fe50e1c8740 1 -- 172.21.15.107:0/3970463217 >> 172.21.15.107:0/3970463217 conn(0x5574c086fb10 msgr2=0x5574c086d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.809+0000 7fe50e1c8740 1 -- 172.21.15.107:0/3970463217 shutdown_connections 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.809+0000 7fe50e1c8740 1 -- 172.21.15.107:0/3970463217 wait complete. 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.809+0000 7fe50e1c8740 1 -- start start 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.809+0000 7fe50e1c8740 1 Processor -- start 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0879f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0866d50 0x5574c087a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe50e1c8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 0x5574c0876f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b0d30 con 0x5574c0866d50 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b0660 con 0x5574c08632e0 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe50e1c8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041ad8c0 con 0x5574c0867670 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe50e1c8740 10 client.0 _ll_register_callbacks cb 0x7fe4fc098850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe508c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0879f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe509600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 0x5574c0876f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe508c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0879f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:43796/0 (socket says 172.21.15.107:43796) 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe503e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0866d50 0x5574c087a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe508c00640 1 -- 172.21.15.107:0/2791665510 learned_addr learned my addr 172.21.15.107:0/2791665510 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe508c00640 1 -- 172.21.15.107:0/2791665510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 msgr2=0x5574c0876f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe508c00640 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 0x5574c0876f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe508c00640 1 -- 172.21.15.107:0/2791665510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0866d50 msgr2=0x5574c087a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:00.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.810+0000 7fe508c00640 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0866d50 0x5574c087a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.811+0000 7fe508c00640 1 -- 172.21.15.107:0/2791665510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c087ab70 con 0x5574c08632e0 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.811+0000 7fe509600640 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 0x5574c0876f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.811+0000 7fe508c00640 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0879f70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80099f0 tx=0x7fe4f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d299cafb936c109 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.811+0000 7fe4ea000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.811+0000 7fe4ea000640 1 -- 172.21.15.107:0/2791665510 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8003af0 con 0x5574c08632e0 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.811+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2791665510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4fc09c5b0 con 0x5574c08632e0 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.811+0000 7fe4ea000640 1 -- 172.21.15.107:0/2791665510 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fe4f8003c90 con 0x5574c08632e0 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.811+0000 7fe4ea000640 1 -- 172.21.15.107:0/2791665510 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe4f8005020 con 0x5574c08632e0 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.811+0000 7fe50e1c8740 10 client.27377 fetch_fsmap learned FSMap version 48 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.811+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2791665510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5574c08f5890 con 0x5574c08632e0 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.812+0000 7fe4ea000640 1 -- 172.21.15.107:0/2791665510 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc09c5b0 con 0x5574c08632e0 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.812+0000 7fe4ea000640 1 -- 172.21.15.107:0/2791665510 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fe4f803a020 con 0x5574c08632e0 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.812+0000 7fe50e1c8740 10 client.27377 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:32:00.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.812+0000 7fe50e1c8740 10 client.27377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:00.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.812+0000 7fe50e1c8740 20 client.27377 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:00.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.812+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574c0871f30 0x5574c08f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:00.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.812+0000 7fe50e1c8740 4 client.27377 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:32:00.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.812+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2791665510 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574c07d4e60 con 0x5574c0871f30 2024-09-19T04:32:00.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.812+0000 7fe503e00640 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574c0871f30 0x5574c08f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:00.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe503e00640 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574c0871f30 0x5574c08f7fe0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d682e2d6fbf5387 server_cookie=f3d602b4ff433392 in_seq=0 out_seq=0 2024-09-19T04:32:00.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe4ea000640 10 client.27377 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:00.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe4ea000640 1 -- 172.21.15.107:0/2791665510 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5574c07d4e60 con 0x5574c0871f30 2024-09-19T04:32:00.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe4ea000640 10 client.27377 handle_command_reply: tid=0 2024-09-19T04:32:00.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 client.27377 shutdown 2024-09-19T04:32:00.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2791665510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 msgr2=0x5574c0879f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:00.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0879f70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80099f0 tx=0x7fe4f80046c0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2791665510 shutdown_connections 2024-09-19T04:32:00.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5574c0871f30 0x5574c08f7fe0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5574c0867670 0x5574c0876f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574c0866d50 0x5574c087a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 --2- 172.21.15.107:0/2791665510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574c08632e0 0x5574c0879f70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe4ea000640 0 client.27377 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:00.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2791665510 >> 172.21.15.107:0/2791665510 conn(0x5574c086fb10 msgr2=0x5574c086dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:00.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2791665510 shutdown_connections 2024-09-19T04:32:00.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 1 -- 172.21.15.107:0/2791665510 wait complete. 2024-09-19T04:32:00.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.813+0000 7fe50e1c8740 20 client.27377 trim_cache size 0 max 16384 2024-09-19T04:32:00.821 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:32:00.821 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:32:00.821 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:32:00.821 INFO:teuthology.orchestra.run.smithi107.stdout: "0f633d4d-8e30-48ea-a9b7-c0a272861a80": { 2024-09-19T04:32:00.821 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:32:00.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "0f633d4d-8e30-48ea-a9b7-c0a272861a80", 2024-09-19T04:32:00.821 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:32:00.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:32:00.822 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:32:00.822 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:32:00.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 -- 172.21.15.107:0/1643387087 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe4ec0a33b0 msgr2=0x7fe4ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:00.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe4ec0a33b0 0x7fe4ec0a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5574c0864140 tx=0x7fe4f8005f90 comp rx=0 tx=0).stop 2024-09-19T04:32:00.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 -- 172.21.15.107:0/1643387087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5041079c0 msgr2=0x7fe504074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:00.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5041079c0 0x7fe504074f70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00eb10 tx=0x7fe4fc00c2d0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 -- 172.21.15.107:0/1643387087 shutdown_connections 2024-09-19T04:32:00.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fe4ec0a33b0 0x7fe4ec0a5870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5041079c0 0x7fe504074f70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe504102360 0x7fe504074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 --2- 172.21.15.107:0/1643387087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041019c0 0x7fe5040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:00.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 -- 172.21.15.107:0/1643387087 >> 172.21.15.107:0/1643387087 conn(0x7fe5040fd740 msgr2=0x7fe5040fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:00.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.814+0000 7fe50a000640 1 -- 172.21.15.107:0/1643387087 shutdown_connections 2024-09-19T04:32:00.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:00.815+0000 7fe50a000640 1 -- 172.21.15.107:0/1643387087 wait complete. 2024-09-19T04:32:01.245 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:00 smithi107 ceph-mon[30659]: pgmap v363: 97 pgs: 97 active+clean; 327 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-19T04:32:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:00 smithi115 ceph-mon[35477]: pgmap v363: 97 pgs: 97 active+clean; 327 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-19T04:32:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:00 smithi145 ceph-mon[35096]: pgmap v363: 97 pgs: 97 active+clean; 327 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-19T04:32:01.725 INFO:tasks.workunit.client.0.smithi107.stdout: 1 398744 2.92 MB/sec execute 499 sec latency 690.854 ms 2024-09-19T04:32:01.732 INFO:tasks.cephfs.filesystem:scrub status for tag:0f633d4d-8e30-48ea-a9b7-c0a272861a80 - {'path': '/', 'tag': '0f633d4d-8e30-48ea-a9b7-c0a272861a80', 'options': 'recursive,force'} 2024-09-19T04:32:02.725 INFO:tasks.workunit.client.0.smithi107.stdout: 1 399001 2.92 MB/sec execute 500 sec latency 794.573 ms 2024-09-19T04:32:03.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:02 smithi115 ceph-mon[35477]: pgmap v364: 97 pgs: 97 active+clean; 332 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-09-19T04:32:03.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:02 smithi145 ceph-mon[35096]: pgmap v364: 97 pgs: 97 active+clean; 332 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-09-19T04:32:03.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:02 smithi107 ceph-mon[30659]: pgmap v364: 97 pgs: 97 active+clean; 332 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-09-19T04:32:03.725 INFO:tasks.workunit.client.0.smithi107.stdout: 1 399001 2.91 MB/sec execute 501 sec latency 1794.690 ms 2024-09-19T04:32:04.725 INFO:tasks.workunit.client.0.smithi107.stdout: 1 399911 2.91 MB/sec execute 502 sec latency 2299.132 ms 2024-09-19T04:32:05.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:04 smithi115 ceph-mon[35477]: pgmap v365: 97 pgs: 97 active+clean; 332 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-09-19T04:32:05.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:04 smithi145 ceph-mon[35096]: pgmap v365: 97 pgs: 97 active+clean; 332 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-09-19T04:32:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:04 smithi107 ceph-mon[30659]: pgmap v365: 97 pgs: 97 active+clean; 332 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-09-19T04:32:05.725 INFO:tasks.workunit.client.0.smithi107.stdout: 1 401300 2.93 MB/sec execute 503 sec latency 13.612 ms 2024-09-19T04:32:06.725 INFO:tasks.workunit.client.0.smithi107.stdout: 1 402480 2.93 MB/sec execute 504 sec latency 18.116 ms 2024-09-19T04:32:07.268 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:06 smithi107 ceph-mon[30659]: pgmap v366: 97 pgs: 97 active+clean; 335 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 197 op/s 2024-09-19T04:32:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:06 smithi115 ceph-mon[35477]: pgmap v366: 97 pgs: 97 active+clean; 335 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 197 op/s 2024-09-19T04:32:07.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:06 smithi145 ceph-mon[35096]: pgmap v366: 97 pgs: 97 active+clean; 335 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 197 op/s 2024-09-19T04:32:07.725 INFO:tasks.workunit.client.0.smithi107.stdout: 1 402551 2.93 MB/sec execute 505 sec latency 936.336 ms 2024-09-19T04:32:08.725 INFO:tasks.workunit.client.0.smithi107.stdout: 1 402551 2.92 MB/sec execute 506 sec latency 1936.443 ms 2024-09-19T04:32:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:08 smithi115 ceph-mon[35477]: pgmap v367: 97 pgs: 97 active+clean; 335 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-09-19T04:32:08.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:08 smithi115 ceph-mon[35477]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:32:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:08 smithi145 ceph-mon[35096]: pgmap v367: 97 pgs: 97 active+clean; 335 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-09-19T04:32:08.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:08 smithi145 ceph-mon[35096]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:32:08.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:08 smithi107 ceph-mon[30659]: pgmap v367: 97 pgs: 97 active+clean; 335 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-09-19T04:32:08.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:08 smithi107 ceph-mon[30659]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:32:09.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 403193 2.92 MB/sec execute 507 sec latency 2438.838 ms 2024-09-19T04:32:10.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 404722 2.93 MB/sec execute 508 sec latency 12.399 ms 2024-09-19T04:32:10.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:10 smithi115 ceph-mon[35477]: pgmap v369: 97 pgs: 97 active+clean; 335 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-09-19T04:32:10.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:10 smithi145 ceph-mon[35096]: pgmap v369: 97 pgs: 97 active+clean; 335 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-09-19T04:32:10.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:10 smithi107 ceph-mon[30659]: pgmap v369: 97 pgs: 97 active+clean; 335 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-09-19T04:32:11.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 405895 2.94 MB/sec execute 509 sec latency 16.567 ms 2024-09-19T04:32:12.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 406092 2.93 MB/sec execute 510 sec latency 760.162 ms 2024-09-19T04:32:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:12 smithi115 ceph-mon[35477]: pgmap v370: 97 pgs: 97 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-19T04:32:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:32:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:32:12.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:32:12.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:32:12.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:12 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:32:12.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:12 smithi107 ceph-mon[30659]: pgmap v370: 97 pgs: 97 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-19T04:32:12.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:32:12.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:32:12.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:32:12.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:32:12.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:12 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:32:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:12 smithi145 ceph-mon[35096]: pgmap v370: 97 pgs: 97 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-19T04:32:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:32:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:32:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:32:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:32:12.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:12 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:32:13.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 406092 2.93 MB/sec execute 511 sec latency 1760.275 ms 2024-09-19T04:32:14.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 407160 2.93 MB/sec execute 512 sec latency 2264.539 ms 2024-09-19T04:32:14.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:14 smithi115 ceph-mon[35477]: pgmap v371: 97 pgs: 97 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-19T04:32:14.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:14 smithi145 ceph-mon[35096]: pgmap v371: 97 pgs: 97 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-19T04:32:14.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:14 smithi107 ceph-mon[30659]: pgmap v371: 97 pgs: 97 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-19T04:32:15.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 408500 2.94 MB/sec execute 513 sec latency 19.800 ms 2024-09-19T04:32:16.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 409641 2.95 MB/sec execute 514 sec latency 38.885 ms 2024-09-19T04:32:16.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:16 smithi115 ceph-mon[35477]: pgmap v372: 97 pgs: 97 active+clean; 341 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-19T04:32:16.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:16 smithi115 ceph-mon[35477]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:32:16.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:16 smithi145 ceph-mon[35096]: pgmap v372: 97 pgs: 97 active+clean; 341 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-19T04:32:16.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:16 smithi145 ceph-mon[35096]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:32:16.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:16 smithi107 ceph-mon[30659]: pgmap v372: 97 pgs: 97 active+clean; 341 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-19T04:32:16.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:16 smithi107 ceph-mon[30659]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:32:17.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 409641 2.94 MB/sec execute 515 sec latency 1038.928 ms 2024-09-19T04:32:18.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 409641 2.94 MB/sec execute 516 sec latency 2038.968 ms 2024-09-19T04:32:18.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:18 smithi115 ceph-mon[35477]: pgmap v374: 97 pgs: 97 active+clean; 341 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 148 op/s 2024-09-19T04:32:18.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:18 smithi145 ceph-mon[35096]: pgmap v374: 97 pgs: 97 active+clean; 341 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 148 op/s 2024-09-19T04:32:18.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:18 smithi107 ceph-mon[30659]: pgmap v374: 97 pgs: 97 active+clean; 341 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 148 op/s 2024-09-19T04:32:19.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 411304 2.95 MB/sec execute 517 sec latency 2274.831 ms 2024-09-19T04:32:20.726 INFO:tasks.workunit.client.0.smithi107.stdout: 1 412405 2.95 MB/sec execute 518 sec latency 16.686 ms 2024-09-19T04:32:20.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:20 smithi115 ceph-mon[35477]: pgmap v375: 97 pgs: 97 active+clean; 341 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-19T04:32:20.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:20 smithi145 ceph-mon[35096]: pgmap v375: 97 pgs: 97 active+clean; 341 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-19T04:32:20.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:20 smithi107 ceph-mon[30659]: pgmap v375: 97 pgs: 97 active+clean; 341 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-19T04:32:21.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 413183 2.96 MB/sec execute 519 sec latency 341.191 ms 2024-09-19T04:32:22.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 413183 2.95 MB/sec execute 520 sec latency 1341.257 ms 2024-09-19T04:32:22.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:22 smithi145 ceph-mon[35096]: pgmap v376: 97 pgs: 97 active+clean; 348 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-09-19T04:32:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:22 smithi107 ceph-mon[30659]: pgmap v376: 97 pgs: 97 active+clean; 348 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-09-19T04:32:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:22 smithi115 ceph-mon[35477]: pgmap v376: 97 pgs: 97 active+clean; 348 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-09-19T04:32:23.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 413183 2.95 MB/sec execute 521 sec latency 2341.388 ms 2024-09-19T04:32:24.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 414109 2.95 MB/sec execute 522 sec latency 2842.174 ms 2024-09-19T04:32:24.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:24 smithi145 ceph-mon[35096]: pgmap v377: 97 pgs: 97 active+clean; 348 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-09-19T04:32:24.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:24 smithi107 ceph-mon[30659]: pgmap v377: 97 pgs: 97 active+clean; 348 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-09-19T04:32:24.998 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:24 smithi115 ceph-mon[35477]: pgmap v377: 97 pgs: 97 active+clean; 348 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-09-19T04:32:25.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 415545 2.96 MB/sec execute 523 sec latency 11.998 ms 2024-09-19T04:32:26.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 416732 2.97 MB/sec execute 524 sec latency 16.699 ms 2024-09-19T04:32:26.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:26 smithi145 ceph-mon[35096]: pgmap v378: 97 pgs: 97 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-09-19T04:32:26.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:26 smithi107 ceph-mon[30659]: pgmap v378: 97 pgs: 97 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-09-19T04:32:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:26 smithi115 ceph-mon[35477]: pgmap v378: 97 pgs: 97 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-09-19T04:32:27.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 416732 2.96 MB/sec execute 525 sec latency 1003.883 ms 2024-09-19T04:32:28.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 416732 2.95 MB/sec execute 526 sec latency 2003.998 ms 2024-09-19T04:32:28.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:28 smithi145 ceph-mon[35096]: pgmap v379: 97 pgs: 97 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-09-19T04:32:28.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:28 smithi107 ceph-mon[30659]: pgmap v379: 97 pgs: 97 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-09-19T04:32:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:28 smithi115 ceph-mon[35477]: pgmap v379: 97 pgs: 97 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-09-19T04:32:29.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 417800 2.96 MB/sec execute 527 sec latency 2505.286 ms 2024-09-19T04:32:30.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 419119 2.97 MB/sec execute 528 sec latency 10.866 ms 2024-09-19T04:32:30.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:30 smithi145 ceph-mon[35096]: pgmap v380: 97 pgs: 97 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-09-19T04:32:30.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:30 smithi107 ceph-mon[30659]: pgmap v380: 97 pgs: 97 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-09-19T04:32:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:30 smithi115 ceph-mon[35477]: pgmap v380: 97 pgs: 97 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-09-19T04:32:31.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 420275 2.97 MB/sec execute 529 sec latency 16.782 ms 2024-09-19T04:32:31.733 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:32:32.105 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:32.728 INFO:tasks.workunit.client.0.smithi107.stdout: 1 420275 2.97 MB/sec execute 530 sec latency 1001.516 ms 2024-09-19T04:32:32.801 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:32 smithi107 ceph-mon[30659]: pgmap v381: 97 pgs: 97 active+clean; 356 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 177 op/s 2024-09-19T04:32:32.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.798+0000 7f866b200640 1 -- 172.21.15.107:0/344144581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664102790 msgr2=0x7f86641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:32.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.798+0000 7f866b200640 1 --2- 172.21.15.107:0/344144581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664102790 0x7f86641067e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8650009900 tx=0x7f865002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.799+0000 7f866b200640 1 -- 172.21.15.107:0/344144581 shutdown_connections 2024-09-19T04:32:32.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.799+0000 7f866b200640 1 --2- 172.21.15.107:0/344144581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8664106f10 0x7f866410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.799+0000 7f866b200640 1 --2- 172.21.15.107:0/344144581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664102790 0x7f86641067e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.799+0000 7f866b200640 1 --2- 172.21.15.107:0/344144581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8664101de0 0x7f86641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.799+0000 7f866b200640 1 -- 172.21.15.107:0/344144581 >> 172.21.15.107:0/344144581 conn(0x7f86640fd8c0 msgr2=0x7f86640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:32.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.799+0000 7f866b200640 1 -- 172.21.15.107:0/344144581 shutdown_connections 2024-09-19T04:32:32.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.799+0000 7f866b200640 1 -- 172.21.15.107:0/344144581 wait complete. 2024-09-19T04:32:32.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.800+0000 7f866b200640 1 Processor -- start 2024-09-19T04:32:32.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.800+0000 7f866b200640 1 -- start start 2024-09-19T04:32:32.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.800+0000 7f866b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664101de0 0x7f866419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.800+0000 7f866b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8664102790 0x7f866419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.800+0000 7f866b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8664106f10 0x7f866419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.800+0000 7f866b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8664075320 con 0x7f8664101de0 2024-09-19T04:32:32.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.800+0000 7f866b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86640751a0 con 0x7f8664102790 2024-09-19T04:32:32.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.800+0000 7f866b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f86640754a0 con 0x7f8664106f10 2024-09-19T04:32:32.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.800+0000 7f8663e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664101de0 0x7f866419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.801+0000 7f8663e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664101de0 0x7f866419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54600/0 (socket says 172.21.15.107:54600) 2024-09-19T04:32:32.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.801+0000 7f8663e00640 1 -- 172.21.15.107:0/1921640019 learned_addr learned my addr 172.21.15.107:0/1921640019 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:32.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.801+0000 7f8663400640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8664102790 0x7f866419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.801+0000 7f8669200640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8664106f10 0x7f866419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.801+0000 7f8663e00640 1 -- 172.21.15.107:0/1921640019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8664106f10 msgr2=0x7f866419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:32.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.801+0000 7f8663e00640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8664106f10 0x7f866419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.801+0000 7f8663e00640 1 -- 172.21.15.107:0/1921640019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8664102790 msgr2=0x7f866419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:32.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.801+0000 7f8663e00640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8664102790 0x7f866419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.801+0000 7f8663e00640 1 -- 172.21.15.107:0/1921640019 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866419f600 con 0x7f8664101de0 2024-09-19T04:32:32.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.801+0000 7f8663e00640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664101de0 0x7f866419a650 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f865400ea80 tx=0x7f865400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.802+0000 7f8660c00640 1 -- 172.21.15.107:0/1921640019 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865400ce40 con 0x7f8664101de0 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.802+0000 7f8660c00640 1 -- 172.21.15.107:0/1921640019 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8654005020 con 0x7f8664101de0 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.802+0000 7f8660c00640 1 -- 172.21.15.107:0/1921640019 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654011760 con 0x7f8664101de0 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.802+0000 7f866b200640 1 -- 172.21.15.107:0/1921640019 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866419f8f0 con 0x7f8664101de0 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.802+0000 7f866b200640 1 -- 172.21.15.107:0/1921640019 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86641a71d0 con 0x7f8664101de0 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.806+0000 7f866de2e740 1 Processor -- start 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.806+0000 7f866de2e740 1 -- start start 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.806+0000 7f8660c00640 1 -- 172.21.15.107:0/1921640019 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f86540051d0 con 0x7f8664101de0 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.806+0000 7f866de2e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d42fff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.806+0000 7f866de2e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 0x5650d430a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.806+0000 7f866de2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 0x5650d430d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.806+0000 7f866de2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5650d42663e0 con 0x5650d430ad70 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.806+0000 7f866de2e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5650d420dd80 con 0x5650d43004b0 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.806+0000 7f866de2e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5650d42623a0 con 0x5650d42ffb90 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8669200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d42fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8663e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 0x5650d430a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8669200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d42fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37404/0 (socket says 172.21.15.107:37404) 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8669200640 1 -- 172.21.15.107:0/1943825184 learned_addr learned my addr 172.21.15.107:0/1943825184 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8663400640 1 --2- 172.21.15.107:0/1943825184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 0x5650d430d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8669200640 1 -- 172.21.15.107:0/1943825184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 msgr2=0x5650d430a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8669200640 1 --2- 172.21.15.107:0/1943825184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 0x5650d430a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8669200640 1 -- 172.21.15.107:0/1943825184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 msgr2=0x5650d430d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8669200640 1 --2- 172.21.15.107:0/1943825184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 0x5650d430d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8669200640 1 -- 172.21.15.107:0/1943825184 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650d422cae0 con 0x5650d42ffb90 2024-09-19T04:32:32.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.807+0000 7f8669200640 1 --2- 172.21.15.107:0/1943825184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d42fff70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f865c009c20 tx=0x7f865c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca367ef056c092c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:32.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f8659600640 1 -- 172.21.15.107:0/1943825184 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c01e070 con 0x5650d42ffb90 2024-09-19T04:32:32.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f8659600640 1 -- 172.21.15.107:0/1943825184 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f865c019b10 con 0x5650d42ffb90 2024-09-19T04:32:32.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f8660c00640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f86480a3430 0x7f86480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f8659600640 1 -- 172.21.15.107:0/1943825184 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c00eae0 con 0x5650d42ffb90 2024-09-19T04:32:32.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f8660c00640 1 -- 172.21.15.107:0/1921640019 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f86540d0250 con 0x7f8664101de0 2024-09-19T04:32:32.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f8663e00640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f86480a3430 0x7f86480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f866de2e740 1 -- 172.21.15.107:0/1943825184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 msgr2=0x5650d42fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:32.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f866de2e740 1 --2- 172.21.15.107:0/1943825184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d42fff70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f865c009c20 tx=0x7f865c011810 comp rx=0 tx=0).stop 2024-09-19T04:32:32.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f866de2e740 1 -- 172.21.15.107:0/1943825184 shutdown_connections 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f866de2e740 1 --2- 172.21.15.107:0/1943825184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 0x5650d430d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f866de2e740 1 --2- 172.21.15.107:0/1943825184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 0x5650d430a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.808+0000 7f866de2e740 1 --2- 172.21.15.107:0/1943825184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d42fff70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.809+0000 7f866de2e740 1 -- 172.21.15.107:0/1943825184 >> 172.21.15.107:0/1943825184 conn(0x5650d42fc120 msgr2=0x5650d42fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.809+0000 7f8663e00640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f86480a3430 0x7f86480a58f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f86540cda70 tx=0x7f86540d3da0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.809+0000 7f866de2e740 1 -- 172.21.15.107:0/1943825184 shutdown_connections 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.809+0000 7f866de2e740 1 -- 172.21.15.107:0/1943825184 wait complete. 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.809+0000 7f866de2e740 1 -- start start 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.809+0000 7f866de2e740 1 Processor -- start 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f866de2e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d4307cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f866de2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 0x5650d4308230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f866de2e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d430ad70 0x5650d4308770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f866de2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5650d430d9e0 con 0x5650d43004b0 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f866de2e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5650d42210f0 con 0x5650d430ad70 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f866de2e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5650d430db60 con 0x5650d42ffb90 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f866de2e740 10 client.0 _ll_register_callbacks cb 0x5650d42129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f8669200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d4307cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f8663400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 0x5650d4308230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f8669200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d4307cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37406/0 (socket says 172.21.15.107:37406) 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f8663400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 0x5650d4308230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54626/0 (socket says 172.21.15.107:54626) 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f8669200640 1 -- 172.21.15.107:0/2390277965 learned_addr learned my addr 172.21.15.107:0/2390277965 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:32.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.810+0000 7f8669200640 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d430ad70 0x5650d4308770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f8669200640 1 -- 172.21.15.107:0/2390277965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d430ad70 msgr2=0x5650d4308770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f8669200640 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d430ad70 0x5650d4308770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f8669200640 1 -- 172.21.15.107:0/2390277965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 msgr2=0x5650d4308230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f8669200640 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 0x5650d4308230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f8669200640 1 -- 172.21.15.107:0/2390277965 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650d3fd73c0 con 0x5650d42ffb90 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f8663400640 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 0x5650d4308230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f8669200640 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d4307cf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f865c00e5e0 tx=0x7f865c009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3f20ac43f8c879 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f864e000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f864e000640 1 -- 172.21.15.107:0/2390277965 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c01e070 con 0x5650d42ffb90 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f866de2e740 1 -- 172.21.15.107:0/2390277965 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865c019b10 con 0x5650d42ffb90 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f864e000640 1 -- 172.21.15.107:0/2390277965 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f865c005460 con 0x5650d42ffb90 2024-09-19T04:32:32.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f864e000640 1 -- 172.21.15.107:0/2390277965 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f865c004030 con 0x5650d42ffb90 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f866de2e740 10 client.27334 fetch_fsmap learned FSMap version 48 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.811+0000 7f866de2e740 1 -- 172.21.15.107:0/2390277965 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5650d4319710 con 0x5650d42ffb90 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.812+0000 7f864e000640 1 -- 172.21.15.107:0/2390277965 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f865c019b10 con 0x5650d42ffb90 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.812+0000 7f864e000640 1 -- 172.21.15.107:0/2390277965 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f865c02f030 con 0x5650d42ffb90 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.812+0000 7f866de2e740 10 client.27334 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.812+0000 7f866de2e740 10 client.27334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.812+0000 7f866de2e740 20 client.27334 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.812+0000 7f866de2e740 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5650d431aac0 0x5650d431ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.812+0000 7f866de2e740 4 client.27334 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.812+0000 7f866de2e740 1 -- 172.21.15.107:0/2390277965 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650d4273950 con 0x5650d431aac0 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.813+0000 7f8663400640 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5650d431aac0 0x5650d431ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.813+0000 7f8663400640 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5650d431aac0 0x5650d431ceb0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=324af11c630bfea0 server_cookie=55cd855df8a905c8 in_seq=0 out_seq=0 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.813+0000 7f864e000640 10 client.27334 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:32.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f864e000640 1 -- 172.21.15.107:0/2390277965 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650d4273950 con 0x5650d431aac0 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f864e000640 10 client.27334 handle_command_reply: tid=0 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 client.27334 shutdown 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 -- 172.21.15.107:0/2390277965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 msgr2=0x5650d4307cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d4307cf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f865c00e5e0 tx=0x7f865c009680 comp rx=0 tx=0).stop 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 -- 172.21.15.107:0/2390277965 shutdown_connections 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5650d431aac0 0x5650d431ceb0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d430ad70 0x5650d4308770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 0x5650d4308230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f864e000640 0 client.27334 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 --2- 172.21.15.107:0/2390277965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d4307cf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 -- 172.21.15.107:0/2390277965 >> 172.21.15.107:0/2390277965 conn(0x5650d42fc120 msgr2=0x5650d4301e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:32.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 -- 172.21.15.107:0/2390277965 shutdown_connections 2024-09-19T04:32:32.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 1 -- 172.21.15.107:0/2390277965 wait complete. 2024-09-19T04:32:32.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.814+0000 7f866de2e740 20 client.27334 trim_cache size 0 max 16384 2024-09-19T04:32:32.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.832+0000 7f866de2e740 1 Processor -- start 2024-09-19T04:32:32.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.832+0000 7f866de2e740 1 -- start start 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f866de2e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d42ffb90 0x5650d43025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f866de2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 0x5650d4309bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f866de2e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d430ad70 0x5650d43225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f866de2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f86641afa40 con 0x5650d43004b0 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f866de2e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86641b1580 con 0x5650d42ffb90 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f866de2e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f86641b0110 con 0x5650d430ad70 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f8669200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d42ffb90 0x5650d43025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f8669200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d42ffb90 0x5650d43025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38442/0 (socket says 172.21.15.107:38442) 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f8669200640 1 -- 172.21.15.107:0/2612110344 learned_addr learned my addr 172.21.15.107:0/2612110344 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f8663400640 1 --2- 172.21.15.107:0/2612110344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 0x5650d4309bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f8669200640 1 --2- 172.21.15.107:0/2612110344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d430ad70 0x5650d43225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f8669200640 1 -- 172.21.15.107:0/2612110344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d430ad70 msgr2=0x5650d43225f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f8669200640 1 --2- 172.21.15.107:0/2612110344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d430ad70 0x5650d43225f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f8669200640 1 -- 172.21.15.107:0/2612110344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 msgr2=0x5650d4309bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f8669200640 1 --2- 172.21.15.107:0/2612110344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 0x5650d4309bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.833+0000 7f8669200640 1 -- 172.21.15.107:0/2612110344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650d43073c0 con 0x5650d42ffb90 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.834+0000 7f8669200640 1 --2- 172.21.15.107:0/2612110344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d42ffb90 0x5650d43025f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f865c009c20 tx=0x7f865c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84d5cf514ee6b436 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.834+0000 7f8658c00640 1 -- 172.21.15.107:0/2612110344 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c01e070 con 0x5650d42ffb90 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.834+0000 7f8658c00640 1 -- 172.21.15.107:0/2612110344 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f865c011d10 con 0x5650d42ffb90 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.834+0000 7f8658c00640 1 -- 172.21.15.107:0/2612110344 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c00e610 con 0x5650d42ffb90 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 -- 172.21.15.107:0/2612110344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d42ffb90 msgr2=0x5650d43025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 --2- 172.21.15.107:0/2612110344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d42ffb90 0x5650d43025f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f865c009c20 tx=0x7f865c005970 comp rx=0 tx=0).stop 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 -- 172.21.15.107:0/2612110344 shutdown_connections 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 --2- 172.21.15.107:0/2612110344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d430ad70 0x5650d43225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 --2- 172.21.15.107:0/2612110344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d43004b0 0x5650d4309bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 --2- 172.21.15.107:0/2612110344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d42ffb90 0x5650d43025f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 -- 172.21.15.107:0/2612110344 >> 172.21.15.107:0/2612110344 conn(0x5650d42fc120 msgr2=0x5650d4306cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:32.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 -- 172.21.15.107:0/2612110344 shutdown_connections 2024-09-19T04:32:32.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 -- 172.21.15.107:0/2612110344 wait complete. 2024-09-19T04:32:32.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 -- start start 2024-09-19T04:32:32.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.835+0000 7f866de2e740 1 Processor -- start 2024-09-19T04:32:32.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f866de2e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d4312e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f866de2e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 0x5650d43133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f866de2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 0x5650d430e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f866de2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f86641b0eb0 con 0x5650d430ad70 2024-09-19T04:32:32.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f866de2e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86641b07e0 con 0x5650d43004b0 2024-09-19T04:32:32.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f866de2e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f86641ada40 con 0x5650d42ffb90 2024-09-19T04:32:32.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f866de2e740 10 client.0 _ll_register_callbacks cb 0x7f86540d3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:32.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f8663400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 0x5650d43133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f8663400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 0x5650d43133b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38446/0 (socket says 172.21.15.107:38446) 2024-09-19T04:32:32.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f8663400640 1 -- 172.21.15.107:0/1665245881 learned_addr learned my addr 172.21.15.107:0/1665245881 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:32.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f8669200640 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 0x5650d430e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.836+0000 7f8669200640 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d4312e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f8663400640 1 -- 172.21.15.107:0/1665245881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 msgr2=0x5650d4312e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f8663400640 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d4312e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f8663400640 1 -- 172.21.15.107:0/1665245881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 msgr2=0x5650d430e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f8663400640 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 0x5650d430e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f8663400640 1 -- 172.21.15.107:0/1665245881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650d430eb10 con 0x5650d43004b0 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f8669200640 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 0x5650d430e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f8663400640 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 0x5650d43133b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8650009230 tx=0x7f8650005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccfb3ea0d1afe04a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f864e000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f864e000640 1 -- 172.21.15.107:0/1665245881 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650035040 con 0x5650d43004b0 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f864e000640 1 -- 172.21.15.107:0/1665245881 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f8650005450 con 0x5650d43004b0 2024-09-19T04:32:32.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.837+0000 7f866de2e740 1 -- 172.21.15.107:0/1665245881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865c011d10 con 0x5650d43004b0 2024-09-19T04:32:32.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f864e000640 1 -- 172.21.15.107:0/1665245881 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8650005650 con 0x5650d43004b0 2024-09-19T04:32:32.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f866de2e740 10 client.27395 fetch_fsmap learned FSMap version 48 2024-09-19T04:32:32.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f866de2e740 1 -- 172.21.15.107:0/1665245881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5650d43850b0 con 0x5650d43004b0 2024-09-19T04:32:32.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f864e000640 1 -- 172.21.15.107:0/1665245881 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f865c011d10 con 0x5650d43004b0 2024-09-19T04:32:32.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f864e000640 1 -- 172.21.15.107:0/1665245881 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f86500319b0 con 0x5650d43004b0 2024-09-19T04:32:32.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f866de2e740 10 client.27395 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:32:32.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f866de2e740 10 client.27395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:32.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f866de2e740 20 client.27395 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:32.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f866de2e740 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5650d4385860 0x5650d4387c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:32.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f866de2e740 4 client.27395 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:32:32.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.838+0000 7f866de2e740 1 -- 172.21.15.107:0/1665245881 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650d426de60 con 0x5650d4385860 2024-09-19T04:32:32.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f8669200640 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5650d4385860 0x5650d4387c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:32.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f8669200640 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5650d4385860 0x5650d4387c80 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6157ab9100247702 server_cookie=a947d215af46758b in_seq=0 out_seq=0 2024-09-19T04:32:32.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f864e000640 10 client.27395 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:32.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f864e000640 1 -- 172.21.15.107:0/1665245881 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5650d426de60 con 0x5650d4385860 2024-09-19T04:32:32.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f864e000640 10 client.27395 handle_command_reply: tid=0 2024-09-19T04:32:32.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f866de2e740 1 client.27395 shutdown 2024-09-19T04:32:32.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f866de2e740 1 -- 172.21.15.107:0/1665245881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 msgr2=0x5650d43133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:32.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f866de2e740 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 0x5650d43133b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8650009230 tx=0x7f8650005e40 comp rx=0 tx=0).stop 2024-09-19T04:32:32.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f866de2e740 1 -- 172.21.15.107:0/1665245881 shutdown_connections 2024-09-19T04:32:32.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f866de2e740 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5650d4385860 0x5650d4387c80 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f866de2e740 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650d430ad70 0x5650d430e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f866de2e740 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650d43004b0 0x5650d43133b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f866de2e740 1 --2- 172.21.15.107:0/1665245881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5650d42ffb90 0x5650d4312e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f864e000640 0 client.27395 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:32.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f866de2e740 1 -- 172.21.15.107:0/1665245881 >> 172.21.15.107:0/1665245881 conn(0x5650d42fc120 msgr2=0x5650d42fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:32.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.839+0000 7f866de2e740 1 -- 172.21.15.107:0/1665245881 shutdown_connections 2024-09-19T04:32:32.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866de2e740 1 -- 172.21.15.107:0/1665245881 wait complete. 2024-09-19T04:32:32.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866de2e740 20 client.27395 trim_cache size 0 max 16384 2024-09-19T04:32:32.859 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:32:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:32:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:32:32.859 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:32:32.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866b200640 1 -- 172.21.15.107:0/1921640019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f86480a3430 msgr2=0x7f86480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:32.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866b200640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f86480a3430 0x7f86480a58f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f86540cda70 tx=0x7f86540d3da0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866b200640 1 -- 172.21.15.107:0/1921640019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664101de0 msgr2=0x7f866419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:32.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866b200640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664101de0 0x7f866419a650 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f865400ea80 tx=0x7f865400ef40 comp rx=0 tx=0).stop 2024-09-19T04:32:32.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866b200640 1 -- 172.21.15.107:0/1921640019 shutdown_connections 2024-09-19T04:32:32.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866b200640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f86480a3430 0x7f86480a58f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866b200640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8664106f10 0x7f866419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866b200640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8664102790 0x7f866419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866b200640 1 --2- 172.21.15.107:0/1921640019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664101de0 0x7f866419a650 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:32.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.840+0000 7f866b200640 1 -- 172.21.15.107:0/1921640019 >> 172.21.15.107:0/1921640019 conn(0x7f86640fd8c0 msgr2=0x7f86640fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:32.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.841+0000 7f866b200640 1 -- 172.21.15.107:0/1921640019 shutdown_connections 2024-09-19T04:32:32.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:32.841+0000 7f866b200640 1 -- 172.21.15.107:0/1921640019 wait complete. 2024-09-19T04:32:32.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:32 smithi145 ceph-mon[35096]: pgmap v381: 97 pgs: 97 active+clean; 356 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 177 op/s 2024-09-19T04:32:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:32 smithi115 ceph-mon[35477]: pgmap v381: 97 pgs: 97 active+clean; 356 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 177 op/s 2024-09-19T04:32:33.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 420275 2.96 MB/sec execute 531 sec latency 2001.583 ms 2024-09-19T04:32:33.765 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:32:33.765 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:32:34.144 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:34.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.705+0000 7f6bc2c00640 1 -- 172.21.15.107:0/3659721393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bbc1079c0 msgr2=0x7f6bbc107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:34.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.705+0000 7f6bc2c00640 1 --2- 172.21.15.107:0/3659721393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bbc1079c0 0x7f6bbc107da0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009930 tx=0x7f6bb002a420 comp rx=0 tx=0).stop 2024-09-19T04:32:34.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.705+0000 7f6bc2c00640 1 -- 172.21.15.107:0/3659721393 shutdown_connections 2024-09-19T04:32:34.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.705+0000 7f6bc2c00640 1 --2- 172.21.15.107:0/3659721393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bbc102360 0x7f6bbc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.705+0000 7f6bc2c00640 1 --2- 172.21.15.107:0/3659721393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc1019c0 0x7f6bbc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.705+0000 7f6bc2c00640 1 --2- 172.21.15.107:0/3659721393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bbc1079c0 0x7f6bbc107da0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.705+0000 7f6bc2c00640 1 -- 172.21.15.107:0/3659721393 >> 172.21.15.107:0/3659721393 conn(0x7f6bbc0fd740 msgr2=0x7f6bbc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:34.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.705+0000 7f6bc2c00640 1 -- 172.21.15.107:0/3659721393 shutdown_connections 2024-09-19T04:32:34.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.705+0000 7f6bc2c00640 1 -- 172.21.15.107:0/3659721393 wait complete. 2024-09-19T04:32:34.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.706+0000 7f6bc2c00640 1 Processor -- start 2024-09-19T04:32:34.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc2c00640 1 -- start start 2024-09-19T04:32:34.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc2c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc1019c0 0x7f6bbc1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc2c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bbc102360 0x7f6bbc198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc2c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bbc1079c0 0x7f6bbc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc2c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc111d80 con 0x7f6bbc1079c0 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc2c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc111c00 con 0x7f6bbc1019c0 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc2c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc111f00 con 0x7f6bbc102360 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc0e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bbc102360 0x7f6bbc198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc0e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bbc102360 0x7f6bbc198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37452/0 (socket says 172.21.15.107:37452) 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc0e00640 1 -- 172.21.15.107:0/551856580 learned_addr learned my addr 172.21.15.107:0/551856580 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.707+0000 7f6bc1800640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc1019c0 0x7f6bbc1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.708+0000 7f6bc2200640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bbc1079c0 0x7f6bbc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.708+0000 7f6bc0e00640 1 -- 172.21.15.107:0/551856580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc1019c0 msgr2=0x7f6bbc1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.708+0000 7f6bc0e00640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc1019c0 0x7f6bbc1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.708+0000 7f6bc0e00640 1 -- 172.21.15.107:0/551856580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bbc1079c0 msgr2=0x7f6bbc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:34.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.708+0000 7f6bc0e00640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bbc1079c0 0x7f6bbc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.708+0000 7f6bc0e00640 1 -- 172.21.15.107:0/551856580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc19d270 con 0x7f6bbc102360 2024-09-19T04:32:34.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.708+0000 7f6bc0e00640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bbc102360 0x7f6bbc198800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800e9f0 tx=0x7f6ba800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:34.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.708+0000 7f6bba000640 1 -- 172.21.15.107:0/551856580 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800cd80 con 0x7f6bbc102360 2024-09-19T04:32:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.709+0000 7f6bc2c00640 1 -- 172.21.15.107:0/551856580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc19d560 con 0x7f6bbc102360 2024-09-19T04:32:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.709+0000 7f6bba000640 1 -- 172.21.15.107:0/551856580 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba8010910 con 0x7f6bbc102360 2024-09-19T04:32:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.709+0000 7f6bba000640 1 -- 172.21.15.107:0/551856580 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80188f0 con 0x7f6bbc102360 2024-09-19T04:32:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.709+0000 7f6bc2c00640 1 -- 172.21.15.107:0/551856580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc1a4e40 con 0x7f6bbc102360 2024-09-19T04:32:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.710+0000 7f6bc2c00640 1 -- 172.21.15.107:0/551856580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bbc101e20 con 0x7f6bbc102360 2024-09-19T04:32:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.713+0000 7f6bba000640 1 -- 172.21.15.107:0/551856580 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f6ba8010ac0 con 0x7f6bbc102360 2024-09-19T04:32:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.716+0000 7f6bba000640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6ba40a3430 0x7f6ba40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.716+0000 7f6bba000640 1 -- 172.21.15.107:0/551856580 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f6ba80d0150 con 0x7f6bbc102360 2024-09-19T04:32:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.716+0000 7f6bc1800640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6ba40a3430 0x7f6ba40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.717+0000 7f6bc1800640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6ba40a3430 0x7f6ba40a58f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009900 tx=0x7f6bb00058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:34.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.719+0000 7f6bba000640 1 -- 172.21.15.107:0/551856580 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ba809caf0 con 0x7f6bbc102360 2024-09-19T04:32:34.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 421301 2.97 MB/sec execute 532 sec latency 2503.183 ms 2024-09-19T04:32:34.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.962+0000 7f6bc2c00640 1 -- 172.21.15.107:0/551856580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bbc1099a0 con 0x7f6bbc102360 2024-09-19T04:32:34.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:34 smithi107 ceph-mon[30659]: pgmap v382: 97 pgs: 97 active+clean; 356 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-09-19T04:32:34.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.963+0000 7f6bba000640 1 -- 172.21.15.107:0/551856580 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11992 (secure 0 0 0) 0x7f6ba809c3f0 con 0x7f6bbc102360 2024-09-19T04:32:34.966 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:32:34.966 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":48,"btime":"2024-09-19T04:29:19:894497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:29:18.944591+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24490,24496,24500,24539,24517]},"id":1}]} 2024-09-19T04:32:34.966 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 48 2024-09-19T04:32:34.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 -- 172.21.15.107:0/551856580 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6ba40a3430 msgr2=0x7f6ba40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:34.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6ba40a3430 0x7f6ba40a58f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009900 tx=0x7f6bb00058d0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 -- 172.21.15.107:0/551856580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bbc102360 msgr2=0x7f6bbc198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:34.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bbc102360 0x7f6bbc198800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800e9f0 tx=0x7f6ba800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 -- 172.21.15.107:0/551856580 shutdown_connections 2024-09-19T04:32:34.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f6ba40a3430 0x7f6ba40a58f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bbc1079c0 0x7f6bbc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6bbc102360 0x7f6bbc198800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 --2- 172.21.15.107:0/551856580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc1019c0 0x7f6bbc1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:34.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 -- 172.21.15.107:0/551856580 >> 172.21.15.107:0/551856580 conn(0x7f6bbc0fd740 msgr2=0x7f6bbc0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:34.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 -- 172.21.15.107:0/551856580 shutdown_connections 2024-09-19T04:32:34.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:34.967+0000 7f6bc2c00640 1 -- 172.21.15.107:0/551856580 wait complete. 2024-09-19T04:32:34.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:34 smithi145 ceph-mon[35096]: pgmap v382: 97 pgs: 97 active+clean; 356 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-09-19T04:32:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:34 smithi115 ceph-mon[35477]: pgmap v382: 97 pgs: 97 active+clean; 356 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-09-19T04:32:35.721 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:32:35.727 INFO:tasks.workunit.client.0.smithi107.stdout: 1 422597 2.98 MB/sec execute 533 sec latency 14.495 ms 2024-09-19T04:32:35.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:35 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/551856580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:32:35.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:35 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/551856580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:32:36.148 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:36.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:35 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/551856580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:32:36.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.724+0000 7fc397800640 1 -- 172.21.15.107:0/27843413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390076720 msgr2=0x7fc390076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:36.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.724+0000 7fc397800640 1 --2- 172.21.15.107:0/27843413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390076720 0x7fc390076b00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc384009900 tx=0x7fc38402a410 comp rx=0 tx=0).stop 2024-09-19T04:32:36.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.724+0000 7fc397800640 1 -- 172.21.15.107:0/27843413 shutdown_connections 2024-09-19T04:32:36.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.724+0000 7fc397800640 1 --2- 172.21.15.107:0/27843413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc390112a30 0x7fc390114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.724+0000 7fc397800640 1 --2- 172.21.15.107:0/27843413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390077040 0x7fc3901124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.724+0000 7fc397800640 1 --2- 172.21.15.107:0/27843413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390076720 0x7fc390076b00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.724+0000 7fc397800640 1 -- 172.21.15.107:0/27843413 >> 172.21.15.107:0/27843413 conn(0x7fc3900ffa90 msgr2=0x7fc390101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:36.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.724+0000 7fc397800640 1 -- 172.21.15.107:0/27843413 shutdown_connections 2024-09-19T04:32:36.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.724+0000 7fc397800640 1 -- 172.21.15.107:0/27843413 wait complete. 2024-09-19T04:32:36.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.726+0000 7fc397800640 1 Processor -- start 2024-09-19T04:32:36.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.726+0000 7fc397800640 1 -- start start 2024-09-19T04:32:36.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 423824 2.98 MB/sec execute 534 sec latency 125.565 ms 2024-09-19T04:32:36.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.726+0000 7fc397800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc390076720 0x7fc3901a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.726+0000 7fc397800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390077040 0x7fc3901a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.726+0000 7fc397800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390112a30 0x7fc39019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.726+0000 7fc397800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901185c0 con 0x7fc390112a30 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.726+0000 7fc397800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc390118440 con 0x7fc390077040 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.726+0000 7fc397800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc390118740 con 0x7fc390076720 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390077040 0x7fc3901a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc394e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc390076720 0x7fc3901a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc394e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc390076720 0x7fc3901a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37460/0 (socket says 172.21.15.107:37460) 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390077040 0x7fc3901a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38472/0 (socket says 172.21.15.107:38472) 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc394e00640 1 -- 172.21.15.107:0/1203234866 learned_addr learned my addr 172.21.15.107:0/1203234866 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:36.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc395800640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390112a30 0x7fc39019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc38fe00640 1 -- 172.21.15.107:0/1203234866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc390076720 msgr2=0x7fc3901a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc38fe00640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc390076720 0x7fc3901a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc38fe00640 1 -- 172.21.15.107:0/1203234866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390112a30 msgr2=0x7fc39019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc38fe00640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390112a30 0x7fc39019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc38fe00640 1 -- 172.21.15.107:0/1203234866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39019efe0 con 0x7fc390077040 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc38fe00640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390077040 0x7fc3901a4c30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00e9f0 tx=0x7fc37c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.727+0000 7fc38d600640 1 -- 172.21.15.107:0/1203234866 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00cd80 con 0x7fc390077040 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.728+0000 7fc397800640 1 -- 172.21.15.107:0/1203234866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39019f2d0 con 0x7fc390077040 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.728+0000 7fc38d600640 1 -- 172.21.15.107:0/1203234866 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc37c0035f0 con 0x7fc390077040 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.728+0000 7fc38d600640 1 -- 172.21.15.107:0/1203234866 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c010670 con 0x7fc390077040 2024-09-19T04:32:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.728+0000 7fc397800640 1 -- 172.21.15.107:0/1203234866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3901ab5c0 con 0x7fc390077040 2024-09-19T04:32:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.731+0000 7fc39a43f740 1 Processor -- start 2024-09-19T04:32:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.731+0000 7fc38d600640 1 -- 172.21.15.107:0/1203234866 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fc37c0037a0 con 0x7fc390077040 2024-09-19T04:32:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.731+0000 7fc39a43f740 1 -- start start 2024-09-19T04:32:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.731+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.731+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0aa590 0x55799d0b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.731+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0b4e50 0x55799d0b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc39a43f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55799d0103e0 con 0x55799d0b4e50 2024-09-19T04:32:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc39a43f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55799cfb7d80 con 0x55799d0aa590 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc39a43f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55799d00c3a0 con 0x55799d0a9c70 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc395800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc395800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0aa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37470/0 (socket says 172.21.15.107:37470) 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc395800640 1 -- 172.21.15.107:0/857551114 learned_addr learned my addr 172.21.15.107:0/857551114 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc394e00640 1 --2- 172.21.15.107:0/857551114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0aa590 0x55799d0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc38fe00640 1 --2- 172.21.15.107:0/857551114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0b4e50 0x55799d0b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc395800640 1 -- 172.21.15.107:0/857551114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0aa590 msgr2=0x55799d0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc395800640 1 --2- 172.21.15.107:0/857551114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0aa590 0x55799d0b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc395800640 1 -- 172.21.15.107:0/857551114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0b4e50 msgr2=0x55799d0b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc395800640 1 --2- 172.21.15.107:0/857551114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0b4e50 0x55799d0b7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc395800640 1 -- 172.21.15.107:0/857551114 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799cfd6ae0 con 0x55799d0a9c70 2024-09-19T04:32:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.732+0000 7fc395800640 1 --2- 172.21.15.107:0/857551114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0aa050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc388009ec0 tx=0x7fc3880107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9cf0455e51649e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:36.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.733+0000 7fc38d600640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc3780a33b0 0x7fc3780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.733+0000 7fc382000640 1 -- 172.21.15.107:0/857551114 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388004d20 con 0x55799d0a9c70 2024-09-19T04:32:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.733+0000 7fc382000640 1 -- 172.21.15.107:0/857551114 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc388003e50 con 0x55799d0a9c70 2024-09-19T04:32:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.733+0000 7fc382000640 1 -- 172.21.15.107:0/857551114 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388005280 con 0x55799d0a9c70 2024-09-19T04:32:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.733+0000 7fc38d600640 1 -- 172.21.15.107:0/1203234866 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fc37c0d00f0 con 0x7fc390077040 2024-09-19T04:32:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.733+0000 7fc394e00640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc3780a33b0 0x7fc3780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.733+0000 7fc39a43f740 1 -- 172.21.15.107:0/857551114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 msgr2=0x55799d0aa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.733+0000 7fc39a43f740 1 --2- 172.21.15.107:0/857551114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0aa050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc388009ec0 tx=0x7fc3880107a0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.734+0000 7fc39a43f740 1 -- 172.21.15.107:0/857551114 shutdown_connections 2024-09-19T04:32:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.734+0000 7fc39a43f740 1 --2- 172.21.15.107:0/857551114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0b4e50 0x55799d0b7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.734+0000 7fc39a43f740 1 --2- 172.21.15.107:0/857551114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0aa590 0x55799d0b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.734+0000 7fc39a43f740 1 --2- 172.21.15.107:0/857551114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0aa050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.734+0000 7fc39a43f740 1 -- 172.21.15.107:0/857551114 >> 172.21.15.107:0/857551114 conn(0x55799d0a6200 msgr2=0x55799d0a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.734+0000 7fc39a43f740 1 -- 172.21.15.107:0/857551114 shutdown_connections 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.734+0000 7fc394e00640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc3780a33b0 0x7fc3780a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55799d0a7060 tx=0x7fc384005a20 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.734+0000 7fc39a43f740 1 -- 172.21.15.107:0/857551114 wait complete. 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.734+0000 7fc39a43f740 1 -- start start 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.734+0000 7fc39a43f740 1 Processor -- start 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0a9c70 0x55799d0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0b4e50 0x55799d0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc39a43f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55799d0b7a20 con 0x55799d0aa590 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc39a43f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55799cfcb0f0 con 0x55799d0a9c70 2024-09-19T04:32:36.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc39a43f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55799d0b7ba0 con 0x55799d0b4e50 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc39a43f740 10 client.0 _ll_register_callbacks cb 0x55799cfb2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54710/0 (socket says 172.21.15.107:54710) 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc38fe00640 1 -- 172.21.15.107:0/2178950860 learned_addr learned my addr 172.21.15.107:0/2178950860 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc394e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0a9c70 0x55799d0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.735+0000 7fc395800640 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0b4e50 0x55799d0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc38fe00640 1 -- 172.21.15.107:0/2178950860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0b4e50 msgr2=0x55799d0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc38fe00640 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0b4e50 0x55799d0b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc38fe00640 1 -- 172.21.15.107:0/2178950860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0a9c70 msgr2=0x55799d0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc38fe00640 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0a9c70 0x55799d0b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc38fe00640 1 -- 172.21.15.107:0/2178950860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799cd813c0 con 0x55799d0aa590 2024-09-19T04:32:36.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc395800640 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0b4e50 0x55799d0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc38fe00640 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0b2270 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0cd860 tx=0x7fc37c09bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a660afdb92038b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc376a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc376a00640 1 -- 172.21.15.107:0/2178950860 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0a0740 con 0x55799d0aa590 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc376a00640 1 -- 172.21.15.107:0/2178950860 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fc37c098a00 con 0x55799d0aa590 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.736+0000 7fc39a43f740 1 -- 172.21.15.107:0/2178950860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc388003e50 con 0x55799d0aa590 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc376a00640 1 -- 172.21.15.107:0/2178950860 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc37c0a5000 con 0x55799d0aa590 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc39a43f740 10 client.17364 fetch_fsmap learned FSMap version 48 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc39a43f740 1 -- 172.21.15.107:0/2178950860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55799d0c3a60 con 0x55799d0aa590 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc376a00640 1 -- 172.21.15.107:0/2178950860 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc388003e50 con 0x55799d0aa590 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc376a00640 1 -- 172.21.15.107:0/2178950860 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fc37c09b7e0 con 0x55799d0aa590 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc39a43f740 10 client.17364 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc39a43f740 10 client.17364 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:32:36.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc39a43f740 20 client.17364 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:36.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55799d0c4f20 0x55799d0c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc39a43f740 4 client.17364 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:36.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.737+0000 7fc39a43f740 1 -- 172.21.15.107:0/2178950860 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55799d01d950 con 0x55799d0c4f20 2024-09-19T04:32:36.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.738+0000 7fc394e00640 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55799d0c4f20 0x55799d0c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.738+0000 7fc394e00640 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55799d0c4f20 0x55799d0c7310 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=acb647f53385ea2b server_cookie=284d4931b2cc2972 in_seq=0 out_seq=0 2024-09-19T04:32:36.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.738+0000 7fc376a00640 10 client.17364 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:32:36.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc376a00640 1 -- 172.21.15.107:0/2178950860 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55799d01d950 con 0x55799d0c4f20 2024-09-19T04:32:36.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc376a00640 10 client.17364 handle_command_reply: tid=0 2024-09-19T04:32:36.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 client.17364 shutdown 2024-09-19T04:32:36.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 -- 172.21.15.107:0/2178950860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 msgr2=0x55799d0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0b2270 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0cd860 tx=0x7fc37c09bf90 comp rx=0 tx=0).stop 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 -- 172.21.15.107:0/2178950860 shutdown_connections 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55799d0c4f20 0x55799d0c7310 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0b4e50 0x55799d0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0b2270 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2178950860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0a9c70 0x55799d0b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 -- 172.21.15.107:0/2178950860 >> 172.21.15.107:0/2178950860 conn(0x55799d0a6200 msgr2=0x55799d0a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc376a00640 0 client.17364 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 -- 172.21.15.107:0/2178950860 shutdown_connections 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.739+0000 7fc39a43f740 1 -- 172.21.15.107:0/2178950860 wait complete. 2024-09-19T04:32:36.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.740+0000 7fc39a43f740 20 client.17364 trim_cache size 0 max 16384 2024-09-19T04:32:36.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.753+0000 7fc39a43f740 1 Processor -- start 2024-09-19T04:32:36.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.753+0000 7fc39a43f740 1 -- start start 2024-09-19T04:32:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.753+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.753+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.753+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 0x55799d0cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.753+0000 7fc39a43f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b3ed0 con 0x55799d0aa590 2024-09-19T04:32:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.753+0000 7fc39a43f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b5a10 con 0x55799d0b4e50 2024-09-19T04:32:36.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.753+0000 7fc39a43f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b45a0 con 0x55799d0a9c70 2024-09-19T04:32:36.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54720/0 (socket says 172.21.15.107:54720) 2024-09-19T04:32:36.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc38fe00640 1 -- 172.21.15.107:0/2279320934 learned_addr learned my addr 172.21.15.107:0/2279320934 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:36.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc394e00640 1 --2- 172.21.15.107:0/2279320934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc395800640 1 --2- 172.21.15.107:0/2279320934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 0x55799d0cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc394e00640 1 -- 172.21.15.107:0/2279320934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 msgr2=0x55799d0cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:36.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc394e00640 1 --2- 172.21.15.107:0/2279320934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 0x55799d0cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc394e00640 1 -- 172.21.15.107:0/2279320934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 msgr2=0x55799d0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc394e00640 1 --2- 172.21.15.107:0/2279320934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc394e00640 1 -- 172.21.15.107:0/2279320934 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799d0a72b0 con 0x55799d0a9c70 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc38fe00640 1 --2- 172.21.15.107:0/2279320934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.754+0000 7fc394e00640 1 --2- 172.21.15.107:0/2279320934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0b3ca0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc39019f9f0 tx=0x7fc38403e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9b5e0c240994b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc381600640 1 -- 172.21.15.107:0/2279320934 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384035040 con 0x55799d0a9c70 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc381600640 1 -- 172.21.15.107:0/2279320934 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc384058070 con 0x55799d0a9c70 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc381600640 1 -- 172.21.15.107:0/2279320934 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3840523b0 con 0x55799d0a9c70 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc39a43f740 1 -- 172.21.15.107:0/2279320934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 msgr2=0x55799d0b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2279320934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0b3ca0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc39019f9f0 tx=0x7fc38403e000 comp rx=0 tx=0).stop 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc39a43f740 1 -- 172.21.15.107:0/2279320934 shutdown_connections 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2279320934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 0x55799d0cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2279320934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2279320934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0b3ca0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc39a43f740 1 -- 172.21.15.107:0/2279320934 >> 172.21.15.107:0/2279320934 conn(0x55799d0a6200 msgr2=0x55799d0b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:36.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc39a43f740 1 -- 172.21.15.107:0/2279320934 shutdown_connections 2024-09-19T04:32:36.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.755+0000 7fc39a43f740 1 -- 172.21.15.107:0/2279320934 wait complete. 2024-09-19T04:32:36.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.756+0000 7fc39a43f740 1 -- start start 2024-09-19T04:32:36.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.756+0000 7fc39a43f740 1 Processor -- start 2024-09-19T04:32:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.756+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0bd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.756+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0b8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.756+0000 7fc39a43f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 0x55799d0b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.756+0000 7fc39a43f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b5340 con 0x55799d0aa590 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.756+0000 7fc39a43f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b4c70 con 0x55799d0b4e50 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.756+0000 7fc39a43f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b1ed0 con 0x55799d0a9c70 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.756+0000 7fc39a43f740 10 client.0 _ll_register_callbacks cb 0x7fc37c003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc394e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0bd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc394e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0bd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37490/0 (socket says 172.21.15.107:37490) 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc394e00640 1 -- 172.21.15.107:0/2501605519 learned_addr learned my addr 172.21.15.107:0/2501605519 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc395800640 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 0x55799d0b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc394e00640 1 -- 172.21.15.107:0/2501605519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 msgr2=0x55799d0b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc394e00640 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 0x55799d0b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc394e00640 1 -- 172.21.15.107:0/2501605519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 msgr2=0x55799d0b8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc394e00640 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0b8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc394e00640 1 -- 172.21.15.107:0/2501605519 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799d0a1770 con 0x55799d0a9c70 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc395800640 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 0x55799d0b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc394e00640 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0bd610 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc38402ab90 tx=0x7fc38402abc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ff005578e59745 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.757+0000 7fc376a00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc376a00640 1 -- 172.21.15.107:0/2501605519 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384062070 con 0x55799d0a9c70 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc39a43f740 1 -- 172.21.15.107:0/2501605519 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc384058070 con 0x55799d0a9c70 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc376a00640 1 -- 172.21.15.107:0/2501605519 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fc3840035f0 con 0x55799d0a9c70 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc376a00640 1 -- 172.21.15.107:0/2501605519 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc38406e050 con 0x55799d0a9c70 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc39a43f740 10 client.27364 fetch_fsmap learned FSMap version 48 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc39a43f740 1 -- 172.21.15.107:0/2501605519 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55799d12fa80 con 0x55799d0a9c70 2024-09-19T04:32:36.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc376a00640 1 -- 172.21.15.107:0/2501605519 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc384058070 con 0x55799d0a9c70 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc376a00640 1 -- 172.21.15.107:0/2501605519 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7fc3840528d0 con 0x55799d0a9c70 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc39a43f740 10 client.27364 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc39a43f740 10 client.27364 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc39a43f740 20 client.27364 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55799d130150 0x55799d132540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc39a43f740 4 client.27364 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.758+0000 7fc39a43f740 1 -- 172.21.15.107:0/2501605519 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55799d017e60 con 0x55799d130150 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc38fe00640 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55799d130150 0x55799d132540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc38fe00640 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55799d130150 0x55799d132540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2cca31f26a0238d7 server_cookie=783e459b9fa0b6ae in_seq=0 out_seq=0 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc376a00640 10 client.27364 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc376a00640 1 -- 172.21.15.107:0/2501605519 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55799d017e60 con 0x55799d130150 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc376a00640 10 client.27364 handle_command_reply: tid=0 2024-09-19T04:32:36.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc39a43f740 1 client.27364 shutdown 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc39a43f740 1 -- 172.21.15.107:0/2501605519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 msgr2=0x55799d0bd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0bd610 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc38402ab90 tx=0x7fc38402abc0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc39a43f740 1 -- 172.21.15.107:0/2501605519 shutdown_connections 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55799d130150 0x55799d132540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55799d0b4e50 0x55799d0b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55799d0aa590 0x55799d0b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.759+0000 7fc39a43f740 1 --2- 172.21.15.107:0/2501605519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55799d0a9c70 0x55799d0bd610 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.760+0000 7fc39a43f740 1 -- 172.21.15.107:0/2501605519 >> 172.21.15.107:0/2501605519 conn(0x55799d0a6200 msgr2=0x55799d0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.760+0000 7fc376a00640 0 client.27364 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.760+0000 7fc39a43f740 1 -- 172.21.15.107:0/2501605519 shutdown_connections 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.760+0000 7fc39a43f740 1 -- 172.21.15.107:0/2501605519 wait complete. 2024-09-19T04:32:36.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.760+0000 7fc39a43f740 20 client.27364 trim_cache size 0 max 16384 2024-09-19T04:32:36.767 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:32:36.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.760+0000 7fc397800640 1 -- 172.21.15.107:0/1203234866 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc3780a33b0 msgr2=0x7fc3780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:36.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.760+0000 7fc397800640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc3780a33b0 0x7fc3780a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55799d0a7060 tx=0x7fc384005a20 comp rx=0 tx=0).stop 2024-09-19T04:32:36.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.760+0000 7fc397800640 1 -- 172.21.15.107:0/1203234866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390077040 msgr2=0x7fc3901a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:36.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.760+0000 7fc397800640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390077040 0x7fc3901a4c30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00e9f0 tx=0x7fc37c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.761+0000 7fc397800640 1 -- 172.21.15.107:0/1203234866 shutdown_connections 2024-09-19T04:32:36.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.761+0000 7fc397800640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc3780a33b0 0x7fc3780a5870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.761+0000 7fc397800640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390112a30 0x7fc39019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.761+0000 7fc397800640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390077040 0x7fc3901a4c30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.761+0000 7fc397800640 1 --2- 172.21.15.107:0/1203234866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc390076720 0x7fc3901a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:36.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.761+0000 7fc397800640 1 -- 172.21.15.107:0/1203234866 >> 172.21.15.107:0/1203234866 conn(0x7fc3900ffa90 msgr2=0x7fc390101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:36.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.761+0000 7fc397800640 1 -- 172.21.15.107:0/1203234866 shutdown_connections 2024-09-19T04:32:36.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:36.761+0000 7fc397800640 1 -- 172.21.15.107:0/1203234866 wait complete. 2024-09-19T04:32:36.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:36 smithi145 ceph-mon[35096]: pgmap v383: 97 pgs: 97 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-09-19T04:32:36.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:36 smithi107 ceph-mon[30659]: pgmap v383: 97 pgs: 97 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-09-19T04:32:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:36 smithi115 ceph-mon[35477]: pgmap v383: 97 pgs: 97 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-09-19T04:32:37.728 INFO:tasks.workunit.client.0.smithi107.stdout: 1 423824 2.98 MB/sec execute 535 sec latency 1125.608 ms 2024-09-19T04:32:37.910 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:32:38.355 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:38.728 INFO:tasks.workunit.client.0.smithi107.stdout: 1 423824 2.97 MB/sec execute 536 sec latency 2125.646 ms 2024-09-19T04:32:38.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.941+0000 7f2229000640 1 -- 172.21.15.107:0/2103233609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2224102360 msgr2=0x7f222410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:38.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.941+0000 7f2229000640 1 --2- 172.21.15.107:0/2103233609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2224102360 0x7f222410e890 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f221c009cd0 tx=0x7f221c02a8e0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.941+0000 7f2229000640 1 -- 172.21.15.107:0/2103233609 shutdown_connections 2024-09-19T04:32:38.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.941+0000 7f2229000640 1 --2- 172.21.15.107:0/2103233609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2224102360 0x7f222410e890 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.941+0000 7f2229000640 1 --2- 172.21.15.107:0/2103233609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22241019c0 0x7f2224101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.941+0000 7f2229000640 1 --2- 172.21.15.107:0/2103233609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22241079c0 0x7f2224107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.941+0000 7f2229000640 1 -- 172.21.15.107:0/2103233609 >> 172.21.15.107:0/2103233609 conn(0x7f22240fd740 msgr2=0x7f22240ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:38.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.941+0000 7f2229000640 1 -- 172.21.15.107:0/2103233609 shutdown_connections 2024-09-19T04:32:38.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.941+0000 7f2229000640 1 -- 172.21.15.107:0/2103233609 wait complete. 2024-09-19T04:32:38.944 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:38 smithi107 ceph-mon[30659]: pgmap v384: 97 pgs: 97 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-19T04:32:38.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.943+0000 7f2229000640 1 Processor -- start 2024-09-19T04:32:38.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.943+0000 7f2229000640 1 -- start start 2024-09-19T04:32:38.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.943+0000 7f2229000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22241019c0 0x7f222419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.943+0000 7f2229000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2224102360 0x7f222419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.943+0000 7f2229000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22241079c0 0x7f222419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.943+0000 7f2229000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2224111dc0 con 0x7f22241019c0 2024-09-19T04:32:38.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.943+0000 7f2229000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2224111c40 con 0x7f2224102360 2024-09-19T04:32:38.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.943+0000 7f2229000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2224111f40 con 0x7f22241079c0 2024-09-19T04:32:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.944+0000 7f2223400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22241019c0 0x7f222419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.944+0000 7f2223400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22241019c0 0x7f222419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43124/0 (socket says 172.21.15.107:43124) 2024-09-19T04:32:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.944+0000 7f2222a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2224102360 0x7f222419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.944+0000 7f2223400640 1 -- 172.21.15.107:0/3065152376 learned_addr learned my addr 172.21.15.107:0/3065152376 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.944+0000 7f2223e00640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22241079c0 0x7f222419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.944+0000 7f2222a00640 1 -- 172.21.15.107:0/3065152376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22241079c0 msgr2=0x7f222419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.944+0000 7f2222a00640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22241079c0 0x7f222419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.944+0000 7f2222a00640 1 -- 172.21.15.107:0/3065152376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22241019c0 msgr2=0x7f222419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.944+0000 7f2222a00640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22241019c0 0x7f222419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.944+0000 7f2222a00640 1 -- 172.21.15.107:0/3065152376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222419f480 con 0x7f2224102360 2024-09-19T04:32:38.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.945+0000 7f2222a00640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2224102360 0x7f222419aa10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f221000e9f0 tx=0x7f221000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:38.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.945+0000 7f221be00640 1 -- 172.21.15.107:0/3065152376 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221000ccc0 con 0x7f2224102360 2024-09-19T04:32:38.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.945+0000 7f2229000640 1 -- 172.21.15.107:0/3065152376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222419f770 con 0x7f2224102360 2024-09-19T04:32:38.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.945+0000 7f221be00640 1 -- 172.21.15.107:0/3065152376 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2210010910 con 0x7f2224102360 2024-09-19T04:32:38.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.945+0000 7f221be00640 1 -- 172.21.15.107:0/3065152376 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210018960 con 0x7f2224102360 2024-09-19T04:32:38.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.945+0000 7f2229000640 1 -- 172.21.15.107:0/3065152376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22241a7050 con 0x7f2224102360 2024-09-19T04:32:38.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.948+0000 7f222d215740 1 Processor -- start 2024-09-19T04:32:38.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.948+0000 7f222d215740 1 -- start start 2024-09-19T04:32:38.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f222d215740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f222d215740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8c8760 0x558e9e8d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f222d215740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8d3020 0x558e9e8d5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f222d215740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558e9e82e3e0 con 0x558e9e8c8760 2024-09-19T04:32:38.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f222d215740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558e9e7d5d80 con 0x558e9e8c7e40 2024-09-19T04:32:38.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f222d215740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558e9e82a3a0 con 0x558e9e8d3020 2024-09-19T04:32:38.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f221be00640 1 -- 172.21.15.107:0/3065152376 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f2210010ac0 con 0x7f2224102360 2024-09-19T04:32:38.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f2223e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f2223e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45918/0 (socket says 172.21.15.107:45918) 2024-09-19T04:32:38.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f2223e00640 1 -- 172.21.15.107:0/376532959 learned_addr learned my addr 172.21.15.107:0/376532959 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:38.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.949+0000 7f2223400640 1 --2- 172.21.15.107:0/376532959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8c8760 0x558e9e8d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.950+0000 7f2223e00640 1 -- 172.21.15.107:0/376532959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8d3020 msgr2=0x558e9e8d5410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:32:38.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.950+0000 7f2223e00640 1 --2- 172.21.15.107:0/376532959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8d3020 0x558e9e8d5410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.950+0000 7f2223e00640 1 -- 172.21.15.107:0/376532959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8c8760 msgr2=0x558e9e8d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:38.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.950+0000 7f2223e00640 1 --2- 172.21.15.107:0/376532959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8c8760 0x558e9e8d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.950+0000 7f2223e00640 1 -- 172.21.15.107:0/376532959 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9e7f4ae0 con 0x558e9e8c7e40 2024-09-19T04:32:38.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.951+0000 7f2223e00640 1 --2- 172.21.15.107:0/376532959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8c8220 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f221c009cd0 tx=0x7f221c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f36de7b99b72c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:38.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.951+0000 7f2218c00640 1 -- 172.21.15.107:0/376532959 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c048070 con 0x558e9e8c7e40 2024-09-19T04:32:38.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.951+0000 7f2218c00640 1 -- 172.21.15.107:0/376532959 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221c036070 con 0x558e9e8c7e40 2024-09-19T04:32:38.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.951+0000 7f2218c00640 1 -- 172.21.15.107:0/376532959 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c0044e0 con 0x558e9e8c7e40 2024-09-19T04:32:38.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.952+0000 7f221be00640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f22080a3430 0x7f22080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.952+0000 7f221be00640 1 -- 172.21.15.107:0/3065152376 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f22100d00f0 con 0x7f2224102360 2024-09-19T04:32:38.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.952+0000 7f222d215740 1 -- 172.21.15.107:0/376532959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 msgr2=0x558e9e8c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:38.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.952+0000 7f222d215740 1 --2- 172.21.15.107:0/376532959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8c8220 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f221c009cd0 tx=0x7f221c009a20 comp rx=0 tx=0).stop 2024-09-19T04:32:38.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.952+0000 7f2223400640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f22080a3430 0x7f22080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.953+0000 7f222d215740 1 -- 172.21.15.107:0/376532959 shutdown_connections 2024-09-19T04:32:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.953+0000 7f222d215740 1 --2- 172.21.15.107:0/376532959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8d3020 0x558e9e8d5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.953+0000 7f222d215740 1 --2- 172.21.15.107:0/376532959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8c8760 0x558e9e8d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.953+0000 7f222d215740 1 --2- 172.21.15.107:0/376532959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8c8220 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.953+0000 7f222d215740 1 -- 172.21.15.107:0/376532959 >> 172.21.15.107:0/376532959 conn(0x558e9e8c43d0 msgr2=0x558e9e8c47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.953+0000 7f222d215740 1 -- 172.21.15.107:0/376532959 shutdown_connections 2024-09-19T04:32:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.953+0000 7f222d215740 1 -- 172.21.15.107:0/376532959 wait complete. 2024-09-19T04:32:38.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.954+0000 7f2223400640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f22080a3430 0x7f22080a58f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x558e9e8c5230 tx=0x7f221400b4a0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:38.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.954+0000 7f222d215740 1 -- start start 2024-09-19T04:32:38.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.954+0000 7f222d215740 1 Processor -- start 2024-09-19T04:32:38.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f222d215740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8c7e40 0x558e9e8cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f222d215740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f222d215740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8d3020 0x558e9e8d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f222d215740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558e9e8d5bf0 con 0x558e9e8c7e40 2024-09-19T04:32:38.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f222d215740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558e9e7e90f0 con 0x558e9e8c8760 2024-09-19T04:32:38.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f222d215740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558e9e8d5d70 con 0x558e9e8d3020 2024-09-19T04:32:38.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f222d215740 10 client.0 _ll_register_callbacks cb 0x558e9e7d0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:38.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f2222a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f2222a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45926/0 (socket says 172.21.15.107:45926) 2024-09-19T04:32:38.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f2222a00640 1 -- 172.21.15.107:0/2628025479 learned_addr learned my addr 172.21.15.107:0/2628025479 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:38.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f2223400640 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8c7e40 0x558e9e8cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.955+0000 7f2223e00640 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8d3020 0x558e9e8d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f2222a00640 1 -- 172.21.15.107:0/2628025479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8d3020 msgr2=0x558e9e8d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:38.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f2222a00640 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8d3020 0x558e9e8d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f2222a00640 1 -- 172.21.15.107:0/2628025479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8c7e40 msgr2=0x558e9e8cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:38.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f2222a00640 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8c7e40 0x558e9e8cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f2222a00640 1 -- 172.21.15.107:0/2628025479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9e59f3c0 con 0x558e9e8c8760 2024-09-19T04:32:38.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f2223e00640 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8d3020 0x558e9e8d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:38.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f2222a00640 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8d04e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f22100988a0 tx=0x7f221009bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99d722920ee31d8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:38.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f220d600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:32:38.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f220d600640 1 -- 172.21.15.107:0/2628025479 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22100cdc90 con 0x558e9e8c8760 2024-09-19T04:32:38.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f220d600640 1 -- 172.21.15.107:0/2628025479 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f221009ba90 con 0x558e9e8c8760 2024-09-19T04:32:38.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.956+0000 7f222d215740 1 -- 172.21.15.107:0/2628025479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221c036070 con 0x558e9e8c8760 2024-09-19T04:32:38.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f220d600640 1 -- 172.21.15.107:0/2628025479 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f22100a5000 con 0x558e9e8c8760 2024-09-19T04:32:38.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f222d215740 10 client.27431 fetch_fsmap learned FSMap version 48 2024-09-19T04:32:38.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f222d215740 1 -- 172.21.15.107:0/2628025479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558e9e8ea4f0 con 0x558e9e8c8760 2024-09-19T04:32:38.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f220d600640 1 -- 172.21.15.107:0/2628025479 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221c036070 con 0x558e9e8c8760 2024-09-19T04:32:38.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f220d600640 1 -- 172.21.15.107:0/2628025479 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f2210098190 con 0x558e9e8c8760 2024-09-19T04:32:38.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f222d215740 10 client.27431 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:32:38.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f222d215740 10 client.27431 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:32:38.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f222d215740 20 client.27431 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:38.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f222d215740 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x558e9e8eb9b0 0x558e9e8edda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f222d215740 4 client.27431 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:38.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.957+0000 7f222d215740 1 -- 172.21.15.107:0/2628025479 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e9e83b950 con 0x558e9e8eb9b0 2024-09-19T04:32:38.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.958+0000 7f2223400640 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x558e9e8eb9b0 0x558e9e8edda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.958+0000 7f2223400640 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x558e9e8eb9b0 0x558e9e8edda0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4603a2ef996ef625 server_cookie=965e8eeeda7c03df in_seq=0 out_seq=0 2024-09-19T04:32:38.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.958+0000 7f220d600640 10 client.27431 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:32:38.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f220d600640 1 -- 172.21.15.107:0/2628025479 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e9e83b950 con 0x558e9e8eb9b0 2024-09-19T04:32:38.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f220d600640 10 client.27431 handle_command_reply: tid=0 2024-09-19T04:32:38.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f222d215740 1 client.27431 shutdown 2024-09-19T04:32:38.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f222d215740 1 -- 172.21.15.107:0/2628025479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 msgr2=0x558e9e8d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:38.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f222d215740 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8d04e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f22100988a0 tx=0x7f221009bda0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f222d215740 1 -- 172.21.15.107:0/2628025479 shutdown_connections 2024-09-19T04:32:38.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f222d215740 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x558e9e8eb9b0 0x558e9e8edda0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f222d215740 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8d3020 0x558e9e8d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f222d215740 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8d04e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f222d215740 1 --2- 172.21.15.107:0/2628025479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8c7e40 0x558e9e8cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f220d600640 0 client.27431 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:32:38.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f222d215740 1 -- 172.21.15.107:0/2628025479 >> 172.21.15.107:0/2628025479 conn(0x558e9e8c43d0 msgr2=0x558e9e8ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:38.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.959+0000 7f222d215740 1 -- 172.21.15.107:0/2628025479 shutdown_connections 2024-09-19T04:32:38.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.960+0000 7f222d215740 1 -- 172.21.15.107:0/2628025479 wait complete. 2024-09-19T04:32:38.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.960+0000 7f222d215740 20 client.27431 trim_cache size 0 max 16384 2024-09-19T04:32:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.974+0000 7f222d215740 1 Processor -- start 2024-09-19T04:32:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.974+0000 7f222d215740 1 -- start start 2024-09-19T04:32:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.974+0000 7f222d215740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.974+0000 7f222d215740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c8760 0x558e9e8f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.974+0000 7f222d215740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 0x558e9e8f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.974+0000 7f222d215740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22241ab470 con 0x558e9e8d3020 2024-09-19T04:32:38.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.974+0000 7f222d215740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f22241acfb0 con 0x558e9e8c7e40 2024-09-19T04:32:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.974+0000 7f222d215740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f22241abb40 con 0x558e9e8c8760 2024-09-19T04:32:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2223400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2222a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c8760 0x558e9e8f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2222a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c8760 0x558e9e8f35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59438/0 (socket says 172.21.15.107:59438) 2024-09-19T04:32:38.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2222a00640 1 -- 172.21.15.107:0/2712099188 learned_addr learned my addr 172.21.15.107:0/2712099188 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:38.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2223400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45938/0 (socket says 172.21.15.107:45938) 2024-09-19T04:32:38.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2223e00640 1 --2- 172.21.15.107:0/2712099188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 0x558e9e8f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2222a00640 1 -- 172.21.15.107:0/2712099188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 msgr2=0x558e9e8cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:38.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2222a00640 1 --2- 172.21.15.107:0/2712099188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2222a00640 1 -- 172.21.15.107:0/2712099188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 msgr2=0x558e9e8f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:38.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2222a00640 1 --2- 172.21.15.107:0/2712099188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 0x558e9e8f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2222a00640 1 -- 172.21.15.107:0/2712099188 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9e8ee360 con 0x558e9e8c8760 2024-09-19T04:32:38.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2223400640 1 --2- 172.21.15.107:0/2712099188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:38.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2223e00640 1 --2- 172.21.15.107:0/2712099188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 0x558e9e8f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:38.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.975+0000 7f2222a00640 1 --2- 172.21.15.107:0/2712099188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c8760 0x558e9e8f35f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f221000e9c0 tx=0x7f221000ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb6a061f4de3c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:38.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.976+0000 7f220fe00640 1 -- 172.21.15.107:0/2712099188 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210018d00 con 0x558e9e8c8760 2024-09-19T04:32:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.976+0000 7f220fe00640 1 -- 172.21.15.107:0/2712099188 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221009c440 con 0x558e9e8c8760 2024-09-19T04:32:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.976+0000 7f220fe00640 1 -- 172.21.15.107:0/2712099188 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221009b890 con 0x558e9e8c8760 2024-09-19T04:32:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.977+0000 7f222d215740 1 -- 172.21.15.107:0/2712099188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c8760 msgr2=0x558e9e8f35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.977+0000 7f222d215740 1 --2- 172.21.15.107:0/2712099188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c8760 0x558e9e8f35f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f221000e9c0 tx=0x7f221000ced0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.977+0000 7f222d215740 1 -- 172.21.15.107:0/2712099188 shutdown_connections 2024-09-19T04:32:38.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.977+0000 7f222d215740 1 --2- 172.21.15.107:0/2712099188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 0x558e9e8f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.977+0000 7f222d215740 1 --2- 172.21.15.107:0/2712099188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c8760 0x558e9e8f35f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.977+0000 7f222d215740 1 --2- 172.21.15.107:0/2712099188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c7e40 0x558e9e8cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.977+0000 7f222d215740 1 -- 172.21.15.107:0/2712099188 >> 172.21.15.107:0/2712099188 conn(0x558e9e8c43d0 msgr2=0x558e9e8ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.977+0000 7f222d215740 1 -- 172.21.15.107:0/2712099188 shutdown_connections 2024-09-19T04:32:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.977+0000 7f222d215740 1 -- 172.21.15.107:0/2712099188 wait complete. 2024-09-19T04:32:38.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.978+0000 7f222d215740 1 -- start start 2024-09-19T04:32:38.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.978+0000 7f222d215740 1 Processor -- start 2024-09-19T04:32:38.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.978+0000 7f222d215740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c7e40 0x558e9e8e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.978+0000 7f222d215740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.978+0000 7f222d215740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 0x558e9e8cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.978+0000 7f222d215740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22241ac8e0 con 0x558e9e8d3020 2024-09-19T04:32:38.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.978+0000 7f222d215740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f22241ac210 con 0x558e9e8c8760 2024-09-19T04:32:38.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.978+0000 7f222d215740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f22241a94b0 con 0x558e9e8c7e40 2024-09-19T04:32:38.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.978+0000 7f222d215740 10 client.0 _ll_register_callbacks cb 0x7f221009c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:38.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2223400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c7e40 0x558e9e8e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2222a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2223e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 0x558e9e8cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2222a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45944/0 (socket says 172.21.15.107:45944) 2024-09-19T04:32:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2222a00640 1 -- 172.21.15.107:0/467238277 learned_addr learned my addr 172.21.15.107:0/467238277 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2223400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c7e40 0x558e9e8e3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59448/0 (socket says 172.21.15.107:59448) 2024-09-19T04:32:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2222a00640 1 -- 172.21.15.107:0/467238277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c7e40 msgr2=0x558e9e8e3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:38.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2222a00640 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c7e40 0x558e9e8e3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2222a00640 1 -- 172.21.15.107:0/467238277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 msgr2=0x558e9e8cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2222a00640 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 0x558e9e8cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2222a00640 1 -- 172.21.15.107:0/467238277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9e8c5060 con 0x558e9e8c8760 2024-09-19T04:32:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f2222a00640 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8ccb70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f221009a990 tx=0x7f221009bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bdbc4610306d274 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.979+0000 7f220d600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:32:38.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f220d600640 1 -- 172.21.15.107:0/467238277 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221009b890 con 0x558e9e8c8760 2024-09-19T04:32:38.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f220d600640 1 -- 172.21.15.107:0/467238277 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f221009ba30 con 0x558e9e8c8760 2024-09-19T04:32:38.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f222d215740 1 -- 172.21.15.107:0/467238277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221009c440 con 0x558e9e8c8760 2024-09-19T04:32:38.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f220d600640 1 -- 172.21.15.107:0/467238277 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2210010430 con 0x558e9e8c8760 2024-09-19T04:32:38.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f222d215740 10 client.27437 fetch_fsmap learned FSMap version 48 2024-09-19T04:32:38.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f222d215740 1 -- 172.21.15.107:0/467238277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558e9e956260 con 0x558e9e8c8760 2024-09-19T04:32:38.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f220d600640 1 -- 172.21.15.107:0/467238277 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221009c440 con 0x558e9e8c8760 2024-09-19T04:32:38.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f220d600640 1 -- 172.21.15.107:0/467238277 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8362+0+0 (secure 0 0 0) 0x7f22100a18d0 con 0x558e9e8c8760 2024-09-19T04:32:38.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f222d215740 10 client.27437 fetch_fsmap finished waiting for FSMap version 48 2024-09-19T04:32:38.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f222d215740 10 client.27437 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:32:38.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f222d215740 20 client.27437 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:38.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f222d215740 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x558e9e956690 0x558e9e958a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:38.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f222d215740 4 client.27437 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:38.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.980+0000 7f222d215740 1 -- 172.21.15.107:0/467238277 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e9e835e60 con 0x558e9e956690 2024-09-19T04:32:38.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.981+0000 7f2223400640 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x558e9e956690 0x558e9e958a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:38.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.981+0000 7f2223400640 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x558e9e956690 0x558e9e958a80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9a9f1e8a1edfa95 server_cookie=8abb1458557f84ee in_seq=0 out_seq=0 2024-09-19T04:32:38.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.981+0000 7f220d600640 10 client.27437 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:32:38.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f220d600640 1 -- 172.21.15.107:0/467238277 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e9e835e60 con 0x558e9e956690 2024-09-19T04:32:38.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f220d600640 10 client.27437 handle_command_reply: tid=0 2024-09-19T04:32:38.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 client.27437 shutdown 2024-09-19T04:32:38.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 -- 172.21.15.107:0/467238277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 msgr2=0x558e9e8ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:38.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8ccb70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f221009a990 tx=0x7f221009bed0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 -- 172.21.15.107:0/467238277 shutdown_connections 2024-09-19T04:32:38.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x558e9e956690 0x558e9e958a80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e9e8d3020 0x558e9e8cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9e8c8760 0x558e9e8ccb70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f220d600640 0 client.27437 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:32:38.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 --2- 172.21.15.107:0/467238277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e9e8c7e40 0x558e9e8e3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:38.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 -- 172.21.15.107:0/467238277 >> 172.21.15.107:0/467238277 conn(0x558e9e8c43d0 msgr2=0x558e9e8d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:39.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 -- 172.21.15.107:0/467238277 shutdown_connections 2024-09-19T04:32:39.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 1 -- 172.21.15.107:0/467238277 wait complete. 2024-09-19T04:32:39.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.982+0000 7f222d215740 20 client.27437 trim_cache size 0 max 16384 2024-09-19T04:32:39.000 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:32:39.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 -- 172.21.15.107:0/3065152376 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f22080a3430 msgr2=0x7f22080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f22080a3430 0x7f22080a58f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x558e9e8c5230 tx=0x7f221400b4a0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 -- 172.21.15.107:0/3065152376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2224102360 msgr2=0x7f222419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:39.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2224102360 0x7f222419aa10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f221000e9f0 tx=0x7f221000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 -- 172.21.15.107:0/3065152376 shutdown_connections 2024-09-19T04:32:39.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f22080a3430 0x7f22080a58f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22241079c0 0x7f222419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2224102360 0x7f222419aa10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 --2- 172.21.15.107:0/3065152376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22241019c0 0x7f222419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:39.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 -- 172.21.15.107:0/3065152376 >> 172.21.15.107:0/3065152376 conn(0x7f22240fd740 msgr2=0x7f222410d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:39.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 -- 172.21.15.107:0/3065152376 shutdown_connections 2024-09-19T04:32:39.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:38.983+0000 7f2229000640 1 -- 172.21.15.107:0/3065152376 wait complete. 2024-09-19T04:32:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:38 smithi115 ceph-mon[35477]: pgmap v384: 97 pgs: 97 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-19T04:32:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:38 smithi145 ceph-mon[35096]: pgmap v384: 97 pgs: 97 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-19T04:32:39.728 INFO:tasks.workunit.client.0.smithi107.stdout: 1 425480 2.98 MB/sec execute 537 sec latency 2359.609 ms 2024-09-19T04:32:39.869 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:32:40.243 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:40.728 INFO:tasks.workunit.client.0.smithi107.stdout: 1 425487 2.98 MB/sec execute 538 sec latency 995.624 ms 2024-09-19T04:32:40.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.827+0000 7f7403200640 1 -- 172.21.15.107:0/105258143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7404103e30 msgr2=0x7f7404104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:40.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.827+0000 7f7403200640 1 --2- 172.21.15.107:0/105258143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7404103e30 0x7f7404104210 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009900 tx=0x7f73f402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.827+0000 7f7403200640 1 -- 172.21.15.107:0/105258143 shutdown_connections 2024-09-19T04:32:40.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.827+0000 7f7403200640 1 --2- 172.21.15.107:0/105258143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7404108f60 0x7f7404110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.827+0000 7f7403200640 1 --2- 172.21.15.107:0/105258143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74041047e0 0x7f7404108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.827+0000 7f7403200640 1 --2- 172.21.15.107:0/105258143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7404103e30 0x7f7404104210 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.827+0000 7f7403200640 1 -- 172.21.15.107:0/105258143 >> 172.21.15.107:0/105258143 conn(0x7f74040ff8d0 msgr2=0x7f7404101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:40.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.827+0000 7f7403200640 1 -- 172.21.15.107:0/105258143 shutdown_connections 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.827+0000 7f7403200640 1 -- 172.21.15.107:0/105258143 wait complete. 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.828+0000 7f7403200640 1 Processor -- start 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.828+0000 7f7403200640 1 -- start start 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7403200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7404103e30 0x7f740419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7403200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f74041047e0 0x7f740419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7403200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7404108f60 0x7f740419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7403200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7404113fd0 con 0x7f7404103e30 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7403200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7404113e50 con 0x7f7404108f60 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7403200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7404114150 con 0x7f74041047e0 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7401e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7404103e30 0x7f740419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7401400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f74041047e0 0x7f740419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7401e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7404103e30 0x7f740419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43194/0 (socket says 172.21.15.107:43194) 2024-09-19T04:32:40.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7401e00640 1 -- 172.21.15.107:0/1023686938 learned_addr learned my addr 172.21.15.107:0/1023686938 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:40.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7402800640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7404108f60 0x7f740419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7402800640 1 -- 172.21.15.107:0/1023686938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f74041047e0 msgr2=0x7f740419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:40.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7402800640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f74041047e0 0x7f740419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7402800640 1 -- 172.21.15.107:0/1023686938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7404103e30 msgr2=0x7f740419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:40.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.829+0000 7f7402800640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7404103e30 0x7f740419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.830+0000 7f7402800640 1 -- 172.21.15.107:0/1023686938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740419f440 con 0x7f7404108f60 2024-09-19T04:32:40.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.830+0000 7f7401e00640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7404103e30 0x7f740419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:40.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.830+0000 7f7402800640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7404108f60 0x7f740419ed60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00e940 tx=0x7f73fc00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.830+0000 7f73faa00640 1 -- 172.21.15.107:0/1023686938 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc004d20 con 0x7f7404108f60 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.830+0000 7f73faa00640 1 -- 172.21.15.107:0/1023686938 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73fc005020 con 0x7f7404108f60 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.830+0000 7f7403200640 1 -- 172.21.15.107:0/1023686938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740419f730 con 0x7f7404108f60 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.830+0000 7f73faa00640 1 -- 172.21.15.107:0/1023686938 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0116b0 con 0x7f7404108f60 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.830+0000 7f7403200640 1 -- 172.21.15.107:0/1023686938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74041a7010 con 0x7f7404108f60 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.833+0000 7f740b9be740 1 Processor -- start 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.833+0000 7f73faa00640 1 -- 172.21.15.107:0/1023686938 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f73fc0054d0 con 0x7f7404108f60 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.833+0000 7f740b9be740 1 -- start start 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f740b9be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f481e40 0x558d1f482220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f740b9be740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 0x558d1f48cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f740b9be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f48d020 0x558d1f48f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f740b9be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558d1f3e83e0 con 0x558d1f481e40 2024-09-19T04:32:40.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f740b9be740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558d1f38fd80 con 0x558d1f482760 2024-09-19T04:32:40.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f740b9be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558d1f3e43a0 con 0x558d1f48d020 2024-09-19T04:32:40.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f7401400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f481e40 0x558d1f482220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f7401e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 0x558d1f48cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f7401e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 0x558d1f48cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45974/0 (socket says 172.21.15.107:45974) 2024-09-19T04:32:40.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f7401e00640 1 -- 172.21.15.107:0/135306519 learned_addr learned my addr 172.21.15.107:0/135306519 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:40.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f7401400640 1 --2- 172.21.15.107:0/135306519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f48d020 0x558d1f48f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f7401e00640 1 -- 172.21.15.107:0/135306519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f48d020 msgr2=0x558d1f48f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:40.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.834+0000 7f7401e00640 1 --2- 172.21.15.107:0/135306519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f48d020 0x558d1f48f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f7401e00640 1 -- 172.21.15.107:0/135306519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f481e40 msgr2=0x558d1f482220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:40.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f7401e00640 1 --2- 172.21.15.107:0/135306519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f481e40 0x558d1f482220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f7401e00640 1 -- 172.21.15.107:0/135306519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1f3aeae0 con 0x558d1f482760 2024-09-19T04:32:40.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f7401400640 1 --2- 172.21.15.107:0/135306519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f48d020 0x558d1f48f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:40.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f7401e00640 1 --2- 172.21.15.107:0/135306519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 0x558d1f48cae0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f73f4005c40 tx=0x7f73f4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4a49f7115521d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:40.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f73faa00640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f73e80a3430 0x7f73e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f73faa00640 1 -- 172.21.15.107:0/1023686938 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f73fc0cfd10 con 0x7f7404108f60 2024-09-19T04:32:40.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f73ef400640 1 -- 172.21.15.107:0/135306519 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4040070 con 0x558d1f482760 2024-09-19T04:32:40.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f73ef400640 1 -- 172.21.15.107:0/135306519 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f40047a0 con 0x558d1f482760 2024-09-19T04:32:40.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f73ef400640 1 -- 172.21.15.107:0/135306519 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4035040 con 0x558d1f482760 2024-09-19T04:32:40.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.835+0000 7f7401400640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f73e80a3430 0x7f73e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.836+0000 7f740b9be740 1 -- 172.21.15.107:0/135306519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 msgr2=0x558d1f48cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:40.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.836+0000 7f740b9be740 1 --2- 172.21.15.107:0/135306519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 0x558d1f48cae0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f73f4005c40 tx=0x7f73f4005e80 comp rx=0 tx=0).stop 2024-09-19T04:32:40.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.836+0000 7f7401400640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f73e80a3430 0x7f73e80a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x558d1f47f750 tx=0x7f73f0015040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:40.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.836+0000 7f740b9be740 1 -- 172.21.15.107:0/135306519 shutdown_connections 2024-09-19T04:32:40.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.836+0000 7f740b9be740 1 --2- 172.21.15.107:0/135306519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f48d020 0x558d1f48f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.836+0000 7f740b9be740 1 --2- 172.21.15.107:0/135306519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 0x558d1f48cae0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.836+0000 7f740b9be740 1 --2- 172.21.15.107:0/135306519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f481e40 0x558d1f482220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.836+0000 7f740b9be740 1 -- 172.21.15.107:0/135306519 >> 172.21.15.107:0/135306519 conn(0x558d1f47e3d0 msgr2=0x558d1f47e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.836+0000 7f740b9be740 1 -- 172.21.15.107:0/135306519 shutdown_connections 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.836+0000 7f740b9be740 1 -- 172.21.15.107:0/135306519 wait complete. 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.837+0000 7f740b9be740 1 -- start start 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.837+0000 7f740b9be740 1 Processor -- start 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.837+0000 7f740b9be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f489f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.837+0000 7f740b9be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f482760 0x558d1f48a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.837+0000 7f740b9be740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f48d020 0x558d1f48aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.837+0000 7f740b9be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558d1f48fbf0 con 0x558d1f482760 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.837+0000 7f740b9be740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558d1f3a30f0 con 0x558d1f48d020 2024-09-19T04:32:40.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.837+0000 7f740b9be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558d1f48fd70 con 0x558d1f481e40 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.837+0000 7f740b9be740 10 client.0 _ll_register_callbacks cb 0x558d1f38abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7401400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f482760 0x558d1f48a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7401e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f489f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7401400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f482760 0x558d1f48a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43218/0 (socket says 172.21.15.107:43218) 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7401400640 1 -- 172.21.15.107:0/2900072420 learned_addr learned my addr 172.21.15.107:0/2900072420 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7402800640 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f48d020 0x558d1f48aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7402800640 1 -- 172.21.15.107:0/2900072420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 msgr2=0x558d1f489f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7402800640 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f489f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7402800640 1 -- 172.21.15.107:0/2900072420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f482760 msgr2=0x558d1f48a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7402800640 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f482760 0x558d1f48a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7402800640 1 -- 172.21.15.107:0/2900072420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1f1593c0 con 0x558d1f48d020 2024-09-19T04:32:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7401e00640 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f489f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f7402800640 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f48d020 0x558d1f48aa00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0cd3f0 tx=0x7f73fc09bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2878f471980d6a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.838+0000 7f73e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f73e3e00640 1 -- 172.21.15.107:0/2900072420 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc01f030 con 0x558d1f48d020 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f73e3e00640 1 -- 172.21.15.107:0/2900072420 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f73fc0a4790 con 0x558d1f48d020 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f740b9be740 1 -- 172.21.15.107:0/2900072420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f40047a0 con 0x558d1f48d020 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f73e3e00640 1 -- 172.21.15.107:0/2900072420 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f73fc0bc070 con 0x558d1f48d020 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f740b9be740 10 client.27449 fetch_fsmap learned FSMap version 49 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f740b9be740 1 -- 172.21.15.107:0/2900072420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x558d1f49bc20 con 0x558d1f48d020 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f73e3e00640 1 -- 172.21.15.107:0/2900072420 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f40047a0 con 0x558d1f48d020 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f73e3e00640 1 -- 172.21.15.107:0/2900072420 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f73fc00f040 con 0x558d1f48d020 2024-09-19T04:32:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f740b9be740 10 client.27449 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f740b9be740 10 client.27449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f740b9be740 20 client.27449 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f740b9be740 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x558d1f49d0e0 0x558d1f49f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f740b9be740 4 client.27449 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.839+0000 7f740b9be740 1 -- 172.21.15.107:0/2900072420 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d1f3f5950 con 0x558d1f49d0e0 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.840+0000 7f7401e00640 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x558d1f49d0e0 0x558d1f49f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.840+0000 7f7401e00640 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x558d1f49d0e0 0x558d1f49f510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1a0a695adb1fed1 server_cookie=13599425eeb49a7c in_seq=0 out_seq=0 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.840+0000 7f73e3e00640 10 client.27449 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f73e3e00640 1 -- 172.21.15.107:0/2900072420 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d1f3f5950 con 0x558d1f49d0e0 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f73e3e00640 10 client.27449 handle_command_reply: tid=0 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f740b9be740 1 client.27449 shutdown 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f740b9be740 1 -- 172.21.15.107:0/2900072420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f48d020 msgr2=0x558d1f48aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:40.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f740b9be740 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f48d020 0x558d1f48aa00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0cd3f0 tx=0x7f73fc09bf40 comp rx=0 tx=0).stop 2024-09-19T04:32:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f740b9be740 1 -- 172.21.15.107:0/2900072420 shutdown_connections 2024-09-19T04:32:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f740b9be740 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x558d1f49d0e0 0x558d1f49f510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f740b9be740 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f48d020 0x558d1f48aa00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f73e3e00640 0 client.27449 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:32:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f740b9be740 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f482760 0x558d1f48a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f740b9be740 1 --2- 172.21.15.107:0/2900072420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f489f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.841+0000 7f740b9be740 1 -- 172.21.15.107:0/2900072420 >> 172.21.15.107:0/2900072420 conn(0x558d1f47e3d0 msgr2=0x558d1f484100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.842+0000 7f740b9be740 1 -- 172.21.15.107:0/2900072420 shutdown_connections 2024-09-19T04:32:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.842+0000 7f740b9be740 1 -- 172.21.15.107:0/2900072420 wait complete. 2024-09-19T04:32:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.842+0000 7f740b9be740 20 client.27449 trim_cache size 0 max 16384 2024-09-19T04:32:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.855+0000 7f740b9be740 1 Processor -- start 2024-09-19T04:32:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.855+0000 7f740b9be740 1 -- start start 2024-09-19T04:32:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.855+0000 7f740b9be740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f481e40 0x558d1f4848a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.855+0000 7f740b9be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f482760 0x558d1f48bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.855+0000 7f740b9be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 0x558d1f4a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.855+0000 7f740b9be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74041af8b0 con 0x558d1f48d020 2024-09-19T04:32:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.855+0000 7f740b9be740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f74041b1420 con 0x558d1f481e40 2024-09-19T04:32:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.855+0000 7f740b9be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f74041affb0 con 0x558d1f482760 2024-09-19T04:32:40.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7401e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f481e40 0x558d1f4848a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7401e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f481e40 0x558d1f4848a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:45996/0 (socket says 172.21.15.107:45996) 2024-09-19T04:32:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7401e00640 1 -- 172.21.15.107:0/2128953402 learned_addr learned my addr 172.21.15.107:0/2128953402 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7402800640 1 --2- 172.21.15.107:0/2128953402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 0x558d1f4a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7401400640 1 --2- 172.21.15.107:0/2128953402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f482760 0x558d1f48bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7401e00640 1 -- 172.21.15.107:0/2128953402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f482760 msgr2=0x558d1f48bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7401e00640 1 --2- 172.21.15.107:0/2128953402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f482760 0x558d1f48bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7401e00640 1 -- 172.21.15.107:0/2128953402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 msgr2=0x558d1f4a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7401e00640 1 --2- 172.21.15.107:0/2128953402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 0x558d1f4a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7401e00640 1 -- 172.21.15.107:0/2128953402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1f47f060 con 0x558d1f481e40 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.856+0000 7f7401e00640 1 --2- 172.21.15.107:0/2128953402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f481e40 0x558d1f4848a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f73f402a9e0 tx=0x7f73f40035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5e7a43e1e03fbb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f73eea00640 1 -- 172.21.15.107:0/2128953402 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4040040 con 0x558d1f481e40 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f73eea00640 1 -- 172.21.15.107:0/2128953402 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f4004d50 con 0x558d1f481e40 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f73eea00640 1 -- 172.21.15.107:0/2128953402 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4035040 con 0x558d1f481e40 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f740b9be740 1 -- 172.21.15.107:0/2128953402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f481e40 msgr2=0x558d1f4848a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f740b9be740 1 --2- 172.21.15.107:0/2128953402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f481e40 0x558d1f4848a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f73f402a9e0 tx=0x7f73f40035f0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f740b9be740 1 -- 172.21.15.107:0/2128953402 shutdown_connections 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f740b9be740 1 --2- 172.21.15.107:0/2128953402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 0x558d1f4a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f740b9be740 1 --2- 172.21.15.107:0/2128953402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f482760 0x558d1f48bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f740b9be740 1 --2- 172.21.15.107:0/2128953402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f481e40 0x558d1f4848a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f740b9be740 1 -- 172.21.15.107:0/2128953402 >> 172.21.15.107:0/2128953402 conn(0x558d1f47e3d0 msgr2=0x558d1f48b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:40.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f740b9be740 1 -- 172.21.15.107:0/2128953402 shutdown_connections 2024-09-19T04:32:40.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.857+0000 7f740b9be740 1 -- 172.21.15.107:0/2128953402 wait complete. 2024-09-19T04:32:40.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.858+0000 7f740b9be740 1 -- start start 2024-09-19T04:32:40.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.858+0000 7f740b9be740 1 Processor -- start 2024-09-19T04:32:40.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.858+0000 7f740b9be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f490700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.858+0000 7f740b9be740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 0x558d1f490c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.858+0000 7f740b9be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 0x558d1f4ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.858+0000 7f740b9be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74041b0d50 con 0x558d1f48d020 2024-09-19T04:32:40.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.858+0000 7f740b9be740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f74041b0680 con 0x558d1f482760 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.858+0000 7f740b9be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f74041ad8b0 con 0x558d1f481e40 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f740b9be740 10 client.0 _ll_register_callbacks cb 0x7f73fc004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7401e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f490700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7401e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f490700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59498/0 (socket says 172.21.15.107:59498) 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7401e00640 1 -- 172.21.15.107:0/872527522 learned_addr learned my addr 172.21.15.107:0/872527522 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7401400640 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 0x558d1f490c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7402800640 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 0x558d1f4ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7401e00640 1 -- 172.21.15.107:0/872527522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 msgr2=0x558d1f490c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7401e00640 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 0x558d1f490c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7401e00640 1 -- 172.21.15.107:0/872527522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 msgr2=0x558d1f4ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7401e00640 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 0x558d1f4ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7401e00640 1 -- 172.21.15.107:0/872527522 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1f500420 con 0x558d1f481e40 2024-09-19T04:32:40.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7402800640 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 0x558d1f4ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:40.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f7401e00640 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f490700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f73f40314f0 tx=0x7f73f40039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5838f14223ec25d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:40.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.859+0000 7f73e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:32:40.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.860+0000 7f73e3e00640 1 -- 172.21.15.107:0/872527522 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4035030 con 0x558d1f481e40 2024-09-19T04:32:40.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.860+0000 7f740b9be740 1 -- 172.21.15.107:0/872527522 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f4004d50 con 0x558d1f481e40 2024-09-19T04:32:40.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.860+0000 7f73e3e00640 1 -- 172.21.15.107:0/872527522 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f73f4003c00 con 0x558d1f481e40 2024-09-19T04:32:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.860+0000 7f73e3e00640 1 -- 172.21.15.107:0/872527522 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f73f4068050 con 0x558d1f481e40 2024-09-19T04:32:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.860+0000 7f740b9be740 10 client.27394 fetch_fsmap learned FSMap version 49 2024-09-19T04:32:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.860+0000 7f740b9be740 1 -- 172.21.15.107:0/872527522 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x558d1f5077b0 con 0x558d1f481e40 2024-09-19T04:32:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.860+0000 7f73e3e00640 1 -- 172.21.15.107:0/872527522 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f4004d50 con 0x558d1f481e40 2024-09-19T04:32:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.861+0000 7f73e3e00640 1 -- 172.21.15.107:0/872527522 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f73f40315b0 con 0x558d1f481e40 2024-09-19T04:32:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.861+0000 7f740b9be740 10 client.27394 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:32:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.861+0000 7f740b9be740 10 client.27394 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:32:40.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.861+0000 7f740b9be740 20 client.27394 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.861+0000 7f740b9be740 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x558d1f5079e0 0x558d1f509dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.861+0000 7f740b9be740 4 client.27394 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.861+0000 7f740b9be740 1 -- 172.21.15.107:0/872527522 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d1f3efe60 con 0x558d1f5079e0 2024-09-19T04:32:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.861+0000 7f7401400640 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x558d1f5079e0 0x558d1f509dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f7401400640 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x558d1f5079e0 0x558d1f509dd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9e4f14263a69730 server_cookie=3c9ce5f0cf058c23 in_seq=0 out_seq=0 2024-09-19T04:32:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f73e3e00640 10 client.27394 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:32:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f73e3e00640 1 -- 172.21.15.107:0/872527522 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d1f3efe60 con 0x558d1f5079e0 2024-09-19T04:32:40.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f73e3e00640 10 client.27394 handle_command_reply: tid=0 2024-09-19T04:32:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 client.27394 shutdown 2024-09-19T04:32:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 -- 172.21.15.107:0/872527522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 msgr2=0x558d1f490700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f490700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f73f40314f0 tx=0x7f73f40039c0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 -- 172.21.15.107:0/872527522 shutdown_connections 2024-09-19T04:32:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x558d1f5079e0 0x558d1f509dd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d1f48d020 0x558d1f4ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d1f482760 0x558d1f490c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f73e3e00640 0 client.27394 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:32:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 --2- 172.21.15.107:0/872527522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558d1f481e40 0x558d1f490700 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 -- 172.21.15.107:0/872527522 >> 172.21.15.107:0/872527522 conn(0x558d1f47e3d0 msgr2=0x558d1f48a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 -- 172.21.15.107:0/872527522 shutdown_connections 2024-09-19T04:32:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 1 -- 172.21.15.107:0/872527522 wait complete. 2024-09-19T04:32:40.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.862+0000 7f740b9be740 20 client.27394 trim_cache size 0 max 16384 2024-09-19T04:32:40.874 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:32:40.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 -- 172.21.15.107:0/1023686938 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f73e80a3430 msgr2=0x7f73e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:40.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f73e80a3430 0x7f73e80a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x558d1f47f750 tx=0x7f73f0015040 comp rx=0 tx=0).stop 2024-09-19T04:32:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 -- 172.21.15.107:0/1023686938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7404108f60 msgr2=0x7f740419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7404108f60 0x7f740419ed60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00e940 tx=0x7f73fc00ee00 comp rx=0 tx=0).stop 2024-09-19T04:32:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 -- 172.21.15.107:0/1023686938 shutdown_connections 2024-09-19T04:32:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f73e80a3430 0x7f73e80a58f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7404108f60 0x7f740419ed60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f74041047e0 0x7f740419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 --2- 172.21.15.107:0/1023686938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7404103e30 0x7f740419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 -- 172.21.15.107:0/1023686938 >> 172.21.15.107:0/1023686938 conn(0x7f74040ff8d0 msgr2=0x7f7404100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 -- 172.21.15.107:0/1023686938 shutdown_connections 2024-09-19T04:32:40.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:40.863+0000 7f7403200640 1 -- 172.21.15.107:0/1023686938 wait complete. 2024-09-19T04:32:40.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:40 smithi107 ceph-mon[30659]: pgmap v385: 97 pgs: 97 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-19T04:32:40.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:40 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:32:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:40 smithi115 ceph-mon[35477]: pgmap v385: 97 pgs: 97 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-19T04:32:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:40 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:32:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:40 smithi145 ceph-mon[35096]: pgmap v385: 97 pgs: 97 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-19T04:32:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:40 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:32:41.728 INFO:tasks.workunit.client.0.smithi107.stdout: 1 425487 2.97 MB/sec execute 539 sec latency 1995.922 ms 2024-09-19T04:32:41.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:32:42.215 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:42.728 INFO:tasks.workunit.client.0.smithi107.stdout: 1 425487 2.97 MB/sec execute 540 sec latency 2996.011 ms 2024-09-19T04:32:43.019 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:42 smithi107 ceph-mon[30659]: pgmap v386: 97 pgs: 97 active+clean; 365 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-19T04:32:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.032+0000 7f7995400640 1 -- 172.21.15.107:0/4242029960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7990104800 msgr2=0x7f7990108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.032+0000 7f7995400640 1 --2- 172.21.15.107:0/4242029960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7990104800 0x7f7990108960 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7978009900 tx=0x7f797802a3f0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.032+0000 7f7995400640 1 -- 172.21.15.107:0/4242029960 shutdown_connections 2024-09-19T04:32:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.032+0000 7f7995400640 1 --2- 172.21.15.107:0/4242029960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7990109090 0x7f7990110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.032+0000 7f7995400640 1 --2- 172.21.15.107:0/4242029960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7990104800 0x7f7990108960 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.032+0000 7f7995400640 1 --2- 172.21.15.107:0/4242029960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7990103e50 0x7f7990104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.032+0000 7f7995400640 1 -- 172.21.15.107:0/4242029960 >> 172.21.15.107:0/4242029960 conn(0x7f79900ff8b0 msgr2=0x7f7990101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.032+0000 7f7995400640 1 -- 172.21.15.107:0/4242029960 shutdown_connections 2024-09-19T04:32:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.032+0000 7f7995400640 1 -- 172.21.15.107:0/4242029960 wait complete. 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.033+0000 7f7995400640 1 Processor -- start 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.033+0000 7f7995400640 1 -- start start 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.033+0000 7f7995400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7990103e50 0x7f799019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.033+0000 7f7995400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7990104800 0x7f799019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.033+0000 7f7995400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7990109090 0x7f799019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.033+0000 7f7995400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79901140e0 con 0x7f7990103e50 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.033+0000 7f7995400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7990113f60 con 0x7f7990104800 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.033+0000 7f7995400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7990114260 con 0x7f7990109090 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f798fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7990103e50 0x7f799019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f7994a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7990109090 0x7f799019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f798fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7990103e50 0x7f799019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43246/0 (socket says 172.21.15.107:43246) 2024-09-19T04:32:43.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f798fe00640 1 -- 172.21.15.107:0/3001208608 learned_addr learned my addr 172.21.15.107:0/3001208608 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:43.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f7987e00640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7990104800 0x7f799019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f798fe00640 1 -- 172.21.15.107:0/3001208608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7990109090 msgr2=0x7f799019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:43.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f798fe00640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7990109090 0x7f799019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f798fe00640 1 -- 172.21.15.107:0/3001208608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7990104800 msgr2=0x7f799019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:43.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f798fe00640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7990104800 0x7f799019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f798fe00640 1 -- 172.21.15.107:0/3001208608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799019f550 con 0x7f7990103e50 2024-09-19T04:32:43.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.034+0000 7f798fe00640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7990103e50 0x7f799019a5a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f798000d950 tx=0x7f798000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:43.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.035+0000 7f798d600640 1 -- 172.21.15.107:0/3001208608 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980017030 con 0x7f7990103e50 2024-09-19T04:32:43.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.035+0000 7f798d600640 1 -- 172.21.15.107:0/3001208608 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7980004020 con 0x7f7990103e50 2024-09-19T04:32:43.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.035+0000 7f7995400640 1 -- 172.21.15.107:0/3001208608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799019f840 con 0x7f7990103e50 2024-09-19T04:32:43.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.035+0000 7f798d600640 1 -- 172.21.15.107:0/3001208608 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79800047a0 con 0x7f7990103e50 2024-09-19T04:32:43.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.035+0000 7f7995400640 1 -- 172.21.15.107:0/3001208608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79901a7120 con 0x7f7990103e50 2024-09-19T04:32:43.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.038+0000 7f798d600640 1 -- 172.21.15.107:0/3001208608 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f79800041d0 con 0x7f7990103e50 2024-09-19T04:32:43.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.038+0000 7f7999480740 1 Processor -- start 2024-09-19T04:32:43.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.038+0000 7f7999480740 1 -- start start 2024-09-19T04:32:43.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.038+0000 7f7999480740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe62220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.038+0000 7f7999480740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 0x564f6fe6cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.038+0000 7f7999480740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe6d020 0x564f6fe6f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.038+0000 7f7999480740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564f6fdc83e0 con 0x564f6fe61e40 2024-09-19T04:32:43.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.038+0000 7f7999480740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f6fd6fd80 con 0x564f6fe62760 2024-09-19T04:32:43.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.038+0000 7f7999480740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564f6fdc43a0 con 0x564f6fe6d020 2024-09-19T04:32:43.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.039+0000 7f7994a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe62220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.039+0000 7f7994a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe62220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43258/0 (socket says 172.21.15.107:43258) 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.039+0000 7f7994a00640 1 -- 172.21.15.107:0/2419574866 learned_addr learned my addr 172.21.15.107:0/2419574866 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.039+0000 7f798fe00640 1 --2- 172.21.15.107:0/2419574866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 0x564f6fe6cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.039+0000 7f7987e00640 1 --2- 172.21.15.107:0/2419574866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe6d020 0x564f6fe6f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.039+0000 7f7994a00640 1 -- 172.21.15.107:0/2419574866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe6d020 msgr2=0x564f6fe6f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.039+0000 7f7994a00640 1 --2- 172.21.15.107:0/2419574866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe6d020 0x564f6fe6f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.039+0000 7f7994a00640 1 -- 172.21.15.107:0/2419574866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 msgr2=0x564f6fe6cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.039+0000 7f7994a00640 1 --2- 172.21.15.107:0/2419574866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 0x564f6fe6cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.039+0000 7f7994a00640 1 -- 172.21.15.107:0/2419574866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f6fd8eae0 con 0x564f6fe61e40 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.040+0000 7f798d600640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f79740a3430 0x7f79740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.040+0000 7f7994a00640 1 --2- 172.21.15.107:0/2419574866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe62220 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7988009f70 tx=0x7f7988010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7674b7ddf7d581ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.040+0000 7f798d600640 1 -- 172.21.15.107:0/3001208608 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f79800cf620 con 0x7f7990103e50 2024-09-19T04:32:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.040+0000 7f7985200640 1 -- 172.21.15.107:0/2419574866 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988016070 con 0x564f6fe61e40 2024-09-19T04:32:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.040+0000 7f7985200640 1 -- 172.21.15.107:0/2419574866 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7988004d20 con 0x564f6fe61e40 2024-09-19T04:32:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.040+0000 7f7985200640 1 -- 172.21.15.107:0/2419574866 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988012950 con 0x564f6fe61e40 2024-09-19T04:32:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.040+0000 7f798fe00640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f79740a3430 0x7f79740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f7999480740 1 -- 172.21.15.107:0/2419574866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 msgr2=0x564f6fe62220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f798fe00640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f79740a3430 0x7f79740a58f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x564f6fe5f230 tx=0x7f79800d5530 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f7999480740 1 --2- 172.21.15.107:0/2419574866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe62220 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7988009f70 tx=0x7f7988010800 comp rx=0 tx=0).stop 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f7999480740 1 -- 172.21.15.107:0/2419574866 shutdown_connections 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f7999480740 1 --2- 172.21.15.107:0/2419574866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe6d020 0x564f6fe6f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f7999480740 1 --2- 172.21.15.107:0/2419574866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 0x564f6fe6cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f7999480740 1 --2- 172.21.15.107:0/2419574866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe62220 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f7999480740 1 -- 172.21.15.107:0/2419574866 >> 172.21.15.107:0/2419574866 conn(0x564f6fe5e3d0 msgr2=0x564f6fe5e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f7999480740 1 -- 172.21.15.107:0/2419574866 shutdown_connections 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f7999480740 1 -- 172.21.15.107:0/2419574866 wait complete. 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.041+0000 7f7999480740 1 -- start start 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.042+0000 7f7999480740 1 Processor -- start 2024-09-19T04:32:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.042+0000 7f7999480740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe69f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.042+0000 7f7999480740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 0x564f6fe6a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.042+0000 7f7999480740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe6d020 0x564f6fe6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.042+0000 7f7999480740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564f6fe6fbf0 con 0x564f6fe61e40 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.042+0000 7f7999480740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f6fd830f0 con 0x564f6fe62760 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.042+0000 7f7999480740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564f6fe6fd70 con 0x564f6fe6d020 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.042+0000 7f7999480740 10 client.0 _ll_register_callbacks cb 0x564f6fd6abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.042+0000 7f7994a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe69f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.042+0000 7f7987e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 0x564f6fe6a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7987e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 0x564f6fe6a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:46052/0 (socket says 172.21.15.107:46052) 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7987e00640 1 -- 172.21.15.107:0/3479911006 learned_addr learned my addr 172.21.15.107:0/3479911006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7994a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe69f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43262/0 (socket says 172.21.15.107:43262) 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7994a00640 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe6d020 0x564f6fe6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7987e00640 1 -- 172.21.15.107:0/3479911006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe6d020 msgr2=0x564f6fe6a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7987e00640 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe6d020 0x564f6fe6a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7987e00640 1 -- 172.21.15.107:0/3479911006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 msgr2=0x564f6fe69f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7987e00640 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe69f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7987e00640 1 -- 172.21.15.107:0/3479911006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6fb393c0 con 0x564f6fe62760 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7994a00640 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe69f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f7987e00640 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 0x564f6fe6a480 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7978033000 tx=0x7f79780044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa2202bbb5549d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.043+0000 7f797d600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.044+0000 7f797d600640 1 -- 172.21.15.107:0/3479911006 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978035040 con 0x564f6fe62760 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.044+0000 7f7999480740 1 -- 172.21.15.107:0/3479911006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7988004d20 con 0x564f6fe62760 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.044+0000 7f797d600640 1 -- 172.21.15.107:0/3479911006 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f7978005590 con 0x564f6fe62760 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.044+0000 7f797d600640 1 -- 172.21.15.107:0/3479911006 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f79780046c0 con 0x564f6fe62760 2024-09-19T04:32:43.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.044+0000 7f7999480740 10 client.27467 fetch_fsmap learned FSMap version 49 2024-09-19T04:32:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.044+0000 7f7999480740 1 -- 172.21.15.107:0/3479911006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564f6fe7bcb0 con 0x564f6fe62760 2024-09-19T04:32:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.044+0000 7f797d600640 1 -- 172.21.15.107:0/3479911006 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7988004d20 con 0x564f6fe62760 2024-09-19T04:32:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.044+0000 7f797d600640 1 -- 172.21.15.107:0/3479911006 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f7978031b50 con 0x564f6fe62760 2024-09-19T04:32:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.045+0000 7f7999480740 10 client.27467 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:32:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.045+0000 7f7999480740 10 client.27467 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:32:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.045+0000 7f7999480740 20 client.27467 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.045+0000 7f7999480740 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x564f6fe7d0c0 0x564f6fe7f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.045+0000 7f7999480740 4 client.27467 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.045+0000 7f7999480740 1 -- 172.21.15.107:0/3479911006 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f6fdd5950 con 0x564f6fe7d0c0 2024-09-19T04:32:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.045+0000 7f7994a00640 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x564f6fe7d0c0 0x564f6fe7f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.046+0000 7f7994a00640 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x564f6fe7d0c0 0x564f6fe7f4b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f77b809bbc860c3 server_cookie=3252e5d777946db6 in_seq=0 out_seq=0 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.046+0000 7f797d600640 10 client.27467 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.046+0000 7f797d600640 1 -- 172.21.15.107:0/3479911006 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f6fdd5950 con 0x564f6fe7d0c0 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.046+0000 7f797d600640 10 client.27467 handle_command_reply: tid=0 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.046+0000 7f7999480740 1 client.27467 shutdown 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 1 -- 172.21.15.107:0/3479911006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 msgr2=0x564f6fe6a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 0x564f6fe6a480 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7978033000 tx=0x7f79780044f0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 1 -- 172.21.15.107:0/3479911006 shutdown_connections 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x564f6fe7d0c0 0x564f6fe7f4b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe6d020 0x564f6fe6a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe62760 0x564f6fe6a480 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f797d600640 0 client.27467 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:32:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 1 --2- 172.21.15.107:0/3479911006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe61e40 0x564f6fe69f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 1 -- 172.21.15.107:0/3479911006 >> 172.21.15.107:0/3479911006 conn(0x564f6fe5e3d0 msgr2=0x564f6fe64100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 1 -- 172.21.15.107:0/3479911006 shutdown_connections 2024-09-19T04:32:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 1 -- 172.21.15.107:0/3479911006 wait complete. 2024-09-19T04:32:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.047+0000 7f7999480740 20 client.27467 trim_cache size 0 max 16384 2024-09-19T04:32:43.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.060+0000 7f7999480740 1 Processor -- start 2024-09-19T04:32:43.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.060+0000 7f7999480740 1 -- start start 2024-09-19T04:32:43.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7999480740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe61e40 0x564f6fe648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7999480740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe62760 0x564f6fe6beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7999480740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe6d020 0x564f6fe84d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7999480740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79901af910 con 0x564f6fe6d020 2024-09-19T04:32:43.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7999480740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f79901b1480 con 0x564f6fe61e40 2024-09-19T04:32:43.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7999480740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f79901b0010 con 0x564f6fe62760 2024-09-19T04:32:43.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7994a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe61e40 0x564f6fe648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7994a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe6d020 0x564f6fe84d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7994a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe61e40 0x564f6fe648a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:46068/0 (socket says 172.21.15.107:46068) 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7994a00640 1 -- 172.21.15.107:0/1833585167 learned_addr learned my addr 172.21.15.107:0/1833585167 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7987e00640 1 --2- 172.21.15.107:0/1833585167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe62760 0x564f6fe6beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7994a00640 1 -- 172.21.15.107:0/1833585167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe62760 msgr2=0x564f6fe6beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.061+0000 7f7994a00640 1 --2- 172.21.15.107:0/1833585167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe62760 0x564f6fe6beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.062+0000 7f7994a00640 1 -- 172.21.15.107:0/1833585167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe6d020 msgr2=0x564f6fe84d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.062+0000 7f7994a00640 1 --2- 172.21.15.107:0/1833585167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe6d020 0x564f6fe84d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.062+0000 7f7994a00640 1 -- 172.21.15.107:0/1833585167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f6fe7fb30 con 0x564f6fe61e40 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.062+0000 7f7987e00640 1 --2- 172.21.15.107:0/1833585167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe62760 0x564f6fe6beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.062+0000 7f7994a00640 1 --2- 172.21.15.107:0/1833585167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe61e40 0x564f6fe648a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f79880124c0 tx=0x7f798801d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f1ed33f4503f706 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.062+0000 7f797fe00640 1 -- 172.21.15.107:0/1833585167 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798802f070 con 0x564f6fe61e40 2024-09-19T04:32:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.062+0000 7f797fe00640 1 -- 172.21.15.107:0/1833585167 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7988025040 con 0x564f6fe61e40 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.062+0000 7f797fe00640 1 -- 172.21.15.107:0/1833585167 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988024070 con 0x564f6fe61e40 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 -- 172.21.15.107:0/1833585167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe61e40 msgr2=0x564f6fe648a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 --2- 172.21.15.107:0/1833585167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe61e40 0x564f6fe648a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f79880124c0 tx=0x7f798801d040 comp rx=0 tx=0).stop 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 -- 172.21.15.107:0/1833585167 shutdown_connections 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 --2- 172.21.15.107:0/1833585167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe6d020 0x564f6fe84d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 --2- 172.21.15.107:0/1833585167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe62760 0x564f6fe6beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 --2- 172.21.15.107:0/1833585167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe61e40 0x564f6fe648a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 -- 172.21.15.107:0/1833585167 >> 172.21.15.107:0/1833585167 conn(0x564f6fe5e3d0 msgr2=0x564f6fe6b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 -- 172.21.15.107:0/1833585167 shutdown_connections 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 -- 172.21.15.107:0/1833585167 wait complete. 2024-09-19T04:32:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 -- start start 2024-09-19T04:32:43.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.063+0000 7f7999480740 1 Processor -- start 2024-09-19T04:32:43.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7999480740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe61e40 0x564f6fe755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7999480740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe62760 0x564f6fe708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7999480740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe6d020 0x564f6fe70e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7999480740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79901b0db0 con 0x564f6fe62760 2024-09-19T04:32:43.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7999480740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f79901b06e0 con 0x564f6fe6d020 2024-09-19T04:32:43.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7999480740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f79901ad910 con 0x564f6fe61e40 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7999480740 10 client.0 _ll_register_callbacks cb 0x7f7978009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7987e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe62760 0x564f6fe708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7987e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe62760 0x564f6fe708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43286/0 (socket says 172.21.15.107:43286) 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7987e00640 1 -- 172.21.15.107:0/3470876547 learned_addr learned my addr 172.21.15.107:0/3470876547 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7994a00640 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe6d020 0x564f6fe70e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.064+0000 7f7994a00640 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe61e40 0x564f6fe755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f7987e00640 1 -- 172.21.15.107:0/3470876547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe61e40 msgr2=0x564f6fe755b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f7987e00640 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe61e40 0x564f6fe755b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f7987e00640 1 -- 172.21.15.107:0/3470876547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe6d020 msgr2=0x564f6fe70e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f7987e00640 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe6d020 0x564f6fe70e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f7987e00640 1 -- 172.21.15.107:0/3470876547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6fe714c0 con 0x564f6fe62760 2024-09-19T04:32:43.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f7994a00640 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe61e40 0x564f6fe755b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f7987e00640 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe62760 0x564f6fe708e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7978006010 tx=0x7f79780055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e795c15b133f34fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f797d600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f797d600640 1 -- 172.21.15.107:0/3470876547 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978035070 con 0x564f6fe62760 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f797d600640 1 -- 172.21.15.107:0/3470876547 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f7978003e20 con 0x564f6fe62760 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f7999480740 1 -- 172.21.15.107:0/3470876547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7988025040 con 0x564f6fe62760 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f797d600640 1 -- 172.21.15.107:0/3470876547 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7978053040 con 0x564f6fe62760 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f7999480740 10 client.17424 fetch_fsmap learned FSMap version 49 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.065+0000 7f7999480740 1 -- 172.21.15.107:0/3470876547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564f6fee7be0 con 0x564f6fe62760 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.066+0000 7f797d600640 1 -- 172.21.15.107:0/3470876547 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7988025040 con 0x564f6fe62760 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.066+0000 7f797d600640 1 -- 172.21.15.107:0/3470876547 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f79780315b0 con 0x564f6fe62760 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.066+0000 7f7999480740 10 client.17424 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.066+0000 7f7999480740 10 client.17424 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:32:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.066+0000 7f7999480740 20 client.17424 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.066+0000 7f7999480740 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x564f6fee8390 0x564f6feea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.066+0000 7f7999480740 4 client.17424 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.066+0000 7f7999480740 1 -- 172.21.15.107:0/3470876547 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f6fdcfe60 con 0x564f6fee8390 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.066+0000 7f7994a00640 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x564f6fee8390 0x564f6feea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7994a00640 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x564f6fee8390 0x564f6feea7b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8257a8650884ad70 server_cookie=be7c2d5a9b1d7de3 in_seq=0 out_seq=0 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f797d600640 10 client.17424 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f797d600640 1 -- 172.21.15.107:0/3470876547 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f6fdcfe60 con 0x564f6fee8390 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f797d600640 10 client.17424 handle_command_reply: tid=0 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7999480740 1 client.17424 shutdown 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7999480740 1 -- 172.21.15.107:0/3470876547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe62760 msgr2=0x564f6fe708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7999480740 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe62760 0x564f6fe708e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7978006010 tx=0x7f79780055c0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7999480740 1 -- 172.21.15.107:0/3470876547 shutdown_connections 2024-09-19T04:32:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7999480740 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x564f6fee8390 0x564f6feea7b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7999480740 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6fe6d020 0x564f6fe70e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7999480740 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f6fe62760 0x564f6fe708e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7999480740 1 --2- 172.21.15.107:0/3470876547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f6fe61e40 0x564f6fe755b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f797d600640 0 client.17424 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:32:43.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7999480740 1 -- 172.21.15.107:0/3470876547 >> 172.21.15.107:0/3470876547 conn(0x564f6fe5e3d0 msgr2=0x564f6fe69f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:43.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.067+0000 7f7999480740 1 -- 172.21.15.107:0/3470876547 shutdown_connections 2024-09-19T04:32:43.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.068+0000 7f7999480740 1 -- 172.21.15.107:0/3470876547 wait complete. 2024-09-19T04:32:43.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.068+0000 7f7999480740 20 client.17424 trim_cache size 0 max 16384 2024-09-19T04:32:43.072 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:32:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.068+0000 7f7995400640 1 -- 172.21.15.107:0/3001208608 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f79740a3430 msgr2=0x7f79740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.068+0000 7f7995400640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f79740a3430 0x7f79740a58f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x564f6fe5f230 tx=0x7f79800d5530 comp rx=0 tx=0).stop 2024-09-19T04:32:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.068+0000 7f7995400640 1 -- 172.21.15.107:0/3001208608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7990103e50 msgr2=0x7f799019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.068+0000 7f7995400640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7990103e50 0x7f799019a5a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f798000d950 tx=0x7f798000de10 comp rx=0 tx=0).stop 2024-09-19T04:32:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.069+0000 7f7995400640 1 -- 172.21.15.107:0/3001208608 shutdown_connections 2024-09-19T04:32:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.069+0000 7f7995400640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f79740a3430 0x7f79740a58f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.069+0000 7f7995400640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7990109090 0x7f799019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.069+0000 7f7995400640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7990104800 0x7f799019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.069+0000 7f7995400640 1 --2- 172.21.15.107:0/3001208608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7990103e50 0x7f799019a5a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.069+0000 7f7995400640 1 -- 172.21.15.107:0/3001208608 >> 172.21.15.107:0/3001208608 conn(0x7f79900ff8b0 msgr2=0x7f7990100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.069+0000 7f7995400640 1 -- 172.21.15.107:0/3001208608 shutdown_connections 2024-09-19T04:32:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:43.069+0000 7f7995400640 1 -- 172.21.15.107:0/3001208608 wait complete. 2024-09-19T04:32:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:42 smithi115 ceph-mon[35477]: pgmap v386: 97 pgs: 97 active+clean; 365 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-19T04:32:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:42 smithi145 ceph-mon[35096]: pgmap v386: 97 pgs: 97 active+clean; 365 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-19T04:32:43.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 425487 2.96 MB/sec execute 541 sec latency 3996.052 ms 2024-09-19T04:32:44.194 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:32:44.599 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:44.728 INFO:tasks.workunit.client.0.smithi107.stdout: 1 426200 2.96 MB/sec execute 542 sec latency 4246.051 ms 2024-09-19T04:32:45.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:44 smithi107 ceph-mon[30659]: pgmap v387: 97 pgs: 97 active+clean; 365 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-09-19T04:32:45.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:44 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:32:45.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.186+0000 7f0f61e00640 1 -- 172.21.15.107:0/3538247003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f5c1047e0 msgr2=0x7f0f5c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:45.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.186+0000 7f0f61e00640 1 --2- 172.21.15.107:0/3538247003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f5c1047e0 0x7f0f5c108830 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009900 tx=0x7f0f4802a410 comp rx=0 tx=0).stop 2024-09-19T04:32:45.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.186+0000 7f0f61e00640 1 -- 172.21.15.107:0/3538247003 shutdown_connections 2024-09-19T04:32:45.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.186+0000 7f0f61e00640 1 --2- 172.21.15.107:0/3538247003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f5c108f60 0x7f0f5c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.186+0000 7f0f61e00640 1 --2- 172.21.15.107:0/3538247003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f5c1047e0 0x7f0f5c108830 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.186+0000 7f0f61e00640 1 --2- 172.21.15.107:0/3538247003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c103e30 0x7f0f5c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.186+0000 7f0f61e00640 1 -- 172.21.15.107:0/3538247003 >> 172.21.15.107:0/3538247003 conn(0x7f0f5c0ff910 msgr2=0x7f0f5c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:45.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.186+0000 7f0f61e00640 1 -- 172.21.15.107:0/3538247003 shutdown_connections 2024-09-19T04:32:45.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.186+0000 7f0f61e00640 1 -- 172.21.15.107:0/3538247003 wait complete. 2024-09-19T04:32:45.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.187+0000 7f0f61e00640 1 Processor -- start 2024-09-19T04:32:45.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.187+0000 7f0f61e00640 1 -- start start 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f61e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c103e30 0x7f0f5c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f61e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f5c1047e0 0x7f0f5c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f61e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f5c108f60 0x7f0f5c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f61e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c114010 con 0x7f0f5c1047e0 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f61e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c113e90 con 0x7f0f5c103e30 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f61e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c114190 con 0x7f0f5c108f60 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f60a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c103e30 0x7f0f5c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f5c1047e0 0x7f0f5c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f60a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c103e30 0x7f0f5c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:46090/0 (socket says 172.21.15.107:46090) 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f60a00640 1 -- 172.21.15.107:0/849903500 learned_addr learned my addr 172.21.15.107:0/849903500 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f61400640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f5c108f60 0x7f0f5c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f60a00640 1 -- 172.21.15.107:0/849903500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f5c108f60 msgr2=0x7f0f5c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:45.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f60a00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f5c108f60 0x7f0f5c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.188+0000 7f0f60a00640 1 -- 172.21.15.107:0/849903500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f5c1047e0 msgr2=0x7f0f5c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:45.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.189+0000 7f0f60a00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f5c1047e0 0x7f0f5c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.189+0000 7f0f60a00640 1 -- 172.21.15.107:0/849903500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c19f480 con 0x7f0f5c103e30 2024-09-19T04:32:45.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.189+0000 7f0f60a00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c103e30 0x7f0f5c19a4d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000e9f0 tx=0x7f0f5000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:45.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.189+0000 7f0f59600640 1 -- 172.21.15.107:0/849903500 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5000cd80 con 0x7f0f5c103e30 2024-09-19T04:32:45.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.189+0000 7f0f61e00640 1 -- 172.21.15.107:0/849903500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c19f770 con 0x7f0f5c103e30 2024-09-19T04:32:45.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.189+0000 7f0f59600640 1 -- 172.21.15.107:0/849903500 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f50005500 con 0x7f0f5c103e30 2024-09-19T04:32:45.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.189+0000 7f0f59600640 1 -- 172.21.15.107:0/849903500 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50010690 con 0x7f0f5c103e30 2024-09-19T04:32:45.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.189+0000 7f0f61e00640 1 -- 172.21.15.107:0/849903500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c1a7050 con 0x7f0f5c103e30 2024-09-19T04:32:45.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.192+0000 7f0f59600640 1 -- 172.21.15.107:0/849903500 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f0f500046c0 con 0x7f0f5c103e30 2024-09-19T04:32:45.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.192+0000 7f0f65e6f740 1 Processor -- start 2024-09-19T04:32:45.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.193+0000 7f0f65e6f740 1 -- start start 2024-09-19T04:32:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.193+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8257e40 0x55ffb8258220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.193+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8258760 0x55ffb8262ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.193+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8265410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.193+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ffb81be3e0 con 0x55ffb8258760 2024-09-19T04:32:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.193+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ffb8165d80 con 0x55ffb8263020 2024-09-19T04:32:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.193+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ffb81ba3a0 con 0x55ffb8257e40 2024-09-19T04:32:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.193+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8265410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8265410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:46094/0 (socket says 172.21.15.107:46094) 2024-09-19T04:32:45.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f5be00640 1 -- 172.21.15.107:0/3414802079 learned_addr learned my addr 172.21.15.107:0/3414802079 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:45.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f61400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8257e40 0x55ffb8258220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f60a00640 1 --2- 172.21.15.107:0/3414802079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8258760 0x55ffb8262ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f61400640 1 -- 172.21.15.107:0/3414802079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 msgr2=0x55ffb8265410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f61400640 1 --2- 172.21.15.107:0/3414802079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8265410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f61400640 1 -- 172.21.15.107:0/3414802079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8258760 msgr2=0x55ffb8262ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f61400640 1 --2- 172.21.15.107:0/3414802079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8258760 0x55ffb8262ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f61400640 1 -- 172.21.15.107:0/3414802079 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb8184ae0 con 0x55ffb8257e40 2024-09-19T04:32:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f5be00640 1 --2- 172.21.15.107:0/3414802079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8265410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f60a00640 1 --2- 172.21.15.107:0/3414802079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8258760 0x55ffb8262ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:45.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.194+0000 7f0f61400640 1 --2- 172.21.15.107:0/3414802079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8257e40 0x55ffb8258220 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009eb0 tx=0x7f0f540107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d74b1ba02adc8b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.195+0000 7f0f4dc00640 1 -- 172.21.15.107:0/3414802079 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54003b70 con 0x55ffb8257e40 2024-09-19T04:32:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.195+0000 7f0f4dc00640 1 -- 172.21.15.107:0/3414802079 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f54015070 con 0x55ffb8257e40 2024-09-19T04:32:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.195+0000 7f0f4dc00640 1 -- 172.21.15.107:0/3414802079 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5401a940 con 0x55ffb8257e40 2024-09-19T04:32:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.195+0000 7f0f59600640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0f440a33b0 0x7f0f440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.195+0000 7f0f59600640 1 -- 172.21.15.107:0/849903500 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f0f500d0140 con 0x7f0f5c103e30 2024-09-19T04:32:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.195+0000 7f0f60a00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0f440a33b0 0x7f0f440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.195+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3414802079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8257e40 msgr2=0x55ffb8258220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.195+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/3414802079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8257e40 0x55ffb8258220 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009eb0 tx=0x7f0f540107a0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:44 smithi115 ceph-mon[35477]: pgmap v387: 97 pgs: 97 active+clean; 365 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-09-19T04:32:45.198 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:44 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:32:45.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.196+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3414802079 shutdown_connections 2024-09-19T04:32:45.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.196+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/3414802079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8265410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.196+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/3414802079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8258760 0x55ffb8262ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.196+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/3414802079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8257e40 0x55ffb8258220 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.196+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3414802079 >> 172.21.15.107:0/3414802079 conn(0x55ffb82543d0 msgr2=0x55ffb82547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:45.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.196+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3414802079 shutdown_connections 2024-09-19T04:32:45.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.196+0000 7f0f60a00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0f440a33b0 0x7f0f440a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55ffb8255230 tx=0x7f0f50098040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:45.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.196+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3414802079 wait complete. 2024-09-19T04:32:45.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.197+0000 7f0f65e6f740 1 -- start start 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.197+0000 7f0f65e6f740 1 Processor -- start 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.197+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb825ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.197+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8258760 0x55ffb82604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8263020 0x55ffb82609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ffb8265bf0 con 0x55ffb8257e40 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ffb81790f0 con 0x55ffb8258760 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ffb8265d70 con 0x55ffb8263020 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f65e6f740 10 client.0 _ll_register_callbacks cb 0x55ffb8160bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f61400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb825ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f61400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb825ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43320/0 (socket says 172.21.15.107:43320) 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f61400640 1 -- 172.21.15.107:0/3439049850 learned_addr learned my addr 172.21.15.107:0/3439049850 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:45.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f5be00640 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8258760 0x55ffb82604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f61400640 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8263020 0x55ffb82609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f5be00640 1 -- 172.21.15.107:0/3439049850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8263020 msgr2=0x55ffb82609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f5be00640 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8263020 0x55ffb82609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f5be00640 1 -- 172.21.15.107:0/3439049850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 msgr2=0x55ffb825ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f5be00640 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb825ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.198+0000 7f0f5be00640 1 -- 172.21.15.107:0/3439049850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb7f2f3c0 con 0x55ffb8258760 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.199+0000 7f0f5be00640 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8258760 0x55ffb82604a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0f48031550 tx=0x7f0f48004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f35d741bab716e37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.199+0000 7f0f42000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.199+0000 7f0f42000640 1 -- 172.21.15.107:0/3439049850 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48035040 con 0x55ffb8258760 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.199+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3439049850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f54015070 con 0x55ffb8258760 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.199+0000 7f0f42000640 1 -- 172.21.15.107:0/3439049850 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f0f48005450 con 0x55ffb8258760 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.199+0000 7f0f42000640 1 -- 172.21.15.107:0/3439049850 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0f48005650 con 0x55ffb8258760 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.199+0000 7f0f65e6f740 10 client.27485 fetch_fsmap learned FSMap version 49 2024-09-19T04:32:45.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.199+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3439049850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ffb827a4e0 con 0x55ffb8258760 2024-09-19T04:32:45.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.199+0000 7f0f42000640 1 -- 172.21.15.107:0/3439049850 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f54015070 con 0x55ffb8258760 2024-09-19T04:32:45.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.200+0000 7f0f42000640 1 -- 172.21.15.107:0/3439049850 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f0f48003a20 con 0x55ffb8258760 2024-09-19T04:32:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.200+0000 7f0f65e6f740 10 client.27485 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:32:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.200+0000 7f0f65e6f740 10 client.27485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.200+0000 7f0f65e6f740 20 client.27485 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.200+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ffb827b8f0 0x55ffb827dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.200+0000 7f0f65e6f740 4 client.27485 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.200+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3439049850 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffb81cb950 con 0x55ffb827b8f0 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.200+0000 7f0f61400640 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ffb827b8f0 0x55ffb827dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.201+0000 7f0f61400640 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ffb827b8f0 0x55ffb827dd00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26b17375fd616d19 server_cookie=ff2d806f8be44843 in_seq=0 out_seq=0 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.201+0000 7f0f42000640 10 client.27485 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.201+0000 7f0f42000640 1 -- 172.21.15.107:0/3439049850 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffb81cb950 con 0x55ffb827b8f0 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.201+0000 7f0f42000640 10 client.27485 handle_command_reply: tid=0 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.201+0000 7f0f65e6f740 1 client.27485 shutdown 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3439049850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8258760 msgr2=0x55ffb82604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8258760 0x55ffb82604a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0f48031550 tx=0x7f0f48004270 comp rx=0 tx=0).stop 2024-09-19T04:32:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3439049850 shutdown_connections 2024-09-19T04:32:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ffb827b8f0 0x55ffb827dd00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8263020 0x55ffb82609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8258760 0x55ffb82604a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/3439049850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb825ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3439049850 >> 172.21.15.107:0/3439049850 conn(0x55ffb82543d0 msgr2=0x55ffb825a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f42000640 0 client.27485 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3439049850 shutdown_connections 2024-09-19T04:32:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 1 -- 172.21.15.107:0/3439049850 wait complete. 2024-09-19T04:32:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.202+0000 7f0f65e6f740 20 client.27485 trim_cache size 0 max 16384 2024-09-19T04:32:45.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.220+0000 7f0f65e6f740 1 Processor -- start 2024-09-19T04:32:45.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.220+0000 7f0f65e6f740 1 -- start start 2024-09-19T04:32:45.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb8261ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb82835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8283b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c1ab500 con 0x55ffb8257e40 2024-09-19T04:32:45.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c1ad070 con 0x55ffb8263020 2024-09-19T04:32:45.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c1abc00 con 0x55ffb8258760 2024-09-19T04:32:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb82835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f61400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb8261ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb82835d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59616/0 (socket says 172.21.15.107:59616) 2024-09-19T04:32:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f5be00640 1 -- 172.21.15.107:0/963132731 learned_addr learned my addr 172.21.15.107:0/963132731 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f61400640 1 --2- 172.21.15.107:0/963132731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8283b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f5be00640 1 -- 172.21.15.107:0/963132731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 msgr2=0x55ffb8283b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f5be00640 1 --2- 172.21.15.107:0/963132731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8283b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f5be00640 1 -- 172.21.15.107:0/963132731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 msgr2=0x55ffb8261ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f5be00640 1 --2- 172.21.15.107:0/963132731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb8261ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f5be00640 1 -- 172.21.15.107:0/963132731 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb8255060 con 0x55ffb8258760 2024-09-19T04:32:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.221+0000 7f0f61400640 1 --2- 172.21.15.107:0/963132731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8283b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.222+0000 7f0f5be00640 1 --2- 172.21.15.107:0/963132731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb82835d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802aa00 tx=0x7f0f48005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff64d8286df42f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.222+0000 7f0f4d200640 1 -- 172.21.15.107:0/963132731 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48035070 con 0x55ffb8258760 2024-09-19T04:32:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.222+0000 7f0f4d200640 1 -- 172.21.15.107:0/963132731 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f48040070 con 0x55ffb8258760 2024-09-19T04:32:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.222+0000 7f0f4d200640 1 -- 172.21.15.107:0/963132731 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48052070 con 0x55ffb8258760 2024-09-19T04:32:45.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:44 smithi145 ceph-mon[35096]: pgmap v387: 97 pgs: 97 active+clean; 365 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-09-19T04:32:45.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:44 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:32:45.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.223+0000 7f0f65e6f740 1 -- 172.21.15.107:0/963132731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 msgr2=0x55ffb82835d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:45.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.223+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/963132731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb82835d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802aa00 tx=0x7f0f48005b50 comp rx=0 tx=0).stop 2024-09-19T04:32:45.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.223+0000 7f0f65e6f740 1 -- 172.21.15.107:0/963132731 shutdown_connections 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.223+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/963132731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb8283b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.223+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/963132731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb82835d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.223+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/963132731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb8261ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.223+0000 7f0f65e6f740 1 -- 172.21.15.107:0/963132731 >> 172.21.15.107:0/963132731 conn(0x55ffb82543d0 msgr2=0x55ffb82617c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.223+0000 7f0f65e6f740 1 -- 172.21.15.107:0/963132731 shutdown_connections 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.224+0000 7f0f65e6f740 1 -- 172.21.15.107:0/963132731 wait complete. 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.224+0000 7f0f65e6f740 1 -- start start 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.224+0000 7f0f65e6f740 1 Processor -- start 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb826fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb8270180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f65e6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb82e55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c1ac9a0 con 0x55ffb8257e40 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c1ac2d0 con 0x55ffb8263020 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f65e6f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c1a9500 con 0x55ffb8258760 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f65e6f740 10 client.0 _ll_register_callbacks cb 0x7f0f48009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f61400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb826fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f61400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb826fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43342/0 (socket says 172.21.15.107:43342) 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb8270180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f61400640 1 -- 172.21.15.107:0/455650350 learned_addr learned my addr 172.21.15.107:0/455650350 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f61400640 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb82e55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f5be00640 1 -- 172.21.15.107:0/455650350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 msgr2=0x55ffb82e55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.225+0000 7f0f5be00640 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb82e55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.226+0000 7f0f5be00640 1 -- 172.21.15.107:0/455650350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 msgr2=0x55ffb826fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.226+0000 7f0f5be00640 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb826fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.226+0000 7f0f5be00640 1 -- 172.21.15.107:0/455650350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb8253c80 con 0x55ffb8258760 2024-09-19T04:32:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.226+0000 7f0f5be00640 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb8270180 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802a9d0 tx=0x7f0f48005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a3b00f446b9a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.226+0000 7f0f42000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:32:45.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.226+0000 7f0f42000640 1 -- 172.21.15.107:0/455650350 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48052030 con 0x55ffb8258760 2024-09-19T04:32:45.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.226+0000 7f0f65e6f740 1 -- 172.21.15.107:0/455650350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f48040070 con 0x55ffb8258760 2024-09-19T04:32:45.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.226+0000 7f0f42000640 1 -- 172.21.15.107:0/455650350 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f0f48003800 con 0x55ffb8258760 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f42000640 1 -- 172.21.15.107:0/455650350 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0f48067050 con 0x55ffb8258760 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f65e6f740 10 client.27430 fetch_fsmap learned FSMap version 49 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f65e6f740 1 -- 172.21.15.107:0/455650350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ffb82e6ae0 con 0x55ffb8258760 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f42000640 1 -- 172.21.15.107:0/455650350 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f48040070 con 0x55ffb8258760 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f42000640 1 -- 172.21.15.107:0/455650350 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f0f48045470 con 0x55ffb8258760 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f65e6f740 10 client.27430 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f65e6f740 10 client.27430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f65e6f740 20 client.27430 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ffb82e7120 0x55ffb82e9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f65e6f740 4 client.27430 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f65e6f740 1 -- 172.21.15.107:0/455650350 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffb81c5e60 con 0x55ffb82e7120 2024-09-19T04:32:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.227+0000 7f0f61400640 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ffb82e7120 0x55ffb82e9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f61400640 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ffb82e7120 0x55ffb82e9540 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf8259cfafc71c server_cookie=84afa1923a74bdd9 in_seq=0 out_seq=0 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f42000640 10 client.27430 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f42000640 1 -- 172.21.15.107:0/455650350 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffb81c5e60 con 0x55ffb82e7120 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f42000640 10 client.27430 handle_command_reply: tid=0 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f65e6f740 1 client.27430 shutdown 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f65e6f740 1 -- 172.21.15.107:0/455650350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 msgr2=0x55ffb8270180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb8270180 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802a9d0 tx=0x7f0f48005c90 comp rx=0 tx=0).stop 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f65e6f740 1 -- 172.21.15.107:0/455650350 shutdown_connections 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55ffb82e7120 0x55ffb82e9540 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffb8263020 0x55ffb82e55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ffb8258760 0x55ffb8270180 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f65e6f740 1 --2- 172.21.15.107:0/455650350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ffb8257e40 0x55ffb826fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f65e6f740 1 -- 172.21.15.107:0/455650350 >> 172.21.15.107:0/455650350 conn(0x55ffb82543d0 msgr2=0x55ffb8253490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:45.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.228+0000 7f0f42000640 0 client.27430 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:45.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.229+0000 7f0f65e6f740 1 -- 172.21.15.107:0/455650350 shutdown_connections 2024-09-19T04:32:45.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.229+0000 7f0f65e6f740 1 -- 172.21.15.107:0/455650350 wait complete. 2024-09-19T04:32:45.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.229+0000 7f0f65e6f740 20 client.27430 trim_cache size 0 max 16384 2024-09-19T04:32:45.233 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:32:45.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.229+0000 7f0f61e00640 1 -- 172.21.15.107:0/849903500 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0f440a33b0 msgr2=0x7f0f440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:45.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.229+0000 7f0f61e00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0f440a33b0 0x7f0f440a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55ffb8255230 tx=0x7f0f50098040 comp rx=0 tx=0).stop 2024-09-19T04:32:45.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.229+0000 7f0f61e00640 1 -- 172.21.15.107:0/849903500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c103e30 msgr2=0x7f0f5c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:45.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.229+0000 7f0f61e00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c103e30 0x7f0f5c19a4d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000e9f0 tx=0x7f0f5000edf0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.229+0000 7f0f61e00640 1 -- 172.21.15.107:0/849903500 shutdown_connections 2024-09-19T04:32:45.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.230+0000 7f0f61e00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f0f440a33b0 0x7f0f440a5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.230+0000 7f0f61e00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f5c108f60 0x7f0f5c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.230+0000 7f0f61e00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f5c1047e0 0x7f0f5c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.230+0000 7f0f61e00640 1 --2- 172.21.15.107:0/849903500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c103e30 0x7f0f5c19a4d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:45.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.230+0000 7f0f61e00640 1 -- 172.21.15.107:0/849903500 >> 172.21.15.107:0/849903500 conn(0x7f0f5c0ff910 msgr2=0x7f0f5c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:45.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.230+0000 7f0f61e00640 1 -- 172.21.15.107:0/849903500 shutdown_connections 2024-09-19T04:32:45.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:45.230+0000 7f0f61e00640 1 -- 172.21.15.107:0/849903500 wait complete. 2024-09-19T04:32:45.728 INFO:tasks.workunit.client.0.smithi107.stdout: 1 427365 2.97 MB/sec execute 543 sec latency 169.563 ms 2024-09-19T04:32:45.782 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:45 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:32:45.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:45 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:32:45.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:45 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:32:46.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:45 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:32:46.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:45 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:32:46.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:45 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:32:46.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:45 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:32:46.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:45 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:32:46.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:45 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:32:46.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 427365 2.96 MB/sec execute 544 sec latency 1169.736 ms 2024-09-19T04:32:47.103 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:32:47.104 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:32:47.155 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:46 smithi107 ceph-mon[30659]: pgmap v388: 97 pgs: 97 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-19T04:32:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:46 smithi115 ceph-mon[35477]: pgmap v388: 97 pgs: 97 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-19T04:32:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:46 smithi145 ceph-mon[35096]: pgmap v388: 97 pgs: 97 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-19T04:32:47.477 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:47.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 427365 2.96 MB/sec execute 545 sec latency 2169.857 ms 2024-09-19T04:32:48.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.210+0000 7fd697a00640 1 -- 172.21.15.107:0/4139405463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69010e3c0 msgr2=0x7fd690110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:48.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.210+0000 7fd697a00640 1 --2- 172.21.15.107:0/4139405463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69010e3c0 0x7fd690110810 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd688009900 tx=0x7fd68802a490 comp rx=0 tx=0).stop 2024-09-19T04:32:48.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.211+0000 7fd697a00640 1 -- 172.21.15.107:0/4139405463 shutdown_connections 2024-09-19T04:32:48.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.211+0000 7fd697a00640 1 --2- 172.21.15.107:0/4139405463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69010e3c0 0x7fd690110810 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.211+0000 7fd697a00640 1 --2- 172.21.15.107:0/4139405463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd690074f40 0x7fd69010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.211+0000 7fd697a00640 1 --2- 172.21.15.107:0/4139405463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd690074620 0x7fd690074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.211+0000 7fd697a00640 1 -- 172.21.15.107:0/4139405463 >> 172.21.15.107:0/4139405463 conn(0x7fd6900fd8c0 msgr2=0x7fd6900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.211+0000 7fd697a00640 1 -- 172.21.15.107:0/4139405463 shutdown_connections 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.211+0000 7fd697a00640 1 -- 172.21.15.107:0/4139405463 wait complete. 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.212+0000 7fd697a00640 1 Processor -- start 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.212+0000 7fd697a00640 1 -- start start 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.212+0000 7fd697a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd690074620 0x7fd69019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.212+0000 7fd697a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd690074f40 0x7fd69019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.212+0000 7fd697a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69010e3c0 0x7fd69019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.212+0000 7fd697a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901141c0 con 0x7fd69010e3c0 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.212+0000 7fd697a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd690114040 con 0x7fd690074f40 2024-09-19T04:32:48.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.212+0000 7fd697a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd690114340 con 0x7fd690074620 2024-09-19T04:32:48.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd695a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69010e3c0 0x7fd69019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd695000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd690074620 0x7fd69019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd695000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd690074620 0x7fd69019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59654/0 (socket says 172.21.15.107:59654) 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd695000640 1 -- 172.21.15.107:0/1167317149 learned_addr learned my addr 172.21.15.107:0/1167317149 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd68fe00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd690074f40 0x7fd69019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd68fe00640 1 -- 172.21.15.107:0/1167317149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd690074620 msgr2=0x7fd69019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd68fe00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd690074620 0x7fd69019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd68fe00640 1 -- 172.21.15.107:0/1167317149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69010e3c0 msgr2=0x7fd69019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd68fe00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69010e3c0 0x7fd69019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd68fe00640 1 -- 172.21.15.107:0/1167317149 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69019f620 con 0x7fd690074f40 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd695000640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd690074620 0x7fd69019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd68fe00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd690074f40 0x7fd69019abb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00d720 tx=0x7fd67c00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.213+0000 7fd68d600640 1 -- 172.21.15.107:0/1167317149 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c017070 con 0x7fd690074f40 2024-09-19T04:32:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.214+0000 7fd697a00640 1 -- 172.21.15.107:0/1167317149 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69019f910 con 0x7fd690074f40 2024-09-19T04:32:48.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.214+0000 7fd68d600640 1 -- 172.21.15.107:0/1167317149 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c00de30 con 0x7fd690074f40 2024-09-19T04:32:48.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.214+0000 7fd68d600640 1 -- 172.21.15.107:0/1167317149 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0047a0 con 0x7fd690074f40 2024-09-19T04:32:48.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.214+0000 7fd697a00640 1 -- 172.21.15.107:0/1167317149 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6901a71f0 con 0x7fd690074f40 2024-09-19T04:32:48.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.217+0000 7fd69a43d740 1 Processor -- start 2024-09-19T04:32:48.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.217+0000 7fd69a43d740 1 -- start start 2024-09-19T04:32:48.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.217+0000 7fd68d600640 1 -- 172.21.15.107:0/1167317149 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fd67c0099d0 con 0x7fd690074f40 2024-09-19T04:32:48.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.217+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba9524ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.217+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba95253e0 0x55bba952f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.217+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 0x55bba9532090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.217+0000 7fd69a43d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bba948b290 con 0x55bba952fca0 2024-09-19T04:32:48.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.217+0000 7fd69a43d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bba9432b40 con 0x55bba9524ac0 2024-09-19T04:32:48.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.217+0000 7fd69a43d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bba9487250 con 0x55bba95253e0 2024-09-19T04:32:48.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.217+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 0x55bba9532090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.218+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 0x55bba9532090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43378/0 (socket says 172.21.15.107:43378) 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.218+0000 7fd68fe00640 1 -- 172.21.15.107:0/1862646498 learned_addr learned my addr 172.21.15.107:0/1862646498 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.218+0000 7fd695000640 1 --2- 172.21.15.107:0/1862646498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba95253e0 0x55bba952f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.218+0000 7fd695a00640 1 --2- 172.21.15.107:0/1862646498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba9524ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.218+0000 7fd695000640 1 -- 172.21.15.107:0/1862646498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 msgr2=0x55bba9524ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.218+0000 7fd695000640 1 --2- 172.21.15.107:0/1862646498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba9524ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.218+0000 7fd695000640 1 -- 172.21.15.107:0/1862646498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 msgr2=0x55bba9532090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.218+0000 7fd695000640 1 --2- 172.21.15.107:0/1862646498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 0x55bba9532090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.218+0000 7fd695000640 1 -- 172.21.15.107:0/1862646498 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bba91fc6a0 con 0x55bba95253e0 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.218+0000 7fd68fe00640 1 --2- 172.21.15.107:0/1862646498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 0x55bba9532090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.219+0000 7fd695000640 1 --2- 172.21.15.107:0/1862646498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba95253e0 0x55bba952f760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd680011870 tx=0x7fd680011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52c15c3dde19bf68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:48.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.219+0000 7fd68d600640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6740a33b0 0x7fd6740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.219+0000 7fd686000640 1 -- 172.21.15.107:0/1862646498 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800112f0 con 0x55bba95253e0 2024-09-19T04:32:48.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.219+0000 7fd686000640 1 -- 172.21.15.107:0/1862646498 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680003e50 con 0x55bba95253e0 2024-09-19T04:32:48.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.219+0000 7fd686000640 1 -- 172.21.15.107:0/1862646498 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800044d0 con 0x55bba95253e0 2024-09-19T04:32:48.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.219+0000 7fd68d600640 1 -- 172.21.15.107:0/1167317149 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fd67c0cf8c0 con 0x7fd690074f40 2024-09-19T04:32:48.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.219+0000 7fd695a00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6740a33b0 0x7fd6740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.220+0000 7fd69a43d740 1 -- 172.21.15.107:0/1862646498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba95253e0 msgr2=0x55bba952f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:48.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.220+0000 7fd69a43d740 1 --2- 172.21.15.107:0/1862646498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba95253e0 0x55bba952f760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd680011870 tx=0x7fd680011d30 comp rx=0 tx=0).stop 2024-09-19T04:32:48.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.220+0000 7fd69a43d740 1 -- 172.21.15.107:0/1862646498 shutdown_connections 2024-09-19T04:32:48.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.220+0000 7fd69a43d740 1 --2- 172.21.15.107:0/1862646498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 0x55bba9532090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.220+0000 7fd69a43d740 1 --2- 172.21.15.107:0/1862646498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba95253e0 0x55bba952f760 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.220+0000 7fd69a43d740 1 --2- 172.21.15.107:0/1862646498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba9524ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.220+0000 7fd69a43d740 1 -- 172.21.15.107:0/1862646498 >> 172.21.15.107:0/1862646498 conn(0x55bba9521050 msgr2=0x55bba9521430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:48.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.220+0000 7fd695a00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6740a33b0 0x7fd6740a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd688031550 tx=0x7fd6880046c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:48.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.220+0000 7fd69a43d740 1 -- 172.21.15.107:0/1862646498 shutdown_connections 2024-09-19T04:32:48.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.220+0000 7fd69a43d740 1 -- 172.21.15.107:0/1862646498 wait complete. 2024-09-19T04:32:48.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.221+0000 7fd69a43d740 1 -- start start 2024-09-19T04:32:48.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.221+0000 7fd69a43d740 1 Processor -- start 2024-09-19T04:32:48.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9524ac0 0x55bba952f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba95253e0 0x55bba952c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 0x55bba952ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd69a43d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bba9532910 con 0x55bba952fca0 2024-09-19T04:32:48.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd69a43d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bba9445fa0 con 0x55bba95253e0 2024-09-19T04:32:48.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd69a43d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bba9532a90 con 0x55bba9524ac0 2024-09-19T04:32:48.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd69a43d740 10 client.0 _ll_register_callbacks cb 0x55bba9199ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:48.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba95253e0 0x55bba952c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd695a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 0x55bba952ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba95253e0 0x55bba952c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:46154/0 (socket says 172.21.15.107:46154) 2024-09-19T04:32:48.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.222+0000 7fd68fe00640 1 -- 172.21.15.107:0/4225180650 learned_addr learned my addr 172.21.15.107:0/4225180650 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:48.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.223+0000 7fd695000640 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9524ac0 0x55bba952f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.223+0000 7fd68fe00640 1 -- 172.21.15.107:0/4225180650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9524ac0 msgr2=0x55bba952f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:48.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.223+0000 7fd68fe00640 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9524ac0 0x55bba952f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.223+0000 7fd68fe00640 1 -- 172.21.15.107:0/4225180650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 msgr2=0x55bba952ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:48.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.223+0000 7fd68fe00640 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 0x55bba952ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.223+0000 7fd68fe00640 1 -- 172.21.15.107:0/4225180650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bba952d4c0 con 0x55bba95253e0 2024-09-19T04:32:48.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.223+0000 7fd68fe00640 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba95253e0 0x55bba952c780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0d52d0 tx=0x7fd67c09a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a8abfd52c481e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:48.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.223+0000 7fd67aa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:32:48.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.224+0000 7fd67aa00640 1 -- 172.21.15.107:0/4225180650 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c084e60 con 0x55bba95253e0 2024-09-19T04:32:48.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.224+0000 7fd67aa00640 1 -- 172.21.15.107:0/4225180650 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fd67c0cd250 con 0x55bba95253e0 2024-09-19T04:32:48.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.224+0000 7fd69a43d740 1 -- 172.21.15.107:0/4225180650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd680003e50 con 0x55bba95253e0 2024-09-19T04:32:48.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.224+0000 7fd67aa00640 1 -- 172.21.15.107:0/4225180650 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd67c09b6a0 con 0x55bba95253e0 2024-09-19T04:32:48.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.224+0000 7fd69a43d740 10 client.27500 fetch_fsmap learned FSMap version 49 2024-09-19T04:32:48.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.224+0000 7fd69a43d740 1 -- 172.21.15.107:0/4225180650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55bba953bd80 con 0x55bba95253e0 2024-09-19T04:32:48.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.225+0000 7fd67aa00640 1 -- 172.21.15.107:0/4225180650 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680003e50 con 0x55bba95253e0 2024-09-19T04:32:48.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.225+0000 7fd67aa00640 1 -- 172.21.15.107:0/4225180650 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7fd67c0860b0 con 0x55bba95253e0 2024-09-19T04:32:48.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.225+0000 7fd69a43d740 10 client.27500 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:32:48.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.225+0000 7fd69a43d740 10 client.27500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:48.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.225+0000 7fd69a43d740 20 client.27500 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:48.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.225+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55bba953d240 0x55bba953f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.225+0000 7fd69a43d740 4 client.27500 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:48.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.225+0000 7fd69a43d740 1 -- 172.21.15.107:0/4225180650 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bba9498800 con 0x55bba953d240 2024-09-19T04:32:48.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.226+0000 7fd695000640 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55bba953d240 0x55bba953f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.227+0000 7fd695000640 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55bba953d240 0x55bba953f6e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8029f79e05d9bcae server_cookie=ef910928f7a068b9 in_seq=0 out_seq=0 2024-09-19T04:32:48.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.227+0000 7fd67aa00640 10 client.27500 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:48.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.228+0000 7fd67aa00640 1 -- 172.21.15.107:0/4225180650 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bba9498800 con 0x55bba953d240 2024-09-19T04:32:48.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.228+0000 7fd67aa00640 10 client.27500 handle_command_reply: tid=0 2024-09-19T04:32:48.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.228+0000 7fd69a43d740 1 client.27500 shutdown 2024-09-19T04:32:48.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.229+0000 7fd69a43d740 1 -- 172.21.15.107:0/4225180650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba95253e0 msgr2=0x55bba952c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:48.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.229+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba95253e0 0x55bba952c780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0d52d0 tx=0x7fd67c09a070 comp rx=0 tx=0).stop 2024-09-19T04:32:48.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.229+0000 7fd69a43d740 1 -- 172.21.15.107:0/4225180650 shutdown_connections 2024-09-19T04:32:48.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.229+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55bba953d240 0x55bba953f6e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.229+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba952fca0 0x55bba952ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.229+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba95253e0 0x55bba952c780 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.229+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4225180650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9524ac0 0x55bba952f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.229+0000 7fd67aa00640 0 client.27500 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:48.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.229+0000 7fd69a43d740 1 -- 172.21.15.107:0/4225180650 >> 172.21.15.107:0/4225180650 conn(0x55bba9521050 msgr2=0x55bba95201c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:48.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.230+0000 7fd69a43d740 1 -- 172.21.15.107:0/4225180650 shutdown_connections 2024-09-19T04:32:48.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.230+0000 7fd69a43d740 1 -- 172.21.15.107:0/4225180650 wait complete. 2024-09-19T04:32:48.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.231+0000 7fd69a43d740 20 client.27500 trim_cache size 0 max 16384 2024-09-19T04:32:48.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.257+0000 7fd69a43d740 1 Processor -- start 2024-09-19T04:32:48.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.257+0000 7fd69a43d740 1 -- start start 2024-09-19T04:32:48.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.257+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 0x55bba952e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.257+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba9529a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.257+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 0x55bba9529f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.257+0000 7fd69a43d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901ab750 con 0x55bba95253e0 2024-09-19T04:32:48.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.257+0000 7fd69a43d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901ad2c0 con 0x55bba9524ac0 2024-09-19T04:32:48.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.257+0000 7fd69a43d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901abe50 con 0x55bba9521050 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.258+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba9529a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.258+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba9529a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:46158/0 (socket says 172.21.15.107:46158) 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.258+0000 7fd68fe00640 1 -- 172.21.15.107:0/2332666889 learned_addr learned my addr 172.21.15.107:0/2332666889 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.258+0000 7fd695a00640 1 --2- 172.21.15.107:0/2332666889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 0x55bba9529f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.258+0000 7fd695000640 1 --2- 172.21.15.107:0/2332666889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 0x55bba952e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.258+0000 7fd68fe00640 1 -- 172.21.15.107:0/2332666889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 msgr2=0x55bba952e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.258+0000 7fd68fe00640 1 --2- 172.21.15.107:0/2332666889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 0x55bba952e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.258+0000 7fd68fe00640 1 -- 172.21.15.107:0/2332666889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 msgr2=0x55bba9529f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.258+0000 7fd68fe00640 1 --2- 172.21.15.107:0/2332666889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 0x55bba9529f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.258+0000 7fd68fe00640 1 -- 172.21.15.107:0/2332666889 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bba9524840 con 0x55bba9524ac0 2024-09-19T04:32:48.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.259+0000 7fd68fe00640 1 --2- 172.21.15.107:0/2332666889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba9529a00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd67c098530 tx=0x7fd67c005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b14c246d37b75c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:48.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.259+0000 7fd685600640 1 -- 172.21.15.107:0/2332666889 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c084e60 con 0x55bba9524ac0 2024-09-19T04:32:48.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.259+0000 7fd685600640 1 -- 172.21.15.107:0/2332666889 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c004220 con 0x55bba9524ac0 2024-09-19T04:32:48.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.259+0000 7fd685600640 1 -- 172.21.15.107:0/2332666889 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c09b4c0 con 0x55bba9524ac0 2024-09-19T04:32:48.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.260+0000 7fd69a43d740 1 -- 172.21.15.107:0/2332666889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 msgr2=0x55bba9529a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:48.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.260+0000 7fd69a43d740 1 --2- 172.21.15.107:0/2332666889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba9529a00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd67c098530 tx=0x7fd67c005020 comp rx=0 tx=0).stop 2024-09-19T04:32:48.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.260+0000 7fd69a43d740 1 -- 172.21.15.107:0/2332666889 shutdown_connections 2024-09-19T04:32:48.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.260+0000 7fd69a43d740 1 --2- 172.21.15.107:0/2332666889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 0x55bba9529f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.260+0000 7fd69a43d740 1 --2- 172.21.15.107:0/2332666889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba9529a00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.260+0000 7fd69a43d740 1 --2- 172.21.15.107:0/2332666889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 0x55bba952e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.260+0000 7fd69a43d740 1 -- 172.21.15.107:0/2332666889 >> 172.21.15.107:0/2332666889 conn(0x55bba952ce20 msgr2=0x55bba952dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:48.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.260+0000 7fd69a43d740 1 -- 172.21.15.107:0/2332666889 shutdown_connections 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.260+0000 7fd69a43d740 1 -- 172.21.15.107:0/2332666889 wait complete. 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.260+0000 7fd69a43d740 1 -- start start 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd69a43d740 1 Processor -- start 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 0x55bba954a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba954ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd69a43d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 0x55bba95a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd69a43d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901acbf0 con 0x55bba95253e0 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd69a43d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901ac520 con 0x55bba9524ac0 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd69a43d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901a9750 con 0x55bba9521050 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd69a43d740 10 client.0 _ll_register_callbacks cb 0x7fd67c0ccf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd695000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 0x55bba954a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd695a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 0x55bba95a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.261+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba954ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.262+0000 7fd695a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 0x55bba95a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43404/0 (socket says 172.21.15.107:43404) 2024-09-19T04:32:48.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.262+0000 7fd695a00640 1 -- 172.21.15.107:0/4139697393 learned_addr learned my addr 172.21.15.107:0/4139697393 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:48.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.262+0000 7fd695a00640 1 -- 172.21.15.107:0/4139697393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 msgr2=0x55bba954a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:48.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.262+0000 7fd695a00640 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 0x55bba954a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.262+0000 7fd695a00640 1 -- 172.21.15.107:0/4139697393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 msgr2=0x55bba954ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:48.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.262+0000 7fd695a00640 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba954ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.262+0000 7fd695a00640 1 -- 172.21.15.107:0/4139697393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bba95a4570 con 0x55bba95253e0 2024-09-19T04:32:48.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.262+0000 7fd695000640 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 0x55bba954a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:48.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.262+0000 7fd695a00640 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 0x55bba95a3ed0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd688033000 tx=0x7fd688009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3498ba3a5b727da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:48.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.262+0000 7fd67aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:32:48.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.263+0000 7fd67aa00640 1 -- 172.21.15.107:0/4139697393 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688035040 con 0x55bba95253e0 2024-09-19T04:32:48.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.263+0000 7fd67aa00640 1 -- 172.21.15.107:0/4139697393 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fd688003800 con 0x55bba95253e0 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.263+0000 7fd69a43d740 1 -- 172.21.15.107:0/4139697393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67c004220 con 0x55bba95253e0 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.263+0000 7fd67aa00640 1 -- 172.21.15.107:0/4139697393 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd688003a00 con 0x55bba95253e0 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.263+0000 7fd69a43d740 10 client.17460 fetch_fsmap learned FSMap version 49 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.263+0000 7fd69a43d740 1 -- 172.21.15.107:0/4139697393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55bba95ab9a0 con 0x55bba95253e0 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.263+0000 7fd67aa00640 1 -- 172.21.15.107:0/4139697393 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c004220 con 0x55bba95253e0 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.264+0000 7fd67aa00640 1 -- 172.21.15.107:0/4139697393 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7fd68806d070 con 0x55bba95253e0 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.264+0000 7fd69a43d740 10 client.17460 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.264+0000 7fd69a43d740 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.264+0000 7fd69a43d740 20 client.17460 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.264+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55bba952fca0 0x55bba95ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.264+0000 7fd69a43d740 4 client.17460 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.264+0000 7fd69a43d740 1 -- 172.21.15.107:0/4139697393 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bba9492d10 con 0x55bba952fca0 2024-09-19T04:32:48.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.264+0000 7fd695000640 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55bba952fca0 0x55bba95ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:48.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.265+0000 7fd695000640 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55bba952fca0 0x55bba95ae1e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67a9cabdd9fc407 server_cookie=795e1227387d3aa8 in_seq=0 out_seq=0 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.265+0000 7fd67aa00640 10 client.17460 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd67aa00640 1 -- 172.21.15.107:0/4139697393 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bba9492d10 con 0x55bba952fca0 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd67aa00640 10 client.17460 handle_command_reply: tid=0 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 client.17460 shutdown 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 -- 172.21.15.107:0/4139697393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 msgr2=0x55bba95a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 0x55bba95a3ed0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd688033000 tx=0x7fd688009260 comp rx=0 tx=0).stop 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 -- 172.21.15.107:0/4139697393 shutdown_connections 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55bba952fca0 0x55bba95ae1e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bba95253e0 0x55bba95a3ed0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba9524ac0 0x55bba954ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 --2- 172.21.15.107:0/4139697393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bba9521050 0x55bba954a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd67aa00640 0 client.17460 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:48.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 -- 172.21.15.107:0/4139697393 >> 172.21.15.107:0/4139697393 conn(0x55bba952ce20 msgr2=0x55bba953b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:48.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 -- 172.21.15.107:0/4139697393 shutdown_connections 2024-09-19T04:32:48.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.266+0000 7fd69a43d740 1 -- 172.21.15.107:0/4139697393 wait complete. 2024-09-19T04:32:48.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd69a43d740 20 client.17460 trim_cache size 0 max 16384 2024-09-19T04:32:48.270 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:32:48.270 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:32:48.270 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "4d0f11af-7cce-477f-b507-91a8046c7634", 2024-09-19T04:32:48.270 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:32:48.270 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:32:48.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd697a00640 1 -- 172.21.15.107:0/1167317149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6740a33b0 msgr2=0x7fd6740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:48.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd697a00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6740a33b0 0x7fd6740a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd688031550 tx=0x7fd6880046c0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd697a00640 1 -- 172.21.15.107:0/1167317149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd690074f40 msgr2=0x7fd69019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:48.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd697a00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd690074f40 0x7fd69019abb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00d720 tx=0x7fd67c00dbe0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd697a00640 1 -- 172.21.15.107:0/1167317149 shutdown_connections 2024-09-19T04:32:48.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd697a00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd6740a33b0 0x7fd6740a5870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd697a00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69010e3c0 0x7fd69019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd697a00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd690074f40 0x7fd69019abb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd697a00640 1 --2- 172.21.15.107:0/1167317149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd690074620 0x7fd69019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:48.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.267+0000 7fd697a00640 1 -- 172.21.15.107:0/1167317149 >> 172.21.15.107:0/1167317149 conn(0x7fd6900fd8c0 msgr2=0x7fd6900fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:48.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.268+0000 7fd697a00640 1 -- 172.21.15.107:0/1167317149 shutdown_connections 2024-09-19T04:32:48.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:48.268+0000 7fd697a00640 1 -- 172.21.15.107:0/1167317149 wait complete. 2024-09-19T04:32:48.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 427368 2.95 MB/sec execute 546 sec latency 2776.355 ms 2024-09-19T04:32:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:48 smithi115 ceph-mon[35477]: pgmap v389: 97 pgs: 97 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-09-19T04:32:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:48 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:48 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:48 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:48 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:48 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:48 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:49.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:48 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:48 smithi145 ceph-mon[35096]: pgmap v389: 97 pgs: 97 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-09-19T04:32:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:48 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:48 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:48 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:48 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:48 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:48 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:48 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:48 smithi107 ceph-mon[30659]: pgmap v389: 97 pgs: 97 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-09-19T04:32:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:48 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:49.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:48 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:49.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:48 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:49.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:48 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:49.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:48 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:49.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:48 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:49.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:48 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:49.331 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:32:49.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 429282 2.97 MB/sec execute 547 sec latency 493.026 ms 2024-09-19T04:32:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:32:50.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:32:50.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:50.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:50.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:50.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:50.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:32:50.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:49 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.125 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:32:50.125 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.126 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:50.126 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:50.126 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:50.126 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.126 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.126 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:50.126 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:32:50.126 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.128 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.128 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.128 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:49 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:32:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:32:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:49 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:32:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.309+0000 7f24ce000640 1 -- 172.21.15.107:0/940908342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24c8102790 msgr2=0x7f24c81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.309+0000 7f24ce000640 1 --2- 172.21.15.107:0/940908342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24c8102790 0x7f24c81067e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009900 tx=0x7f24b402a410 comp rx=0 tx=0).stop 2024-09-19T04:32:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.309+0000 7f24ce000640 1 -- 172.21.15.107:0/940908342 shutdown_connections 2024-09-19T04:32:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.309+0000 7f24ce000640 1 --2- 172.21.15.107:0/940908342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8106f10 0x7f24c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.309+0000 7f24ce000640 1 --2- 172.21.15.107:0/940908342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24c8102790 0x7f24c81067e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.309+0000 7f24ce000640 1 --2- 172.21.15.107:0/940908342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c8101de0 0x7f24c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.309+0000 7f24ce000640 1 -- 172.21.15.107:0/940908342 >> 172.21.15.107:0/940908342 conn(0x7f24c80fd8c0 msgr2=0x7f24c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.309+0000 7f24ce000640 1 -- 172.21.15.107:0/940908342 shutdown_connections 2024-09-19T04:32:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.309+0000 7f24ce000640 1 -- 172.21.15.107:0/940908342 wait complete. 2024-09-19T04:32:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.310+0000 7f24ce000640 1 Processor -- start 2024-09-19T04:32:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.310+0000 7f24ce000640 1 -- start start 2024-09-19T04:32:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24ce000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8101de0 0x7f24c819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24ce000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c8102790 0x7f24c819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24ce000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24c8106f10 0x7f24c819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24ce000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8111f70 con 0x7f24c8101de0 2024-09-19T04:32:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24ce000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8111df0 con 0x7f24c8102790 2024-09-19T04:32:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24ce000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f24c81120f0 con 0x7f24c8106f10 2024-09-19T04:32:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24c7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24c8106f10 0x7f24c819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24c7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24c8106f10 0x7f24c819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38106/0 (socket says 172.21.15.107:38106) 2024-09-19T04:32:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24c7400640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8101de0 0x7f24c819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24c7e00640 1 -- 172.21.15.107:0/289648107 learned_addr learned my addr 172.21.15.107:0/289648107 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.311+0000 7f24c6a00640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c8102790 0x7f24c819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24c7400640 1 -- 172.21.15.107:0/289648107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24c8106f10 msgr2=0x7f24c819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24c7400640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24c8106f10 0x7f24c819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24c7400640 1 -- 172.21.15.107:0/289648107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c8102790 msgr2=0x7f24c819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24c7400640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c8102790 0x7f24c819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24c7400640 1 -- 172.21.15.107:0/289648107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c819f600 con 0x7f24c8101de0 2024-09-19T04:32:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24c7e00640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24c8106f10 0x7f24c819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24c7400640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8101de0 0x7f24c819a650 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f24b800e9f0 tx=0x7f24b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24bfe00640 1 -- 172.21.15.107:0/289648107 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b800ccc0 con 0x7f24c8101de0 2024-09-19T04:32:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24bfe00640 1 -- 172.21.15.107:0/289648107 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b8010910 con 0x7f24c8101de0 2024-09-19T04:32:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24bfe00640 1 -- 172.21.15.107:0/289648107 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8018960 con 0x7f24c8101de0 2024-09-19T04:32:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24ce000640 1 -- 172.21.15.107:0/289648107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c819f8f0 con 0x7f24c8101de0 2024-09-19T04:32:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.312+0000 7f24ce000640 1 -- 172.21.15.107:0/289648107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c81a71d0 con 0x7f24c8101de0 2024-09-19T04:32:50.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.315+0000 7f24d0c08740 1 Processor -- start 2024-09-19T04:32:50.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.315+0000 7f24bfe00640 1 -- 172.21.15.107:0/289648107 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f24b8010ac0 con 0x7f24c8101de0 2024-09-19T04:32:50.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.315+0000 7f24d0c08740 1 -- start start 2024-09-19T04:32:50.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561378352b90 0x561378352f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613783534b0 0x56137835d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 0x561378360160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24d0c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5613782b93e0 con 0x56137835dd70 2024-09-19T04:32:50.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24d0c08740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561378260d80 con 0x5613783534b0 2024-09-19T04:32:50.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24d0c08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5613782b53a0 con 0x561378352b90 2024-09-19T04:32:50.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 0x561378360160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613783534b0 0x56137835d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24c7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561378352b90 0x561378352f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 0x561378360160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52112/0 (socket says 172.21.15.107:52112) 2024-09-19T04:32:50.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.316+0000 7f24c6a00640 1 -- 172.21.15.107:0/972646817 learned_addr learned my addr 172.21.15.107:0/972646817 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:50.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.317+0000 7f24c7400640 1 -- 172.21.15.107:0/972646817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561378352b90 msgr2=0x561378352f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.317+0000 7f24c7400640 1 --2- 172.21.15.107:0/972646817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561378352b90 0x561378352f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.317+0000 7f24c7400640 1 -- 172.21.15.107:0/972646817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 msgr2=0x561378360160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.317+0000 7f24c7400640 1 --2- 172.21.15.107:0/972646817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 0x561378360160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.317+0000 7f24c7400640 1 -- 172.21.15.107:0/972646817 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137827fae0 con 0x5613783534b0 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.317+0000 7f24c6a00640 1 --2- 172.21.15.107:0/972646817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 0x561378360160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.317+0000 7f24c7400640 1 --2- 172.21.15.107:0/972646817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613783534b0 0x56137835d830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f24b80cd800 tx=0x7f24b80d86c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27bdbf2c95adb46d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.317+0000 7f24b3e00640 1 -- 172.21.15.107:0/972646817 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b80e3070 con 0x5613783534b0 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.317+0000 7f24b3e00640 1 -- 172.21.15.107:0/972646817 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b80d09c0 con 0x5613783534b0 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.317+0000 7f24b3e00640 1 -- 172.21.15.107:0/972646817 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b80de900 con 0x5613783534b0 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24bfe00640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24ac0a33b0 0x7f24ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24bfe00640 1 -- 172.21.15.107:0/289648107 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f24b80d00f0 con 0x7f24c8101de0 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24c7e00640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24ac0a33b0 0x7f24ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24d0c08740 1 -- 172.21.15.107:0/972646817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613783534b0 msgr2=0x56137835d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24d0c08740 1 --2- 172.21.15.107:0/972646817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613783534b0 0x56137835d830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f24b80cd800 tx=0x7f24b80d86c0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24d0c08740 1 -- 172.21.15.107:0/972646817 shutdown_connections 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24d0c08740 1 --2- 172.21.15.107:0/972646817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 0x561378360160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24d0c08740 1 --2- 172.21.15.107:0/972646817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5613783534b0 0x56137835d830 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24d0c08740 1 --2- 172.21.15.107:0/972646817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561378352b90 0x561378352f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24d0c08740 1 -- 172.21.15.107:0/972646817 >> 172.21.15.107:0/972646817 conn(0x56137834f120 msgr2=0x56137834f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.318+0000 7f24d0c08740 1 -- 172.21.15.107:0/972646817 shutdown_connections 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.319+0000 7f24d0c08740 1 -- 172.21.15.107:0/972646817 wait complete. 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.319+0000 7f24c7e00640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24ac0a33b0 0x7f24ac0a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f24c00049d0 tx=0x7f24c0009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:50.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.319+0000 7f24d0c08740 1 -- start start 2024-09-19T04:32:50.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.319+0000 7f24d0c08740 1 Processor -- start 2024-09-19T04:32:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x56137835ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613783534b0 0x56137835b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56137835dd70 0x56137835b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24d0c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5613783609e0 con 0x5613783534b0 2024-09-19T04:32:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24d0c08740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5613782740f0 con 0x561378352b90 2024-09-19T04:32:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24d0c08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561378360b60 con 0x56137835dd70 2024-09-19T04:32:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24d0c08740 10 client.0 _ll_register_callbacks cb 0x5613782659e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x56137835ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x56137835ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38740/0 (socket says 172.21.15.107:38740) 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c7400640 1 -- 172.21.15.107:0/2688695197 learned_addr learned my addr 172.21.15.107:0/2688695197 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c6a00640 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613783534b0 0x56137835b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c7400640 1 -- 172.21.15.107:0/2688695197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56137835dd70 msgr2=0x56137835b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c7e00640 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56137835dd70 0x56137835b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c7400640 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56137835dd70 0x56137835b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c7400640 1 -- 172.21.15.107:0/2688695197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613783534b0 msgr2=0x56137835b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c7400640 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613783534b0 0x56137835b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c7400640 1 -- 172.21.15.107:0/2688695197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137802a3c0 con 0x561378352b90 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c6a00640 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613783534b0 0x56137835b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.320+0000 7f24c7e00640 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56137835dd70 0x56137835b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:32:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.321+0000 7f24c7400640 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x56137835ac80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f24b8098e10 tx=0x7f24b80d0370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ecebd2d32925f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.321+0000 7f24b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:32:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.321+0000 7f24b0c00640 1 -- 172.21.15.107:0/2688695197 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b80986e0 con 0x561378352b90 2024-09-19T04:32:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.321+0000 7f24b0c00640 1 -- 172.21.15.107:0/2688695197 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f24b80d3380 con 0x561378352b90 2024-09-19T04:32:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.321+0000 7f24d0c08740 1 -- 172.21.15.107:0/2688695197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b80d09c0 con 0x561378352b90 2024-09-19T04:32:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.321+0000 7f24b0c00640 1 -- 172.21.15.107:0/2688695197 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f24b809bda0 con 0x561378352b90 2024-09-19T04:32:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.321+0000 7f24d0c08740 10 client.27518 fetch_fsmap learned FSMap version 49 2024-09-19T04:32:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.321+0000 7f24d0c08740 1 -- 172.21.15.107:0/2688695197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5613783609e0 con 0x561378352b90 2024-09-19T04:32:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.321+0000 7f24b0c00640 1 -- 172.21.15.107:0/2688695197 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b80d09c0 con 0x561378352b90 2024-09-19T04:32:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.322+0000 7f24b0c00640 1 -- 172.21.15.107:0/2688695197 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f24b80cda80 con 0x561378352b90 2024-09-19T04:32:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.322+0000 7f24d0c08740 10 client.27518 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.322+0000 7f24d0c08740 10 client.27518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.322+0000 7f24d0c08740 20 client.27518 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.322+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56137836d840 0x56137836fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.322+0000 7f24d0c08740 4 client.27518 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.322+0000 7f24d0c08740 1 -- 172.21.15.107:0/2688695197 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613782c6950 con 0x56137836d840 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.322+0000 7f24c6a00640 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56137836d840 0x56137836fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.323+0000 7f24c6a00640 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56137836d840 0x56137836fc30 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81471a98c155118 server_cookie=ecac539518992cf9 in_seq=0 out_seq=0 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.323+0000 7f24b0c00640 10 client.27518 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.323+0000 7f24b0c00640 1 -- 172.21.15.107:0/2688695197 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613782c6950 con 0x56137836d840 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.323+0000 7f24b0c00640 10 client.27518 handle_command_reply: tid=0 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.323+0000 7f24d0c08740 1 client.27518 shutdown 2024-09-19T04:32:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 1 -- 172.21.15.107:0/2688695197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 msgr2=0x56137835ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x56137835ac80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f24b8098e10 tx=0x7f24b80d0370 comp rx=0 tx=0).stop 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 1 -- 172.21.15.107:0/2688695197 shutdown_connections 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x56137836d840 0x56137836fc30 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56137835dd70 0x56137835b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613783534b0 0x56137835b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24b0c00640 0 client.27518 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2688695197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x56137835ac80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 1 -- 172.21.15.107:0/2688695197 >> 172.21.15.107:0/2688695197 conn(0x56137834f120 msgr2=0x561378354e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 1 -- 172.21.15.107:0/2688695197 shutdown_connections 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 1 -- 172.21.15.107:0/2688695197 wait complete. 2024-09-19T04:32:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.324+0000 7f24d0c08740 20 client.27518 trim_cache size 0 max 16384 2024-09-19T04:32:50.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.342+0000 7f24d0c08740 1 Processor -- start 2024-09-19T04:32:50.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.342+0000 7f24d0c08740 1 -- start start 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561378352b90 0x5613783555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 0x561378359550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56137835dd70 0x56137835b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24d0c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24c81afb20 con 0x561378352b90 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24d0c08740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24c81b1690 con 0x56137835dd70 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24d0c08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f24c81b0220 con 0x5613783534b0 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 0x561378359550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24c7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56137835dd70 0x56137835b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24c7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561378352b90 0x5613783555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 0x561378359550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38122/0 (socket says 172.21.15.107:38122) 2024-09-19T04:32:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24c6a00640 1 -- 172.21.15.107:0/608862342 learned_addr learned my addr 172.21.15.107:0/608862342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:50.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24c7e00640 1 -- 172.21.15.107:0/608862342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 msgr2=0x561378359550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:50.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24c7e00640 1 --2- 172.21.15.107:0/608862342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 0x561378359550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24c7e00640 1 -- 172.21.15.107:0/608862342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561378352b90 msgr2=0x5613783555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:50.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24c7e00640 1 --2- 172.21.15.107:0/608862342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561378352b90 0x5613783555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.343+0000 7f24c7e00640 1 -- 172.21.15.107:0/608862342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137834ebc0 con 0x56137835dd70 2024-09-19T04:32:50.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.344+0000 7f24c7e00640 1 --2- 172.21.15.107:0/608862342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56137835dd70 0x56137835b350 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24c00123d0 tx=0x7f24c0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eab76b85b50e27fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.344+0000 7f24b3400640 1 -- 172.21.15.107:0/608862342 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c002d070 con 0x56137835dd70 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.344+0000 7f24b3400640 1 -- 172.21.15.107:0/608862342 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c0011e20 con 0x56137835dd70 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.344+0000 7f24b3400640 1 -- 172.21.15.107:0/608862342 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0028cf0 con 0x56137835dd70 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 -- 172.21.15.107:0/608862342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56137835dd70 msgr2=0x56137835b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 --2- 172.21.15.107:0/608862342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56137835dd70 0x56137835b350 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24c00123d0 tx=0x7f24c0017000 comp rx=0 tx=0).stop 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 -- 172.21.15.107:0/608862342 shutdown_connections 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 --2- 172.21.15.107:0/608862342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56137835dd70 0x56137835b350 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 --2- 172.21.15.107:0/608862342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 0x561378359550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 --2- 172.21.15.107:0/608862342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561378352b90 0x5613783555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 -- 172.21.15.107:0/608862342 >> 172.21.15.107:0/608862342 conn(0x56137834f120 msgr2=0x56137835ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:50.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 -- 172.21.15.107:0/608862342 shutdown_connections 2024-09-19T04:32:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 -- 172.21.15.107:0/608862342 wait complete. 2024-09-19T04:32:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 -- start start 2024-09-19T04:32:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.345+0000 7f24d0c08740 1 Processor -- start 2024-09-19T04:32:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x561378361f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 0x561378362490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24d0c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 0x5613783d7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24d0c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24c81b0fc0 con 0x56137835dd70 2024-09-19T04:32:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24d0c08740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24c81b08f0 con 0x561378352b90 2024-09-19T04:32:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24d0c08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f24c81adb20 con 0x5613783534b0 2024-09-19T04:32:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24d0c08740 10 client.0 _ll_register_callbacks cb 0x7f24b80d8170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:32:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x561378361f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24c7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x561378361f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38768/0 (socket says 172.21.15.107:38768) 2024-09-19T04:32:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24c7400640 1 -- 172.21.15.107:0/2398177099 learned_addr learned my addr 172.21.15.107:0/2398177099 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24c7e00640 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 0x5613783d7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.346+0000 7f24c6a00640 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 0x561378362490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24c7400640 1 -- 172.21.15.107:0/2398177099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 msgr2=0x561378362490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24c7400640 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 0x561378362490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24c7400640 1 -- 172.21.15.107:0/2398177099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 msgr2=0x5613783d7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24c7400640 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 0x5613783d7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24c7400640 1 -- 172.21.15.107:0/2398177099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613783d7770 con 0x561378352b90 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24c6a00640 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 0x561378362490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24c7400640 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x561378361f50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f24b809c3d0 tx=0x7f24b80d86c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1666719765bdb4ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24b0c00640 1 -- 172.21.15.107:0/2398177099 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b809bf80 con 0x561378352b90 2024-09-19T04:32:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24b0c00640 1 -- 172.21.15.107:0/2398177099 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f24b80cda80 con 0x561378352b90 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24d0c08740 1 -- 172.21.15.107:0/2398177099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c0011e20 con 0x561378352b90 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24b0c00640 1 -- 172.21.15.107:0/2398177099 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f24b809c210 con 0x561378352b90 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24d0c08740 10 client.27524 fetch_fsmap learned FSMap version 49 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.347+0000 7f24d0c08740 1 -- 172.21.15.107:0/2398177099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5613783d8920 con 0x561378352b90 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.348+0000 7f24b0c00640 1 -- 172.21.15.107:0/2398177099 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c0011e20 con 0x561378352b90 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.348+0000 7f24b0c00640 1 -- 172.21.15.107:0/2398177099 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f24b80b5150 con 0x561378352b90 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.348+0000 7f24d0c08740 10 client.27524 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.348+0000 7f24d0c08740 10 client.27524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.348+0000 7f24d0c08740 20 client.27524 populate_metadata read hostname 'smithi107' 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.348+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613783d8df0 0x5613783db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.348+0000 7f24d0c08740 4 client.27524 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.348+0000 7f24d0c08740 1 -- 172.21.15.107:0/2398177099 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5613782c0e60 con 0x5613783d8df0 2024-09-19T04:32:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.348+0000 7f24c6a00640 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613783d8df0 0x5613783db1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24c6a00640 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613783d8df0 0x5613783db1e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1162c226901e0d8 server_cookie=c8508a3b1e07841 in_seq=0 out_seq=0 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24b0c00640 10 client.27524 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24b0c00640 1 -- 172.21.15.107:0/2398177099 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5613782c0e60 con 0x5613783d8df0 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24b0c00640 10 client.27524 handle_command_reply: tid=0 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 client.27524 shutdown 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 -- 172.21.15.107:0/2398177099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 msgr2=0x561378361f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x561378361f50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f24b809c3d0 tx=0x7f24b80d86c0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 -- 172.21.15.107:0/2398177099 shutdown_connections 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x5613783d8df0 0x5613783db1e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56137835dd70 0x5613783d7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613783534b0 0x561378362490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 --2- 172.21.15.107:0/2398177099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561378352b90 0x561378361f50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24b0c00640 0 client.27524 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:32:50.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 -- 172.21.15.107:0/2398177099 >> 172.21.15.107:0/2398177099 conn(0x56137834f120 msgr2=0x56137835b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:50.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 -- 172.21.15.107:0/2398177099 shutdown_connections 2024-09-19T04:32:50.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 1 -- 172.21.15.107:0/2398177099 wait complete. 2024-09-19T04:32:50.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.349+0000 7f24d0c08740 20 client.27524 trim_cache size 0 max 16384 2024-09-19T04:32:50.356 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:32:50.356 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:32:50.356 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:32:50.357 INFO:teuthology.orchestra.run.smithi107.stdout: "4d0f11af-7cce-477f-b507-91a8046c7634": { 2024-09-19T04:32:50.357 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:32:50.357 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "4d0f11af-7cce-477f-b507-91a8046c7634", 2024-09-19T04:32:50.357 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:32:50.357 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:32:50.357 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:32:50.357 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:32:50.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 -- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24ac0a33b0 msgr2=0x7f24ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:50.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24ac0a33b0 0x7f24ac0a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f24c00049d0 tx=0x7f24c0009500 comp rx=0 tx=0).stop 2024-09-19T04:32:50.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 -- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8101de0 msgr2=0x7f24c819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:32:50.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8101de0 0x7f24c819a650 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f24b800e9f0 tx=0x7f24b800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 -- 172.21.15.107:0/289648107 shutdown_connections 2024-09-19T04:32:50.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f24ac0a33b0 0x7f24ac0a5870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24c8106f10 0x7f24c819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c8102790 0x7f24c819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 --2- 172.21.15.107:0/289648107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8101de0 0x7f24c819a650 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:32:50.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 -- 172.21.15.107:0/289648107 >> 172.21.15.107:0/289648107 conn(0x7f24c80fd8c0 msgr2=0x7f24c80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:32:50.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 -- 172.21.15.107:0/289648107 shutdown_connections 2024-09-19T04:32:50.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:32:50.350+0000 7f24ce000640 1 -- 172.21.15.107:0/289648107 wait complete. 2024-09-19T04:32:50.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 430295 2.97 MB/sec execute 548 sec latency 15.537 ms 2024-09-19T04:32:50.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:50 smithi107 ceph-mon[30659]: pgmap v390: 97 pgs: 97 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-09-19T04:32:51.097 INFO:tasks.cephfs.filesystem:scrub status for tag:4d0f11af-7cce-477f-b507-91a8046c7634 - {'path': '/', 'tag': '4d0f11af-7cce-477f-b507-91a8046c7634', 'options': 'recursive,force'} 2024-09-19T04:32:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:50 smithi115 ceph-mon[35477]: pgmap v390: 97 pgs: 97 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-09-19T04:32:51.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:50 smithi145 ceph-mon[35096]: pgmap v390: 97 pgs: 97 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-09-19T04:32:51.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 430914 2.97 MB/sec execute 549 sec latency 446.231 ms 2024-09-19T04:32:52.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 430914 2.96 MB/sec execute 550 sec latency 1446.367 ms 2024-09-19T04:32:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:52 smithi115 ceph-mon[35477]: pgmap v391: 97 pgs: 97 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-09-19T04:32:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:52 smithi145 ceph-mon[35096]: pgmap v391: 97 pgs: 97 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-09-19T04:32:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:52 smithi107 ceph-mon[30659]: pgmap v391: 97 pgs: 97 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-09-19T04:32:53.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 430914 2.96 MB/sec execute 551 sec latency 2446.447 ms 2024-09-19T04:32:54.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 432395 2.97 MB/sec execute 552 sec latency 2678.921 ms 2024-09-19T04:32:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:54 smithi115 ceph-mon[35477]: pgmap v392: 97 pgs: 97 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-09-19T04:32:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:54 smithi145 ceph-mon[35096]: pgmap v392: 97 pgs: 97 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-09-19T04:32:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:54 smithi107 ceph-mon[30659]: pgmap v392: 97 pgs: 97 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-09-19T04:32:55.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 433555 2.97 MB/sec execute 553 sec latency 15.566 ms 2024-09-19T04:32:56.729 INFO:tasks.workunit.client.0.smithi107.stdout: 1 434458 2.98 MB/sec execute 554 sec latency 228.996 ms 2024-09-19T04:32:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:56 smithi115 ceph-mon[35477]: pgmap v393: 97 pgs: 97 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-09-19T04:32:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:56 smithi145 ceph-mon[35096]: pgmap v393: 97 pgs: 97 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-09-19T04:32:57.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:56 smithi107 ceph-mon[30659]: pgmap v393: 97 pgs: 97 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-09-19T04:32:57.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 434458 2.97 MB/sec execute 555 sec latency 1229.148 ms 2024-09-19T04:32:58.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 434458 2.97 MB/sec execute 556 sec latency 2229.197 ms 2024-09-19T04:32:59.031 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:32:58 smithi107 ceph-mon[30659]: pgmap v394: 97 pgs: 97 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-19T04:32:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:32:58 smithi115 ceph-mon[35477]: pgmap v394: 97 pgs: 97 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-19T04:32:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:32:58 smithi145 ceph-mon[35096]: pgmap v394: 97 pgs: 97 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-19T04:32:59.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 435483 2.97 MB/sec execute 557 sec latency 2731.273 ms 2024-09-19T04:33:00.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 436140 2.97 MB/sec execute 558 sec latency 695.091 ms 2024-09-19T04:33:01.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:00 smithi115 ceph-mon[35477]: pgmap v395: 97 pgs: 97 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-19T04:33:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:00 smithi145 ceph-mon[35096]: pgmap v395: 97 pgs: 97 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-19T04:33:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:00 smithi107 ceph-mon[30659]: pgmap v395: 97 pgs: 97 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-19T04:33:01.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 436487 2.97 MB/sec execute 559 sec latency 1450.850 ms 2024-09-19T04:33:02.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 437054 2.97 MB/sec execute 560 sec latency 36.068 ms 2024-09-19T04:33:03.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:02 smithi107 ceph-mon[30659]: pgmap v396: 97 pgs: 97 active+clean; 378 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-09-19T04:33:03.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:02 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:33:03.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:02 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:33:03.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:02 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:33:03.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:02 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:33:03.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:02 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:33:03.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:02 smithi107 ceph-mon[30659]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:33:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:02 smithi115 ceph-mon[35477]: pgmap v396: 97 pgs: 97 active+clean; 378 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-09-19T04:33:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:02 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:33:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:02 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:33:03.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:02 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:33:03.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:02 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:33:03.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:02 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:33:03.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:02 smithi115 ceph-mon[35477]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:33:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:02 smithi145 ceph-mon[35096]: pgmap v396: 97 pgs: 97 active+clean; 378 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-09-19T04:33:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:02 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:33:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:02 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:33:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:02 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:33:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:02 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:33:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:02 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:33:03.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:02 smithi145 ceph-mon[35096]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:33:03.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 438005 2.97 MB/sec execute 561 sec latency 420.119 ms 2024-09-19T04:33:04.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 439091 2.98 MB/sec execute 562 sec latency 914.585 ms 2024-09-19T04:33:05.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:04 smithi115 ceph-mon[35477]: pgmap v398: 97 pgs: 97 active+clean; 378 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-19T04:33:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:04 smithi145 ceph-mon[35096]: pgmap v398: 97 pgs: 97 active+clean; 378 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-19T04:33:05.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:04 smithi107 ceph-mon[30659]: pgmap v398: 97 pgs: 97 active+clean; 378 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-19T04:33:05.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 440391 2.99 MB/sec execute 563 sec latency 57.240 ms 2024-09-19T04:33:06.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 441547 2.99 MB/sec execute 564 sec latency 18.566 ms 2024-09-19T04:33:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:06 smithi115 ceph-mon[35477]: pgmap v399: 97 pgs: 97 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-09-19T04:33:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:06 smithi115 ceph-mon[35477]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T04:33:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:06 smithi145 ceph-mon[35096]: pgmap v399: 97 pgs: 97 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-09-19T04:33:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:06 smithi145 ceph-mon[35096]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T04:33:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:06 smithi107 ceph-mon[30659]: pgmap v399: 97 pgs: 97 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-09-19T04:33:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:06 smithi107 ceph-mon[30659]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T04:33:07.730 INFO:tasks.workunit.client.0.smithi107.stdout: 1 441547 2.99 MB/sec execute 565 sec latency 1018.265 ms 2024-09-19T04:33:08.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 441547 2.98 MB/sec execute 566 sec latency 2018.372 ms 2024-09-19T04:33:08.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:08 smithi107 ceph-mon[30659]: pgmap v401: 97 pgs: 97 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-09-19T04:33:09.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:08 smithi115 ceph-mon[35477]: pgmap v401: 97 pgs: 97 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-09-19T04:33:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:08 smithi145 ceph-mon[35096]: pgmap v401: 97 pgs: 97 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-09-19T04:33:09.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 442606 2.99 MB/sec execute 567 sec latency 2515.207 ms 2024-09-19T04:33:10.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 443907 3.00 MB/sec execute 568 sec latency 11.186 ms 2024-09-19T04:33:11.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:10 smithi115 ceph-mon[35477]: pgmap v402: 97 pgs: 97 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-19T04:33:11.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:10 smithi145 ceph-mon[35096]: pgmap v402: 97 pgs: 97 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-19T04:33:11.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:10 smithi107 ceph-mon[30659]: pgmap v402: 97 pgs: 97 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-19T04:33:11.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 445096 3.00 MB/sec execute 569 sec latency 23.947 ms 2024-09-19T04:33:12.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 445096 2.99 MB/sec execute 570 sec latency 1024.090 ms 2024-09-19T04:33:13.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:12 smithi115 ceph-mon[35477]: pgmap v403: 97 pgs: 97 active+clean; 388 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-19T04:33:13.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:12 smithi145 ceph-mon[35096]: pgmap v403: 97 pgs: 97 active+clean; 388 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-19T04:33:13.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:12 smithi107 ceph-mon[30659]: pgmap v403: 97 pgs: 97 active+clean; 388 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-19T04:33:13.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 445096 2.99 MB/sec execute 571 sec latency 2024.245 ms 2024-09-19T04:33:14.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 446578 3.00 MB/sec execute 572 sec latency 2320.107 ms 2024-09-19T04:33:15.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:14 smithi107 ceph-mon[30659]: pgmap v404: 97 pgs: 97 active+clean; 388 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-09-19T04:33:15.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:14 smithi115 ceph-mon[35477]: pgmap v404: 97 pgs: 97 active+clean; 388 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-09-19T04:33:15.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:14 smithi145 ceph-mon[35096]: pgmap v404: 97 pgs: 97 active+clean; 388 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-09-19T04:33:15.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 447668 3.00 MB/sec execute 573 sec latency 16.600 ms 2024-09-19T04:33:16.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 448638 3.01 MB/sec execute 574 sec latency 192.265 ms 2024-09-19T04:33:16.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:16 smithi107 ceph-mon[30659]: pgmap v405: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-09-19T04:33:17.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:16 smithi115 ceph-mon[35477]: pgmap v405: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-09-19T04:33:17.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:16 smithi145 ceph-mon[35096]: pgmap v405: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-09-19T04:33:17.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 448638 3.00 MB/sec execute 575 sec latency 1192.375 ms 2024-09-19T04:33:18.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 448638 3.00 MB/sec execute 576 sec latency 2192.517 ms 2024-09-19T04:33:19.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:18 smithi107 ceph-mon[30659]: pgmap v406: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-09-19T04:33:19.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:18 smithi115 ceph-mon[35477]: pgmap v406: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-09-19T04:33:19.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:18 smithi145 ceph-mon[35096]: pgmap v406: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-09-19T04:33:19.731 INFO:tasks.workunit.client.0.smithi107.stdout: 1 449437 3.00 MB/sec execute 577 sec latency 2693.561 ms 2024-09-19T04:33:20.732 INFO:tasks.workunit.client.0.smithi107.stdout: 1 450934 3.01 MB/sec execute 578 sec latency 20.514 ms 2024-09-19T04:33:21.099 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:33:21.152 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:20 smithi107 ceph-mon[30659]: pgmap v407: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-09-19T04:33:21.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:20 smithi115 ceph-mon[35477]: pgmap v407: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-09-19T04:33:21.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:20 smithi145 ceph-mon[35096]: pgmap v407: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-09-19T04:33:21.496 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:21.732 INFO:tasks.workunit.client.0.smithi107.stdout: 1 452117 3.01 MB/sec execute 579 sec latency 14.919 ms 2024-09-19T04:33:22.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.072+0000 7fd9d2200640 1 -- 172.21.15.107:0/2080682044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1049a0 msgr2=0x7fd9cc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.072+0000 7fd9d2200640 1 --2- 172.21.15.107:0/2080682044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1049a0 0x7fd9cc1089f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009900 tx=0x7fd9b402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.072+0000 7fd9d2200640 1 -- 172.21.15.107:0/2080682044 shutdown_connections 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.072+0000 7fd9d2200640 1 --2- 172.21.15.107:0/2080682044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9cc109120 0x7fd9cc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.072+0000 7fd9d2200640 1 --2- 172.21.15.107:0/2080682044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1049a0 0x7fd9cc1089f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.072+0000 7fd9d2200640 1 --2- 172.21.15.107:0/2080682044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc103ff0 0x7fd9cc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.072+0000 7fd9d2200640 1 -- 172.21.15.107:0/2080682044 >> 172.21.15.107:0/2080682044 conn(0x7fd9cc0ffa90 msgr2=0x7fd9cc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.072+0000 7fd9d2200640 1 -- 172.21.15.107:0/2080682044 shutdown_connections 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.072+0000 7fd9d2200640 1 -- 172.21.15.107:0/2080682044 wait complete. 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.073+0000 7fd9d2200640 1 Processor -- start 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.073+0000 7fd9d2200640 1 -- start start 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.073+0000 7fd9d2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc103ff0 0x7fd9cc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.073+0000 7fd9d2200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc1049a0 0x7fd9cc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.073+0000 7fd9d2200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9cc109120 0x7fd9cc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9d2200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc114120 con 0x7fd9cc103ff0 2024-09-19T04:33:22.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9d2200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc113fa0 con 0x7fd9cc1049a0 2024-09-19T04:33:22.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9d2200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1142a0 con 0x7fd9cc109120 2024-09-19T04:33:22.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc103ff0 0x7fd9cc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc103ff0 0x7fd9cc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42856/0 (socket says 172.21.15.107:42856) 2024-09-19T04:33:22.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9cb400640 1 -- 172.21.15.107:0/3620629690 learned_addr learned my addr 172.21.15.107:0/3620629690 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:22.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc1049a0 0x7fd9cc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9cc109120 0x7fd9cc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9cb400640 1 -- 172.21.15.107:0/3620629690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9cc109120 msgr2=0x7fd9cc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9cb400640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9cc109120 0x7fd9cc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9cb400640 1 -- 172.21.15.107:0/3620629690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc1049a0 msgr2=0x7fd9cc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9cb400640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc1049a0 0x7fd9cc19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9cb400640 1 -- 172.21.15.107:0/3620629690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc19f5d0 con 0x7fd9cc103ff0 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.074+0000 7fd9cb400640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc103ff0 0x7fd9cc19a620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00e9f0 tx=0x7fd9bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.075+0000 7fd9c8c00640 1 -- 172.21.15.107:0/3620629690 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00cd80 con 0x7fd9cc103ff0 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.075+0000 7fd9c8c00640 1 -- 172.21.15.107:0/3620629690 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9bc010910 con 0x7fd9cc103ff0 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.075+0000 7fd9d2200640 1 -- 172.21.15.107:0/3620629690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9cc19f8c0 con 0x7fd9cc103ff0 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.075+0000 7fd9c8c00640 1 -- 172.21.15.107:0/3620629690 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc018990 con 0x7fd9cc103ff0 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.075+0000 7fd9d2200640 1 -- 172.21.15.107:0/3620629690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9cc1a71a0 con 0x7fd9cc103ff0 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.079+0000 7fd9d4cd6740 1 Processor -- start 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.079+0000 7fd9d4cd6740 1 -- start start 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.079+0000 7fd9c8c00640 1 -- 172.21.15.107:0/3620629690 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fd9bc010ac0 con 0x7fd9cc103ff0 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.079+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d1ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.079+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d2a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.079+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d2ad70 0x556746d2d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.079+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556746c863e0 con 0x556746d2ad70 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.079+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556746c2dd80 con 0x556746d1fb90 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.079+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556746c823a0 con 0x556746d204b0 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.080+0000 7fd9cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d1ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.080+0000 7fd9cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d1ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:60502/0 (socket says 172.21.15.107:60502) 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.080+0000 7fd9cbe00640 1 -- 172.21.15.107:0/2445814816 learned_addr learned my addr 172.21.15.107:0/2445814816 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.080+0000 7fd9cb400640 1 --2- 172.21.15.107:0/2445814816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d2a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.080+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/2445814816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d2ad70 0x556746d2d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.080+0000 7fd9cbe00640 1 -- 172.21.15.107:0/2445814816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 msgr2=0x556746d2a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.080+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/2445814816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d2a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.080+0000 7fd9cbe00640 1 -- 172.21.15.107:0/2445814816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d2ad70 msgr2=0x556746d2d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:22.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.080+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/2445814816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d2ad70 0x556746d2d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.080+0000 7fd9cbe00640 1 -- 172.21.15.107:0/2445814816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556746c4cae0 con 0x556746d1fb90 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.081+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/2445814816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d1ff70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009f90 tx=0x7fd9c4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15e89e685e4d9bd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.081+0000 7fd9c8c00640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd9b00a33b0 0x7fd9b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.081+0000 7fd9bbe00640 1 -- 172.21.15.107:0/2445814816 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4011700 con 0x556746d1fb90 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.081+0000 7fd9bbe00640 1 -- 172.21.15.107:0/2445814816 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c4011d80 con 0x556746d1fb90 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.081+0000 7fd9bbe00640 1 -- 172.21.15.107:0/2445814816 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4018380 con 0x556746d1fb90 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.081+0000 7fd9c8c00640 1 -- 172.21.15.107:0/3620629690 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fd9bc0d0070 con 0x7fd9cc103ff0 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.081+0000 7fd9cb400640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd9b00a33b0 0x7fd9b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2445814816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 msgr2=0x556746d1ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/2445814816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d1ff70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009f90 tx=0x7fd9c4010860 comp rx=0 tx=0).stop 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2445814816 shutdown_connections 2024-09-19T04:33:22.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/2445814816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d2ad70 0x556746d2d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/2445814816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d2a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/2445814816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d1ff70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2445814816 >> 172.21.15.107:0/2445814816 conn(0x556746d1c120 msgr2=0x556746d1c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9cb400640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd9b00a33b0 0x7fd9b00a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x556746d1cf80 tx=0x7fd9bc0d3b20 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2445814816 shutdown_connections 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2445814816 wait complete. 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 -- start start 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.082+0000 7fd9d4cd6740 1 Processor -- start 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d1fb90 0x556746d27c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d2ad70 0x556746d286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556746d2d9e0 con 0x556746d1fb90 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556746c410f0 con 0x556746d2ad70 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556746d2db60 con 0x556746d204b0 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9d4cd6740 10 client.0 _ll_register_callbacks cb 0x556746c329e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9cbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d1fb90 0x556746d27c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9cbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d1fb90 0x556746d27c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42876/0 (socket says 172.21.15.107:42876) 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9cbe00640 1 -- 172.21.15.107:0/1683259225 learned_addr learned my addr 172.21.15.107:0/1683259225 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.083+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d2ad70 0x556746d286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9cbe00640 1 -- 172.21.15.107:0/1683259225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 msgr2=0x556746d281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9cbe00640 1 -- 172.21.15.107:0/1683259225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d2ad70 msgr2=0x556746d286f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:22.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d2ad70 0x556746d286f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9cbe00640 1 -- 172.21.15.107:0/1683259225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567469f73c0 con 0x556746d1fb90 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d1fb90 0x556746d27c70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4006010 tx=0x7fd9c4011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe33b0176e49708 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9b8c00640 1 -- 172.21.15.107:0/1683259225 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40195e0 con 0x556746d1fb90 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9b8c00640 1 -- 172.21.15.107:0/1683259225 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fd9c4019de0 con 0x556746d1fb90 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.084+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/1683259225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c4011d80 con 0x556746d1fb90 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9b8c00640 1 -- 172.21.15.107:0/1683259225 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd9c4004a40 con 0x556746d1fb90 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9d4cd6740 10 client.17490 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/1683259225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556746d396a0 con 0x556746d1fb90 2024-09-19T04:33:22.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9b8c00640 1 -- 172.21.15.107:0/1683259225 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c4011d80 con 0x556746d1fb90 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9b8c00640 1 -- 172.21.15.107:0/1683259225 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7fd9c402f030 con 0x556746d1fb90 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9d4cd6740 10 client.17490 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9d4cd6740 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9d4cd6740 20 client.17490 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556746d3ab00 0x556746d3cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9d4cd6740 4 client.17490 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.085+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/1683259225 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556746c93950 con 0x556746d3ab00 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.086+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556746d3ab00 0x556746d3cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.086+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556746d3ab00 0x556746d3cef0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a69e9b455425c8 server_cookie=f5444091d2c92dec in_seq=0 out_seq=0 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.086+0000 7fd9b8c00640 10 client.17490 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9b8c00640 1 -- 172.21.15.107:0/1683259225 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556746c93950 con 0x556746d3ab00 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9b8c00640 10 client.17490 handle_command_reply: tid=0 2024-09-19T04:33:22.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 client.17490 shutdown 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/1683259225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d1fb90 msgr2=0x556746d27c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d1fb90 0x556746d27c70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4006010 tx=0x7fd9c4011f50 comp rx=0 tx=0).stop 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/1683259225 shutdown_connections 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556746d3ab00 0x556746d3cef0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d2ad70 0x556746d286f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/1683259225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d1fb90 0x556746d27c70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/1683259225 >> 172.21.15.107:0/1683259225 conn(0x556746d1c120 msgr2=0x556746d21e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9b8c00640 0 client.17490 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/1683259225 shutdown_connections 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/1683259225 wait complete. 2024-09-19T04:33:22.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.087+0000 7fd9d4cd6740 20 client.17490 trim_cache size 0 max 16384 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9d4cd6740 1 Processor -- start 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9d4cd6740 1 -- start start 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d27570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d42490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d2ad70 0x556746d429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1afaa0 con 0x556746d2ad70 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1b1610 con 0x556746d1fb90 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1b01a0 con 0x556746d204b0 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9cbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d2ad70 0x556746d429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d42490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d27570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d42490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42842/0 (socket says 172.21.15.107:42842) 2024-09-19T04:33:22.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.105+0000 7fd9c3e00640 1 -- 172.21.15.107:0/3195098565 learned_addr learned my addr 172.21.15.107:0/3195098565 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.106+0000 7fd9c3e00640 1 -- 172.21.15.107:0/3195098565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 msgr2=0x556746d27570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.106+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/3195098565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d27570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.106+0000 7fd9c3e00640 1 -- 172.21.15.107:0/3195098565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d2ad70 msgr2=0x556746d429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.106+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/3195098565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d2ad70 0x556746d429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.106+0000 7fd9c3e00640 1 -- 172.21.15.107:0/3195098565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556746d202e0 con 0x556746d204b0 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.106+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/3195098565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d27570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.106+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/3195098565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d42490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009900 tx=0x7fd9b4031a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aea7817433ac87df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.106+0000 7fd9bb400640 1 -- 172.21.15.107:0/3195098565 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4035040 con 0x556746d204b0 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.106+0000 7fd9bb400640 1 -- 172.21.15.107:0/3195098565 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b4031a70 con 0x556746d204b0 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.106+0000 7fd9bb400640 1 -- 172.21.15.107:0/3195098565 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4004bc0 con 0x556746d204b0 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/3195098565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 msgr2=0x556746d42490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/3195098565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d42490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009900 tx=0x7fd9b4031a00 comp rx=0 tx=0).stop 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/3195098565 shutdown_connections 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/3195098565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d2ad70 0x556746d429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/3195098565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d204b0 0x556746d42490 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/3195098565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d27570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/3195098565 >> 172.21.15.107:0/3195098565 conn(0x556746d1c120 msgr2=0x556746d29cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/3195098565 shutdown_connections 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/3195098565 wait complete. 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 -- start start 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.107+0000 7fd9d4cd6740 1 Processor -- start 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d32f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d204b0 0x556746d334d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9d4cd6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d2ad70 0x556746d244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1b0f40 con 0x556746d204b0 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1b0870 con 0x556746d1fb90 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9d4cd6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1adaa0 con 0x556746d2ad70 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9d4cd6740 10 client.0 _ll_register_callbacks cb 0x7fd9c4009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d204b0 0x556746d334d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d204b0 0x556746d334d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42908/0 (socket says 172.21.15.107:42908) 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9c3e00640 1 -- 172.21.15.107:0/2352912134 learned_addr learned my addr 172.21.15.107:0/2352912134 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d32f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d2ad70 0x556746d244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9c3e00640 1 -- 172.21.15.107:0/2352912134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d2ad70 msgr2=0x556746d244a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.108+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d2ad70 0x556746d244a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9c3e00640 1 -- 172.21.15.107:0/2352912134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 msgr2=0x556746d32f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:22.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d32f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9c3e00640 1 -- 172.21.15.107:0/2352912134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556746d24b10 con 0x556746d204b0 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d2ad70 0x556746d244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9c3e00640 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d204b0 0x556746d334d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4000c00 tx=0x7fd9b40317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a8bd5731ad9a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9b8c00640 1 -- 172.21.15.107:0/2352912134 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4056020 con 0x556746d204b0 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9b8c00640 1 -- 172.21.15.107:0/2352912134 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fd9b40037f0 con 0x556746d204b0 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2352912134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9b4031a70 con 0x556746d204b0 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9b8c00640 1 -- 172.21.15.107:0/2352912134 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd9b4061050 con 0x556746d204b0 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9d4cd6740 10 client.17496 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.109+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2352912134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556746da4dd0 con 0x556746d204b0 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.110+0000 7fd9b8c00640 1 -- 172.21.15.107:0/2352912134 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b4031a70 con 0x556746d204b0 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.110+0000 7fd9b8c00640 1 -- 172.21.15.107:0/2352912134 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7fd9b403d450 con 0x556746d204b0 2024-09-19T04:33:22.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.110+0000 7fd9d4cd6740 10 client.17496 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:22.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.110+0000 7fd9d4cd6740 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:33:22.254 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:33:22.254 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:33:22.254 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:33:22.254 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.110+0000 7fd9d4cd6740 20 client.17496 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.110+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556746da5510 0x556746da7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.110+0000 7fd9d4cd6740 4 client.17496 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.110+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2352912134 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556746c8de60 con 0x556746da5510 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.110+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556746da5510 0x556746da7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9cbe00640 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556746da5510 0x556746da7930 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76308ff683b6823b server_cookie=f7e02d9ede295d5b in_seq=0 out_seq=0 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9b8c00640 10 client.17496 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9b8c00640 1 -- 172.21.15.107:0/2352912134 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556746c8de60 con 0x556746da5510 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9b8c00640 10 client.17496 handle_command_reply: tid=0 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9d4cd6740 1 client.17496 shutdown 2024-09-19T04:33:22.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2352912134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d204b0 msgr2=0x556746d334d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d204b0 0x556746d334d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4000c00 tx=0x7fd9b40317c0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2352912134 shutdown_connections 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x556746da5510 0x556746da7930 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556746d2ad70 0x556746d244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556746d204b0 0x556746d334d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9d4cd6740 1 --2- 172.21.15.107:0/2352912134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556746d1fb90 0x556746d32f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9b8c00640 0 client.17496 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2352912134 >> 172.21.15.107:0/2352912134 conn(0x556746d1c120 msgr2=0x556746d1a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.111+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2352912134 shutdown_connections 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d4cd6740 1 -- 172.21.15.107:0/2352912134 wait complete. 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d4cd6740 20 client.17496 trim_cache size 0 max 16384 2024-09-19T04:33:22.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 -- 172.21.15.107:0/3620629690 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd9b00a33b0 msgr2=0x7fd9b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd9b00a33b0 0x7fd9b00a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x556746d1cf80 tx=0x7fd9bc0d3b20 comp rx=0 tx=0).stop 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 -- 172.21.15.107:0/3620629690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc103ff0 msgr2=0x7fd9cc19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc103ff0 0x7fd9cc19a620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00e9f0 tx=0x7fd9bc00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 -- 172.21.15.107:0/3620629690 shutdown_connections 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fd9b00a33b0 0x7fd9b00a5870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9cc109120 0x7fd9cc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc1049a0 0x7fd9cc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 --2- 172.21.15.107:0/3620629690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc103ff0 0x7fd9cc19a620 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 -- 172.21.15.107:0/3620629690 >> 172.21.15.107:0/3620629690 conn(0x7fd9cc0ffa90 msgr2=0x7fd9cc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.112+0000 7fd9d2200640 1 -- 172.21.15.107:0/3620629690 shutdown_connections 2024-09-19T04:33:22.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:22.113+0000 7fd9d2200640 1 -- 172.21.15.107:0/3620629690 wait complete. 2024-09-19T04:33:22.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 452187 3.01 MB/sec execute 580 sec latency 947.048 ms 2024-09-19T04:33:22.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:22 smithi107 ceph-mon[30659]: pgmap v408: 97 pgs: 97 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-09-19T04:33:23.189 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:33:23.190 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:33:23.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:22 smithi115 ceph-mon[35477]: pgmap v408: 97 pgs: 97 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-09-19T04:33:23.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:22 smithi145 ceph-mon[35096]: pgmap v408: 97 pgs: 97 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-09-19T04:33:23.559 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:23.736 INFO:tasks.workunit.client.0.smithi107.stdout: 1 452187 3.00 MB/sec execute 581 sec latency 1947.093 ms 2024-09-19T04:33:24.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.142+0000 7f8508600640 1 -- 172.21.15.107:0/1493875753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8500101dd0 msgr2=0x7f85001021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:24.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.142+0000 7f8508600640 1 --2- 172.21.15.107:0/1493875753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8500101dd0 0x7f85001021b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009900 tx=0x7f84f402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.142+0000 7f8508600640 1 -- 172.21.15.107:0/1493875753 shutdown_connections 2024-09-19T04:33:24.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.142+0000 7f8508600640 1 --2- 172.21.15.107:0/1493875753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8500106f00 0x7f850010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.142+0000 7f8508600640 1 --2- 172.21.15.107:0/1493875753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500102780 0x7f85001067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.142+0000 7f8508600640 1 --2- 172.21.15.107:0/1493875753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8500101dd0 0x7f85001021b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.142+0000 7f8508600640 1 -- 172.21.15.107:0/1493875753 >> 172.21.15.107:0/1493875753 conn(0x7f85000fd8b0 msgr2=0x7f85000ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.145+0000 7f8508600640 1 -- 172.21.15.107:0/1493875753 shutdown_connections 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.145+0000 7f8508600640 1 -- 172.21.15.107:0/1493875753 wait complete. 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 Processor -- start 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 -- start start 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8500101dd0 0x7f8500198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8500102780 0x7f85001989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500106f00 0x7f850019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8500111f60 con 0x7f8500102780 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8500111de0 con 0x7f8500101dd0 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f85001120e0 con 0x7f8500106f00 2024-09-19T04:33:24.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8505c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8500101dd0 0x7f8500198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8505200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8500102780 0x7f85001989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8505200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8500102780 0x7f85001989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42920/0 (socket says 172.21.15.107:42920) 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8505200640 1 -- 172.21.15.107:0/219890425 learned_addr learned my addr 172.21.15.107:0/219890425 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8506600640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500106f00 0x7f850019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8505c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8500101dd0 0x7f8500198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:60568/0 (socket says 172.21.15.107:60568) 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8506600640 1 -- 172.21.15.107:0/219890425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8500101dd0 msgr2=0x7f8500198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8506600640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8500101dd0 0x7f8500198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8506600640 1 -- 172.21.15.107:0/219890425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8500102780 msgr2=0x7f85001989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8506600640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8500102780 0x7f85001989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8506600640 1 -- 172.21.15.107:0/219890425 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850019d430 con 0x7f8500106f00 2024-09-19T04:33:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8506600640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500106f00 0x7f850019cd50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f84f800ec50 tx=0x7f84f800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:24.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f84fe000640 1 -- 172.21.15.107:0/219890425 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8003b70 con 0x7f8500106f00 2024-09-19T04:33:24.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 -- 172.21.15.107:0/219890425 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850019d720 con 0x7f8500106f00 2024-09-19T04:33:24.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f84fe000640 1 -- 172.21.15.107:0/219890425 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f8003d10 con 0x7f8500106f00 2024-09-19T04:33:24.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f84fe000640 1 -- 172.21.15.107:0/219890425 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8010890 con 0x7f8500106f00 2024-09-19T04:33:24.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 -- 172.21.15.107:0/219890425 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8500074420 con 0x7f8500106f00 2024-09-19T04:33:24.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.146+0000 7f8508600640 1 -- 172.21.15.107:0/219890425 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85001032b0 con 0x7f8500106f00 2024-09-19T04:33:24.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.152+0000 7f84fe000640 1 -- 172.21.15.107:0/219890425 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f84f8004b40 con 0x7f8500106f00 2024-09-19T04:33:24.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.152+0000 7f84fe000640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84e80a33b0 0x7f84e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:24.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.152+0000 7f84fe000640 1 -- 172.21.15.107:0/219890425 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f84f80d04f0 con 0x7f8500106f00 2024-09-19T04:33:24.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.152+0000 7f8505c00640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84e80a33b0 0x7f84e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:24.553 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:33:24.553 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":49,"btime":"2024-09-19T04:32:39:725165+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:32:38.948815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:33:24.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.152+0000 7f8505c00640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84e80a33b0 0x7f84e80a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009230 tx=0x7f84f40058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:24.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.152+0000 7f84fe000640 1 -- 172.21.15.107:0/219890425 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84f809ce90 con 0x7f8500106f00 2024-09-19T04:33:24.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.382+0000 7f8508600640 1 -- 172.21.15.107:0/219890425 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85001021b0 con 0x7f8500106f00 2024-09-19T04:33:24.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.382+0000 7f84fe000640 1 -- 172.21.15.107:0/219890425 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11993 (secure 0 0 0) 0x7f85001021b0 con 0x7f8500106f00 2024-09-19T04:33:24.553 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 49 2024-09-19T04:33:24.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 -- 172.21.15.107:0/219890425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84e80a33b0 msgr2=0x7f84e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:24.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84e80a33b0 0x7f84e80a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009230 tx=0x7f84f40058d0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 -- 172.21.15.107:0/219890425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500106f00 msgr2=0x7f850019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:24.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500106f00 0x7f850019cd50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f84f800ec50 tx=0x7f84f800c6a0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 -- 172.21.15.107:0/219890425 shutdown_connections 2024-09-19T04:33:24.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f84e80a33b0 0x7f84e80a5870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500106f00 0x7f850019cd50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8500102780 0x7f85001989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 --2- 172.21.15.107:0/219890425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8500101dd0 0x7f8500198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:24.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 -- 172.21.15.107:0/219890425 >> 172.21.15.107:0/219890425 conn(0x7f85000fd8b0 msgr2=0x7f85000feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:24.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 -- 172.21.15.107:0/219890425 shutdown_connections 2024-09-19T04:33:24.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:24.387+0000 7f8508600640 1 -- 172.21.15.107:0/219890425 wait complete. 2024-09-19T04:33:24.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 452369 3.00 MB/sec execute 582 sec latency 2806.574 ms 2024-09-19T04:33:25.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:24 smithi115 ceph-mon[35477]: pgmap v409: 97 pgs: 97 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-09-19T04:33:25.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:24 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/219890425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:33:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:24 smithi145 ceph-mon[35096]: pgmap v409: 97 pgs: 97 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-09-19T04:33:25.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:24 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/219890425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:33:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:24 smithi107 ceph-mon[30659]: pgmap v409: 97 pgs: 97 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-09-19T04:33:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:24 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/219890425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:33:25.424 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:33:25.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 454272 3.02 MB/sec execute 583 sec latency 16.762 ms 2024-09-19T04:33:25.807 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.390+0000 7f2b09800640 1 -- 172.21.15.107:0/1164794192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074590 msgr2=0x7f2b04074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.390+0000 7f2b09800640 1 --2- 172.21.15.107:0/1164794192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074590 0x7f2b04074970 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009900 tx=0x7f2af402a480 comp rx=0 tx=0).stop 2024-09-19T04:33:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.390+0000 7f2b09800640 1 -- 172.21.15.107:0/1164794192 shutdown_connections 2024-09-19T04:33:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.390+0000 7f2b09800640 1 --2- 172.21.15.107:0/1164794192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b041129d0 0x7f2b04114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.390+0000 7f2b09800640 1 --2- 172.21.15.107:0/1164794192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b04074eb0 0x7f2b04112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.390+0000 7f2b09800640 1 --2- 172.21.15.107:0/1164794192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074590 0x7f2b04074970 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.390+0000 7f2b09800640 1 -- 172.21.15.107:0/1164794192 >> 172.21.15.107:0/1164794192 conn(0x7f2b040fd6e0 msgr2=0x7f2b040ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.390+0000 7f2b09800640 1 -- 172.21.15.107:0/1164794192 shutdown_connections 2024-09-19T04:33:26.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.390+0000 7f2b09800640 1 -- 172.21.15.107:0/1164794192 wait complete. 2024-09-19T04:33:26.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.391+0000 7f2b09800640 1 Processor -- start 2024-09-19T04:33:26.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.391+0000 7f2b09800640 1 -- start start 2024-09-19T04:33:26.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b09800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04074590 0x7f2b041a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b09800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b04074eb0 0x7f2b041a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b09800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b041129d0 0x7f2b0419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b09800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04118390 con 0x7f2b04074eb0 2024-09-19T04:33:26.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b09800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04118210 con 0x7f2b041129d0 2024-09-19T04:33:26.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b09800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04118510 con 0x7f2b04074590 2024-09-19T04:33:26.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b08e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b041129d0 0x7f2b0419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b08e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b041129d0 0x7f2b0419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:60588/0 (socket says 172.21.15.107:60588) 2024-09-19T04:33:26.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b08e00640 1 -- 172.21.15.107:0/485015101 learned_addr learned my addr 172.21.15.107:0/485015101 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:26.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2afbe00640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b04074eb0 0x7f2b041a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b03e00640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04074590 0x7f2b041a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b08e00640 1 -- 172.21.15.107:0/485015101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04074590 msgr2=0x7f2b041a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:26.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b08e00640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04074590 0x7f2b041a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b08e00640 1 -- 172.21.15.107:0/485015101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b04074eb0 msgr2=0x7f2b041a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:26.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b08e00640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b04074eb0 0x7f2b041a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.392+0000 7f2b08e00640 1 -- 172.21.15.107:0/485015101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0419ef10 con 0x7f2b041129d0 2024-09-19T04:33:26.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.393+0000 7f2b08e00640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b041129d0 0x7f2b0419e6a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009c90 tx=0x7f2afc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:26.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.393+0000 7f2b01600640 1 -- 172.21.15.107:0/485015101 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc010630 con 0x7f2b041129d0 2024-09-19T04:33:26.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.393+0000 7f2b01600640 1 -- 172.21.15.107:0/485015101 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2afc010cb0 con 0x7f2b041129d0 2024-09-19T04:33:26.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.393+0000 7f2b09800640 1 -- 172.21.15.107:0/485015101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0419f200 con 0x7f2b041129d0 2024-09-19T04:33:26.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.393+0000 7f2b01600640 1 -- 172.21.15.107:0/485015101 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc017d20 con 0x7f2b041129d0 2024-09-19T04:33:26.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.393+0000 7f2b09800640 1 -- 172.21.15.107:0/485015101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b041ab380 con 0x7f2b041129d0 2024-09-19T04:33:26.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.396+0000 7f2b0d83d740 1 Processor -- start 2024-09-19T04:33:26.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.396+0000 7f2b01600640 1 -- 172.21.15.107:0/485015101 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f2afc0107d0 con 0x7f2b041129d0 2024-09-19T04:33:26.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.396+0000 7f2b0d83d740 1 -- start start 2024-09-19T04:33:26.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2ee40 0x55f7edb2f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2f760 0x55f7edb39ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edb3c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f7eda953e0 con 0x55f7edb2ee40 2024-09-19T04:33:26.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f7eda3cd80 con 0x55f7edb3a020 2024-09-19T04:33:26.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f7eda913a0 con 0x55f7edb2f760 2024-09-19T04:33:26.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2afbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2ee40 0x55f7edb2f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2f760 0x55f7edb39ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2f760 0x55f7edb39ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42880/0 (socket says 172.21.15.107:42880) 2024-09-19T04:33:26.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2b03e00640 1 -- 172.21.15.107:0/3727618271 learned_addr learned my addr 172.21.15.107:0/3727618271 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:26.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.397+0000 7f2afbe00640 1 --2- 172.21.15.107:0/3727618271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edb3c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2b03e00640 1 -- 172.21.15.107:0/3727618271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 msgr2=0x55f7edb3c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:26.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2b03e00640 1 --2- 172.21.15.107:0/3727618271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edb3c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2b03e00640 1 -- 172.21.15.107:0/3727618271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2ee40 msgr2=0x55f7edb2f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:26.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2b03e00640 1 --2- 172.21.15.107:0/3727618271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2ee40 0x55f7edb2f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2b03e00640 1 -- 172.21.15.107:0/3727618271 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7eda5bae0 con 0x55f7edb2f760 2024-09-19T04:33:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2b01600640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2ae80a33b0 0x7f2ae80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2b03e00640 1 --2- 172.21.15.107:0/3727618271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2f760 0x55f7edb39ae0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009900 tx=0x7f2af4005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8c288b974420f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2b01600640 1 -- 172.21.15.107:0/485015101 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f2afc0d0510 con 0x7f2b041129d0 2024-09-19T04:33:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2afbe00640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2ae80a33b0 0x7f2ae80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2af9600640 1 -- 172.21.15.107:0/3727618271 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4040070 con 0x55f7edb2f760 2024-09-19T04:33:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2af9600640 1 -- 172.21.15.107:0/3727618271 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4031870 con 0x55f7edb2f760 2024-09-19T04:33:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.398+0000 7f2af9600640 1 -- 172.21.15.107:0/3727618271 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4035040 con 0x55f7edb2f760 2024-09-19T04:33:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.399+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3727618271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2f760 msgr2=0x55f7edb39ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.399+0000 7f2afbe00640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2ae80a33b0 0x7f2ae80a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55f7edb2c750 tx=0x7f2aec005850 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.399+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/3727618271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2f760 0x55f7edb39ae0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009900 tx=0x7f2af4005be0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.399+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3727618271 shutdown_connections 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.399+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/3727618271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edb3c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.399+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/3727618271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2f760 0x55f7edb39ae0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.399+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/3727618271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2ee40 0x55f7edb2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.399+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3727618271 >> 172.21.15.107:0/3727618271 conn(0x55f7edb2b3d0 msgr2=0x55f7edb2b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.399+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3727618271 shutdown_connections 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.399+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3727618271 wait complete. 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.400+0000 7f2b0d83d740 1 -- start start 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.400+0000 7f2b0d83d740 1 Processor -- start 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.400+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2ee40 0x55f7edb36f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.400+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb2f760 0x55f7edb37440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.400+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb3a020 0x55f7edb37980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.400+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f7edb3cbf0 con 0x55f7edb2ee40 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.400+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f7eda500f0 con 0x55f7edb2f760 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.400+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f7edb3cd70 con 0x55f7edb3a020 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b0d83d740 10 client.0 _ll_register_callbacks cb 0x55f7eda37bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2afbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb2f760 0x55f7edb37440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b08e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb3a020 0x55f7edb37980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b08e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb3a020 0x55f7edb37980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42884/0 (socket says 172.21.15.107:42884) 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b08e00640 1 -- 172.21.15.107:0/3237024807 learned_addr learned my addr 172.21.15.107:0/3237024807 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2afbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb2f760 0x55f7edb37440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:60604/0 (socket says 172.21.15.107:60604) 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b03e00640 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2ee40 0x55f7edb36f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b08e00640 1 -- 172.21.15.107:0/3237024807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb2f760 msgr2=0x55f7edb37440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b08e00640 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb2f760 0x55f7edb37440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b08e00640 1 -- 172.21.15.107:0/3237024807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2ee40 msgr2=0x55f7edb36f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b08e00640 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2ee40 0x55f7edb36f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b08e00640 1 -- 172.21.15.107:0/3237024807 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7ed8063c0 con 0x55f7edb3a020 2024-09-19T04:33:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2b08e00640 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb3a020 0x55f7edb37980 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2afc09c7a0 tx=0x7f2afc09c5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d16c81610fac017 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.401+0000 7f2af2000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:33:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2af2000640 1 -- 172.21.15.107:0/3237024807 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0b5060 con 0x55f7edb3a020 2024-09-19T04:33:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2af2000640 1 -- 172.21.15.107:0/3237024807 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f2afc0d0070 con 0x55f7edb3a020 2024-09-19T04:33:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3237024807 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2af4031870 con 0x55f7edb3a020 2024-09-19T04:33:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2af2000640 1 -- 172.21.15.107:0/3237024807 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2afc09cf10 con 0x55f7edb3a020 2024-09-19T04:33:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2b0d83d740 10 client.27499 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3237024807 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f7edb48cc0 con 0x55f7edb3a020 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2af2000640 1 -- 172.21.15.107:0/3237024807 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4031870 con 0x55f7edb3a020 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2af2000640 1 -- 172.21.15.107:0/3237024807 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f2afc09db40 con 0x55f7edb3a020 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2b0d83d740 10 client.27499 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2b0d83d740 10 client.27499 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2b0d83d740 20 client.27499 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55f7edb4a0d0 0x55f7edb4c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2b0d83d740 4 client.27499 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.402+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3237024807 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7edaa2950 con 0x55f7edb4a0d0 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.403+0000 7f2b03e00640 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55f7edb4a0d0 0x55f7edb4c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.403+0000 7f2b03e00640 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55f7edb4a0d0 0x55f7edb4c520 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6d3fd033bbfe41b1 server_cookie=17e94c9d4ba51835 in_seq=0 out_seq=0 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.403+0000 7f2af2000640 10 client.27499 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:33:26.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2af2000640 1 -- 172.21.15.107:0/3237024807 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7edaa2950 con 0x55f7edb4a0d0 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2af2000640 10 client.27499 handle_command_reply: tid=0 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2b0d83d740 1 client.27499 shutdown 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3237024807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb3a020 msgr2=0x55f7edb37980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb3a020 0x55f7edb37980 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2afc09c7a0 tx=0x7f2afc09c5e0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3237024807 shutdown_connections 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55f7edb4a0d0 0x55f7edb4c520 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb3a020 0x55f7edb37980 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb2f760 0x55f7edb37440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/3237024807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2ee40 0x55f7edb36f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3237024807 >> 172.21.15.107:0/3237024807 conn(0x55f7edb2b3d0 msgr2=0x55f7edb31100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2af2000640 0 client.27499 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:33:26.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.404+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3237024807 shutdown_connections 2024-09-19T04:33:26.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.405+0000 7f2b0d83d740 1 -- 172.21.15.107:0/3237024807 wait complete. 2024-09-19T04:33:26.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.405+0000 7f2b0d83d740 20 client.27499 trim_cache size 0 max 16384 2024-09-19T04:33:26.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.418+0000 7f2b0d83d740 1 Processor -- start 2024-09-19T04:33:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b0d83d740 1 -- start start 2024-09-19T04:33:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2f760 0x55f7edb38e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edb51df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041b3c20 con 0x55f7edb2f760 2024-09-19T04:33:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041b5790 con 0x55f7edb3a020 2024-09-19T04:33:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041b4320 con 0x55f7edb2ee40 2024-09-19T04:33:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2afbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2f760 0x55f7edb38e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb318f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42886/0 (socket says 172.21.15.107:42886) 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b03e00640 1 -- 172.21.15.107:0/1434681737 learned_addr learned my addr 172.21.15.107:0/1434681737 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b08e00640 1 --2- 172.21.15.107:0/1434681737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edb51df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b03e00640 1 -- 172.21.15.107:0/1434681737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 msgr2=0x55f7edb51df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b03e00640 1 --2- 172.21.15.107:0/1434681737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edb51df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b03e00640 1 -- 172.21.15.107:0/1434681737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2f760 msgr2=0x55f7edb38e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.419+0000 7f2b03e00640 1 --2- 172.21.15.107:0/1434681737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2f760 0x55f7edb38e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.420+0000 7f2b03e00640 1 -- 172.21.15.107:0/1434681737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7edb2f590 con 0x55f7edb2ee40 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.420+0000 7f2b03e00640 1 --2- 172.21.15.107:0/1434681737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb318f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2af40053f0 tx=0x7f2af4047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1258757491ace5a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.420+0000 7f2af8c00640 1 -- 172.21.15.107:0/1434681737 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4040040 con 0x55f7edb2ee40 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.420+0000 7f2af8c00640 1 -- 172.21.15.107:0/1434681737 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4031870 con 0x55f7edb2ee40 2024-09-19T04:33:26.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.420+0000 7f2af8c00640 1 -- 172.21.15.107:0/1434681737 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4035040 con 0x55f7edb2ee40 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1434681737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 msgr2=0x55f7edb318f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/1434681737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb318f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2af40053f0 tx=0x7f2af4047040 comp rx=0 tx=0).stop 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1434681737 shutdown_connections 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/1434681737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edb51df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/1434681737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2f760 0x55f7edb38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/1434681737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb318f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1434681737 >> 172.21.15.107:0/1434681737 conn(0x55f7edb2b3d0 msgr2=0x55f7edb38760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1434681737 shutdown_connections 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1434681737 wait complete. 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 -- start start 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.421+0000 7f2b0d83d740 1 Processor -- start 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb3d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2f760 0x55f7edb3dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b0d83d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edbacdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041b50c0 con 0x55f7edb2f760 2024-09-19T04:33:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041b49f0 con 0x55f7edb3a020 2024-09-19T04:33:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b0d83d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041b1c20 con 0x55f7edb2ee40 2024-09-19T04:33:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b0d83d740 10 client.0 _ll_register_callbacks cb 0x7f2afc0176c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb3d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb3d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42890/0 (socket says 172.21.15.107:42890) 2024-09-19T04:33:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2afbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2f760 0x55f7edb3dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b03e00640 1 -- 172.21.15.107:0/1279199662 learned_addr learned my addr 172.21.15.107:0/1279199662 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b08e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edbacdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b08e00640 1 -- 172.21.15.107:0/1279199662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 msgr2=0x55f7edb3d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:26.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b08e00640 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb3d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b08e00640 1 -- 172.21.15.107:0/1279199662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2f760 msgr2=0x55f7edb3dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b08e00640 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2f760 0x55f7edb3dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b08e00640 1 -- 172.21.15.107:0/1279199662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7edbad490 con 0x55f7edb3a020 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.422+0000 7f2b03e00640 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb3d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.423+0000 7f2b08e00640 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edbacdf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00cba0 tx=0x7f2afc0177c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b53087f3b59bef94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.423+0000 7f2af2000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.423+0000 7f2af2000640 1 -- 172.21.15.107:0/1279199662 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc010e60 con 0x55f7edb3a020 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.423+0000 7f2af2000640 1 -- 172.21.15.107:0/1279199662 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f2afc004d40 con 0x55f7edb3a020 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.423+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1279199662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2af4031870 con 0x55f7edb3a020 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.423+0000 7f2af2000640 1 -- 172.21.15.107:0/1279199662 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2afc09c810 con 0x55f7edb3a020 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.423+0000 7f2b0d83d740 10 client.27566 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.423+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1279199662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f7edbb4800 con 0x55f7edb3a020 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.423+0000 7f2af2000640 1 -- 172.21.15.107:0/1279199662 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4031870 con 0x55f7edb3a020 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.423+0000 7f2af2000640 1 -- 172.21.15.107:0/1279199662 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f2afc09bc60 con 0x55f7edb3a020 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.424+0000 7f2b0d83d740 10 client.27566 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.424+0000 7f2b0d83d740 10 client.27566 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.424+0000 7f2b0d83d740 20 client.27566 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.424+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55f7edbb4ef0 0x55f7edbb7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.424+0000 7f2b0d83d740 4 client.27566 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.424+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1279199662 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7eda9ce60 con 0x55f7edbb4ef0 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.424+0000 7f2b03e00640 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55f7edbb4ef0 0x55f7edbb7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.424+0000 7f2b03e00640 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55f7edbb4ef0 0x55f7edbb7310 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9e3ba94c450f9f0 server_cookie=5b3f6337501978bd in_seq=0 out_seq=0 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.424+0000 7f2af2000640 10 client.27566 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2af2000640 1 -- 172.21.15.107:0/1279199662 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7eda9ce60 con 0x55f7edbb4ef0 2024-09-19T04:33:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2af2000640 10 client.27566 handle_command_reply: tid=0 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 client.27566 shutdown 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1279199662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 msgr2=0x55f7edbacdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edbacdf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00cba0 tx=0x7f2afc0177c0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1279199662 shutdown_connections 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55f7edbb4ef0 0x55f7edbb7310 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7edb3a020 0x55f7edbacdf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7edb2f760 0x55f7edb3dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2af2000640 0 client.27566 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 --2- 172.21.15.107:0/1279199662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7edb2ee40 0x55f7edb3d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1279199662 >> 172.21.15.107:0/1279199662 conn(0x55f7edb2b3d0 msgr2=0x55f7edb376e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1279199662 shutdown_connections 2024-09-19T04:33:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 1 -- 172.21.15.107:0/1279199662 wait complete. 2024-09-19T04:33:26.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.425+0000 7f2b0d83d740 20 client.27566 trim_cache size 0 max 16384 2024-09-19T04:33:26.433 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:33:26.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 -- 172.21.15.107:0/485015101 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2ae80a33b0 msgr2=0x7f2ae80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:26.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2ae80a33b0 0x7f2ae80a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55f7edb2c750 tx=0x7f2aec005850 comp rx=0 tx=0).stop 2024-09-19T04:33:26.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 -- 172.21.15.107:0/485015101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b041129d0 msgr2=0x7f2b0419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:26.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b041129d0 0x7f2b0419e6a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009c90 tx=0x7f2afc00c6a0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 -- 172.21.15.107:0/485015101 shutdown_connections 2024-09-19T04:33:26.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2ae80a33b0 0x7f2ae80a5870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b041129d0 0x7f2b0419e6a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b04074eb0 0x7f2b041a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 --2- 172.21.15.107:0/485015101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04074590 0x7f2b041a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 -- 172.21.15.107:0/485015101 >> 172.21.15.107:0/485015101 conn(0x7f2b040fd6e0 msgr2=0x7f2b040fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 -- 172.21.15.107:0/485015101 shutdown_connections 2024-09-19T04:33:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:26.426+0000 7f2b09800640 1 -- 172.21.15.107:0/485015101 wait complete. 2024-09-19T04:33:26.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 455381 3.02 MB/sec execute 584 sec latency 20.985 ms 2024-09-19T04:33:27.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:26 smithi115 ceph-mon[35477]: pgmap v410: 97 pgs: 97 active+clean; 405 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 162 op/s 2024-09-19T04:33:27.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:26 smithi145 ceph-mon[35096]: pgmap v410: 97 pgs: 97 active+clean; 405 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 162 op/s 2024-09-19T04:33:27.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:26 smithi107 ceph-mon[30659]: pgmap v410: 97 pgs: 97 active+clean; 405 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 162 op/s 2024-09-19T04:33:27.426 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:33:27.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 455730 3.02 MB/sec execute 585 sec latency 538.146 ms 2024-09-19T04:33:27.797 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:28.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.370+0000 7f0106000640 1 -- 172.21.15.107:0/1550227608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100074590 msgr2=0x7f0100074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:28.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.370+0000 7f0106000640 1 --2- 172.21.15.107:0/1550227608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100074590 0x7f0100074970 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009900 tx=0x7f00f002a480 comp rx=0 tx=0).stop 2024-09-19T04:33:28.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.370+0000 7f0106000640 1 -- 172.21.15.107:0/1550227608 shutdown_connections 2024-09-19T04:33:28.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.370+0000 7f0106000640 1 --2- 172.21.15.107:0/1550227608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100112b90 0x7f0100114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.370+0000 7f0106000640 1 --2- 172.21.15.107:0/1550227608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0100074eb0 0x7f0100112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.370+0000 7f0106000640 1 --2- 172.21.15.107:0/1550227608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100074590 0x7f0100074970 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.370+0000 7f0106000640 1 -- 172.21.15.107:0/1550227608 >> 172.21.15.107:0/1550227608 conn(0x7f01000fd8c0 msgr2=0x7f01000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:28.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.370+0000 7f0106000640 1 -- 172.21.15.107:0/1550227608 shutdown_connections 2024-09-19T04:33:28.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.370+0000 7f0106000640 1 -- 172.21.15.107:0/1550227608 wait complete. 2024-09-19T04:33:28.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.371+0000 7f0106000640 1 Processor -- start 2024-09-19T04:33:28.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.371+0000 7f0106000640 1 -- start start 2024-09-19T04:33:28.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f0106000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100074590 0x7f010019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f0106000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0100074eb0 0x7f010019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f0106000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100112b90 0x7f01001a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f0106000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f01001185c0 con 0x7f0100112b90 2024-09-19T04:33:28.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f0106000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0100118440 con 0x7f0100074590 2024-09-19T04:33:28.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f0106000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0100118740 con 0x7f0100074eb0 2024-09-19T04:33:28.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100112b90 0x7f01001a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100112b90 0x7f01001a3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43018/0 (socket says 172.21.15.107:43018) 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f00ffe00640 1 -- 172.21.15.107:0/897707801 learned_addr learned my addr 172.21.15.107:0/897707801 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f00ff400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100074590 0x7f010019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.372+0000 7f00f7e00640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0100074eb0 0x7f010019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f00ff400640 1 -- 172.21.15.107:0/897707801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0100074eb0 msgr2=0x7f010019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f00ff400640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0100074eb0 0x7f010019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f00ff400640 1 -- 172.21.15.107:0/897707801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100112b90 msgr2=0x7f01001a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f00ff400640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100112b90 0x7f01001a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f00ff400640 1 -- 172.21.15.107:0/897707801 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01001a3b80 con 0x7f0100074590 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f00f7e00640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0100074eb0 0x7f010019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f00ff400640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100074590 0x7f010019eb50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f00f002a9a0 tx=0x7f00f0003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:28.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f00fcc00640 1 -- 172.21.15.107:0/897707801 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f003f070 con 0x7f0100074590 2024-09-19T04:33:28.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f0106000640 1 -- 172.21.15.107:0/897707801 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01001a3e10 con 0x7f0100074590 2024-09-19T04:33:28.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f00fcc00640 1 -- 172.21.15.107:0/897707801 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f0031c90 con 0x7f0100074590 2024-09-19T04:33:28.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f00fcc00640 1 -- 172.21.15.107:0/897707801 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0035030 con 0x7f0100074590 2024-09-19T04:33:28.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.373+0000 7f0106000640 1 -- 172.21.15.107:0/897707801 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01001ab700 con 0x7f0100074590 2024-09-19T04:33:28.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.377+0000 7f0108c46740 1 Processor -- start 2024-09-19T04:33:28.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.377+0000 7f0108c46740 1 -- start start 2024-09-19T04:33:28.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f0108c46740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb62c70 0x556c0eb63050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f0108c46740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb63590 0x556c0eb6d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f0108c46740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb6de50 0x556c0eb70240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f0108c46740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556c0eac93e0 con 0x556c0eb62c70 2024-09-19T04:33:28.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f0108c46740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556c0ea70d80 con 0x556c0eb63590 2024-09-19T04:33:28.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f0108c46740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556c0eac53a0 con 0x556c0eb6de50 2024-09-19T04:33:28.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb62c70 0x556c0eb63050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb62c70 0x556c0eb63050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43026/0 (socket says 172.21.15.107:43026) 2024-09-19T04:33:28.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f00ffe00640 1 -- 172.21.15.107:0/1389225729 learned_addr learned my addr 172.21.15.107:0/1389225729 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:28.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f00fcc00640 1 -- 172.21.15.107:0/897707801 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f00f0042070 con 0x7f0100074590 2024-09-19T04:33:28.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f00f7e00640 1 --2- 172.21.15.107:0/1389225729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb6de50 0x556c0eb70240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f00ffe00640 1 -- 172.21.15.107:0/1389225729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb6de50 msgr2=0x556c0eb70240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:28.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f00ffe00640 1 --2- 172.21.15.107:0/1389225729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb6de50 0x556c0eb70240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f00ffe00640 1 -- 172.21.15.107:0/1389225729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb63590 msgr2=0x556c0eb6d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:28.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f00ffe00640 1 --2- 172.21.15.107:0/1389225729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb63590 0x556c0eb6d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.378+0000 7f00ffe00640 1 -- 172.21.15.107:0/1389225729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0ea8fae0 con 0x556c0eb62c70 2024-09-19T04:33:28.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.379+0000 7f00ffe00640 1 --2- 172.21.15.107:0/1389225729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb62c70 0x556c0eb63050 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f00f8011470 tx=0x7f00f8011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224b7613873cc29d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:28.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.379+0000 7f00efe00640 1 -- 172.21.15.107:0/1389225729 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8004b20 con 0x556c0eb62c70 2024-09-19T04:33:28.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.379+0000 7f00efe00640 1 -- 172.21.15.107:0/1389225729 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f8003e50 con 0x556c0eb62c70 2024-09-19T04:33:28.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.379+0000 7f00efe00640 1 -- 172.21.15.107:0/1389225729 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f800f600 con 0x556c0eb62c70 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.380+0000 7f0108c46740 1 -- 172.21.15.107:0/1389225729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb62c70 msgr2=0x556c0eb63050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.380+0000 7f0108c46740 1 --2- 172.21.15.107:0/1389225729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb62c70 0x556c0eb63050 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f00f8011470 tx=0x7f00f8011930 comp rx=0 tx=0).stop 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.380+0000 7f00fcc00640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f00e40a33b0 0x7f00e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.380+0000 7f00fcc00640 1 -- 172.21.15.107:0/897707801 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f00f00f0670 con 0x7f0100074590 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.380+0000 7f0108c46740 1 -- 172.21.15.107:0/1389225729 shutdown_connections 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.380+0000 7f0108c46740 1 --2- 172.21.15.107:0/1389225729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb6de50 0x556c0eb70240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.380+0000 7f0108c46740 1 --2- 172.21.15.107:0/1389225729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb63590 0x556c0eb6d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.380+0000 7f0108c46740 1 --2- 172.21.15.107:0/1389225729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb62c70 0x556c0eb63050 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.380+0000 7f0108c46740 1 -- 172.21.15.107:0/1389225729 >> 172.21.15.107:0/1389225729 conn(0x556c0eb5f200 msgr2=0x556c0eb5f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.380+0000 7f00ff400640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f00e40a33b0 0x7f00e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.381+0000 7f0108c46740 1 -- 172.21.15.107:0/1389225729 shutdown_connections 2024-09-19T04:33:28.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.381+0000 7f0108c46740 1 -- 172.21.15.107:0/1389225729 wait complete. 2024-09-19T04:33:28.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.381+0000 7f00ff400640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f00e40a33b0 0x7f00e40a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x556c0eb60060 tx=0x7f00f0005d50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:28.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.381+0000 7f0108c46740 1 -- start start 2024-09-19T04:33:28.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.382+0000 7f0108c46740 1 Processor -- start 2024-09-19T04:33:28.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.382+0000 7f0108c46740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb6acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.382+0000 7f0108c46740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb63590 0x556c0eb6b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.382+0000 7f0108c46740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb6de50 0x556c0eb6b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.382+0000 7f0108c46740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556c0eb70a20 con 0x556c0eb6de50 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.382+0000 7f0108c46740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556c0ea840f0 con 0x556c0eb63590 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.382+0000 7f0108c46740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556c0eb70ba0 con 0x556c0eb62c70 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.382+0000 7f0108c46740 10 client.0 _ll_register_callbacks cb 0x556c0ea6bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.382+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb6acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.382+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb6de50 0x556c0eb6b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb6de50 0x556c0eb6b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43032/0 (socket says 172.21.15.107:43032) 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00ffe00640 1 -- 172.21.15.107:0/4261037928 learned_addr learned my addr 172.21.15.107:0/4261037928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00f7e00640 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb63590 0x556c0eb6b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00f7e00640 1 -- 172.21.15.107:0/4261037928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 msgr2=0x556c0eb6acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00f7e00640 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb6acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00f7e00640 1 -- 172.21.15.107:0/4261037928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb6de50 msgr2=0x556c0eb6b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00f7e00640 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb6de50 0x556c0eb6b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00f7e00640 1 -- 172.21.15.107:0/4261037928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0e83a3c0 con 0x556c0eb63590 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00ffe00640 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb6de50 0x556c0eb6b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00f7e00640 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb63590 0x556c0eb6b200 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f010019fac0 tx=0x7f00e8012e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c5f397d8df9742 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.383+0000 7f00ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.384+0000 7f00ecc00640 1 -- 172.21.15.107:0/4261037928 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8004050 con 0x556c0eb63590 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.384+0000 7f00ecc00640 1 -- 172.21.15.107:0/4261037928 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f00e80041f0 con 0x556c0eb63590 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.384+0000 7f0108c46740 1 -- 172.21.15.107:0/4261037928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f8003e50 con 0x556c0eb63590 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.384+0000 7f00ecc00640 1 -- 172.21.15.107:0/4261037928 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f00e80043f0 con 0x556c0eb63590 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.384+0000 7f0108c46740 10 client.27575 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.384+0000 7f0108c46740 1 -- 172.21.15.107:0/4261037928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556c0eb7ca40 con 0x556c0eb63590 2024-09-19T04:33:28.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.384+0000 7f00ecc00640 1 -- 172.21.15.107:0/4261037928 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f8003e50 con 0x556c0eb63590 2024-09-19T04:33:28.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.385+0000 7f00ecc00640 1 -- 172.21.15.107:0/4261037928 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f00e8003cb0 con 0x556c0eb63590 2024-09-19T04:33:28.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.385+0000 7f0108c46740 10 client.27575 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:28.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.385+0000 7f0108c46740 10 client.27575 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:33:28.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.385+0000 7f0108c46740 20 client.27575 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.385+0000 7f0108c46740 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x556c0eb7e090 0x556c0eb80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.385+0000 7f0108c46740 4 client.27575 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.385+0000 7f0108c46740 1 -- 172.21.15.107:0/4261037928 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c0ead6950 con 0x556c0eb7e090 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.385+0000 7f00ffe00640 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x556c0eb7e090 0x556c0eb80480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.386+0000 7f00ffe00640 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x556c0eb7e090 0x556c0eb80480 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a54393a15a18503 server_cookie=40d9740d053f7be0 in_seq=0 out_seq=0 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.386+0000 7f00ecc00640 10 client.27575 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f00ecc00640 1 -- 172.21.15.107:0/4261037928 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c0ead6950 con 0x556c0eb7e090 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f00ecc00640 10 client.27575 handle_command_reply: tid=0 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 client.27575 shutdown 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 -- 172.21.15.107:0/4261037928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb63590 msgr2=0x556c0eb6b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb63590 0x556c0eb6b200 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f010019fac0 tx=0x7f00e8012e70 comp rx=0 tx=0).stop 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 -- 172.21.15.107:0/4261037928 shutdown_connections 2024-09-19T04:33:28.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x556c0eb7e090 0x556c0eb80480 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb6de50 0x556c0eb6b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb63590 0x556c0eb6b200 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 --2- 172.21.15.107:0/4261037928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb6acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 -- 172.21.15.107:0/4261037928 >> 172.21.15.107:0/4261037928 conn(0x556c0eb5f200 msgr2=0x556c0eb64f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:28.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f00ecc00640 0 client.27575 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:33:28.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 -- 172.21.15.107:0/4261037928 shutdown_connections 2024-09-19T04:33:28.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 1 -- 172.21.15.107:0/4261037928 wait complete. 2024-09-19T04:33:28.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.387+0000 7f0108c46740 20 client.27575 trim_cache size 0 max 16384 2024-09-19T04:33:28.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f0108c46740 1 Processor -- start 2024-09-19T04:33:28.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f0108c46740 1 -- start start 2024-09-19T04:33:28.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f0108c46740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb6cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f0108c46740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb85c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f0108c46740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb6de50 0x556c0eb861a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f0108c46740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f01001b3f60 con 0x556c0eb63590 2024-09-19T04:33:28.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f0108c46740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01001b5ad0 con 0x556c0eb6de50 2024-09-19T04:33:28.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f0108c46740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f01001b4660 con 0x556c0eb62c70 2024-09-19T04:33:28.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb85c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb85c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43046/0 (socket says 172.21.15.107:43046) 2024-09-19T04:33:28.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f00f7e00640 1 -- 172.21.15.107:0/391828885 learned_addr learned my addr 172.21.15.107:0/391828885 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:28.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.401+0000 7f00ffe00640 1 --2- 172.21.15.107:0/391828885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb6cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.402+0000 7f00ffe00640 1 --2- 172.21.15.107:0/391828885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb6de50 0x556c0eb861a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.402+0000 7f00f7e00640 1 -- 172.21.15.107:0/391828885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 msgr2=0x556c0eb6cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:28.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.402+0000 7f00f7e00640 1 --2- 172.21.15.107:0/391828885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb6cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.402+0000 7f00f7e00640 1 -- 172.21.15.107:0/391828885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb6de50 msgr2=0x556c0eb861a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:28.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.402+0000 7f00f7e00640 1 --2- 172.21.15.107:0/391828885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb6de50 0x556c0eb861a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.402+0000 7f00f7e00640 1 -- 172.21.15.107:0/391828885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0eb5f030 con 0x556c0eb63590 2024-09-19T04:33:28.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.402+0000 7f00f7e00640 1 --2- 172.21.15.107:0/391828885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb85c60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f00e8000c00 tx=0x7f00e8010850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a746561f58697abb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:28.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.402+0000 7f00ef400640 1 -- 172.21.15.107:0/391828885 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8029040 con 0x556c0eb63590 2024-09-19T04:33:28.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.402+0000 7f00ef400640 1 -- 172.21.15.107:0/391828885 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e8010b40 con 0x556c0eb63590 2024-09-19T04:33:28.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.402+0000 7f00ef400640 1 -- 172.21.15.107:0/391828885 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8005330 con 0x556c0eb63590 2024-09-19T04:33:28.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.403+0000 7f0108c46740 1 -- 172.21.15.107:0/391828885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 msgr2=0x556c0eb85c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:28.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.403+0000 7f0108c46740 1 --2- 172.21.15.107:0/391828885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb85c60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f00e8000c00 tx=0x7f00e8010850 comp rx=0 tx=0).stop 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.403+0000 7f0108c46740 1 -- 172.21.15.107:0/391828885 shutdown_connections 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.403+0000 7f0108c46740 1 --2- 172.21.15.107:0/391828885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb6de50 0x556c0eb861a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.403+0000 7f0108c46740 1 --2- 172.21.15.107:0/391828885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb85c60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.403+0000 7f0108c46740 1 --2- 172.21.15.107:0/391828885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb6cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.403+0000 7f0108c46740 1 -- 172.21.15.107:0/391828885 >> 172.21.15.107:0/391828885 conn(0x556c0eb5f200 msgr2=0x556c0eb6c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.403+0000 7f0108c46740 1 -- 172.21.15.107:0/391828885 shutdown_connections 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.403+0000 7f0108c46740 1 -- 172.21.15.107:0/391828885 wait complete. 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.404+0000 7f0108c46740 1 -- start start 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.404+0000 7f0108c46740 1 Processor -- start 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.404+0000 7f0108c46740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb67cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.404+0000 7f0108c46740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.404+0000 7f0108c46740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb6de50 0x556c0eb68730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.404+0000 7f0108c46740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f01001b5400 con 0x556c0eb63590 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.404+0000 7f0108c46740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01001b4d30 con 0x556c0eb6de50 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.404+0000 7f0108c46740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f01001b1f60 con 0x556c0eb62c70 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.404+0000 7f0108c46740 10 client.0 _ll_register_callbacks cb 0x7f00e8009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb67cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb681f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43056/0 (socket says 172.21.15.107:43056) 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb67cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42938/0 (socket says 172.21.15.107:42938) 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00f7e00640 1 -- 172.21.15.107:0/204650310 learned_addr learned my addr 172.21.15.107:0/204650310 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00ffe00640 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb6de50 0x556c0eb68730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00f7e00640 1 -- 172.21.15.107:0/204650310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 msgr2=0x556c0eb67cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00f7e00640 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb67cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00f7e00640 1 -- 172.21.15.107:0/204650310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb6de50 msgr2=0x556c0eb68730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00f7e00640 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb6de50 0x556c0eb68730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00f7e00640 1 -- 172.21.15.107:0/204650310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0eb5fe90 con 0x556c0eb63590 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00ffe00640 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb67cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00f7e00640 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb681f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f00e800a010 tx=0x7f00e8009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bf7160985d8870 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.405+0000 7f00ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.406+0000 7f00ecc00640 1 -- 172.21.15.107:0/204650310 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8029070 con 0x556c0eb63590 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.406+0000 7f0108c46740 1 -- 172.21.15.107:0/204650310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e8010b40 con 0x556c0eb63590 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.406+0000 7f00ecc00640 1 -- 172.21.15.107:0/204650310 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f00e8018040 con 0x556c0eb63590 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.406+0000 7f00ecc00640 1 -- 172.21.15.107:0/204650310 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f00e800fb40 con 0x556c0eb63590 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.406+0000 7f0108c46740 10 client.17538 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:28.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.406+0000 7f0108c46740 1 -- 172.21.15.107:0/204650310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556c0ebe8d00 con 0x556c0eb63590 2024-09-19T04:33:28.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.406+0000 7f00ecc00640 1 -- 172.21.15.107:0/204650310 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e8010b40 con 0x556c0eb63590 2024-09-19T04:33:28.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.407+0000 7f00ecc00640 1 -- 172.21.15.107:0/204650310 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f00e8040020 con 0x556c0eb63590 2024-09-19T04:33:28.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.407+0000 7f0108c46740 10 client.17538 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:28.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.407+0000 7f0108c46740 10 client.17538 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:33:28.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.407+0000 7f0108c46740 20 client.17538 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:28.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.407+0000 7f0108c46740 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x556c0ebe9300 0x556c0ebeb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:28.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.407+0000 7f0108c46740 4 client.17538 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:33:28.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.407+0000 7f0108c46740 1 -- 172.21.15.107:0/204650310 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c0ead0e60 con 0x556c0ebe9300 2024-09-19T04:33:28.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.407+0000 7f00ffe00640 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x556c0ebe9300 0x556c0ebeb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f00ffe00640 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x556c0ebe9300 0x556c0ebeb6f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bec1d5ffd48cacdc server_cookie=5a069a35e6d3988d in_seq=0 out_seq=0 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f00ecc00640 10 client.17538 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f00ecc00640 1 -- 172.21.15.107:0/204650310 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c0ead0e60 con 0x556c0ebe9300 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f00ecc00640 10 client.17538 handle_command_reply: tid=0 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 client.17538 shutdown 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 -- 172.21.15.107:0/204650310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 msgr2=0x556c0eb681f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb681f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f00e800a010 tx=0x7f00e8009340 comp rx=0 tx=0).stop 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 -- 172.21.15.107:0/204650310 shutdown_connections 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x556c0ebe9300 0x556c0ebeb6f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c0eb6de50 0x556c0eb68730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c0eb63590 0x556c0eb681f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 --2- 172.21.15.107:0/204650310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c0eb62c70 0x556c0eb67cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 -- 172.21.15.107:0/204650310 >> 172.21.15.107:0/204650310 conn(0x556c0eb5f200 msgr2=0x556c0eb6cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:28.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f00ecc00640 0 client.17538 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:33:28.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 -- 172.21.15.107:0/204650310 shutdown_connections 2024-09-19T04:33:28.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 1 -- 172.21.15.107:0/204650310 wait complete. 2024-09-19T04:33:28.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.408+0000 7f0108c46740 20 client.17538 trim_cache size 0 max 16384 2024-09-19T04:33:28.414 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 -- 172.21.15.107:0/897707801 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f00e40a33b0 msgr2=0x7f00e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f00e40a33b0 0x7f00e40a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x556c0eb60060 tx=0x7f00f0005d50 comp rx=0 tx=0).stop 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 -- 172.21.15.107:0/897707801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100074590 msgr2=0x7f010019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100074590 0x7f010019eb50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f00f002a9a0 tx=0x7f00f0003b20 comp rx=0 tx=0).stop 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 -- 172.21.15.107:0/897707801 shutdown_connections 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f00e40a33b0 0x7f00e40a5870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100112b90 0x7f01001a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0100074eb0 0x7f010019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 --2- 172.21.15.107:0/897707801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100074590 0x7f010019eb50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 -- 172.21.15.107:0/897707801 >> 172.21.15.107:0/897707801 conn(0x7f01000fd8c0 msgr2=0x7f01000ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 -- 172.21.15.107:0/897707801 shutdown_connections 2024-09-19T04:33:28.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:28.409+0000 7f0106000640 1 -- 172.21.15.107:0/897707801 wait complete. 2024-09-19T04:33:28.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 455730 3.01 MB/sec execute 586 sec latency 1538.246 ms 2024-09-19T04:33:29.116 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:28 smithi107 ceph-mon[30659]: pgmap v411: 97 pgs: 97 active+clean; 405 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-19T04:33:29.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:28 smithi115 ceph-mon[35477]: pgmap v411: 97 pgs: 97 active+clean; 405 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-19T04:33:29.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:28 smithi145 ceph-mon[35096]: pgmap v411: 97 pgs: 97 active+clean; 405 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-19T04:33:29.518 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:33:29.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 455733 3.01 MB/sec execute 587 sec latency 2449.476 ms 2024-09-19T04:33:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:30.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.460+0000 7f7519400640 1 -- 172.21.15.107:0/547725805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75141019c0 msgr2=0x7f7514101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:30.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.460+0000 7f7519400640 1 --2- 172.21.15.107:0/547725805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75141019c0 0x7f7514101e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7500009930 tx=0x7f750002a440 comp rx=0 tx=0).stop 2024-09-19T04:33:30.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.460+0000 7f7519400640 1 -- 172.21.15.107:0/547725805 shutdown_connections 2024-09-19T04:33:30.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.460+0000 7f7519400640 1 --2- 172.21.15.107:0/547725805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7514102360 0x7f751410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.460+0000 7f7519400640 1 --2- 172.21.15.107:0/547725805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75141019c0 0x7f7514101e20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.460+0000 7f7519400640 1 --2- 172.21.15.107:0/547725805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75141079c0 0x7f7514107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.460+0000 7f7519400640 1 -- 172.21.15.107:0/547725805 >> 172.21.15.107:0/547725805 conn(0x7f75140fd740 msgr2=0x7f75140ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:30.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.460+0000 7f7519400640 1 -- 172.21.15.107:0/547725805 shutdown_connections 2024-09-19T04:33:30.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.461+0000 7f7519400640 1 -- 172.21.15.107:0/547725805 wait complete. 2024-09-19T04:33:30.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.461+0000 7f7519400640 1 Processor -- start 2024-09-19T04:33:30.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7519400640 1 -- start start 2024-09-19T04:33:30.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7519400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75141019c0 0x7f751419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7519400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7514102360 0x7f751419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7519400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75141079c0 0x7f751419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7519400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7514111dc0 con 0x7f7514102360 2024-09-19T04:33:30.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7519400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7514111c40 con 0x7f75141079c0 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7519400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7514111f40 con 0x7f75141019c0 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7513e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75141019c0 0x7f751419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7513400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7514102360 0x7f751419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7513e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75141019c0 0x7f751419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51078/0 (socket says 172.21.15.107:51078) 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7513e00640 1 -- 172.21.15.107:0/1826109065 learned_addr learned my addr 172.21.15.107:0/1826109065 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.462+0000 7f7518a00640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75141079c0 0x7f751419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.463+0000 7f7513e00640 1 -- 172.21.15.107:0/1826109065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75141079c0 msgr2=0x7f751419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.463+0000 7f7513e00640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75141079c0 0x7f751419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.463+0000 7f7513e00640 1 -- 172.21.15.107:0/1826109065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7514102360 msgr2=0x7f751419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.463+0000 7f7513e00640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7514102360 0x7f751419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.463+0000 7f7513e00640 1 -- 172.21.15.107:0/1826109065 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751419f480 con 0x7f75141019c0 2024-09-19T04:33:30.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.463+0000 7f7513e00640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75141019c0 0x7f751419a4d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f750400e9f0 tx=0x7f750400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:30.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.463+0000 7f7510c00640 1 -- 172.21.15.107:0/1826109065 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750400cd80 con 0x7f75141019c0 2024-09-19T04:33:30.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.463+0000 7f7510c00640 1 -- 172.21.15.107:0/1826109065 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7504005500 con 0x7f75141019c0 2024-09-19T04:33:30.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.463+0000 7f7519400640 1 -- 172.21.15.107:0/1826109065 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751419f770 con 0x7f75141019c0 2024-09-19T04:33:30.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.463+0000 7f7510c00640 1 -- 172.21.15.107:0/1826109065 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504010690 con 0x7f75141019c0 2024-09-19T04:33:30.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.464+0000 7f7519400640 1 -- 172.21.15.107:0/1826109065 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75141a7050 con 0x7f75141019c0 2024-09-19T04:33:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.467+0000 7f7510c00640 1 -- 172.21.15.107:0/1826109065 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f75040046c0 con 0x7f75141019c0 2024-09-19T04:33:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.467+0000 7f751d45b740 1 Processor -- start 2024-09-19T04:33:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.467+0000 7f751d45b740 1 -- start start 2024-09-19T04:33:30.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.467+0000 7f751d45b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d769e40 0x563f9d76a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.467+0000 7f751d45b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d76a760 0x563f9d774ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.467+0000 7f751d45b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d775020 0x563f9d777410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.467+0000 7f751d45b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f9d6d03e0 con 0x563f9d769e40 2024-09-19T04:33:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.467+0000 7f751d45b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563f9d677d80 con 0x563f9d775020 2024-09-19T04:33:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.467+0000 7f751d45b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563f9d6cc3a0 con 0x563f9d76a760 2024-09-19T04:33:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7513e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d76a760 0x563f9d774ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7518a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d769e40 0x563f9d76a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7513e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d76a760 0x563f9d774ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51092/0 (socket says 172.21.15.107:51092) 2024-09-19T04:33:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7513e00640 1 -- 172.21.15.107:0/3009619101 learned_addr learned my addr 172.21.15.107:0/3009619101 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:30.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7513400640 1 --2- 172.21.15.107:0/3009619101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d775020 0x563f9d777410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7518a00640 1 -- 172.21.15.107:0/3009619101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d76a760 msgr2=0x563f9d774ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:30.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7518a00640 1 --2- 172.21.15.107:0/3009619101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d76a760 0x563f9d774ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7518a00640 1 -- 172.21.15.107:0/3009619101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d775020 msgr2=0x563f9d777410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:30.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7518a00640 1 --2- 172.21.15.107:0/3009619101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d775020 0x563f9d777410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7518a00640 1 -- 172.21.15.107:0/3009619101 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f9d696ae0 con 0x563f9d769e40 2024-09-19T04:33:30.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7513e00640 1 --2- 172.21.15.107:0/3009619101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d76a760 0x563f9d774ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:30.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7513400640 1 --2- 172.21.15.107:0/3009619101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d775020 0x563f9d777410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:30.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.468+0000 7f7510c00640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f74f80a3430 0x7f74f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.469+0000 7f7518a00640 1 --2- 172.21.15.107:0/3009619101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d769e40 0x563f9d76a220 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f750c009f00 tx=0x7f750c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb29de1086f8a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:30.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.469+0000 7f7510c00640 1 -- 172.21.15.107:0/1826109065 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f75040d00b0 con 0x7f75141019c0 2024-09-19T04:33:30.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.469+0000 7f7513e00640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f74f80a3430 0x7f74f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.469+0000 7f7509200640 1 -- 172.21.15.107:0/3009619101 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c004d20 con 0x563f9d769e40 2024-09-19T04:33:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.469+0000 7f7509200640 1 -- 172.21.15.107:0/3009619101 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750c004030 con 0x563f9d769e40 2024-09-19T04:33:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.469+0000 7f7509200640 1 -- 172.21.15.107:0/3009619101 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c005260 con 0x563f9d769e40 2024-09-19T04:33:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.470+0000 7f751d45b740 1 -- 172.21.15.107:0/3009619101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d769e40 msgr2=0x563f9d76a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.470+0000 7f7513e00640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f74f80a3430 0x7f74f80a58f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7504098d20 tx=0x7f75040bc040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.470+0000 7f751d45b740 1 --2- 172.21.15.107:0/3009619101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d769e40 0x563f9d76a220 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f750c009f00 tx=0x7f750c011810 comp rx=0 tx=0).stop 2024-09-19T04:33:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.470+0000 7f751d45b740 1 -- 172.21.15.107:0/3009619101 shutdown_connections 2024-09-19T04:33:30.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.470+0000 7f751d45b740 1 --2- 172.21.15.107:0/3009619101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d775020 0x563f9d777410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.470+0000 7f751d45b740 1 --2- 172.21.15.107:0/3009619101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d76a760 0x563f9d774ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.470+0000 7f751d45b740 1 --2- 172.21.15.107:0/3009619101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d769e40 0x563f9d76a220 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.470+0000 7f751d45b740 1 -- 172.21.15.107:0/3009619101 >> 172.21.15.107:0/3009619101 conn(0x563f9d7663d0 msgr2=0x563f9d7667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.470+0000 7f751d45b740 1 -- 172.21.15.107:0/3009619101 shutdown_connections 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.470+0000 7f751d45b740 1 -- 172.21.15.107:0/3009619101 wait complete. 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.471+0000 7f751d45b740 1 -- start start 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.471+0000 7f751d45b740 1 Processor -- start 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.471+0000 7f751d45b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d769e40 0x563f9d771f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.471+0000 7f751d45b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d76a760 0x563f9d772440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.471+0000 7f751d45b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d775020 0x563f9d772980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.471+0000 7f751d45b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f9d777bf0 con 0x563f9d775020 2024-09-19T04:33:30.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.471+0000 7f751d45b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563f9d68b0f0 con 0x563f9d76a760 2024-09-19T04:33:30.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.471+0000 7f751d45b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563f9d777d70 con 0x563f9d769e40 2024-09-19T04:33:30.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.471+0000 7f751d45b740 10 client.0 _ll_register_callbacks cb 0x563f9d672bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7513400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d76a760 0x563f9d772440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7513400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d76a760 0x563f9d772440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39528/0 (socket says 172.21.15.107:39528) 2024-09-19T04:33:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7513400640 1 -- 172.21.15.107:0/346499538 learned_addr learned my addr 172.21.15.107:0/346499538 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7518a00640 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d769e40 0x563f9d771f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7518a00640 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d775020 0x563f9d772980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7513400640 1 -- 172.21.15.107:0/346499538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d769e40 msgr2=0x563f9d771f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7513400640 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d769e40 0x563f9d771f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7513400640 1 -- 172.21.15.107:0/346499538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d775020 msgr2=0x563f9d772980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7513400640 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d775020 0x563f9d772980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7513400640 1 -- 172.21.15.107:0/346499538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f9d4413c0 con 0x563f9d76a760 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f7513400640 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d76a760 0x563f9d772440 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f750002a960 tx=0x7f75000317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50ad4618fcf66794 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.472+0000 7f74fd600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.473+0000 7f74fd600640 1 -- 172.21.15.107:0/346499538 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500035040 con 0x563f9d76a760 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.473+0000 7f74fd600640 1 -- 172.21.15.107:0/346499538 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f7500004240 con 0x563f9d76a760 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.473+0000 7f751d45b740 1 -- 172.21.15.107:0/346499538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f750c004030 con 0x563f9d76a760 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.473+0000 7f74fd600640 1 -- 172.21.15.107:0/346499538 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7500004440 con 0x563f9d76a760 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.473+0000 7f751d45b740 10 client.27593 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.473+0000 7f751d45b740 1 -- 172.21.15.107:0/346499538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x563f9d783c30 con 0x563f9d76a760 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.473+0000 7f74fd600640 1 -- 172.21.15.107:0/346499538 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750c004030 con 0x563f9d76a760 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.474+0000 7f74fd600640 1 -- 172.21.15.107:0/346499538 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f7500046600 con 0x563f9d76a760 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.474+0000 7f751d45b740 10 client.27593 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.474+0000 7f751d45b740 10 client.27593 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:33:30.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.474+0000 7f751d45b740 20 client.27593 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.474+0000 7f751d45b740 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x563f9d785140 0x563f9d787560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.474+0000 7f751d45b740 4 client.27593 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.474+0000 7f751d45b740 1 -- 172.21.15.107:0/346499538 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f9d6dd950 con 0x563f9d785140 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.474+0000 7f7518a00640 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x563f9d785140 0x563f9d787560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.475+0000 7f7518a00640 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x563f9d785140 0x563f9d787560 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0e5c64236e1d7b8 server_cookie=5ba9a7ffd003f49f in_seq=0 out_seq=0 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.475+0000 7f74fd600640 10 client.27593 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.475+0000 7f74fd600640 1 -- 172.21.15.107:0/346499538 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f9d6dd950 con 0x563f9d785140 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.475+0000 7f74fd600640 10 client.27593 handle_command_reply: tid=0 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.475+0000 7f751d45b740 1 client.27593 shutdown 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 1 -- 172.21.15.107:0/346499538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d76a760 msgr2=0x563f9d772440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d76a760 0x563f9d772440 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f750002a960 tx=0x7f75000317e0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 1 -- 172.21.15.107:0/346499538 shutdown_connections 2024-09-19T04:33:30.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x563f9d785140 0x563f9d787560 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d775020 0x563f9d772980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d76a760 0x563f9d772440 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 1 --2- 172.21.15.107:0/346499538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d769e40 0x563f9d771f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 1 -- 172.21.15.107:0/346499538 >> 172.21.15.107:0/346499538 conn(0x563f9d7663d0 msgr2=0x563f9d76c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:30.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f74fd600640 0 client.27593 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:33:30.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 1 -- 172.21.15.107:0/346499538 shutdown_connections 2024-09-19T04:33:30.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 1 -- 172.21.15.107:0/346499538 wait complete. 2024-09-19T04:33:30.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.476+0000 7f751d45b740 20 client.27593 trim_cache size 0 max 16384 2024-09-19T04:33:30.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f751d45b740 1 Processor -- start 2024-09-19T04:33:30.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f751d45b740 1 -- start start 2024-09-19T04:33:30.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f751d45b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d7727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f751d45b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 0x563f9d78ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f751d45b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 0x563f9d78d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f751d45b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75141af850 con 0x563f9d76a760 2024-09-19T04:33:30.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f751d45b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75141b13c0 con 0x563f9d769e40 2024-09-19T04:33:30.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f751d45b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75141aff50 con 0x563f9d775020 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f7518a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d7727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f7518a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d7727d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39532/0 (socket says 172.21.15.107:39532) 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f7518a00640 1 -- 172.21.15.107:0/2637219392 learned_addr learned my addr 172.21.15.107:0/2637219392 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f7513400640 1 --2- 172.21.15.107:0/2637219392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 0x563f9d78ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.492+0000 7f7518a00640 1 --2- 172.21.15.107:0/2637219392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 0x563f9d78d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.493+0000 7f7518a00640 1 -- 172.21.15.107:0/2637219392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 msgr2=0x563f9d78d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.493+0000 7f7518a00640 1 --2- 172.21.15.107:0/2637219392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 0x563f9d78d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.493+0000 7f7518a00640 1 -- 172.21.15.107:0/2637219392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 msgr2=0x563f9d78ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.493+0000 7f7518a00640 1 --2- 172.21.15.107:0/2637219392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 0x563f9d78ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.493+0000 7f7518a00640 1 -- 172.21.15.107:0/2637219392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f9d777ad0 con 0x563f9d769e40 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.493+0000 7f7518a00640 1 --2- 172.21.15.107:0/2637219392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d7727d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f750c005b40 tx=0x7f750c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0279d6e910eb8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:30.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.493+0000 7f74ffe00640 1 -- 172.21.15.107:0/2637219392 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c01e040 con 0x563f9d769e40 2024-09-19T04:33:30.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.493+0000 7f74ffe00640 1 -- 172.21.15.107:0/2637219392 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750c00f480 con 0x563f9d769e40 2024-09-19T04:33:30.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.493+0000 7f74ffe00640 1 -- 172.21.15.107:0/2637219392 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c019650 con 0x563f9d769e40 2024-09-19T04:33:30.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.494+0000 7f751d45b740 1 -- 172.21.15.107:0/2637219392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 msgr2=0x563f9d7727d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:30.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.494+0000 7f751d45b740 1 --2- 172.21.15.107:0/2637219392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d7727d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f750c005b40 tx=0x7f750c009500 comp rx=0 tx=0).stop 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.494+0000 7f751d45b740 1 -- 172.21.15.107:0/2637219392 shutdown_connections 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.494+0000 7f751d45b740 1 --2- 172.21.15.107:0/2637219392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 0x563f9d78d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.494+0000 7f751d45b740 1 --2- 172.21.15.107:0/2637219392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 0x563f9d78ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.494+0000 7f751d45b740 1 --2- 172.21.15.107:0/2637219392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d7727d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.494+0000 7f751d45b740 1 -- 172.21.15.107:0/2637219392 >> 172.21.15.107:0/2637219392 conn(0x563f9d7663d0 msgr2=0x563f9d76c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.494+0000 7f751d45b740 1 -- 172.21.15.107:0/2637219392 shutdown_connections 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.494+0000 7f751d45b740 1 -- 172.21.15.107:0/2637219392 wait complete. 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.495+0000 7f751d45b740 1 -- start start 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.495+0000 7f751d45b740 1 Processor -- start 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.495+0000 7f751d45b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d77d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.495+0000 7f751d45b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 0x563f9d77d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.495+0000 7f751d45b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 0x563f9d76e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.495+0000 7f751d45b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75141b0cf0 con 0x563f9d76a760 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.495+0000 7f751d45b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75141b0620 con 0x563f9d769e40 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.495+0000 7f751d45b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75141ad850 con 0x563f9d775020 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.495+0000 7f751d45b740 10 client.0 _ll_register_callbacks cb 0x7f7500009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7518a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d77d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7518a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d77d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39538/0 (socket says 172.21.15.107:39538) 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7518a00640 1 -- 172.21.15.107:0/122556463 learned_addr learned my addr 172.21.15.107:0/122556463 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7518a00640 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 0x563f9d76e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7513400640 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 0x563f9d77d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7513400640 1 -- 172.21.15.107:0/122556463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 msgr2=0x563f9d76e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:30.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7513400640 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 0x563f9d76e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7513400640 1 -- 172.21.15.107:0/122556463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 msgr2=0x563f9d77d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:30.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7513400640 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d77d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7513400640 1 -- 172.21.15.107:0/122556463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f9d76ef10 con 0x563f9d76a760 2024-09-19T04:33:30.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7518a00640 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 0x563f9d76e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:30.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f7513400640 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 0x563f9d77d6e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7500000c00 tx=0x7f7500046e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a6ffe46f39f470 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:30.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.496+0000 7f74fd600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:33:30.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.497+0000 7f74fd600640 1 -- 172.21.15.107:0/122556463 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750005c060 con 0x563f9d76a760 2024-09-19T04:33:30.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.497+0000 7f74fd600640 1 -- 172.21.15.107:0/122556463 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f7500046680 con 0x563f9d76a760 2024-09-19T04:33:30.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.497+0000 7f751d45b740 1 -- 172.21.15.107:0/122556463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f750c00f480 con 0x563f9d76a760 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.497+0000 7f74fd600640 1 -- 172.21.15.107:0/122556463 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7500035050 con 0x563f9d76a760 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.497+0000 7f751d45b740 10 client.17556 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.497+0000 7f751d45b740 1 -- 172.21.15.107:0/122556463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x563f9d7ef860 con 0x563f9d76a760 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.497+0000 7f74fd600640 1 -- 172.21.15.107:0/122556463 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750c00f480 con 0x563f9d76a760 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.498+0000 7f74fd600640 1 -- 172.21.15.107:0/122556463 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f7500056070 con 0x563f9d76a760 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.498+0000 7f751d45b740 10 client.17556 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.498+0000 7f751d45b740 10 client.17556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.498+0000 7f751d45b740 20 client.17556 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.498+0000 7f751d45b740 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x563f9d7f0010 0x563f9d7f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.498+0000 7f751d45b740 4 client.17556 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.498+0000 7f751d45b740 1 -- 172.21.15.107:0/122556463 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f9d6d7e60 con 0x563f9d7f0010 2024-09-19T04:33:30.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.498+0000 7f7518a00640 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x563f9d7f0010 0x563f9d7f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:30.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.499+0000 7f74fd600640 10 client.17556 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:33:30.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.499+0000 7f7518a00640 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x563f9d7f0010 0x563f9d7f2430 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7cf565f9b26de5 server_cookie=4e61880710ce3e05 in_seq=0 out_seq=0 2024-09-19T04:33:30.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.499+0000 7f74fd600640 1 -- 172.21.15.107:0/122556463 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f9d6d7e60 con 0x563f9d7f0010 2024-09-19T04:33:30.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.499+0000 7f74fd600640 10 client.17556 handle_command_reply: tid=0 2024-09-19T04:33:30.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.499+0000 7f751d45b740 1 client.17556 shutdown 2024-09-19T04:33:30.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.499+0000 7f751d45b740 1 -- 172.21.15.107:0/122556463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 msgr2=0x563f9d77d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:30.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.499+0000 7f751d45b740 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 0x563f9d77d6e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7500000c00 tx=0x7f7500046e00 comp rx=0 tx=0).stop 2024-09-19T04:33:30.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.500+0000 7f751d45b740 1 -- 172.21.15.107:0/122556463 shutdown_connections 2024-09-19T04:33:30.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.500+0000 7f751d45b740 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x563f9d7f0010 0x563f9d7f2430 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.500+0000 7f751d45b740 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f9d775020 0x563f9d76e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.500+0000 7f751d45b740 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f9d76a760 0x563f9d77d6e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.500+0000 7f751d45b740 1 --2- 172.21.15.107:0/122556463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f9d769e40 0x563f9d77d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.500+0000 7f751d45b740 1 -- 172.21.15.107:0/122556463 >> 172.21.15.107:0/122556463 conn(0x563f9d7663d0 msgr2=0x563f9d7707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:30.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.500+0000 7f74fd600640 0 client.17556 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:33:30.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.500+0000 7f751d45b740 1 -- 172.21.15.107:0/122556463 shutdown_connections 2024-09-19T04:33:30.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.500+0000 7f751d45b740 1 -- 172.21.15.107:0/122556463 wait complete. 2024-09-19T04:33:30.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.500+0000 7f751d45b740 20 client.17556 trim_cache size 0 max 16384 2024-09-19T04:33:30.511 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:33:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.501+0000 7f7519400640 1 -- 172.21.15.107:0/1826109065 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f74f80a3430 msgr2=0x7f74f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.501+0000 7f7519400640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f74f80a3430 0x7f74f80a58f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7504098d20 tx=0x7f75040bc040 comp rx=0 tx=0).stop 2024-09-19T04:33:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.501+0000 7f7519400640 1 -- 172.21.15.107:0/1826109065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75141019c0 msgr2=0x7f751419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.501+0000 7f7519400640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75141019c0 0x7f751419a4d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f750400e9f0 tx=0x7f750400edf0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.501+0000 7f7519400640 1 -- 172.21.15.107:0/1826109065 shutdown_connections 2024-09-19T04:33:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.501+0000 7f7519400640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f74f80a3430 0x7f74f80a58f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.501+0000 7f7519400640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75141079c0 0x7f751419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.501+0000 7f7519400640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7514102360 0x7f751419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.501+0000 7f7519400640 1 --2- 172.21.15.107:0/1826109065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75141019c0 0x7f751419a4d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.501+0000 7f7519400640 1 -- 172.21.15.107:0/1826109065 >> 172.21.15.107:0/1826109065 conn(0x7f75140fd740 msgr2=0x7f75140fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.502+0000 7f7519400640 1 -- 172.21.15.107:0/1826109065 shutdown_connections 2024-09-19T04:33:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:30.502+0000 7f7519400640 1 -- 172.21.15.107:0/1826109065 wait complete. 2024-09-19T04:33:30.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 457722 3.02 MB/sec execute 588 sec latency 100.910 ms 2024-09-19T04:33:31.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:30 smithi107 ceph-mon[30659]: pgmap v412: 97 pgs: 97 active+clean; 405 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-19T04:33:31.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:30 smithi115 ceph-mon[35477]: pgmap v412: 97 pgs: 97 active+clean; 405 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-19T04:33:31.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:30 smithi145 ceph-mon[35096]: pgmap v412: 97 pgs: 97 active+clean; 405 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-19T04:33:31.377 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:33:31.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458345 3.02 MB/sec execute 589 sec latency 39.209 ms 2024-09-19T04:33:31.791 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:32.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.381+0000 7f672fe00640 1 -- 172.21.15.107:0/1694577605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6728102790 msgr2=0x7f67281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.381+0000 7f672fe00640 1 --2- 172.21.15.107:0/1694577605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6728102790 0x7f67281067e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6714009860 tx=0x7f671402a340 comp rx=0 tx=0).stop 2024-09-19T04:33:32.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.381+0000 7f672fe00640 1 -- 172.21.15.107:0/1694577605 shutdown_connections 2024-09-19T04:33:32.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.381+0000 7f672fe00640 1 --2- 172.21.15.107:0/1694577605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728106f10 0x7f672810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.381+0000 7f672fe00640 1 --2- 172.21.15.107:0/1694577605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6728102790 0x7f67281067e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.381+0000 7f672fe00640 1 --2- 172.21.15.107:0/1694577605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6728101de0 0x7f67281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.381+0000 7f672fe00640 1 -- 172.21.15.107:0/1694577605 >> 172.21.15.107:0/1694577605 conn(0x7f67280fd8c0 msgr2=0x7f67280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.381+0000 7f672fe00640 1 -- 172.21.15.107:0/1694577605 shutdown_connections 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.381+0000 7f672fe00640 1 -- 172.21.15.107:0/1694577605 wait complete. 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.382+0000 7f672fe00640 1 Processor -- start 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.382+0000 7f672fe00640 1 -- start start 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6728101de0 0x7f672819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6728102790 0x7f672819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728106f10 0x7f672819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6728111f70 con 0x7f6728106f10 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672fe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6728111df0 con 0x7f6728102790 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f67281120f0 con 0x7f6728101de0 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728106f10 0x7f672819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728106f10 0x7f672819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37372/0 (socket says 172.21.15.107:37372) 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672de00640 1 -- 172.21.15.107:0/2428472535 learned_addr learned my addr 172.21.15.107:0/2428472535 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672d400640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6728101de0 0x7f672819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672ca00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6728102790 0x7f672819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672de00640 1 -- 172.21.15.107:0/2428472535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6728101de0 msgr2=0x7f672819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672de00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6728101de0 0x7f672819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672de00640 1 -- 172.21.15.107:0/2428472535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6728102790 msgr2=0x7f672819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672de00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6728102790 0x7f672819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672de00640 1 -- 172.21.15.107:0/2428472535 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672819f620 con 0x7f6728106f10 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672d400640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6728101de0 0x7f672819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.383+0000 7f672ca00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6728102790 0x7f672819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.384+0000 7f672de00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728106f10 0x7f672819ef40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f672000efc0 tx=0x7f672000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.384+0000 7f6726000640 1 -- 172.21.15.107:0/2428472535 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720003970 con 0x7f6728106f10 2024-09-19T04:33:32.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.384+0000 7f672fe00640 1 -- 172.21.15.107:0/2428472535 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672819f910 con 0x7f6728106f10 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.384+0000 7f6726000640 1 -- 172.21.15.107:0/2428472535 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6720004020 con 0x7f6728106f10 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.384+0000 7f6726000640 1 -- 172.21.15.107:0/2428472535 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672000b9f0 con 0x7f6728106f10 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.384+0000 7f672fe00640 1 -- 172.21.15.107:0/2428472535 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67281a71f0 con 0x7f6728106f10 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.388+0000 7f6732886740 1 Processor -- start 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.388+0000 7f6726000640 1 -- 172.21.15.107:0/2428472535 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f672001d020 con 0x7f6728106f10 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f6732886740 1 -- start start 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f6732886740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb1010050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f6732886740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb101a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f6732886740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb101d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f6732886740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562eb0f763e0 con 0x562eb101ae50 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f6732886740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562eb0f1dd80 con 0x562eb100fc70 2024-09-19T04:33:32.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f6732886740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562eb0f723a0 con 0x562eb1010590 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f672ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb1010050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f672ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb101d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f672d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb101a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f672ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb101d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37388/0 (socket says 172.21.15.107:37388) 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.389+0000 7f672ca00640 1 -- 172.21.15.107:0/2204512177 learned_addr learned my addr 172.21.15.107:0/2204512177 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f672d400640 1 -- 172.21.15.107:0/2204512177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 msgr2=0x562eb1010050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f672d400640 1 --2- 172.21.15.107:0/2204512177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb1010050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f672d400640 1 -- 172.21.15.107:0/2204512177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 msgr2=0x562eb101d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f672d400640 1 --2- 172.21.15.107:0/2204512177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb101d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f672d400640 1 -- 172.21.15.107:0/2204512177 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb0f3cae0 con 0x562eb1010590 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f672ca00640 1 --2- 172.21.15.107:0/2204512177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb101d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f6726000640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f67100a33b0 0x7f67100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f6726000640 1 -- 172.21.15.107:0/2428472535 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f67200cfb70 con 0x7f6728106f10 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f672d400640 1 --2- 172.21.15.107:0/2204512177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb101a910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f671c009fd0 tx=0x7f671c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ed59da5f99300b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f672ca00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f67100a33b0 0x7f67100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f671a600640 1 -- 172.21.15.107:0/2204512177 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c004310 con 0x562eb1010590 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f671a600640 1 -- 172.21.15.107:0/2204512177 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f671c0044b0 con 0x562eb1010590 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.390+0000 7f671a600640 1 -- 172.21.15.107:0/2204512177 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c0035f0 con 0x562eb1010590 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.391+0000 7f6732886740 1 -- 172.21.15.107:0/2204512177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 msgr2=0x562eb101a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.391+0000 7f6732886740 1 --2- 172.21.15.107:0/2204512177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb101a910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f671c009fd0 tx=0x7f671c0118d0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.391+0000 7f672ca00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f67100a33b0 0x7f67100a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f67140095f0 tx=0x7f67140056d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.391+0000 7f6732886740 1 -- 172.21.15.107:0/2204512177 shutdown_connections 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.391+0000 7f6732886740 1 --2- 172.21.15.107:0/2204512177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb101d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.391+0000 7f6732886740 1 --2- 172.21.15.107:0/2204512177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb101a910 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.391+0000 7f6732886740 1 --2- 172.21.15.107:0/2204512177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb1010050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.391+0000 7f6732886740 1 -- 172.21.15.107:0/2204512177 >> 172.21.15.107:0/2204512177 conn(0x562eb100c200 msgr2=0x562eb100c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.391+0000 7f6732886740 1 -- 172.21.15.107:0/2204512177 shutdown_connections 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.391+0000 7f6732886740 1 -- 172.21.15.107:0/2204512177 wait complete. 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.392+0000 7f6732886740 1 -- start start 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.392+0000 7f6732886740 1 Processor -- start 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.392+0000 7f6732886740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb100fc70 0x562eb1017d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.392+0000 7f6732886740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1018250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.392+0000 7f6732886740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb101ae50 0x562eb1018790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.392+0000 7f6732886740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562eb101da20 con 0x562eb100fc70 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.392+0000 7f6732886740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562eb0f310f0 con 0x562eb101ae50 2024-09-19T04:33:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.392+0000 7f6732886740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562eb101dba0 con 0x562eb1010590 2024-09-19T04:33:32.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.392+0000 7f6732886740 10 client.0 _ll_register_callbacks cb 0x562eb0f18bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:32.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb100fc70 0x562eb1017d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1018250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1018250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51156/0 (socket says 172.21.15.107:51156) 2024-09-19T04:33:32.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672ca00640 1 -- 172.21.15.107:0/968255905 learned_addr learned my addr 172.21.15.107:0/968255905 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672de00640 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb101ae50 0x562eb1018790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672ca00640 1 -- 172.21.15.107:0/968255905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb101ae50 msgr2=0x562eb1018790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672ca00640 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb101ae50 0x562eb1018790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672ca00640 1 -- 172.21.15.107:0/968255905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb100fc70 msgr2=0x562eb1017d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672ca00640 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb100fc70 0x562eb1017d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672ca00640 1 -- 172.21.15.107:0/968255905 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb0ce73c0 con 0x562eb1010590 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f672ca00640 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1018250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6714035040 tx=0x7f671402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c1af8d0ed20447d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.393+0000 7f670ea00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f670ea00640 1 -- 172.21.15.107:0/968255905 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671404f020 con 0x562eb1010590 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f6732886740 1 -- 172.21.15.107:0/968255905 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f671c0044b0 con 0x562eb1010590 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f670ea00640 1 -- 172.21.15.107:0/968255905 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f6714003bc0 con 0x562eb1010590 2024-09-19T04:33:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f670ea00640 1 -- 172.21.15.107:0/968255905 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f671405a050 con 0x562eb1010590 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f6732886740 10 client.27553 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f6732886740 1 -- 172.21.15.107:0/968255905 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562eb1029a80 con 0x562eb1010590 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f670ea00640 1 -- 172.21.15.107:0/968255905 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f671c0044b0 con 0x562eb1010590 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f670ea00640 1 -- 172.21.15.107:0/968255905 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f671403a070 con 0x562eb1010590 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f6732886740 10 client.27553 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f6732886740 10 client.27553 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f6732886740 20 client.27553 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f6732886740 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x562eb102af40 0x562eb102d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f6732886740 4 client.27553 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.394+0000 7f6732886740 1 -- 172.21.15.107:0/968255905 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eb0f83950 con 0x562eb102af40 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.395+0000 7f672d400640 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x562eb102af40 0x562eb102d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.395+0000 7f672d400640 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x562eb102af40 0x562eb102d3b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=546d4fe3d77c211b server_cookie=c495c24b75ba2676 in_seq=0 out_seq=0 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.395+0000 7f670ea00640 10 client.27553 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:33:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.396+0000 7f670ea00640 1 -- 172.21.15.107:0/968255905 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562eb0f83950 con 0x562eb102af40 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.396+0000 7f670ea00640 10 client.27553 handle_command_reply: tid=0 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.396+0000 7f6732886740 1 client.27553 shutdown 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.396+0000 7f6732886740 1 -- 172.21.15.107:0/968255905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 msgr2=0x562eb1018250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.396+0000 7f6732886740 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1018250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6714035040 tx=0x7f671402a340 comp rx=0 tx=0).stop 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.397+0000 7f6732886740 1 -- 172.21.15.107:0/968255905 shutdown_connections 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.397+0000 7f6732886740 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x562eb102af40 0x562eb102d3b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.397+0000 7f6732886740 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb101ae50 0x562eb1018790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.397+0000 7f670ea00640 0 client.27553 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.397+0000 7f6732886740 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1018250 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.397+0000 7f6732886740 1 --2- 172.21.15.107:0/968255905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb100fc70 0x562eb1017d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.397+0000 7f6732886740 1 -- 172.21.15.107:0/968255905 >> 172.21.15.107:0/968255905 conn(0x562eb100c200 msgr2=0x562eb1011ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.397+0000 7f6732886740 1 -- 172.21.15.107:0/968255905 shutdown_connections 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.397+0000 7f6732886740 1 -- 172.21.15.107:0/968255905 wait complete. 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.397+0000 7f6732886740 20 client.27553 trim_cache size 0 max 16384 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.410+0000 7f6732886740 1 Processor -- start 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f6732886740 1 -- start start 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f6732886740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb100b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f6732886740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1032c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f6732886740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb10331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f6732886740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f67281af8e0 con 0x562eb101ae50 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f6732886740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b1450 con 0x562eb100fc70 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f6732886740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f67281affe0 con 0x562eb1010590 2024-09-19T04:33:32.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f672ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1032c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f672d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb100b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f672ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1032c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51172/0 (socket says 172.21.15.107:51172) 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f672de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb10331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.411+0000 7f672ca00640 1 -- 172.21.15.107:0/519474783 learned_addr learned my addr 172.21.15.107:0/519474783 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.412+0000 7f672ca00640 1 -- 172.21.15.107:0/519474783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 msgr2=0x562eb100b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.412+0000 7f672ca00640 1 --2- 172.21.15.107:0/519474783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb100b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.412+0000 7f672ca00640 1 -- 172.21.15.107:0/519474783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 msgr2=0x562eb10331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.412+0000 7f672ca00640 1 --2- 172.21.15.107:0/519474783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb10331c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.412+0000 7f672ca00640 1 -- 172.21.15.107:0/519474783 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb1007770 con 0x562eb1010590 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.412+0000 7f672ca00640 1 --2- 172.21.15.107:0/519474783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1032c80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f671404be60 tx=0x7f6714030460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea12e7f0d62adf41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.412+0000 7f6719c00640 1 -- 172.21.15.107:0/519474783 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671404f040 con 0x562eb1010590 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.412+0000 7f6719c00640 1 -- 172.21.15.107:0/519474783 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67140044d0 con 0x562eb1010590 2024-09-19T04:33:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.412+0000 7f6719c00640 1 -- 172.21.15.107:0/519474783 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714037040 con 0x562eb1010590 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 -- 172.21.15.107:0/519474783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 msgr2=0x562eb1032c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 --2- 172.21.15.107:0/519474783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1032c80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f671404be60 tx=0x7f6714030460 comp rx=0 tx=0).stop 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 -- 172.21.15.107:0/519474783 shutdown_connections 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 --2- 172.21.15.107:0/519474783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb10331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 --2- 172.21.15.107:0/519474783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1032c80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 --2- 172.21.15.107:0/519474783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb100b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 -- 172.21.15.107:0/519474783 >> 172.21.15.107:0/519474783 conn(0x562eb100c200 msgr2=0x562eb100ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 -- 172.21.15.107:0/519474783 shutdown_connections 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 -- 172.21.15.107:0/519474783 wait complete. 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 -- start start 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.413+0000 7f6732886740 1 Processor -- start 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f6732886740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb1014f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f6732886740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1015460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f6732886740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb10159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f6732886740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b0d80 con 0x562eb101ae50 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f6732886740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b06b0 con 0x562eb100fc70 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f6732886740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f67281ad8e0 con 0x562eb1010590 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f6732886740 10 client.0 _ll_register_callbacks cb 0x7f67140096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f672d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb1014f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f672de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb10159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f672ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1015460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f672d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb1014f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39586/0 (socket says 172.21.15.107:39586) 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f672d400640 1 -- 172.21.15.107:0/1514511190 learned_addr learned my addr 172.21.15.107:0/1514511190 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f672d400640 1 -- 172.21.15.107:0/1514511190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 msgr2=0x562eb1015460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f672d400640 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1015460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f672d400640 1 -- 172.21.15.107:0/1514511190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 msgr2=0x562eb10159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f672d400640 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb10159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.414+0000 7f672d400640 1 -- 172.21.15.107:0/1514511190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb1094e20 con 0x562eb100fc70 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.415+0000 7f672d400640 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb1014f20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f671c009fd0 tx=0x7f671c0092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9146d1733ba90673 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.415+0000 7f670ea00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.415+0000 7f670ea00640 1 -- 172.21.15.107:0/1514511190 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c032020 con 0x562eb100fc70 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.415+0000 7f6732886740 1 -- 172.21.15.107:0/1514511190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67140044d0 con 0x562eb100fc70 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.415+0000 7f670ea00640 1 -- 172.21.15.107:0/1514511190 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f671c004880 con 0x562eb100fc70 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.415+0000 7f670ea00640 1 -- 172.21.15.107:0/1514511190 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f671c03d050 con 0x562eb100fc70 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.415+0000 7f6732886740 10 client.27617 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.415+0000 7f6732886740 1 -- 172.21.15.107:0/1514511190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562eb10960e0 con 0x562eb100fc70 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.415+0000 7f670ea00640 1 -- 172.21.15.107:0/1514511190 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67140044d0 con 0x562eb100fc70 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f670ea00640 1 -- 172.21.15.107:0/1514511190 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f671c004cb0 con 0x562eb100fc70 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f6732886740 10 client.27617 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:32.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f6732886740 10 client.27617 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f6732886740 20 client.27617 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f6732886740 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x562eb1096720 0x562eb1098b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f6732886740 4 client.27617 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f6732886740 1 -- 172.21.15.107:0/1514511190 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562eb0f7de60 con 0x562eb1096720 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f672ca00640 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x562eb1096720 0x562eb1098b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f670ea00640 10 client.27617 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f672ca00640 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x562eb1096720 0x562eb1098b40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=83515fe29998f137 server_cookie=6d95b589492d4d40 in_seq=0 out_seq=0 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f670ea00640 1 -- 172.21.15.107:0/1514511190 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562eb0f7de60 con 0x562eb1096720 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.416+0000 7f670ea00640 10 client.27617 handle_command_reply: tid=0 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 client.27617 shutdown 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 -- 172.21.15.107:0/1514511190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 msgr2=0x562eb1014f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb1014f20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f671c009fd0 tx=0x7f671c0092f0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 -- 172.21.15.107:0/1514511190 shutdown_connections 2024-09-19T04:33:32.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x562eb1096720 0x562eb1098b40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562eb101ae50 0x562eb10159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f670ea00640 0 client.27617 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:33:32.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562eb1010590 0x562eb1015460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 --2- 172.21.15.107:0/1514511190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eb100fc70 0x562eb1014f20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 -- 172.21.15.107:0/1514511190 >> 172.21.15.107:0/1514511190 conn(0x562eb100c200 msgr2=0x562eb100b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:32.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 -- 172.21.15.107:0/1514511190 shutdown_connections 2024-09-19T04:33:32.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 1 -- 172.21.15.107:0/1514511190 wait complete. 2024-09-19T04:33:32.495 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:33:32.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.417+0000 7f6732886740 20 client.27617 trim_cache size 0 max 16384 2024-09-19T04:33:32.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 -- 172.21.15.107:0/2428472535 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f67100a33b0 msgr2=0x7f67100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f67100a33b0 0x7f67100a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f67140095f0 tx=0x7f67140056d0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 -- 172.21.15.107:0/2428472535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728106f10 msgr2=0x7f672819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:32.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728106f10 0x7f672819ef40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f672000efc0 tx=0x7f672000c800 comp rx=0 tx=0).stop 2024-09-19T04:33:32.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 -- 172.21.15.107:0/2428472535 shutdown_connections 2024-09-19T04:33:32.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f67100a33b0 0x7f67100a5870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728106f10 0x7f672819ef40 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6728102790 0x7f672819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 --2- 172.21.15.107:0/2428472535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6728101de0 0x7f672819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:32.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 -- 172.21.15.107:0/2428472535 >> 172.21.15.107:0/2428472535 conn(0x7f67280fd8c0 msgr2=0x7f67280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:32.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 -- 172.21.15.107:0/2428472535 shutdown_connections 2024-09-19T04:33:32.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:32.418+0000 7f672fe00640 1 -- 172.21.15.107:0/2428472535 wait complete. 2024-09-19T04:33:32.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458725 3.04 MB/sec execute 590 sec latency 634.262 ms 2024-09-19T04:33:33.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:32 smithi115 ceph-mon[35477]: pgmap v413: 97 pgs: 97 active+clean; 410 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-09-19T04:33:33.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:32 smithi145 ceph-mon[35096]: pgmap v413: 97 pgs: 97 active+clean; 410 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-09-19T04:33:33.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:32 smithi107 ceph-mon[30659]: pgmap v413: 97 pgs: 97 active+clean; 410 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-09-19T04:33:33.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:33:33.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458725 3.03 MB/sec execute 591 sec latency 1634.306 ms 2024-09-19T04:33:33.786 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:34.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.357+0000 7f46b1a00640 1 -- 172.21.15.107:0/53687181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46ac1129e0 msgr2=0x7f46ac114dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.357+0000 7f46b1a00640 1 --2- 172.21.15.107:0/53687181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46ac1129e0 0x7f46ac114dd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009ca0 tx=0x7f46a402a950 comp rx=0 tx=0).stop 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.357+0000 7f46b1a00640 1 -- 172.21.15.107:0/53687181 shutdown_connections 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.357+0000 7f46b1a00640 1 --2- 172.21.15.107:0/53687181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46ac1129e0 0x7f46ac114dd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.357+0000 7f46b1a00640 1 --2- 172.21.15.107:0/53687181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46ac074eb0 0x7f46ac1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.357+0000 7f46b1a00640 1 --2- 172.21.15.107:0/53687181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46ac074590 0x7f46ac074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.357+0000 7f46b1a00640 1 -- 172.21.15.107:0/53687181 >> 172.21.15.107:0/53687181 conn(0x7f46ac0fd710 msgr2=0x7f46ac0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.357+0000 7f46b1a00640 1 -- 172.21.15.107:0/53687181 shutdown_connections 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.357+0000 7f46b1a00640 1 -- 172.21.15.107:0/53687181 wait complete. 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.362+0000 7f46b1a00640 1 Processor -- start 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46b1a00640 1 -- start start 2024-09-19T04:33:34.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46b1a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46ac074590 0x7f46ac1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46b1a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46ac074eb0 0x7f46ac1a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46b1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46ac1129e0 0x7f46ac19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46b1a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac118400 con 0x7f46ac1129e0 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46b1a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac118280 con 0x7f46ac074590 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46b1a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac118580 con 0x7f46ac074eb0 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46abe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46ac074590 0x7f46ac1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46abe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46ac074590 0x7f46ac1a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39596/0 (socket says 172.21.15.107:39596) 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46abe00640 1 -- 172.21.15.107:0/399512334 learned_addr learned my addr 172.21.15.107:0/399512334 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.363+0000 7f46b1000640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46ac1129e0 0x7f46ac19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.364+0000 7f46abe00640 1 -- 172.21.15.107:0/399512334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46ac074eb0 msgr2=0x7f46ac1a4b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.364+0000 7f46abe00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46ac074eb0 0x7f46ac1a4b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.364+0000 7f46abe00640 1 -- 172.21.15.107:0/399512334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46ac1129e0 msgr2=0x7f46ac19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.364+0000 7f46abe00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46ac1129e0 0x7f46ac19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.364+0000 7f46abe00640 1 -- 172.21.15.107:0/399512334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac19ef60 con 0x7f46ac074590 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.364+0000 7f46abe00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46ac074590 0x7f46ac1a4620 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f469c00e9f0 tx=0x7f469c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.364+0000 7f46a8c00640 1 -- 172.21.15.107:0/399512334 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c00cd80 con 0x7f46ac074590 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.364+0000 7f46b1a00640 1 -- 172.21.15.107:0/399512334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac19f250 con 0x7f46ac074590 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.364+0000 7f46a8c00640 1 -- 172.21.15.107:0/399512334 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f469c005500 con 0x7f46ac074590 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.364+0000 7f46a8c00640 1 -- 172.21.15.107:0/399512334 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c010690 con 0x7f46ac074590 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.365+0000 7f46b1a00640 1 -- 172.21.15.107:0/399512334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac1ab5a0 con 0x7f46ac074590 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.368+0000 7f46b5a8f740 1 Processor -- start 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.368+0000 7f46a8c00640 1 -- 172.21.15.107:0/399512334 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f469c0046c0 con 0x7f46ac074590 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.368+0000 7f46b5a8f740 1 -- start start 2024-09-19T04:33:34.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.368+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb88220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.368+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb92ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.368+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb95410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.368+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55758faee3e0 con 0x55758fb88760 2024-09-19T04:33:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.368+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55758fa95d80 con 0x55758fb87e40 2024-09-19T04:33:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.368+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55758faea3a0 con 0x55758fb93020 2024-09-19T04:33:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb92ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46b1000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb88220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46b1000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb88220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39612/0 (socket says 172.21.15.107:39612) 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46b1000640 1 -- 172.21.15.107:0/2500611790 learned_addr learned my addr 172.21.15.107:0/2500611790 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46b1000640 1 -- 172.21.15.107:0/2500611790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 msgr2=0x55758fb95410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46b1000640 1 --2- 172.21.15.107:0/2500611790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb95410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46b1000640 1 -- 172.21.15.107:0/2500611790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 msgr2=0x55758fb92ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46b1000640 1 --2- 172.21.15.107:0/2500611790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb92ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46b1000640 1 -- 172.21.15.107:0/2500611790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758fab4ae0 con 0x55758fb87e40 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46abe00640 1 --2- 172.21.15.107:0/2500611790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb92ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.369+0000 7f46a8c00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46900a33b0 0x7f46900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.370+0000 7f46b1000640 1 --2- 172.21.15.107:0/2500611790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb88220 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f46ac076600 tx=0x7f46a4005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce64853f195aca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.370+0000 7f46a8c00640 1 -- 172.21.15.107:0/399512334 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f469c0d0190 con 0x7f46ac074590 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.370+0000 7f46a1600640 1 -- 172.21.15.107:0/2500611790 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4048070 con 0x55758fb87e40 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.370+0000 7f46a1600640 1 -- 172.21.15.107:0/2500611790 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a4003e50 con 0x55758fb87e40 2024-09-19T04:33:34.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.370+0000 7f46a1600640 1 -- 172.21.15.107:0/2500611790 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4034040 con 0x55758fb87e40 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.370+0000 7f46abe00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46900a33b0 0x7f46900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.371+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2500611790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 msgr2=0x55758fb88220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.371+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/2500611790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb88220 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f46ac076600 tx=0x7f46a4005f80 comp rx=0 tx=0).stop 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.371+0000 7f46abe00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46900a33b0 0x7f46900a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f469c0cd9b0 tx=0x7f469c098040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.371+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2500611790 shutdown_connections 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.371+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/2500611790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb95410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.371+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/2500611790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb92ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.371+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/2500611790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb88220 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.371+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2500611790 >> 172.21.15.107:0/2500611790 conn(0x55758fb843d0 msgr2=0x55758fb847b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.371+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2500611790 shutdown_connections 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.371+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2500611790 wait complete. 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.372+0000 7f46b5a8f740 1 -- start start 2024-09-19T04:33:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.372+0000 7f46b5a8f740 1 Processor -- start 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.372+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb8ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.372+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb90460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.372+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.372+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55758fb95bf0 con 0x55758fb88760 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.372+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55758faa90f0 con 0x55758fb87e40 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55758fb95d70 con 0x55758fb93020 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b5a8f740 10 client.0 _ll_register_callbacks cb 0x55758fa90bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb90460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b1000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb8ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb909a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51222/0 (socket says 172.21.15.107:51222) 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b1000640 1 -- 172.21.15.107:0/2292504283 learned_addr learned my addr 172.21.15.107:0/2292504283 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:34.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b1000640 1 -- 172.21.15.107:0/2292504283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 msgr2=0x55758fb8ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b1000640 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb8ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b1000640 1 -- 172.21.15.107:0/2292504283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 msgr2=0x55758fb90460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b1000640 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb90460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.373+0000 7f46b1000640 1 -- 172.21.15.107:0/2292504283 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55758f85f3c0 con 0x55758fb93020 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.374+0000 7f46b1000640 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb909a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f46a402fb70 tx=0x7f46a4005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8580540542192ced server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.374+0000 7f4696000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.374+0000 7f4696000640 1 -- 172.21.15.107:0/2292504283 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4048070 con 0x55758fb93020 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.374+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2292504283 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a4003e50 con 0x55758fb93020 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.374+0000 7f4696000640 1 -- 172.21.15.107:0/2292504283 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f46a4004250 con 0x55758fb93020 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.374+0000 7f4696000640 1 -- 172.21.15.107:0/2292504283 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f46a4003c70 con 0x55758fb93020 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.374+0000 7f46b5a8f740 10 client.27565 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:34.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.374+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2292504283 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55758fb95bf0 con 0x55758fb93020 2024-09-19T04:33:34.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.375+0000 7f4696000640 1 -- 172.21.15.107:0/2292504283 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a4003e50 con 0x55758fb93020 2024-09-19T04:33:34.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.375+0000 7f4696000640 1 -- 172.21.15.107:0/2292504283 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f46a4030c10 con 0x55758fb93020 2024-09-19T04:33:34.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.375+0000 7f46b5a8f740 10 client.27565 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:34.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.375+0000 7f46b5a8f740 10 client.27565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:33:34.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.375+0000 7f46b5a8f740 20 client.27565 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:34.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.375+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55758fba2dc0 0x55758fba51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.375+0000 7f46b5a8f740 4 client.27565 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:33:34.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.375+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2292504283 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55758fafb950 con 0x55758fba2dc0 2024-09-19T04:33:34.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.409+0000 7f46ab400640 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55758fba2dc0 0x55758fba51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.421+0000 7f46ab400640 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55758fba2dc0 0x55758fba51b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1f09b2acf18165 server_cookie=d303a4725c378989 in_seq=0 out_seq=0 2024-09-19T04:33:34.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.421+0000 7f4696000640 10 client.27565 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.422+0000 7f4696000640 1 -- 172.21.15.107:0/2292504283 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55758fafb950 con 0x55758fba2dc0 2024-09-19T04:33:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.422+0000 7f4696000640 10 client.27565 handle_command_reply: tid=0 2024-09-19T04:33:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.422+0000 7f46b5a8f740 1 client.27565 shutdown 2024-09-19T04:33:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2292504283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 msgr2=0x55758fb909a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb909a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f46a402fb70 tx=0x7f46a4005df0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2292504283 shutdown_connections 2024-09-19T04:33:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55758fba2dc0 0x55758fba51b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb909a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb90460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/2292504283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb8ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2292504283 >> 172.21.15.107:0/2292504283 conn(0x55758fb843d0 msgr2=0x55758fb8a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f4696000640 0 client.27565 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2292504283 shutdown_connections 2024-09-19T04:33:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 1 -- 172.21.15.107:0/2292504283 wait complete. 2024-09-19T04:33:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.423+0000 7f46b5a8f740 20 client.27565 trim_cache size 0 max 16384 2024-09-19T04:33:34.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.437+0000 7f46b5a8f740 1 Processor -- start 2024-09-19T04:33:34.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.437+0000 7f46b5a8f740 1 -- start start 2024-09-19T04:33:34.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.437+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb87e40 0x55758fb8a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.437+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb88760 0x55758fb8e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.437+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fbaa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.437+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1b3cc0 con 0x55758fb87e40 2024-09-19T04:33:34.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.437+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1b5830 con 0x55758fb88760 2024-09-19T04:33:34.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.437+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1b43c0 con 0x55758fb93020 2024-09-19T04:33:34.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.437+0000 7f46b1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb87e40 0x55758fb8a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.438+0000 7f46b1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb87e40 0x55758fb8a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37444/0 (socket says 172.21.15.107:37444) 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.438+0000 7f46b1000640 1 -- 172.21.15.107:0/4037494991 learned_addr learned my addr 172.21.15.107:0/4037494991 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.438+0000 7f46ab400640 1 --2- 172.21.15.107:0/4037494991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb88760 0x55758fb8e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.438+0000 7f46b1000640 1 --2- 172.21.15.107:0/4037494991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fbaa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.438+0000 7f46ab400640 1 -- 172.21.15.107:0/4037494991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 msgr2=0x55758fbaa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.438+0000 7f46ab400640 1 --2- 172.21.15.107:0/4037494991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fbaa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.438+0000 7f46ab400640 1 -- 172.21.15.107:0/4037494991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb87e40 msgr2=0x55758fb8a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.438+0000 7f46ab400640 1 --2- 172.21.15.107:0/4037494991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb87e40 0x55758fb8a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.438+0000 7f46ab400640 1 -- 172.21.15.107:0/4037494991 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758fb88590 con 0x55758fb88760 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.438+0000 7f46ab400640 1 --2- 172.21.15.107:0/4037494991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb88760 0x55758fb8e7f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55758fba5780 tx=0x7f4698005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6faeeaf64acdc98f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.439+0000 7f46a0c00640 1 -- 172.21.15.107:0/4037494991 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698009e20 con 0x55758fb88760 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.439+0000 7f46a0c00640 1 -- 172.21.15.107:0/4037494991 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f469801b3a0 con 0x55758fb88760 2024-09-19T04:33:34.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.439+0000 7f46a0c00640 1 -- 172.21.15.107:0/4037494991 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46980126d0 con 0x55758fb88760 2024-09-19T04:33:34.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.439+0000 7f46b5a8f740 1 -- 172.21.15.107:0/4037494991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb88760 msgr2=0x55758fb8e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.439+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/4037494991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb88760 0x55758fb8e7f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55758fba5780 tx=0x7f4698005f90 comp rx=0 tx=0).stop 2024-09-19T04:33:34.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.440+0000 7f46b5a8f740 1 -- 172.21.15.107:0/4037494991 shutdown_connections 2024-09-19T04:33:34.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.440+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/4037494991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fbaa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.440+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/4037494991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb88760 0x55758fb8e7f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.440+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/4037494991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb87e40 0x55758fb8a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.440+0000 7f46b5a8f740 1 -- 172.21.15.107:0/4037494991 >> 172.21.15.107:0/4037494991 conn(0x55758fb843d0 msgr2=0x55758fb834b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.440+0000 7f46b5a8f740 1 -- 172.21.15.107:0/4037494991 shutdown_connections 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.440+0000 7f46b5a8f740 1 -- 172.21.15.107:0/4037494991 wait complete. 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.440+0000 7f46b5a8f740 1 -- start start 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.440+0000 7f46b5a8f740 1 Processor -- start 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb97000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb97540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b5a8f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb97aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1b5160 con 0x55758fb88760 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1b4a90 con 0x55758fb87e40 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b5a8f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1b1cc0 con 0x55758fb93020 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b5a8f740 10 client.0 _ll_register_callbacks cb 0x7f46a4046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:34.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b1000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb97000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b1000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb97000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39648/0 (socket says 172.21.15.107:39648) 2024-09-19T04:33:34.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b1000640 1 -- 172.21.15.107:0/528604232 learned_addr learned my addr 172.21.15.107:0/528604232 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:34.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46b1000640 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb97aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.441+0000 7f46ab400640 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb97540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.442+0000 7f46b1000640 1 -- 172.21.15.107:0/528604232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 msgr2=0x55758fb97aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:34.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.442+0000 7f46b1000640 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb97aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.442+0000 7f46b1000640 1 -- 172.21.15.107:0/528604232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 msgr2=0x55758fb97540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:34.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.442+0000 7f46b1000640 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb97540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.442+0000 7f46b1000640 1 -- 172.21.15.107:0/528604232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55758fc0ca80 con 0x55758fb87e40 2024-09-19T04:33:34.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.442+0000 7f46b1000640 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb97000 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f46a402a990 tx=0x7f46a403a720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a50add304456365b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:34.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.442+0000 7f4696000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:33:34.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.442+0000 7f4696000640 1 -- 172.21.15.107:0/528604232 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a405e030 con 0x55758fb87e40 2024-09-19T04:33:34.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.442+0000 7f4696000640 1 -- 172.21.15.107:0/528604232 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f46a4004250 con 0x55758fb87e40 2024-09-19T04:33:34.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.442+0000 7f46b5a8f740 1 -- 172.21.15.107:0/528604232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f469801b3a0 con 0x55758fb87e40 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f4696000640 1 -- 172.21.15.107:0/528604232 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f46a4048070 con 0x55758fb87e40 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f46b5a8f740 10 client.27635 fetch_fsmap learned FSMap version 49 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f46b5a8f740 1 -- 172.21.15.107:0/528604232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55758fc0dd80 con 0x55758fb87e40 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f4696000640 1 -- 172.21.15.107:0/528604232 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f469801b3a0 con 0x55758fb87e40 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f4696000640 1 -- 172.21.15.107:0/528604232 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f46a406c020 con 0x55758fb87e40 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f46b5a8f740 10 client.27635 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f46b5a8f740 10 client.27635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f46b5a8f740 20 client.27635 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55758fc0e2e0 0x55758fc106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f46b5a8f740 4 client.27635 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.443+0000 7f46b5a8f740 1 -- 172.21.15.107:0/528604232 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55758faf5e60 con 0x55758fc0e2e0 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.444+0000 7f46ab400640 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55758fc0e2e0 0x55758fc106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:34.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.444+0000 7f46ab400640 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55758fc0e2e0 0x55758fc106d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b09db01bbed6d18 server_cookie=1174bacd47557b76 in_seq=0 out_seq=0 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.444+0000 7f4696000640 10 client.27635 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.444+0000 7f4696000640 1 -- 172.21.15.107:0/528604232 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55758faf5e60 con 0x55758fc0e2e0 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.444+0000 7f4696000640 10 client.27635 handle_command_reply: tid=0 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.444+0000 7f46b5a8f740 1 client.27635 shutdown 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.444+0000 7f46b5a8f740 1 -- 172.21.15.107:0/528604232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 msgr2=0x55758fb97000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.444+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb97000 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f46a402a990 tx=0x7f46a403a720 comp rx=0 tx=0).stop 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.445+0000 7f46b5a8f740 1 -- 172.21.15.107:0/528604232 shutdown_connections 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.445+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55758fc0e2e0 0x55758fc106d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.445+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55758fb93020 0x55758fb97aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.445+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55758fb88760 0x55758fb97540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.445+0000 7f46b5a8f740 1 --2- 172.21.15.107:0/528604232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55758fb87e40 0x55758fb97000 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.445+0000 7f46b5a8f740 1 -- 172.21.15.107:0/528604232 >> 172.21.15.107:0/528604232 conn(0x55758fb843d0 msgr2=0x55758fb91780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:34.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.445+0000 7f4696000640 0 client.27635 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:34.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.445+0000 7f46b5a8f740 1 -- 172.21.15.107:0/528604232 shutdown_connections 2024-09-19T04:33:34.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.445+0000 7f46b5a8f740 1 -- 172.21.15.107:0/528604232 wait complete. 2024-09-19T04:33:34.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.445+0000 7f46b5a8f740 20 client.27635 trim_cache size 0 max 16384 2024-09-19T04:33:34.450 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:33:34.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 -- 172.21.15.107:0/399512334 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46900a33b0 msgr2=0x7f46900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46900a33b0 0x7f46900a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f469c0cd9b0 tx=0x7f469c098040 comp rx=0 tx=0).stop 2024-09-19T04:33:34.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 -- 172.21.15.107:0/399512334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46ac074590 msgr2=0x7f46ac1a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:34.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46ac074590 0x7f46ac1a4620 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f469c00e9f0 tx=0x7f469c00edf0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 -- 172.21.15.107:0/399512334 shutdown_connections 2024-09-19T04:33:34.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f46900a33b0 0x7f46900a5870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46ac1129e0 0x7f46ac19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46ac074eb0 0x7f46ac1a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 --2- 172.21.15.107:0/399512334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46ac074590 0x7f46ac1a4620 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:34.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 -- 172.21.15.107:0/399512334 >> 172.21.15.107:0/399512334 conn(0x7f46ac0fd710 msgr2=0x7f46ac0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:34.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 -- 172.21.15.107:0/399512334 shutdown_connections 2024-09-19T04:33:34.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:34.446+0000 7f46b1a00640 1 -- 172.21.15.107:0/399512334 wait complete. 2024-09-19T04:33:34.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458725 3.03 MB/sec execute 592 sec latency 2634.366 ms 2024-09-19T04:33:34.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:34 smithi107 ceph-mon[30659]: pgmap v414: 97 pgs: 97 active+clean; 410 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-09-19T04:33:35.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:34 smithi115 ceph-mon[35477]: pgmap v414: 97 pgs: 97 active+clean; 410 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-09-19T04:33:35.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:34 smithi145 ceph-mon[35096]: pgmap v414: 97 pgs: 97 active+clean; 410 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-09-19T04:33:35.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458725 3.02 MB/sec execute 593 sec latency 3634.417 ms 2024-09-19T04:33:36.666 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T04:33:36.666 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T04:33:36.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458725 3.02 MB/sec execute 594 sec latency 4634.468 ms 2024-09-19T04:33:36.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:36 smithi107 ceph-mon[30659]: pgmap v415: 97 pgs: 97 active+clean; 414 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 150 op/s 2024-09-19T04:33:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:37.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:36 smithi115 ceph-mon[35477]: pgmap v415: 97 pgs: 97 active+clean; 414 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 150 op/s 2024-09-19T04:33:37.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:36 smithi145 ceph-mon[35096]: pgmap v415: 97 pgs: 97 active+clean; 414 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 150 op/s 2024-09-19T04:33:37.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.643+0000 7f32d2200640 1 -- 172.21.15.107:0/3119227871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc112870 msgr2=0x7f32cc114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.643+0000 7f32d2200640 1 --2- 172.21.15.107:0/3119227871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc112870 0x7f32cc114c60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009ca0 tx=0x7f32c402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.643+0000 7f32d2200640 1 -- 172.21.15.107:0/3119227871 shutdown_connections 2024-09-19T04:33:37.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.643+0000 7f32d2200640 1 --2- 172.21.15.107:0/3119227871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc112870 0x7f32cc114c60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.643+0000 7f32d2200640 1 --2- 172.21.15.107:0/3119227871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32cc077040 0x7f32cc112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.643+0000 7f32d2200640 1 --2- 172.21.15.107:0/3119227871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32cc076720 0x7f32cc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.643+0000 7f32d2200640 1 -- 172.21.15.107:0/3119227871 >> 172.21.15.107:0/3119227871 conn(0x7f32cc0ff8d0 msgr2=0x7f32cc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.643+0000 7f32d2200640 1 -- 172.21.15.107:0/3119227871 shutdown_connections 2024-09-19T04:33:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.644+0000 7f32d2200640 1 -- 172.21.15.107:0/3119227871 wait complete. 2024-09-19T04:33:37.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.644+0000 7f32d2200640 1 Processor -- start 2024-09-19T04:33:37.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.645+0000 7f32d2200640 1 -- start start 2024-09-19T04:33:37.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.645+0000 7f32d2200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32cc076720 0x7f32cc1a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.645+0000 7f32d2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc077040 0x7f32cc1a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.645+0000 7f32d2200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32cc112870 0x7f32cc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.645+0000 7f32d2200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1183b0 con 0x7f32cc077040 2024-09-19T04:33:37.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.645+0000 7f32d2200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc118230 con 0x7f32cc112870 2024-09-19T04:33:37.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.645+0000 7f32d2200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc118530 con 0x7f32cc076720 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.645+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc077040 0x7f32cc1a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.646+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc077040 0x7f32cc1a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37472/0 (socket says 172.21.15.107:37472) 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.646+0000 7f32cbe00640 1 -- 172.21.15.107:0/857745072 learned_addr learned my addr 172.21.15.107:0/857745072 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.646+0000 7f32d0e00640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32cc076720 0x7f32cc1a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.646+0000 7f32d1800640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32cc112870 0x7f32cc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.646+0000 7f32d0e00640 1 -- 172.21.15.107:0/857745072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32cc112870 msgr2=0x7f32cc19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.646+0000 7f32d0e00640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32cc112870 0x7f32cc19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.646+0000 7f32d0e00640 1 -- 172.21.15.107:0/857745072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc077040 msgr2=0x7f32cc1a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.646+0000 7f32d0e00640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc077040 0x7f32cc1a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.646+0000 7f32d0e00640 1 -- 172.21.15.107:0/857745072 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc19ef00 con 0x7f32cc076720 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.646+0000 7f32cbe00640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc077040 0x7f32cc1a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:37.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.647+0000 7f32d0e00640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32cc076720 0x7f32cc1a45c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005fd0 tx=0x7f32c000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:37.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.647+0000 7f32c9600640 1 -- 172.21.15.107:0/857745072 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0009c30 con 0x7f32cc076720 2024-09-19T04:33:37.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.647+0000 7f32d2200640 1 -- 172.21.15.107:0/857745072 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc19f1f0 con 0x7f32cc076720 2024-09-19T04:33:37.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.647+0000 7f32c9600640 1 -- 172.21.15.107:0/857745072 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c0009dd0 con 0x7f32cc076720 2024-09-19T04:33:37.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.647+0000 7f32c9600640 1 -- 172.21.15.107:0/857745072 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0010460 con 0x7f32cc076720 2024-09-19T04:33:37.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.647+0000 7f32d2200640 1 -- 172.21.15.107:0/857745072 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1ab540 con 0x7f32cc076720 2024-09-19T04:33:37.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.650+0000 7f32d62b4740 1 Processor -- start 2024-09-19T04:33:37.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.650+0000 7f32d62b4740 1 -- start start 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.650+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c657090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.650+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c6575d0 0x564f2c661950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.650+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c661e90 0x564f2c664280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.650+0000 7f32d62b4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564f2c5bd290 con 0x564f2c6575d0 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.650+0000 7f32d62b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f2c564b40 con 0x564f2c661e90 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.650+0000 7f32d62b4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564f2c5b9250 con 0x564f2c656cb0 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32c9600640 1 -- 172.21.15.107:0/857745072 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f32c0003f50 con 0x7f32cc076720 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32d1800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c657090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32d1800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c657090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51264/0 (socket says 172.21.15.107:51264) 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32d1800640 1 -- 172.21.15.107:0/1589012236 learned_addr learned my addr 172.21.15.107:0/1589012236 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:37.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32cbe00640 1 --2- 172.21.15.107:0/1589012236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c661e90 0x564f2c664280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32cbe00640 1 -- 172.21.15.107:0/1589012236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 msgr2=0x564f2c657090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:37.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32cbe00640 1 --2- 172.21.15.107:0/1589012236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c657090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32d0e00640 1 --2- 172.21.15.107:0/1589012236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c6575d0 0x564f2c661950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32cbe00640 1 -- 172.21.15.107:0/1589012236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c6575d0 msgr2=0x564f2c661950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:37.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32cbe00640 1 --2- 172.21.15.107:0/1589012236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c6575d0 0x564f2c661950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32cbe00640 1 -- 172.21.15.107:0/1589012236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2c32e6a0 con 0x564f2c661e90 2024-09-19T04:33:37.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.651+0000 7f32cbe00640 1 --2- 172.21.15.107:0/1589012236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c661e90 0x564f2c664280 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f32cc19fee0 tx=0x7f32b80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8770cf1060936118 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.652+0000 7f32be000640 1 -- 172.21.15.107:0/1589012236 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b801e070 con 0x564f2c661e90 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.652+0000 7f32be000640 1 -- 172.21.15.107:0/1589012236 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b80035f0 con 0x564f2c661e90 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.652+0000 7f32be000640 1 -- 172.21.15.107:0/1589012236 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b800f800 con 0x564f2c661e90 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.652+0000 7f32c9600640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32b40a33b0 0x7f32b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.652+0000 7f32d62b4740 1 -- 172.21.15.107:0/1589012236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c661e90 msgr2=0x564f2c664280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.652+0000 7f32d62b4740 1 --2- 172.21.15.107:0/1589012236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c661e90 0x564f2c664280 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f32cc19fee0 tx=0x7f32b80118d0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.652+0000 7f32c9600640 1 -- 172.21.15.107:0/857745072 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f32c00cfbf0 con 0x7f32cc076720 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.652+0000 7f32d1800640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32b40a33b0 0x7f32b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.653+0000 7f32d62b4740 1 -- 172.21.15.107:0/1589012236 shutdown_connections 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.653+0000 7f32d62b4740 1 --2- 172.21.15.107:0/1589012236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c661e90 0x564f2c664280 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.653+0000 7f32d62b4740 1 --2- 172.21.15.107:0/1589012236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c6575d0 0x564f2c661950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.653+0000 7f32d62b4740 1 --2- 172.21.15.107:0/1589012236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c657090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.653+0000 7f32d62b4740 1 -- 172.21.15.107:0/1589012236 >> 172.21.15.107:0/1589012236 conn(0x564f2c653240 msgr2=0x564f2c653620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.653+0000 7f32d62b4740 1 -- 172.21.15.107:0/1589012236 shutdown_connections 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.653+0000 7f32d62b4740 1 -- 172.21.15.107:0/1589012236 wait complete. 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.653+0000 7f32d1800640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32b40a33b0 0x7f32b40a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f32c4004af0 tx=0x7f32c4004a80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.653+0000 7f32d62b4740 1 -- start start 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.653+0000 7f32d62b4740 1 Processor -- start 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c661710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c65e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 0x564f2c65ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32d62b4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564f2c664b00 con 0x564f2c661e90 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32d62b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f2c577fa0 con 0x564f2c6575d0 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32d62b4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564f2c664c80 con 0x564f2c656cb0 2024-09-19T04:33:37.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32d62b4740 10 client.0 _ll_register_callbacks cb 0x564f2c2cbba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:37.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c65e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c65e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39664/0 (socket says 172.21.15.107:39664) 2024-09-19T04:33:37.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32d1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 0x564f2c65ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32cbe00640 1 -- 172.21.15.107:0/2096278339 learned_addr learned my addr 172.21.15.107:0/2096278339 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.654+0000 7f32d0e00640 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c661710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.655+0000 7f32cbe00640 1 -- 172.21.15.107:0/2096278339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 msgr2=0x564f2c661710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.655+0000 7f32cbe00640 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c661710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.655+0000 7f32cbe00640 1 -- 172.21.15.107:0/2096278339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 msgr2=0x564f2c65ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.655+0000 7f32cbe00640 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 0x564f2c65ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.655+0000 7f32cbe00640 1 -- 172.21.15.107:0/2096278339 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2c65f6e0 con 0x564f2c6575d0 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.655+0000 7f32cbe00640 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c65e9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f32b8004630 tx=0x7f32b8003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be98ca5a47b99e5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.655+0000 7f32b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.655+0000 7f32b2a00640 1 -- 172.21.15.107:0/2096278339 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8026020 con 0x564f2c6575d0 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.655+0000 7f32b2a00640 1 -- 172.21.15.107:0/2096278339 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f32b8005040 con 0x564f2c6575d0 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.655+0000 7f32d62b4740 1 -- 172.21.15.107:0/2096278339 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b80035f0 con 0x564f2c6575d0 2024-09-19T04:33:37.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32b2a00640 1 -- 172.21.15.107:0/2096278339 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f32b801e040 con 0x564f2c6575d0 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32d62b4740 10 client.27647 fetch_fsmap learned FSMap version 50 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32d62b4740 1 -- 172.21.15.107:0/2096278339 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564f2c670960 con 0x564f2c6575d0 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32b2a00640 1 -- 172.21.15.107:0/2096278339 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b80035f0 con 0x564f2c6575d0 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32b2a00640 1 -- 172.21.15.107:0/2096278339 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 50) ==== 8370+0+0 (secure 0 0 0) 0x7f32b8021030 con 0x564f2c6575d0 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32d62b4740 10 client.27647 fetch_fsmap finished waiting for FSMap version 50 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32d62b4740 10 client.27647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32d62b4740 20 client.27647 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f2c671e20 0x564f2c674210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32d62b4740 4 client.27647 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.656+0000 7f32d62b4740 1 -- 172.21.15.107:0/2096278339 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f2c5ca800 con 0x564f2c671e20 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.657+0000 7f32d0e00640 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f2c671e20 0x564f2c674210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.657+0000 7f32d0e00640 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f2c671e20 0x564f2c674210 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30c6130b7534214 server_cookie=cfe759277b8261aa in_seq=0 out_seq=0 2024-09-19T04:33:37.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.657+0000 7f32b2a00640 10 client.27647 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:37.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.658+0000 7f32b2a00640 1 -- 172.21.15.107:0/2096278339 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f2c5ca800 con 0x564f2c671e20 2024-09-19T04:33:37.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.658+0000 7f32b2a00640 10 client.27647 handle_command_reply: tid=0 2024-09-19T04:33:37.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.658+0000 7f32d62b4740 1 client.27647 shutdown 2024-09-19T04:33:37.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 1 -- 172.21.15.107:0/2096278339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 msgr2=0x564f2c65e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c65e9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f32b8004630 tx=0x7f32b8003a50 comp rx=0 tx=0).stop 2024-09-19T04:33:37.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 1 -- 172.21.15.107:0/2096278339 shutdown_connections 2024-09-19T04:33:37.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f2c671e20 0x564f2c674210 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 0x564f2c65ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c65e9c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32b2a00640 0 client.27647 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:37.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2096278339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c661710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 1 -- 172.21.15.107:0/2096278339 >> 172.21.15.107:0/2096278339 conn(0x564f2c653240 msgr2=0x564f2c658f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:37.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 1 -- 172.21.15.107:0/2096278339 shutdown_connections 2024-09-19T04:33:37.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 1 -- 172.21.15.107:0/2096278339 wait complete. 2024-09-19T04:33:37.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.659+0000 7f32d62b4740 20 client.27647 trim_cache size 0 max 16384 2024-09-19T04:33:37.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d62b4740 1 Processor -- start 2024-09-19T04:33:37.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d62b4740 1 -- start start 2024-09-19T04:33:37.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c65a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c679870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 0x564f2c679db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d62b4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b3d20 con 0x564f2c661e90 2024-09-19T04:33:37.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d62b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b5890 con 0x564f2c6575d0 2024-09-19T04:33:37.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d62b4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b4420 con 0x564f2c656cb0 2024-09-19T04:33:37.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 0x564f2c679db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 0x564f2c679db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37488/0 (socket says 172.21.15.107:37488) 2024-09-19T04:33:37.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.677+0000 7f32d1800640 1 -- 172.21.15.107:0/417708913 learned_addr learned my addr 172.21.15.107:0/417708913 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:37.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.678+0000 7f32d0e00640 1 --2- 172.21.15.107:0/417708913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c65a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.678+0000 7f32cbe00640 1 --2- 172.21.15.107:0/417708913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c679870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.678+0000 7f32d1800640 1 -- 172.21.15.107:0/417708913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 msgr2=0x564f2c65a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:37.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.678+0000 7f32d1800640 1 --2- 172.21.15.107:0/417708913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c65a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.678+0000 7f32d1800640 1 -- 172.21.15.107:0/417708913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 msgr2=0x564f2c679870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.678+0000 7f32d1800640 1 --2- 172.21.15.107:0/417708913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c679870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.678+0000 7f32d1800640 1 -- 172.21.15.107:0/417708913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2c65d9b0 con 0x564f2c661e90 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.678+0000 7f32d0e00640 1 --2- 172.21.15.107:0/417708913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c65a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.678+0000 7f32d1800640 1 --2- 172.21.15.107:0/417708913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 0x564f2c679db0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009ca0 tx=0x7f32c4041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbeeca9b84ed909 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.679+0000 7f32bd600640 1 -- 172.21.15.107:0/417708913 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4003e00 con 0x564f2c661e90 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.679+0000 7f32bd600640 1 -- 172.21.15.107:0/417708913 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c4036040 con 0x564f2c661e90 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.679+0000 7f32bd600640 1 -- 172.21.15.107:0/417708913 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c402f8d0 con 0x564f2c661e90 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.679+0000 7f32d62b4740 1 -- 172.21.15.107:0/417708913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 msgr2=0x564f2c679db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.679+0000 7f32d62b4740 1 --2- 172.21.15.107:0/417708913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 0x564f2c679db0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009ca0 tx=0x7f32c4041000 comp rx=0 tx=0).stop 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.679+0000 7f32d62b4740 1 -- 172.21.15.107:0/417708913 shutdown_connections 2024-09-19T04:33:37.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.679+0000 7f32d62b4740 1 --2- 172.21.15.107:0/417708913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c661e90 0x564f2c679db0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.679+0000 7f32d62b4740 1 --2- 172.21.15.107:0/417708913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c679870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.679+0000 7f32d62b4740 1 --2- 172.21.15.107:0/417708913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c656cb0 0x564f2c65a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.679+0000 7f32d62b4740 1 -- 172.21.15.107:0/417708913 >> 172.21.15.107:0/417708913 conn(0x564f2c653240 msgr2=0x564f2c6598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:37.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.680+0000 7f32d62b4740 1 -- 172.21.15.107:0/417708913 shutdown_connections 2024-09-19T04:33:37.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.680+0000 7f32d62b4740 1 -- 172.21.15.107:0/417708913 wait complete. 2024-09-19T04:33:37.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.680+0000 7f32d62b4740 1 -- start start 2024-09-19T04:33:37.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.680+0000 7f32d62b4740 1 Processor -- start 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c656cb0 0x564f2c6667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c666d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d62b4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c661e90 0x564f2c6d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d62b4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b51c0 con 0x564f2c656cb0 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d62b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b4af0 con 0x564f2c6575d0 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d62b4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b1d20 con 0x564f2c661e90 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d62b4740 10 client.0 _ll_register_callbacks cb 0x7f32b801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d0e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c656cb0 0x564f2c6667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d0e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c656cb0 0x564f2c6667d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37490/0 (socket says 172.21.15.107:37490) 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d0e00640 1 -- 172.21.15.107:0/2652247291 learned_addr learned my addr 172.21.15.107:0/2652247291 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32cbe00640 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c666d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d1800640 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c661e90 0x564f2c6d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d0e00640 1 -- 172.21.15.107:0/2652247291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c661e90 msgr2=0x564f2c6d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d0e00640 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c661e90 0x564f2c6d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d0e00640 1 -- 172.21.15.107:0/2652247291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 msgr2=0x564f2c666d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d0e00640 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c666d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32d0e00640 1 -- 172.21.15.107:0/2652247291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2c6d4ec0 con 0x564f2c656cb0 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.681+0000 7f32cbe00640 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c666d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.682+0000 7f32d0e00640 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c656cb0 0x564f2c6667d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f32c00039a0 tx=0x7f32c00039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55da65019306e79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.682+0000 7f32b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.682+0000 7f32b2a00640 1 -- 172.21.15.107:0/2652247291 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0020040 con 0x564f2c656cb0 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.682+0000 7f32b2a00640 1 -- 172.21.15.107:0/2652247291 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f32c00035f0 con 0x564f2c656cb0 2024-09-19T04:33:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.682+0000 7f32d62b4740 1 -- 172.21.15.107:0/2652247291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32c4036040 con 0x564f2c656cb0 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.682+0000 7f32b2a00640 1 -- 172.21.15.107:0/2652247291 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f32c00985b0 con 0x564f2c656cb0 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.682+0000 7f32d62b4740 10 client.17607 fetch_fsmap learned FSMap version 50 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.682+0000 7f32d62b4740 1 -- 172.21.15.107:0/2652247291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564f2c6dd420 con 0x564f2c656cb0 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.683+0000 7f32b2a00640 1 -- 172.21.15.107:0/2652247291 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c4036040 con 0x564f2c656cb0 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.683+0000 7f32b2a00640 1 -- 172.21.15.107:0/2652247291 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 50) ==== 8370+0+0 (secure 0 0 0) 0x7f32c0081020 con 0x564f2c656cb0 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.683+0000 7f32d62b4740 10 client.17607 fetch_fsmap finished waiting for FSMap version 50 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.683+0000 7f32d62b4740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.683+0000 7f32d62b4740 20 client.17607 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.683+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f2c6ddb40 0x564f2c6dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.683+0000 7f32d62b4740 4 client.17607 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.683+0000 7f32d62b4740 1 -- 172.21.15.107:0/2652247291 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564f2c5c4d10 con 0x564f2c6ddb40 2024-09-19T04:33:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.683+0000 7f32cbe00640 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f2c6ddb40 0x564f2c6dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.684+0000 7f32cbe00640 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f2c6ddb40 0x564f2c6dff30 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7eeee0baf0ff9b server_cookie=76ebd5188e38e4cf in_seq=0 out_seq=0 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.684+0000 7f32b2a00640 10 client.17607 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32b2a00640 1 -- 172.21.15.107:0/2652247291 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564f2c5c4d10 con 0x564f2c6ddb40 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32b2a00640 10 client.17607 handle_command_reply: tid=0 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32d62b4740 1 client.17607 shutdown 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32d62b4740 1 -- 172.21.15.107:0/2652247291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c656cb0 msgr2=0x564f2c6667d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c656cb0 0x564f2c6667d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f32c00039a0 tx=0x7f32c00039f0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32d62b4740 1 -- 172.21.15.107:0/2652247291 shutdown_connections 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x564f2c6ddb40 0x564f2c6dff30 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f2c661e90 0x564f2c6d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32b2a00640 0 client.17607 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2c6575d0 0x564f2c666d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32d62b4740 1 --2- 172.21.15.107:0/2652247291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f2c656cb0 0x564f2c6667d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.685+0000 7f32d62b4740 1 -- 172.21.15.107:0/2652247291 >> 172.21.15.107:0/2652247291 conn(0x564f2c653240 msgr2=0x564f2c652270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d62b4740 1 -- 172.21.15.107:0/2652247291 shutdown_connections 2024-09-19T04:33:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d62b4740 1 -- 172.21.15.107:0/2652247291 wait complete. 2024-09-19T04:33:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d62b4740 20 client.17607 trim_cache size 0 max 16384 2024-09-19T04:33:37.690 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:33:37.690 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-09-19T04:33:37.690 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "6f9464de-68aa-4875-a496-de7f0e68b0dd", 2024-09-19T04:33:37.690 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-09-19T04:33:37.691 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:33:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d2200640 1 -- 172.21.15.107:0/857745072 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32b40a33b0 msgr2=0x7f32b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d2200640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32b40a33b0 0x7f32b40a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f32c4004af0 tx=0x7f32c4004a80 comp rx=0 tx=0).stop 2024-09-19T04:33:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d2200640 1 -- 172.21.15.107:0/857745072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32cc076720 msgr2=0x7f32cc1a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d2200640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32cc076720 0x7f32cc1a45c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005fd0 tx=0x7f32c000dde0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d2200640 1 -- 172.21.15.107:0/857745072 shutdown_connections 2024-09-19T04:33:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d2200640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f32b40a33b0 0x7f32b40a5870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d2200640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32cc112870 0x7f32cc19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d2200640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc077040 0x7f32cc1a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d2200640 1 --2- 172.21.15.107:0/857745072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32cc076720 0x7f32cc1a45c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.686+0000 7f32d2200640 1 -- 172.21.15.107:0/857745072 >> 172.21.15.107:0/857745072 conn(0x7f32cc0ff8d0 msgr2=0x7f32cc101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.687+0000 7f32d2200640 1 -- 172.21.15.107:0/857745072 shutdown_connections 2024-09-19T04:33:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:37.687+0000 7f32d2200640 1 -- 172.21.15.107:0/857745072 wait complete. 2024-09-19T04:33:37.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458725 3.01 MB/sec execute 595 sec latency 5634.534 ms 2024-09-19T04:33:38.090 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:37 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:38.091 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:37 smithi107 ceph-mon[30659]: scrub queued for path: / 2024-09-19T04:33:38.091 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:37 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:38.091 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:37 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:33:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:37 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:38.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:37 smithi115 ceph-mon[35477]: scrub queued for path: / 2024-09-19T04:33:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:37 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:38.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:37 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:33:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:37 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:37 smithi145 ceph-mon[35096]: scrub queued for path: / 2024-09-19T04:33:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:37 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:38.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:37 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:33:38.582 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:33:38.738 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458725 3.01 MB/sec execute 596 sec latency 6634.597 ms 2024-09-19T04:33:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:33:39.093 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: pgmap v416: 97 pgs: 97 active+clean; 414 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-19T04:33:39.093 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:33:39.093 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:33:39.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:33:39.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:33:39.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:33:39.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:33:39.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:38 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: pgmap v416: 97 pgs: 97 active+clean; 414 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-19T04:33:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:33:39.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:33:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:33:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:33:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:33:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:33:39.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:38 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: pgmap v416: 97 pgs: 97 active+clean; 414 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:33:39.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:38 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:39.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- 172.21.15.107:0/3351719753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b20074620 msgr2=0x7f5b20074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:39.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 --2- 172.21.15.107:0/3351719753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b20074620 0x7f5b20074a00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009900 tx=0x7f5b1002a480 comp rx=0 tx=0).stop 2024-09-19T04:33:39.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- 172.21.15.107:0/3351719753 shutdown_connections 2024-09-19T04:33:39.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 --2- 172.21.15.107:0/3351719753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b2010e210 0x7f5b20110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 --2- 172.21.15.107:0/3351719753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b20074f40 0x7f5b2010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 --2- 172.21.15.107:0/3351719753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b20074620 0x7f5b20074a00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- 172.21.15.107:0/3351719753 >> 172.21.15.107:0/3351719753 conn(0x7f5b200fd710 msgr2=0x7f5b200ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:39.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- 172.21.15.107:0/3351719753 shutdown_connections 2024-09-19T04:33:39.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- 172.21.15.107:0/3351719753 wait complete. 2024-09-19T04:33:39.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 Processor -- start 2024-09-19T04:33:39.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- start start 2024-09-19T04:33:39.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b20074620 0x7f5b2019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b20074f40 0x7f5b2019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b2010e210 0x7f5b2019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20114010 con 0x7f5b2010e210 2024-09-19T04:33:39.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20113e90 con 0x7f5b20074620 2024-09-19T04:33:39.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20114190 con 0x7f5b20074f40 2024-09-19T04:33:39.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b2010e210 0x7f5b2019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b2010e210 0x7f5b2019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59772/0 (socket says 172.21.15.107:59772) 2024-09-19T04:33:39.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1f400640 1 -- 172.21.15.107:0/2876318675 learned_addr learned my addr 172.21.15.107:0/2876318675 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:39.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b17e00640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b20074f40 0x7f5b2019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1f400640 1 -- 172.21.15.107:0/2876318675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b20074f40 msgr2=0x7f5b2019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:39.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1f400640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b20074f40 0x7f5b2019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1f400640 1 -- 172.21.15.107:0/2876318675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b20074620 msgr2=0x7f5b2019a590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T04:33:39.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1f400640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b20074620 0x7f5b2019a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1f400640 1 -- 172.21.15.107:0/2876318675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2019f540 con 0x7f5b2010e210 2024-09-19T04:33:39.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1f400640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b2010e210 0x7f5b2019ee60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800eb10 tx=0x7f5b1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:39.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b17400640 1 -- 172.21.15.107:0/2876318675 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18004b20 con 0x7f5b2010e210 2024-09-19T04:33:39.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b17400640 1 -- 172.21.15.107:0/2876318675 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b18004cc0 con 0x7f5b2010e210 2024-09-19T04:33:39.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b17400640 1 -- 172.21.15.107:0/2876318675 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18010760 con 0x7f5b2010e210 2024-09-19T04:33:39.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- 172.21.15.107:0/2876318675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2019f830 con 0x7f5b2010e210 2024-09-19T04:33:39.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.569+0000 7f5b1fe00640 1 -- 172.21.15.107:0/2876318675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b201a7110 con 0x7f5b2010e210 2024-09-19T04:33:39.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17400640 1 -- 172.21.15.107:0/2876318675 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f5b18005020 con 0x7f5b2010e210 2024-09-19T04:33:39.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 Processor -- start 2024-09-19T04:33:39.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 -- start start 2024-09-19T04:33:39.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf735130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 0x561cbf73f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf73ff30 0x561cbf742320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561cbf69b3e0 con 0x561cbf73ff30 2024-09-19T04:33:39.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561cbf642d80 con 0x561cbf734d50 2024-09-19T04:33:39.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561cbf6973a0 con 0x561cbf735670 2024-09-19T04:33:39.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf735130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b1ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 0x561cbf73f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf735130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38800/0 (socket says 172.21.15.107:38800) 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17e00640 1 -- 172.21.15.107:0/598748599 learned_addr learned my addr 172.21.15.107:0/598748599 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17e00640 1 --2- 172.21.15.107:0/598748599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf73ff30 0x561cbf742320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17e00640 1 -- 172.21.15.107:0/598748599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 msgr2=0x561cbf73f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17400640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5b040a33b0 0x7f5b040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17e00640 1 --2- 172.21.15.107:0/598748599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 0x561cbf73f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17e00640 1 -- 172.21.15.107:0/598748599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf73ff30 msgr2=0x561cbf742320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17e00640 1 --2- 172.21.15.107:0/598748599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf73ff30 0x561cbf742320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17e00640 1 -- 172.21.15.107:0/598748599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cbf661ae0 con 0x561cbf734d50 2024-09-19T04:33:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17400640 1 -- 172.21.15.107:0/2876318675 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f5b180d0300 con 0x7f5b2010e210 2024-09-19T04:33:39.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b17e00640 1 --2- 172.21.15.107:0/598748599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf735130 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5b080117e0 tx=0x7f5b080097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bebae12a4340a69e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:39.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b1f400640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5b040a33b0 0x7f5b040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b0f400640 1 -- 172.21.15.107:0/598748599 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800f7c0 con 0x561cbf734d50 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b0f400640 1 -- 172.21.15.107:0/598748599 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b0800fe40 con 0x561cbf734d50 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b0f400640 1 -- 172.21.15.107:0/598748599 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0801d340 con 0x561cbf734d50 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 -- 172.21.15.107:0/598748599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 msgr2=0x561cbf735130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b1f400640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5b040a33b0 0x7f5b040a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5b180cda70 tx=0x7f5b18005b40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 --2- 172.21.15.107:0/598748599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf735130 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5b080117e0 tx=0x7f5b080097d0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 -- 172.21.15.107:0/598748599 shutdown_connections 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 --2- 172.21.15.107:0/598748599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf73ff30 0x561cbf742320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 --2- 172.21.15.107:0/598748599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 0x561cbf73f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 --2- 172.21.15.107:0/598748599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf735130 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 -- 172.21.15.107:0/598748599 >> 172.21.15.107:0/598748599 conn(0x561cbf7312e0 msgr2=0x561cbf7316c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:39.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 -- 172.21.15.107:0/598748599 shutdown_connections 2024-09-19T04:33:39.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 -- 172.21.15.107:0/598748599 wait complete. 2024-09-19T04:33:39.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.576+0000 7f5b28081740 1 -- start start 2024-09-19T04:33:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 Processor -- start 2024-09-19T04:33:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf73cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 0x561cbf73d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf73ff30 0x561cbf73d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561cbf742ba0 con 0x561cbf735670 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561cbf6560f0 con 0x561cbf734d50 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561cbf742d20 con 0x561cbf73ff30 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 10 client.0 _ll_register_callbacks cb 0x561cbf6479e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b17e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf73cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b17e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf73cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:38814/0 (socket says 172.21.15.107:38814) 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b17e00640 1 -- 172.21.15.107:0/2292788023 learned_addr learned my addr 172.21.15.107:0/2292788023 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b1ea00640 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 0x561cbf73d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b17e00640 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf73ff30 0x561cbf73d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b17e00640 1 -- 172.21.15.107:0/2292788023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf73ff30 msgr2=0x561cbf73d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b17e00640 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf73ff30 0x561cbf73d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b17e00640 1 -- 172.21.15.107:0/2292788023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 msgr2=0x561cbf73d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b17e00640 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 0x561cbf73d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b17e00640 1 -- 172.21.15.107:0/2292788023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cbf40c3c0 con 0x561cbf734d50 2024-09-19T04:33:39.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b17e00640 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf73cec0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x561cbf73ff00 tx=0x7f5b0800ea50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9b9d580680347e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:39.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b03e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b03e00640 1 -- 172.21.15.107:0/2292788023 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08004230 con 0x561cbf734d50 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- 172.21.15.107:0/2292788023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b0800fe40 con 0x561cbf734d50 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b03e00640 1 -- 172.21.15.107:0/2292788023 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f5b080043d0 con 0x561cbf734d50 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b03e00640 1 -- 172.21.15.107:0/2292788023 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f5b0800f790 con 0x561cbf734d50 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 10 client.27662 fetch_fsmap learned FSMap version 50 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- 172.21.15.107:0/2292788023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x561cbf74e8f0 con 0x561cbf734d50 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b03e00640 1 -- 172.21.15.107:0/2292788023 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b0800fe40 con 0x561cbf734d50 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b03e00640 1 -- 172.21.15.107:0/2292788023 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 50) ==== 8370+0+0 (secure 0 0 0) 0x7f5b0801d020 con 0x561cbf734d50 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 10 client.27662 fetch_fsmap finished waiting for FSMap version 50 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 10 client.27662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:33:39.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 20 client.27662 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561cbf74fc40 0x561cbf752030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 4 client.27662 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- 172.21.15.107:0/2292788023 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cbf6a8950 con 0x561cbf74fc40 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b1ea00640 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561cbf74fc40 0x561cbf752030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b1ea00640 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561cbf74fc40 0x561cbf752030 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7f5920476e8580 server_cookie=113e4d785da761ac in_seq=0 out_seq=0 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b03e00640 10 client.27662 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b03e00640 1 -- 172.21.15.107:0/2292788023 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cbf6a8950 con 0x561cbf74fc40 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b03e00640 10 client.27662 handle_command_reply: tid=0 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 client.27662 shutdown 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- 172.21.15.107:0/2292788023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 msgr2=0x561cbf73cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf73cec0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x561cbf73ff00 tx=0x7f5b0800ea50 comp rx=0 tx=0).stop 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- 172.21.15.107:0/2292788023 shutdown_connections 2024-09-19T04:33:39.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561cbf74fc40 0x561cbf752030 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf73ff30 0x561cbf73d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 0x561cbf73d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 --2- 172.21.15.107:0/2292788023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf73cec0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b03e00640 0 client.27662 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:39.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- 172.21.15.107:0/2292788023 >> 172.21.15.107:0/2292788023 conn(0x561cbf7312e0 msgr2=0x561cbf736fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:39.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- 172.21.15.107:0/2292788023 shutdown_connections 2024-09-19T04:33:39.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 1 -- 172.21.15.107:0/2292788023 wait complete. 2024-09-19T04:33:39.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.581+0000 7f5b28081740 20 client.27662 trim_cache size 0 max 16384 2024-09-19T04:33:39.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.600+0000 7f5b28081740 1 Processor -- start 2024-09-19T04:33:39.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.600+0000 7f5b28081740 1 -- start start 2024-09-19T04:33:39.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.600+0000 7f5b28081740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf73f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.600+0000 7f5b28081740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 0x561cbf7377b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.600+0000 7f5b28081740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf73ff30 0x561cbf7576c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201af930 con 0x561cbf73ff30 2024-09-19T04:33:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201b14a0 con 0x561cbf734d50 2024-09-19T04:33:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201b0030 con 0x561cbf735670 2024-09-19T04:33:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 0x561cbf7377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 0x561cbf7377b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:54154/0 (socket says 172.21.15.107:54154) 2024-09-19T04:33:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 -- 172.21.15.107:0/2813625175 learned_addr learned my addr 172.21.15.107:0/2813625175 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b17e00640 1 --2- 172.21.15.107:0/2813625175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf73ff30 0x561cbf7576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 -- 172.21.15.107:0/2813625175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 msgr2=0x561cbf73f1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:33:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 --2- 172.21.15.107:0/2813625175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf73f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 -- 172.21.15.107:0/2813625175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf73ff30 msgr2=0x561cbf7576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 --2- 172.21.15.107:0/2813625175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf73ff30 0x561cbf7576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 -- 172.21.15.107:0/2813625175 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cbf731f70 con 0x561cbf735670 2024-09-19T04:33:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 --2- 172.21.15.107:0/2813625175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 0x561cbf7377b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5b1002a9a0 tx=0x7f5b10000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50bf5e2944b5595 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b0ea00640 1 -- 172.21.15.107:0/2813625175 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10049070 con 0x561cbf735670 2024-09-19T04:33:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b0ea00640 1 -- 172.21.15.107:0/2813625175 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b10003ad0 con 0x561cbf735670 2024-09-19T04:33:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b0ea00640 1 -- 172.21.15.107:0/2813625175 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10035040 con 0x561cbf735670 2024-09-19T04:33:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- 172.21.15.107:0/2813625175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 msgr2=0x561cbf7377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 --2- 172.21.15.107:0/2813625175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 0x561cbf7377b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5b1002a9a0 tx=0x7f5b10000bc0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- 172.21.15.107:0/2813625175 shutdown_connections 2024-09-19T04:33:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 --2- 172.21.15.107:0/2813625175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf73ff30 0x561cbf7576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 --2- 172.21.15.107:0/2813625175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf735670 0x561cbf7377b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 --2- 172.21.15.107:0/2813625175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf73f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- 172.21.15.107:0/2813625175 >> 172.21.15.107:0/2813625175 conn(0x561cbf7312e0 msgr2=0x561cbf7303a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- 172.21.15.107:0/2813625175 shutdown_connections 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- 172.21.15.107:0/2813625175 wait complete. 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- start start 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 Processor -- start 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf748320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 0x561cbf748860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf73ff30 0x561cbf7438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201b0dd0 con 0x561cbf735670 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201b0700 con 0x561cbf734d50 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201ad930 con 0x561cbf73ff30 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b28081740 10 client.0 _ll_register_callbacks cb 0x7f5b080116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b17e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf748320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 0x561cbf748860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 0x561cbf748860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59808/0 (socket says 172.21.15.107:59808) 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 -- 172.21.15.107:0/1342425931 learned_addr learned my addr 172.21.15.107:0/1342425931 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b17e00640 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf73ff30 0x561cbf7438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b17e00640 1 -- 172.21.15.107:0/1342425931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf73ff30 msgr2=0x561cbf7438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b17e00640 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf73ff30 0x561cbf7438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b17e00640 1 -- 172.21.15.107:0/1342425931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 msgr2=0x561cbf748860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b17e00640 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 0x561cbf748860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b17e00640 1 -- 172.21.15.107:0/1342425931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cbf7440d0 con 0x561cbf734d50 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b1ea00640 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 0x561cbf748860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b17e00640 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf748320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5b08001eb0 tx=0x7f5b08005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfd39ee33d7d2a38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:33:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b03e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:33:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.605+0000 7f5b03e00640 1 -- 172.21.15.107:0/1342425931 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08038060 con 0x561cbf734d50 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b28081740 1 -- 172.21.15.107:0/1342425931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b10003ad0 con 0x561cbf734d50 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b03e00640 1 -- 172.21.15.107:0/1342425931 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f5b08009780 con 0x561cbf734d50 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b03e00640 1 -- 172.21.15.107:0/1342425931 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f5b08004880 con 0x561cbf734d50 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b28081740 10 client.27665 fetch_fsmap learned FSMap version 50 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b28081740 1 -- 172.21.15.107:0/1342425931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x561cbf7ba310 con 0x561cbf734d50 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b03e00640 1 -- 172.21.15.107:0/1342425931 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b10003ad0 con 0x561cbf734d50 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b03e00640 1 -- 172.21.15.107:0/1342425931 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 50) ==== 8370+0+0 (secure 0 0 0) 0x7f5b0801d040 con 0x561cbf734d50 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b28081740 10 client.27665 fetch_fsmap finished waiting for FSMap version 50 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b28081740 10 client.27665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b28081740 20 client.27665 populate_metadata read hostname 'smithi107' 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b28081740 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561cbf7ba9d0 0x561cbf7bcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b28081740 4 client.27665 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:33:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b28081740 1 -- 172.21.15.107:0/1342425931 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561cbf6a2e60 con 0x561cbf7ba9d0 2024-09-19T04:33:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.610+0000 7f5b1ea00640 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561cbf7ba9d0 0x561cbf7bcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:33:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1ea00640 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561cbf7ba9d0 0x561cbf7bcdc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61516affcb4b586c server_cookie=c86370014f124153 in_seq=0 out_seq=0 2024-09-19T04:33:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b03e00640 10 client.27665 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b03e00640 1 -- 172.21.15.107:0/1342425931 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x561cbf6a2e60 con 0x561cbf7ba9d0 2024-09-19T04:33:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b03e00640 10 client.27665 handle_command_reply: tid=0 2024-09-19T04:33:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 client.27665 shutdown 2024-09-19T04:33:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 -- 172.21.15.107:0/1342425931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 msgr2=0x561cbf748320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf748320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5b08001eb0 tx=0x7f5b08005d90 comp rx=0 tx=0).stop 2024-09-19T04:33:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 -- 172.21.15.107:0/1342425931 shutdown_connections 2024-09-19T04:33:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x561cbf7ba9d0 0x561cbf7bcdc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561cbf73ff30 0x561cbf7438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cbf735670 0x561cbf748860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 --2- 172.21.15.107:0/1342425931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cbf734d50 0x561cbf748320 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 -- 172.21.15.107:0/1342425931 >> 172.21.15.107:0/1342425931 conn(0x561cbf7312e0 msgr2=0x561cbf7303a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b03e00640 0 client.27665 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:33:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 -- 172.21.15.107:0/1342425931 shutdown_connections 2024-09-19T04:33:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 1 -- 172.21.15.107:0/1342425931 wait complete. 2024-09-19T04:33:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b28081740 20 client.27665 trim_cache size 0 max 16384 2024-09-19T04:33:39.651 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:33:39.652 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (1 inodes in the stack)", 2024-09-19T04:33:39.652 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-09-19T04:33:39.652 INFO:teuthology.orchestra.run.smithi107.stdout: "6f9464de-68aa-4875-a496-de7f0e68b0dd": { 2024-09-19T04:33:39.652 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-09-19T04:33:39.652 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "6f9464de-68aa-4875-a496-de7f0e68b0dd", 2024-09-19T04:33:39.652 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-09-19T04:33:39.652 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:33:39.652 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:33:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:33:39.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 -- 172.21.15.107:0/2876318675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5b040a33b0 msgr2=0x7f5b040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:39.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5b040a33b0 0x7f5b040a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5b180cda70 tx=0x7f5b18005b40 comp rx=0 tx=0).stop 2024-09-19T04:33:39.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 -- 172.21.15.107:0/2876318675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b2010e210 msgr2=0x7f5b2019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:33:39.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b2010e210 0x7f5b2019ee60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800eb10 tx=0x7f5b1800c2d0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 -- 172.21.15.107:0/2876318675 shutdown_connections 2024-09-19T04:33:39.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f5b040a33b0 0x7f5b040a5870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b2010e210 0x7f5b2019ee60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b20074f40 0x7f5b2019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 --2- 172.21.15.107:0/2876318675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b20074620 0x7f5b2019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:33:39.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 -- 172.21.15.107:0/2876318675 >> 172.21.15.107:0/2876318675 conn(0x7f5b200fd710 msgr2=0x7f5b200fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:33:39.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 -- 172.21.15.107:0/2876318675 shutdown_connections 2024-09-19T04:33:39.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:33:39.648+0000 7f5b1fe00640 1 -- 172.21.15.107:0/2876318675 wait complete. 2024-09-19T04:33:39.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458725 3.00 MB/sec execute 597 sec latency 7634.641 ms 2024-09-19T04:33:40.721 INFO:tasks.cephfs.filesystem:scrub status for tag:6f9464de-68aa-4875-a496-de7f0e68b0dd - {'path': '/', 'tag': '6f9464de-68aa-4875-a496-de7f0e68b0dd', 'options': 'recursive,force'} 2024-09-19T04:33:40.737 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458725 3.00 MB/sec execute 598 sec latency 8634.750 ms 2024-09-19T04:33:40.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:40 smithi107 ceph-mon[30659]: pgmap v417: 97 pgs: 97 active+clean; 414 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-19T04:33:41.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:40 smithi115 ceph-mon[35477]: pgmap v417: 97 pgs: 97 active+clean; 414 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-19T04:33:41.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:40 smithi145 ceph-mon[35096]: pgmap v417: 97 pgs: 97 active+clean; 414 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-19T04:33:41.738 INFO:tasks.workunit.client.0.smithi107.stdout: 1 458725 2.99 MB/sec execute 599 sec latency 9634.821 ms 2024-09-19T04:33:42.738 INFO:tasks.workunit.client.0.smithi107.stdout: 1 cleanup 600 sec 2024-09-19T04:33:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:42 smithi115 ceph-mon[35477]: pgmap v418: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-19T04:33:43.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:42 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:42 smithi145 ceph-mon[35096]: pgmap v418: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-19T04:33:43.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:42 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:43.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:42 smithi107 ceph-mon[30659]: pgmap v418: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-19T04:33:43.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:42 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:43.738 INFO:tasks.workunit.client.0.smithi107.stdout: 1 cleanup 601 sec 2024-09-19T04:33:44.738 INFO:tasks.workunit.client.0.smithi107.stdout: 1 cleanup 602 sec 2024-09-19T04:33:45.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:44 smithi107 ceph-mon[30659]: pgmap v419: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 745 KiB/s wr, 63 op/s 2024-09-19T04:33:45.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:44 smithi115 ceph-mon[35477]: pgmap v419: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 745 KiB/s wr, 63 op/s 2024-09-19T04:33:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:44 smithi145 ceph-mon[35096]: pgmap v419: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 745 KiB/s wr, 63 op/s 2024-09-19T04:33:45.738 INFO:tasks.workunit.client.0.smithi107.stdout: 1 cleanup 603 sec 2024-09-19T04:33:46.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:45 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:33:46.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:45 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:33:46.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:45 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:33:46.738 INFO:tasks.workunit.client.0.smithi107.stdout: 1 cleanup 604 sec 2024-09-19T04:33:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:46 smithi115 ceph-mon[35477]: pgmap v420: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-09-19T04:33:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:46 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:33:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:46 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:33:47.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:46 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:33:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:46 smithi145 ceph-mon[35096]: pgmap v420: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-09-19T04:33:47.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:46 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:33:47.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:46 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:33:47.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:46 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:33:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:46 smithi107 ceph-mon[30659]: pgmap v420: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-09-19T04:33:47.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:46 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:33:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:46 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:33:47.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:46 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:33:47.738 INFO:tasks.workunit.client.0.smithi107.stdout: 1 cleanup 605 sec 2024-09-19T04:33:48.738 INFO:tasks.workunit.client.0.smithi107.stdout: 1 cleanup 606 sec 2024-09-19T04:33:49.121 INFO:tasks.workunit.client.0.smithi107.stdout: 0 cleanup 606 sec 2024-09-19T04:33:49.121 INFO:tasks.workunit.client.0.smithi107.stdout: 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Operation Count AvgLat MaxLat 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: ---------------------------------------- 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: NTCreateX 57100 4.602 16888.081 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Close 41938 0.004 0.171 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Rename 2418 88.943 4500.036 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Unlink 11534 2.526 4922.843 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Deltree 2 105.891 143.546 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Mkdir 1 0.015 0.015 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Qpathinfo 51756 1.448 4246.040 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Qfileinfo 9059 0.002 0.042 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Qfsinfo 9487 0.211 4.107 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Sfileinfo 4650 0.033 4.177 2024-09-19T04:33:49.122 INFO:tasks.workunit.client.0.smithi107.stdout: Find 20017 0.295 522.292 2024-09-19T04:33:49.123 INFO:tasks.workunit.client.0.smithi107.stdout: WriteX 28433 0.135 36.058 2024-09-19T04:33:49.123 INFO:tasks.workunit.client.0.smithi107.stdout: ReadX 89511 0.015 18.609 2024-09-19T04:33:49.123 INFO:tasks.workunit.client.0.smithi107.stdout: LockX 186 0.695 3.305 2024-09-19T04:33:49.123 INFO:tasks.workunit.client.0.smithi107.stdout: UnlockX 186 0.607 1.402 2024-09-19T04:33:49.123 INFO:tasks.workunit.client.0.smithi107.stdout: Flush 3999 3.206 621.251 2024-09-19T04:33:49.123 INFO:tasks.workunit.client.0.smithi107.stdout: 2024-09-19T04:33:49.123 INFO:tasks.workunit.client.0.smithi107.stdout:Throughput 2.99107 MB/sec 1 clients 1 procs max_latency=9634.821 ms 2024-09-19T04:33:49.123 INFO:tasks.workunit.client.0.smithi107.stderr:+ dbench 10 2024-09-19T04:33:49.124 INFO:tasks.workunit.client.0.smithi107.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T04:33:49.124 INFO:tasks.workunit.client.0.smithi107.stdout: 2024-09-19T04:33:49.124 INFO:tasks.workunit.client.0.smithi107.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T04:33:49.126 INFO:tasks.workunit.client.0.smithi107.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-19T04:33:49.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:48 smithi115 ceph-mon[35477]: pgmap v421: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 731 KiB/s wr, 41 op/s 2024-09-19T04:33:49.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:48 smithi145 ceph-mon[35096]: pgmap v421: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 731 KiB/s wr, 41 op/s 2024-09-19T04:33:49.226 INFO:tasks.workunit.client.0.smithi107.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-19T04:33:49.226 INFO:tasks.workunit.client.0.smithi107.stdout:releasing clients 2024-09-19T04:33:49.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:48 smithi107 ceph-mon[30659]: pgmap v421: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 731 KiB/s wr, 41 op/s 2024-09-19T04:33:49.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:49 smithi107 ceph-mon[30659]: scrub summary: active paths [/] 2024-09-19T04:33:49.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:49 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:49.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:49 smithi107 ceph-mon[30659]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:49 smithi115 ceph-mon[35477]: scrub summary: active paths [/] 2024-09-19T04:33:50.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:49 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:50.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:49 smithi115 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:50.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:49 smithi145 ceph-mon[35096]: scrub summary: active paths [/] 2024-09-19T04:33:50.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:49 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:50.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:49 smithi145 ceph-mon[35096]: scrub summary: idle+waiting paths [/] 2024-09-19T04:33:50.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 225 87.00 MB/sec warmup 1 sec latency 557.211 ms 2024-09-19T04:33:51.125 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:50 smithi145 ceph-mon[35096]: pgmap v422: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 731 KiB/s wr, 41 op/s 2024-09-19T04:33:51.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:50 smithi115 ceph-mon[35477]: pgmap v422: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 731 KiB/s wr, 41 op/s 2024-09-19T04:33:51.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 596 98.16 MB/sec warmup 2 sec latency 81.512 ms 2024-09-19T04:33:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:50 smithi107 ceph-mon[30659]: pgmap v422: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 731 KiB/s wr, 41 op/s 2024-09-19T04:33:52.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 1057 74.14 MB/sec warmup 3 sec latency 119.379 ms 2024-09-19T04:33:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:52 smithi115 ceph-mon[35477]: pgmap v423: 97 pgs: 97 active+clean; 416 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-09-19T04:33:53.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:52 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:52 smithi145 ceph-mon[35096]: pgmap v423: 97 pgs: 97 active+clean; 416 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-09-19T04:33:53.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:52 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:53.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 1679 64.52 MB/sec warmup 4 sec latency 102.220 ms 2024-09-19T04:33:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:52 smithi107 ceph-mon[30659]: pgmap v423: 97 pgs: 97 active+clean; 416 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-09-19T04:33:53.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:52 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:54.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 2417 56.59 MB/sec warmup 5 sec latency 43.289 ms 2024-09-19T04:33:55.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:54 smithi115 ceph-mon[35477]: pgmap v424: 97 pgs: 97 active+clean; 416 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 779 KiB/s wr, 51 op/s 2024-09-19T04:33:55.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:54 smithi145 ceph-mon[35096]: pgmap v424: 97 pgs: 97 active+clean; 416 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 779 KiB/s wr, 51 op/s 2024-09-19T04:33:55.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 2847 54.63 MB/sec warmup 6 sec latency 452.626 ms 2024-09-19T04:33:55.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:54 smithi107 ceph-mon[30659]: pgmap v424: 97 pgs: 97 active+clean; 416 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 779 KiB/s wr, 51 op/s 2024-09-19T04:33:56.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:55 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:56.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:55 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:56.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 3466 53.49 MB/sec warmup 7 sec latency 571.418 ms 2024-09-19T04:33:56.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:55 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:56 smithi115 ceph-mon[35477]: pgmap v425: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-09-19T04:33:57.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:56 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:33:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:56 smithi145 ceph-mon[35096]: pgmap v425: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-09-19T04:33:57.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:56 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:33:57.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 3692 48.31 MB/sec warmup 8 sec latency 1054.542 ms 2024-09-19T04:33:57.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:56 smithi107 ceph-mon[30659]: pgmap v425: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-09-19T04:33:57.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:56 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:33:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:57 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:33:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:57 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:33:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:57 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:33:58.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:57 smithi115 ceph-mon[35477]: scrub summary: idle 2024-09-19T04:33:58.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:57 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:57 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:33:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:57 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:33:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:57 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:33:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:57 smithi145 ceph-mon[35096]: scrub summary: idle 2024-09-19T04:33:58.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:57 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:58.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 3891 43.00 MB/sec warmup 9 sec latency 2054.584 ms 2024-09-19T04:33:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:57 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:33:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:57 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:33:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:57 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:33:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:57 smithi107 ceph-mon[30659]: scrub summary: idle 2024-09-19T04:33:58.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:57 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:33:59.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:33:58 smithi115 ceph-mon[35477]: pgmap v426: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-09-19T04:33:59.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:33:58 smithi145 ceph-mon[35096]: pgmap v426: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-09-19T04:33:59.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 4180 39.33 MB/sec warmup 10 sec latency 3054.632 ms 2024-09-19T04:33:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:33:58 smithi107 ceph-mon[30659]: pgmap v426: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-09-19T04:34:00.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 4554 37.22 MB/sec warmup 11 sec latency 3059.646 ms 2024-09-19T04:34:01.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:00 smithi145 ceph-mon[35096]: pgmap v427: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-09-19T04:34:01.226 INFO:tasks.workunit.client.0.smithi107.stdout: 10 5009 36.36 MB/sec warmup 12 sec latency 566.403 ms 2024-09-19T04:34:01.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:00 smithi107 ceph-mon[30659]: pgmap v427: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-09-19T04:34:01.257 INFO:tasks.workunit.client.0.smithi107.stdout:[5556] write failed on handle 11056 (Resource temporarily unavailable) 2024-09-19T04:34:01.257 INFO:tasks.workunit.client.0.smithi107.stdout:Child failed with status 1 2024-09-19T04:34:01.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:34:01.271 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-19T04:34:01.272 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T04:34:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:00 smithi115 ceph-mon[35477]: pgmap v427: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-09-19T04:34:02.414 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi107 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-19T04:34:02.650 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=469ea1fddc8e4ef2be57d4c0b0b28fe7 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi107 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-19T04:34:02.653 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T04:34:02.663 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T04:34:02.663 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:34:02.663 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-09-19T04:34:02.664 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T04:34:02.727 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T04:34:02.789 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T04:34:02.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.940+0000 7f177b200640 1 -- 172.21.15.107:0/438930154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1774104900 msgr2=0x7f1774108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:02.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.940+0000 7f177b200640 1 --2- 172.21.15.107:0/438930154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1774104900 0x7f1774108940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f175c009900 tx=0x7f175c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:34:02.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.940+0000 7f177b200640 1 -- 172.21.15.107:0/438930154 shutdown_connections 2024-09-19T04:34:02.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.940+0000 7f177b200640 1 --2- 172.21.15.107:0/438930154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17741090a0 0x7f1774110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:02.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.940+0000 7f177b200640 1 --2- 172.21.15.107:0/438930154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1774104900 0x7f1774108940 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:02.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.940+0000 7f177b200640 1 --2- 172.21.15.107:0/438930154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1774103f50 0x7f1774104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:02.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.940+0000 7f177b200640 1 -- 172.21.15.107:0/438930154 >> 172.21.15.107:0/438930154 conn(0x7f17740ff9e0 msgr2=0x7f1774101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:02.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.940+0000 7f177b200640 1 -- 172.21.15.107:0/438930154 shutdown_connections 2024-09-19T04:34:02.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.940+0000 7f177b200640 1 -- 172.21.15.107:0/438930154 wait complete. 2024-09-19T04:34:02.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.941+0000 7f177b200640 1 Processor -- start 2024-09-19T04:34:02.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.941+0000 7f177b200640 1 -- start start 2024-09-19T04:34:02.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f177b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1774103f50 0x7f177419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:02.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f177b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17741090a0 0x7f177419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:02.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f177b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f177419eeb0 0x7f177419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:02.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f177b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17741140a0 con 0x7f1774103f50 2024-09-19T04:34:02.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f177b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1774113f20 con 0x7f17741090a0 2024-09-19T04:34:02.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f177b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1774114220 con 0x7f177419eeb0 2024-09-19T04:34:02.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f1773e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1774103f50 0x7f177419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:02.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f1773e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1774103f50 0x7f177419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41610/0 (socket says 172.21.15.107:41610) 2024-09-19T04:34:02.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f1773e00640 1 -- 172.21.15.107:0/1885989583 learned_addr learned my addr 172.21.15.107:0/1885989583 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:02.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f176be00640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17741090a0 0x7f177419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:02.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f1779200640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f177419eeb0 0x7f177419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:02.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f1773e00640 1 -- 172.21.15.107:0/1885989583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f177419eeb0 msgr2=0x7f177419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:02.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f1773e00640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f177419eeb0 0x7f177419f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:02.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f1773e00640 1 -- 172.21.15.107:0/1885989583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17741090a0 msgr2=0x7f177419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:02.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f1773e00640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17741090a0 0x7f177419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:02.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f1773e00640 1 -- 172.21.15.107:0/1885989583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177419f9a0 con 0x7f1774103f50 2024-09-19T04:34:02.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.942+0000 7f176be00640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17741090a0 0x7f177419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:02.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.943+0000 7f1773e00640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1774103f50 0x7f177419a5f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f176400d950 tx=0x7f176400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:02.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.943+0000 7f1771600640 1 -- 172.21.15.107:0/1885989583 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764017030 con 0x7f1774103f50 2024-09-19T04:34:02.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.943+0000 7f1771600640 1 -- 172.21.15.107:0/1885989583 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1764004020 con 0x7f1774103f50 2024-09-19T04:34:02.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.943+0000 7f177b200640 1 -- 172.21.15.107:0/1885989583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177419fc90 con 0x7f1774103f50 2024-09-19T04:34:02.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.943+0000 7f1771600640 1 -- 172.21.15.107:0/1885989583 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640047a0 con 0x7f1774103f50 2024-09-19T04:34:02.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.943+0000 7f177b200640 1 -- 172.21.15.107:0/1885989583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17741a7450 con 0x7f1774103f50 2024-09-19T04:34:02.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.944+0000 7f177b200640 1 -- 172.21.15.107:0/1885989583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1774104330 con 0x7f1774103f50 2024-09-19T04:34:02.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.946+0000 7f1771600640 1 -- 172.21.15.107:0/1885989583 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f17640041d0 con 0x7f1774103f50 2024-09-19T04:34:02.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.947+0000 7f1771600640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17580a3460 0x7f17580a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:02.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.948+0000 7f1771600640 1 -- 172.21.15.107:0/1885989583 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8528+0+0 (secure 0 0 0) 0x7f17640cf6a0 con 0x7f1774103f50 2024-09-19T04:34:02.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.948+0000 7f176be00640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17580a3460 0x7f17580a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:02.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.948+0000 7f176be00640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17580a3460 0x7f17580a5920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f175c004770 tx=0x7f175c0058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:02.952+0000 7f1771600640 1 -- 172.21.15.107:0/1885989583 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f176409c040 con 0x7f1774103f50 2024-09-19T04:34:02.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:02 smithi107 ceph-mon[30659]: pgmap v428: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-09-19T04:34:02.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:02 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:34:02.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:02 smithi107 ceph-mon[30659]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T04:34:03.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.151+0000 7f177b200640 1 -- 172.21.15.107:0/1885989583 --> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1774104900 con 0x7f17580a3460 2024-09-19T04:34:03.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.161+0000 7f1771600640 1 -- 172.21.15.107:0/1885989583 <== mgr.15114 v2:172.21.15.107:6800/1724794486 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f1774104900 con 0x7f17580a3460 2024-09-19T04:34:03.162 INFO:teuthology.orchestra.run.smithi107.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T04:20:30", "first": "2024-09-19T04:21:00", "last": "2024-09-19T04:34:00", "last_pruned": "2024-09-19T04:34:00", "created_count": 14, "pruned_count": 8, "active": true} 2024-09-19T04:34:03.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 -- 172.21.15.107:0/1885989583 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17580a3460 msgr2=0x7f17580a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:03.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17580a3460 0x7f17580a5920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f175c004770 tx=0x7f175c0058d0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 -- 172.21.15.107:0/1885989583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1774103f50 msgr2=0x7f177419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:03.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1774103f50 0x7f177419a5f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f176400d950 tx=0x7f176400de10 comp rx=0 tx=0).stop 2024-09-19T04:34:03.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 -- 172.21.15.107:0/1885989583 shutdown_connections 2024-09-19T04:34:03.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f17580a3460 0x7f17580a5920 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f177419eeb0 0x7f177419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17741090a0 0x7f177419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 --2- 172.21.15.107:0/1885989583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1774103f50 0x7f177419a5f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 -- 172.21.15.107:0/1885989583 >> 172.21.15.107:0/1885989583 conn(0x7f17740ff9e0 msgr2=0x7f1774100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:03.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 -- 172.21.15.107:0/1885989583 shutdown_connections 2024-09-19T04:34:03.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.163+0000 7f177b200640 1 -- 172.21.15.107:0/1885989583 wait complete. 2024-09-19T04:34:03.181 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T04:34:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:02 smithi145 ceph-mon[35096]: pgmap v428: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-09-19T04:34:03.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:02 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:34:03.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:02 smithi145 ceph-mon[35096]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T04:34:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.335+0000 7f2f66400640 1 -- 172.21.15.107:0/55686298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f60104900 msgr2=0x7f2f60108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.335+0000 7f2f66400640 1 --2- 172.21.15.107:0/55686298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f60104900 0x7f2f60108940 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009860 tx=0x7f2f4802a340 comp rx=0 tx=0).stop 2024-09-19T04:34:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.335+0000 7f2f66400640 1 -- 172.21.15.107:0/55686298 shutdown_connections 2024-09-19T04:34:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.335+0000 7f2f66400640 1 --2- 172.21.15.107:0/55686298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f601090a0 0x7f2f60110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.335+0000 7f2f66400640 1 --2- 172.21.15.107:0/55686298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f60104900 0x7f2f60108940 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.335+0000 7f2f66400640 1 --2- 172.21.15.107:0/55686298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f60103f50 0x7f2f60104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.335+0000 7f2f66400640 1 -- 172.21.15.107:0/55686298 >> 172.21.15.107:0/55686298 conn(0x7f2f600ff9e0 msgr2=0x7f2f60101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.335+0000 7f2f66400640 1 -- 172.21.15.107:0/55686298 shutdown_connections 2024-09-19T04:34:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.335+0000 7f2f66400640 1 -- 172.21.15.107:0/55686298 wait complete. 2024-09-19T04:34:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.336+0000 7f2f66400640 1 Processor -- start 2024-09-19T04:34:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.336+0000 7f2f66400640 1 -- start start 2024-09-19T04:34:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.336+0000 7f2f66400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f60103f50 0x7f2f6019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.336+0000 7f2f66400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f601090a0 0x7f2f6019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.336+0000 7f2f66400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f6019eef0 0x7f2f6019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:03.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.336+0000 7f2f66400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f601140a0 con 0x7f2f601090a0 2024-09-19T04:34:03.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.336+0000 7f2f66400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f60113f20 con 0x7f2f60103f50 2024-09-19T04:34:03.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.336+0000 7f2f66400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2f60114220 con 0x7f2f6019eef0 2024-09-19T04:34:03.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f601090a0 0x7f2f6019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:03.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f601090a0 0x7f2f6019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41624/0 (socket says 172.21.15.107:41624) 2024-09-19T04:34:03.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f57e00640 1 -- 172.21.15.107:0/3357943139 learned_addr learned my addr 172.21.15.107:0/3357943139 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:03.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f5fe00640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f6019eef0 0x7f2f6019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:03.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f5f400640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f60103f50 0x7f2f6019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:03.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f57e00640 1 -- 172.21.15.107:0/3357943139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f6019eef0 msgr2=0x7f2f6019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:03.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f57e00640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f6019eef0 0x7f2f6019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f57e00640 1 -- 172.21.15.107:0/3357943139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f60103f50 msgr2=0x7f2f6019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:03.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f57e00640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f60103f50 0x7f2f6019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f57e00640 1 -- 172.21.15.107:0/3357943139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6019f9e0 con 0x7f2f601090a0 2024-09-19T04:34:03.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.337+0000 7f2f57e00640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f601090a0 0x7f2f6019ab70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2f48035040 tx=0x7f2f48033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:03.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.338+0000 7f2f5cc00640 1 -- 172.21.15.107:0/3357943139 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48037040 con 0x7f2f601090a0 2024-09-19T04:34:03.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.338+0000 7f2f5cc00640 1 -- 172.21.15.107:0/3357943139 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f48040070 con 0x7f2f601090a0 2024-09-19T04:34:03.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.338+0000 7f2f5cc00640 1 -- 172.21.15.107:0/3357943139 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48003a90 con 0x7f2f601090a0 2024-09-19T04:34:03.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.338+0000 7f2f66400640 1 -- 172.21.15.107:0/3357943139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6019fc70 con 0x7f2f601090a0 2024-09-19T04:34:03.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.338+0000 7f2f66400640 1 -- 172.21.15.107:0/3357943139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f601a7490 con 0x7f2f601090a0 2024-09-19T04:34:03.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.339+0000 7f2f66400640 1 -- 172.21.15.107:0/3357943139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f60104330 con 0x7f2f601090a0 2024-09-19T04:34:03.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.341+0000 7f2f5cc00640 1 -- 172.21.15.107:0/3357943139 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f2f48003c30 con 0x7f2f601090a0 2024-09-19T04:34:03.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.342+0000 7f2f5cc00640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2f440a34e0 0x7f2f440a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:03.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.342+0000 7f2f5cc00640 1 -- 172.21.15.107:0/3357943139 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8528+0+0 (secure 0 0 0) 0x7f2f480efc00 con 0x7f2f601090a0 2024-09-19T04:34:03.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.342+0000 7f2f5f400640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2f440a34e0 0x7f2f440a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:03.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.343+0000 7f2f5f400640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2f440a34e0 0x7f2f440a59a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000bbe0 tx=0x7f2f5000b390 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:03.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.346+0000 7f2f5cc00640 1 -- 172.21.15.107:0/3357943139 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f480bc6b0 con 0x7f2f601090a0 2024-09-19T04:34:03.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:02 smithi115 ceph-mon[35477]: pgmap v428: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-09-19T04:34:03.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:02 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:34:03.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:02 smithi115 ceph-mon[35477]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T04:34:03.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.543+0000 7f2f66400640 1 -- 172.21.15.107:0/3357943139 --> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2f60104900 con 0x7f2f440a34e0 2024-09-19T04:34:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.551+0000 7f2f5cc00640 1 -- 172.21.15.107:0/3357943139 <== mgr.15114 v2:172.21.15.107:6800/1724794486 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2f60104900 con 0x7f2f440a34e0 2024-09-19T04:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout:/ 1m 6m3h 2024-09-19T04:34:03.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 -- 172.21.15.107:0/3357943139 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2f440a34e0 msgr2=0x7f2f440a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:03.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2f440a34e0 0x7f2f440a59a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000bbe0 tx=0x7f2f5000b390 comp rx=0 tx=0).stop 2024-09-19T04:34:03.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 -- 172.21.15.107:0/3357943139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f601090a0 msgr2=0x7f2f6019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:03.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f601090a0 0x7f2f6019ab70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2f48035040 tx=0x7f2f48033d90 comp rx=0 tx=0).stop 2024-09-19T04:34:03.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 -- 172.21.15.107:0/3357943139 shutdown_connections 2024-09-19T04:34:03.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2f440a34e0 0x7f2f440a59a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f6019eef0 0x7f2f6019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f601090a0 0x7f2f6019ab70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 --2- 172.21.15.107:0/3357943139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f60103f50 0x7f2f6019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:03.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 -- 172.21.15.107:0/3357943139 >> 172.21.15.107:0/3357943139 conn(0x7f2f600ff9e0 msgr2=0x7f2f60100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:03.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 -- 172.21.15.107:0/3357943139 shutdown_connections 2024-09-19T04:34:03.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:03.553+0000 7f2f66400640 1 -- 172.21.15.107:0/3357943139 wait complete. 2024-09-19T04:34:03.571 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T04:34:03.584 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T04:34:04.207 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:03 smithi107 ceph-mon[30659]: from='client.17631 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:34:04.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:03 smithi145 ceph-mon[35096]: from='client.17631 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:34:04.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:03 smithi115 ceph-mon[35477]: from='client.17631 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:34:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:04 smithi145 ceph-mon[35096]: pgmap v430: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 236 op/s 2024-09-19T04:34:05.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:04 smithi145 ceph-mon[35096]: from='client.17637 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:34:05.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:04 smithi107 ceph-mon[30659]: pgmap v430: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 236 op/s 2024-09-19T04:34:05.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:04 smithi107 ceph-mon[30659]: from='client.17637 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:34:05.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:04 smithi115 ceph-mon[35477]: pgmap v430: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 236 op/s 2024-09-19T04:34:05.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:04 smithi115 ceph-mon[35477]: from='client.17637 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:34:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:06 smithi145 ceph-mon[35096]: pgmap v431: 97 pgs: 97 active+clean; 464 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 196 op/s 2024-09-19T04:34:07.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:06 smithi145 ceph-mon[35096]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T04:34:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:06 smithi107 ceph-mon[30659]: pgmap v431: 97 pgs: 97 active+clean; 464 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 196 op/s 2024-09-19T04:34:07.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:06 smithi107 ceph-mon[30659]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T04:34:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:06 smithi115 ceph-mon[35477]: pgmap v431: 97 pgs: 97 active+clean; 464 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 196 op/s 2024-09-19T04:34:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:06 smithi115 ceph-mon[35477]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T04:34:09.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:08 smithi145 ceph-mon[35096]: pgmap v433: 97 pgs: 97 active+clean; 464 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 245 op/s 2024-09-19T04:34:09.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:08 smithi107 ceph-mon[30659]: pgmap v433: 97 pgs: 97 active+clean; 464 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 245 op/s 2024-09-19T04:34:09.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:08 smithi115 ceph-mon[35477]: pgmap v433: 97 pgs: 97 active+clean; 464 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 245 op/s 2024-09-19T04:34:10.722 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T04:34:11.106 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:11.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:10 smithi107 ceph-mon[30659]: pgmap v434: 97 pgs: 97 active+clean; 464 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.5 MiB/s wr, 107 op/s 2024-09-19T04:34:11.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:10 smithi115 ceph-mon[35477]: pgmap v434: 97 pgs: 97 active+clean; 464 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.5 MiB/s wr, 107 op/s 2024-09-19T04:34:11.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:10 smithi145 ceph-mon[35096]: pgmap v434: 97 pgs: 97 active+clean; 464 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.5 MiB/s wr, 107 op/s 2024-09-19T04:34:11.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.647+0000 7feb83a00640 1 -- 172.21.15.107:0/2301792270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7c101de0 msgr2=0x7feb7c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:11.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.647+0000 7feb83a00640 1 --2- 172.21.15.107:0/2301792270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7c101de0 0x7feb7c1021c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feb70009900 tx=0x7feb7002a410 comp rx=0 tx=0).stop 2024-09-19T04:34:11.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.647+0000 7feb83a00640 1 -- 172.21.15.107:0/2301792270 shutdown_connections 2024-09-19T04:34:11.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.647+0000 7feb83a00640 1 --2- 172.21.15.107:0/2301792270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb7c106f10 0x7feb7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.647+0000 7feb83a00640 1 --2- 172.21.15.107:0/2301792270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb7c102790 0x7feb7c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.647+0000 7feb83a00640 1 --2- 172.21.15.107:0/2301792270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7c101de0 0x7feb7c1021c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.647+0000 7feb83a00640 1 -- 172.21.15.107:0/2301792270 >> 172.21.15.107:0/2301792270 conn(0x7feb7c0fd8c0 msgr2=0x7feb7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:11.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.647+0000 7feb83a00640 1 -- 172.21.15.107:0/2301792270 shutdown_connections 2024-09-19T04:34:11.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.647+0000 7feb83a00640 1 -- 172.21.15.107:0/2301792270 wait complete. 2024-09-19T04:34:11.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.648+0000 7feb83a00640 1 Processor -- start 2024-09-19T04:34:11.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.648+0000 7feb83a00640 1 -- start start 2024-09-19T04:34:11.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.648+0000 7feb83a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb7c101de0 0x7feb7c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.648+0000 7feb83a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb7c102790 0x7feb7c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.648+0000 7feb83a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7c106f10 0x7feb7c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.648+0000 7feb83a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c111fc0 con 0x7feb7c106f10 2024-09-19T04:34:11.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.648+0000 7feb83a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c111e40 con 0x7feb7c101de0 2024-09-19T04:34:11.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.648+0000 7feb83a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c112140 con 0x7feb7c102790 2024-09-19T04:34:11.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb81000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb7c101de0 0x7feb7c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb7be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb7c102790 0x7feb7c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb81000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb7c101de0 0x7feb7c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:39032/0 (socket says 172.21.15.107:39032) 2024-09-19T04:34:11.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb81a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7c106f10 0x7feb7c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb81000640 1 -- 172.21.15.107:0/474405781 learned_addr learned my addr 172.21.15.107:0/474405781 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:11.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb81000640 1 -- 172.21.15.107:0/474405781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb7c102790 msgr2=0x7feb7c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:11.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb81000640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb7c102790 0x7feb7c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb81000640 1 -- 172.21.15.107:0/474405781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7c106f10 msgr2=0x7feb7c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:11.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb81000640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7c106f10 0x7feb7c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb81000640 1 -- 172.21.15.107:0/474405781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c1a6910 con 0x7feb7c101de0 2024-09-19T04:34:11.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.649+0000 7feb81000640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb7c101de0 0x7feb7c0744f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feb70004770 tx=0x7feb70003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:11.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.650+0000 7feb79600640 1 -- 172.21.15.107:0/474405781 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7003f070 con 0x7feb7c101de0 2024-09-19T04:34:11.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.650+0000 7feb79600640 1 -- 172.21.15.107:0/474405781 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb70031ca0 con 0x7feb7c101de0 2024-09-19T04:34:11.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.650+0000 7feb79600640 1 -- 172.21.15.107:0/474405781 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70035030 con 0x7feb7c101de0 2024-09-19T04:34:11.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.650+0000 7feb83a00640 1 -- 172.21.15.107:0/474405781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c1a6ba0 con 0x7feb7c101de0 2024-09-19T04:34:11.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.650+0000 7feb83a00640 1 -- 172.21.15.107:0/474405781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c1a7030 con 0x7feb7c101de0 2024-09-19T04:34:11.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.654+0000 7feb863bb740 1 Processor -- start 2024-09-19T04:34:11.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.654+0000 7feb863bb740 1 -- start start 2024-09-19T04:34:11.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.654+0000 7feb863bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c25df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.654+0000 7feb863bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25e4b0 0x55b98c268830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.654+0000 7feb863bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c26b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.654+0000 7feb863bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b98c1c43e0 con 0x55b98c25db90 2024-09-19T04:34:11.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.654+0000 7feb863bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b98c16bd80 con 0x55b98c25e4b0 2024-09-19T04:34:11.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.654+0000 7feb863bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b98c1c03a0 con 0x55b98c268d70 2024-09-19T04:34:11.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb79600640 1 -- 172.21.15.107:0/474405781 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7feb70042070 con 0x7feb7c101de0 2024-09-19T04:34:11.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb81a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c25df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb81a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c25df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54788/0 (socket says 172.21.15.107:54788) 2024-09-19T04:34:11.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb7be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c26b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb81a00640 1 -- 172.21.15.107:0/1362311881 learned_addr learned my addr 172.21.15.107:0/1362311881 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:11.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb81000640 1 --2- 172.21.15.107:0/1362311881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25e4b0 0x55b98c268830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb81a00640 1 -- 172.21.15.107:0/1362311881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 msgr2=0x55b98c26b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:11.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb81a00640 1 --2- 172.21.15.107:0/1362311881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c26b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb81a00640 1 -- 172.21.15.107:0/1362311881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25e4b0 msgr2=0x55b98c268830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:11.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb81a00640 1 --2- 172.21.15.107:0/1362311881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25e4b0 0x55b98c268830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.655+0000 7feb81a00640 1 -- 172.21.15.107:0/1362311881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98c18aae0 con 0x55b98c25db90 2024-09-19T04:34:11.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.656+0000 7feb81a00640 1 --2- 172.21.15.107:0/1362311881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c25df70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7feb74009f50 tx=0x7feb740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765b02d348c2cd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:11.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.656+0000 7feb6e000640 1 -- 172.21.15.107:0/1362311881 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74004d20 con 0x55b98c25db90 2024-09-19T04:34:11.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.656+0000 7feb6e000640 1 -- 172.21.15.107:0/1362311881 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb74004050 con 0x55b98c25db90 2024-09-19T04:34:11.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.656+0000 7feb6e000640 1 -- 172.21.15.107:0/1362311881 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74005280 con 0x55b98c25db90 2024-09-19T04:34:11.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.656+0000 7feb79600640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feb640a3430 0x7feb640a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb863bb740 1 -- 172.21.15.107:0/1362311881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 msgr2=0x55b98c25df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:11.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb863bb740 1 --2- 172.21.15.107:0/1362311881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c25df70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7feb74009f50 tx=0x7feb740118d0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb79600640 1 -- 172.21.15.107:0/474405781 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7feb700f0210 con 0x7feb7c101de0 2024-09-19T04:34:11.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb863bb740 1 -- 172.21.15.107:0/1362311881 shutdown_connections 2024-09-19T04:34:11.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb863bb740 1 --2- 172.21.15.107:0/1362311881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c26b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb863bb740 1 --2- 172.21.15.107:0/1362311881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25e4b0 0x55b98c268830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb863bb740 1 --2- 172.21.15.107:0/1362311881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c25df70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb863bb740 1 -- 172.21.15.107:0/1362311881 >> 172.21.15.107:0/1362311881 conn(0x55b98c25a120 msgr2=0x55b98c25a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:11.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb81000640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feb640a3430 0x7feb640a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb863bb740 1 -- 172.21.15.107:0/1362311881 shutdown_connections 2024-09-19T04:34:11.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.657+0000 7feb863bb740 1 -- 172.21.15.107:0/1362311881 wait complete. 2024-09-19T04:34:11.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.658+0000 7feb81000640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feb640a3430 0x7feb640a58f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feb700eda30 tx=0x7feb70009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:11.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.658+0000 7feb863bb740 1 -- start start 2024-09-19T04:34:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.658+0000 7feb863bb740 1 Processor -- start 2024-09-19T04:34:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.658+0000 7feb863bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c265c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.658+0000 7feb863bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25e4b0 0x55b98c2661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.658+0000 7feb863bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c266710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.658+0000 7feb863bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b98c26b9e0 con 0x55b98c25db90 2024-09-19T04:34:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.658+0000 7feb863bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b98c17f0f0 con 0x55b98c25e4b0 2024-09-19T04:34:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.658+0000 7feb863bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b98c26bb60 con 0x55b98c268d70 2024-09-19T04:34:11.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.658+0000 7feb863bb740 10 client.0 _ll_register_callbacks cb 0x55b98c1709e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb81a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c265c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb81a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c265c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54792/0 (socket says 172.21.15.107:54792) 2024-09-19T04:34:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb81a00640 1 -- 172.21.15.107:0/4188146267 learned_addr learned my addr 172.21.15.107:0/4188146267 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb7be00640 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25e4b0 0x55b98c2661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb81a00640 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c266710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb81a00640 1 -- 172.21.15.107:0/4188146267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 msgr2=0x55b98c266710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb81a00640 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c266710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb81a00640 1 -- 172.21.15.107:0/4188146267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25e4b0 msgr2=0x55b98c2661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb81a00640 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25e4b0 0x55b98c2661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb81a00640 1 -- 172.21.15.107:0/4188146267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98bf353c0 con 0x55b98c25db90 2024-09-19T04:34:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb81a00640 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c265c90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feb74005c40 tx=0x7feb74004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72b25eb66e372f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.659+0000 7feb62a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:34:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.660+0000 7feb62a00640 1 -- 172.21.15.107:0/4188146267 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7402b070 con 0x55b98c25db90 2024-09-19T04:34:11.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.660+0000 7feb62a00640 1 -- 172.21.15.107:0/4188146267 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7feb74025450 con 0x55b98c25db90 2024-09-19T04:34:11.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.660+0000 7feb863bb740 1 -- 172.21.15.107:0/4188146267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb74004050 con 0x55b98c25db90 2024-09-19T04:34:11.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.660+0000 7feb62a00640 1 -- 172.21.15.107:0/4188146267 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7feb74025650 con 0x55b98c25db90 2024-09-19T04:34:11.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.660+0000 7feb863bb740 10 client.17649 fetch_fsmap learned FSMap version 55 2024-09-19T04:34:11.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.660+0000 7feb863bb740 1 -- 172.21.15.107:0/4188146267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b98c2776c0 con 0x55b98c25db90 2024-09-19T04:34:11.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.660+0000 7feb62a00640 1 -- 172.21.15.107:0/4188146267 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb74004050 con 0x55b98c25db90 2024-09-19T04:34:11.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.660+0000 7feb62a00640 1 -- 172.21.15.107:0/4188146267 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7feb7400f620 con 0x55b98c25db90 2024-09-19T04:34:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.661+0000 7feb863bb740 10 client.17649 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:34:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.661+0000 7feb863bb740 10 client.17649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:34:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.661+0000 7feb863bb740 20 client.17649 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.661+0000 7feb863bb740 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b98c278b20 0x55b98c27af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.661+0000 7feb863bb740 4 client.17649 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:34:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.661+0000 7feb863bb740 1 -- 172.21.15.107:0/4188146267 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b98c1d1950 con 0x55b98c278b20 2024-09-19T04:34:11.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.661+0000 7feb7be00640 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b98c278b20 0x55b98c27af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.662+0000 7feb7be00640 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b98c278b20 0x55b98c27af10 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91bbf9d0d12e8a72 server_cookie=17421dde8e65d753 in_seq=0 out_seq=0 2024-09-19T04:34:11.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.662+0000 7feb62a00640 10 client.17649 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:34:11.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.662+0000 7feb62a00640 1 -- 172.21.15.107:0/4188146267 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b98c1d1950 con 0x55b98c278b20 2024-09-19T04:34:11.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.662+0000 7feb62a00640 10 client.17649 handle_command_reply: tid=0 2024-09-19T04:34:11.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.662+0000 7feb863bb740 1 client.17649 shutdown 2024-09-19T04:34:11.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 1 -- 172.21.15.107:0/4188146267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 msgr2=0x55b98c265c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:11.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c265c90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feb74005c40 tx=0x7feb74004870 comp rx=0 tx=0).stop 2024-09-19T04:34:11.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 1 -- 172.21.15.107:0/4188146267 shutdown_connections 2024-09-19T04:34:11.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b98c278b20 0x55b98c27af10 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c266710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25e4b0 0x55b98c2661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb62a00640 0 client.17649 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:34:11.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 1 --2- 172.21.15.107:0/4188146267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25db90 0x55b98c265c90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 1 -- 172.21.15.107:0/4188146267 >> 172.21.15.107:0/4188146267 conn(0x55b98c25a120 msgr2=0x55b98c25fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:11.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 1 -- 172.21.15.107:0/4188146267 shutdown_connections 2024-09-19T04:34:11.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 1 -- 172.21.15.107:0/4188146267 wait complete. 2024-09-19T04:34:11.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.663+0000 7feb863bb740 20 client.17649 trim_cache size 0 max 16384 2024-09-19T04:34:11.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.681+0000 7feb863bb740 1 Processor -- start 2024-09-19T04:34:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb863bb740 1 -- start start 2024-09-19T04:34:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb863bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25db90 0x55b98c2605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb863bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c25e4b0 0x55b98c2806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb863bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c268d70 0x55b98c280be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb863bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c1afa10 con 0x55b98c268d70 2024-09-19T04:34:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb863bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c1b16d0 con 0x55b98c25db90 2024-09-19T04:34:11.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb863bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c1b01a0 con 0x55b98c25e4b0 2024-09-19T04:34:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb81a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c268d70 0x55b98c280be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb81a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c268d70 0x55b98c280be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54798/0 (socket says 172.21.15.107:54798) 2024-09-19T04:34:11.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb81a00640 1 -- 172.21.15.107:0/305760495 learned_addr learned my addr 172.21.15.107:0/305760495 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb7be00640 1 --2- 172.21.15.107:0/305760495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c25e4b0 0x55b98c2806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.682+0000 7feb81a00640 1 --2- 172.21.15.107:0/305760495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25db90 0x55b98c2605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.683+0000 7feb81a00640 1 -- 172.21.15.107:0/305760495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c25e4b0 msgr2=0x55b98c2806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.683+0000 7feb81a00640 1 --2- 172.21.15.107:0/305760495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c25e4b0 0x55b98c2806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.683+0000 7feb81a00640 1 -- 172.21.15.107:0/305760495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25db90 msgr2=0x55b98c2605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.683+0000 7feb81a00640 1 --2- 172.21.15.107:0/305760495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25db90 0x55b98c2605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.683+0000 7feb81a00640 1 -- 172.21.15.107:0/305760495 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98c268600 con 0x55b98c268d70 2024-09-19T04:34:11.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.683+0000 7feb7be00640 1 --2- 172.21.15.107:0/305760495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c25e4b0 0x55b98c2806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:11.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.683+0000 7feb81a00640 1 --2- 172.21.15.107:0/305760495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c268d70 0x55b98c280be0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feb74005d70 tx=0x7feb740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0a094a9daa2a39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:11.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.683+0000 7feb6d600640 1 -- 172.21.15.107:0/305760495 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7402b040 con 0x55b98c268d70 2024-09-19T04:34:11.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.683+0000 7feb6d600640 1 -- 172.21.15.107:0/305760495 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb74003d40 con 0x55b98c268d70 2024-09-19T04:34:11.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.683+0000 7feb6d600640 1 -- 172.21.15.107:0/305760495 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74025b20 con 0x55b98c268d70 2024-09-19T04:34:11.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.684+0000 7feb863bb740 1 -- 172.21.15.107:0/305760495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c268d70 msgr2=0x55b98c280be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:11.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.684+0000 7feb863bb740 1 --2- 172.21.15.107:0/305760495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c268d70 0x55b98c280be0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feb74005d70 tx=0x7feb740118d0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.684+0000 7feb863bb740 1 -- 172.21.15.107:0/305760495 shutdown_connections 2024-09-19T04:34:11.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.684+0000 7feb863bb740 1 --2- 172.21.15.107:0/305760495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c268d70 0x55b98c280be0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.684+0000 7feb863bb740 1 --2- 172.21.15.107:0/305760495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c25e4b0 0x55b98c2806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.684+0000 7feb863bb740 1 --2- 172.21.15.107:0/305760495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25db90 0x55b98c2605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.684+0000 7feb863bb740 1 -- 172.21.15.107:0/305760495 >> 172.21.15.107:0/305760495 conn(0x55b98c25a120 msgr2=0x55b98c267d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:11.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.684+0000 7feb863bb740 1 -- 172.21.15.107:0/305760495 shutdown_connections 2024-09-19T04:34:11.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.685+0000 7feb863bb740 1 -- 172.21.15.107:0/305760495 wait complete. 2024-09-19T04:34:11.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.685+0000 7feb863bb740 1 -- start start 2024-09-19T04:34:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.685+0000 7feb863bb740 1 Processor -- start 2024-09-19T04:34:11.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb863bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25db90 0x55b98c271490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb863bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25e4b0 0x55b98c26c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb863bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c26cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb863bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c1b0fd0 con 0x55b98c25e4b0 2024-09-19T04:34:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb863bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c1b08a0 con 0x55b98c25db90 2024-09-19T04:34:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb863bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c1ada10 con 0x55b98c268d70 2024-09-19T04:34:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb863bb740 10 client.0 _ll_register_callbacks cb 0x7feb74004290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:11.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb7be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25e4b0 0x55b98c26c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb7be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25e4b0 0x55b98c26c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54814/0 (socket says 172.21.15.107:54814) 2024-09-19T04:34:11.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb81a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25db90 0x55b98c271490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb7be00640 1 -- 172.21.15.107:0/3365302713 learned_addr learned my addr 172.21.15.107:0/3365302713 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:11.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb81a00640 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c26cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb7be00640 1 -- 172.21.15.107:0/3365302713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 msgr2=0x55b98c26cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:11.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb7be00640 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c26cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb7be00640 1 -- 172.21.15.107:0/3365302713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25db90 msgr2=0x55b98c271490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:11.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb7be00640 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25db90 0x55b98c271490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.686+0000 7feb7be00640 1 -- 172.21.15.107:0/3365302713 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98c26d530 con 0x55b98c25e4b0 2024-09-19T04:34:11.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.687+0000 7feb7be00640 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25e4b0 0x55b98c26c7c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feb68004700 tx=0x7feb68010a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809931d690cb7af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:11.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.687+0000 7feb62a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:34:11.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.687+0000 7feb62a00640 1 -- 172.21.15.107:0/3365302713 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6801f990 con 0x55b98c25e4b0 2024-09-19T04:34:11.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.687+0000 7feb62a00640 1 -- 172.21.15.107:0/3365302713 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7feb6801fb30 con 0x55b98c25e4b0 2024-09-19T04:34:11.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.687+0000 7feb863bb740 1 -- 172.21.15.107:0/3365302713 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb74003d40 con 0x55b98c25e4b0 2024-09-19T04:34:11.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.687+0000 7feb62a00640 1 -- 172.21.15.107:0/3365302713 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7feb6801fd30 con 0x55b98c25e4b0 2024-09-19T04:34:11.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.687+0000 7feb863bb740 10 client.17655 fetch_fsmap learned FSMap version 55 2024-09-19T04:34:11.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.687+0000 7feb863bb740 1 -- 172.21.15.107:0/3365302713 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b98c2e36a0 con 0x55b98c25e4b0 2024-09-19T04:34:11.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.688+0000 7feb62a00640 1 -- 172.21.15.107:0/3365302713 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb74003d40 con 0x55b98c25e4b0 2024-09-19T04:34:11.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.688+0000 7feb62a00640 1 -- 172.21.15.107:0/3365302713 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7feb6802d070 con 0x55b98c25e4b0 2024-09-19T04:34:11.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.688+0000 7feb863bb740 10 client.17655 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:34:11.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.688+0000 7feb863bb740 10 client.17655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:34:11.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.688+0000 7feb863bb740 20 client.17655 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:11.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.688+0000 7feb863bb740 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b98c2e3ed0 0x55b98c2e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:11.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.688+0000 7feb863bb740 4 client.17655 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:34:11.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.688+0000 7feb863bb740 1 -- 172.21.15.107:0/3365302713 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b98c1cbe60 con 0x55b98c2e3ed0 2024-09-19T04:34:11.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.688+0000 7feb81a00640 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b98c2e3ed0 0x55b98c2e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:11.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb81a00640 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b98c2e3ed0 0x55b98c2e62f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6808e666a0b5bb4 server_cookie=2f191c5bf6c5fa34 in_seq=0 out_seq=0 2024-09-19T04:34:11.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb62a00640 10 client.17655 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:34:11.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb62a00640 1 -- 172.21.15.107:0/3365302713 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b98c1cbe60 con 0x55b98c2e3ed0 2024-09-19T04:34:11.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb62a00640 10 client.17655 handle_command_reply: tid=0 2024-09-19T04:34:11.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 client.17655 shutdown 2024-09-19T04:34:11.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 -- 172.21.15.107:0/3365302713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25e4b0 msgr2=0x55b98c26c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:11.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25e4b0 0x55b98c26c7c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feb68004700 tx=0x7feb68010a20 comp rx=0 tx=0).stop 2024-09-19T04:34:11.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 -- 172.21.15.107:0/3365302713 shutdown_connections 2024-09-19T04:34:11.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55b98c2e3ed0 0x55b98c2e62f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b98c268d70 0x55b98c26cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b98c25e4b0 0x55b98c26c7c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 --2- 172.21.15.107:0/3365302713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b98c25db90 0x55b98c271490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 -- 172.21.15.107:0/3365302713 >> 172.21.15.107:0/3365302713 conn(0x55b98c25a120 msgr2=0x55b98c2589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb62a00640 0 client.17655 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:34:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 -- 172.21.15.107:0/3365302713 shutdown_connections 2024-09-19T04:34:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.689+0000 7feb863bb740 1 -- 172.21.15.107:0/3365302713 wait complete. 2024-09-19T04:34:11.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb863bb740 20 client.17655 trim_cache size 0 max 16384 2024-09-19T04:34:11.699 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:34:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-09-19T04:34:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-09-19T04:34:11.700 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:34:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb83a00640 1 -- 172.21.15.107:0/474405781 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feb640a3430 msgr2=0x7feb640a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb83a00640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feb640a3430 0x7feb640a58f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feb700eda30 tx=0x7feb70009260 comp rx=0 tx=0).stop 2024-09-19T04:34:11.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb83a00640 1 -- 172.21.15.107:0/474405781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb7c101de0 msgr2=0x7feb7c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb83a00640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb7c101de0 0x7feb7c0744f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feb70004770 tx=0x7feb70003c10 comp rx=0 tx=0).stop 2024-09-19T04:34:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb83a00640 1 -- 172.21.15.107:0/474405781 shutdown_connections 2024-09-19T04:34:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb83a00640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7feb640a3430 0x7feb640a58f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb83a00640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb7c106f10 0x7feb7c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb83a00640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb7c102790 0x7feb7c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb83a00640 1 --2- 172.21.15.107:0/474405781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb7c101de0 0x7feb7c0744f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.690+0000 7feb83a00640 1 -- 172.21.15.107:0/474405781 >> 172.21.15.107:0/474405781 conn(0x7feb7c0fd8c0 msgr2=0x7feb7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:11.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.691+0000 7feb83a00640 1 -- 172.21.15.107:0/474405781 shutdown_connections 2024-09-19T04:34:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:11.691+0000 7feb83a00640 1 -- 172.21.15.107:0/474405781 wait complete. 2024-09-19T04:34:12.615 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:34:12.616 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:34:12.976 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:12 smithi107 ceph-mon[30659]: pgmap v435: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 2.2 MiB/s wr, 110 op/s 2024-09-19T04:34:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:13.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:12 smithi115 ceph-mon[35477]: pgmap v435: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 2.2 MiB/s wr, 110 op/s 2024-09-19T04:34:13.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:12 smithi145 ceph-mon[35096]: pgmap v435: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 2.2 MiB/s wr, 110 op/s 2024-09-19T04:34:13.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.554+0000 7f87de000640 1 -- 172.21.15.107:0/3764104259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87d81129e0 msgr2=0x7f87d8114dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:13.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.554+0000 7f87de000640 1 --2- 172.21.15.107:0/3764104259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87d81129e0 0x7f87d8114dd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009ca0 tx=0x7f87d002a950 comp rx=0 tx=0).stop 2024-09-19T04:34:13.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.554+0000 7f87de000640 1 -- 172.21.15.107:0/3764104259 shutdown_connections 2024-09-19T04:34:13.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.554+0000 7f87de000640 1 --2- 172.21.15.107:0/3764104259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87d81129e0 0x7f87d8114dd0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:13.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.554+0000 7f87de000640 1 --2- 172.21.15.107:0/3764104259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87d8074eb0 0x7f87d81124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:13.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.554+0000 7f87de000640 1 --2- 172.21.15.107:0/3764104259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87d8074590 0x7f87d8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:13.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.554+0000 7f87de000640 1 -- 172.21.15.107:0/3764104259 >> 172.21.15.107:0/3764104259 conn(0x7f87d80fd710 msgr2=0x7f87d80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:13.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.554+0000 7f87de000640 1 -- 172.21.15.107:0/3764104259 shutdown_connections 2024-09-19T04:34:13.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.554+0000 7f87de000640 1 -- 172.21.15.107:0/3764104259 wait complete. 2024-09-19T04:34:13.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.555+0000 7f87de000640 1 Processor -- start 2024-09-19T04:34:13.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.555+0000 7f87de000640 1 -- start start 2024-09-19T04:34:13.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87de000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87d8074590 0x7f87d81a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:13.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87de000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87d8074eb0 0x7f87d81a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:13.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87de000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87d81129e0 0x7f87d819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:13.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87de000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81183f0 con 0x7f87d81129e0 2024-09-19T04:34:13.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87de000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8118270 con 0x7f87d8074eb0 2024-09-19T04:34:13.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87de000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8118570 con 0x7f87d8074590 2024-09-19T04:34:13.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87dd600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87d81129e0 0x7f87d819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:13.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87dd600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87d81129e0 0x7f87d819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54818/0 (socket says 172.21.15.107:54818) 2024-09-19T04:34:13.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87dd600640 1 -- 172.21.15.107:0/280409471 learned_addr learned my addr 172.21.15.107:0/280409471 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:13.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87dcc00640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87d8074590 0x7f87d81a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:13.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.556+0000 7f87d7e00640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87d8074eb0 0x7f87d81a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:13.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87dd600640 1 -- 172.21.15.107:0/280409471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87d8074590 msgr2=0x7f87d81a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:13.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87dd600640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87d8074590 0x7f87d81a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:13.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87dd600640 1 -- 172.21.15.107:0/280409471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87d8074eb0 msgr2=0x7f87d81a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:13.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87dd600640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87d8074eb0 0x7f87d81a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:13.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87dd600640 1 -- 172.21.15.107:0/280409471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d819ef50 con 0x7f87d81129e0 2024-09-19T04:34:13.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87d7e00640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87d8074eb0 0x7f87d81a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:13.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87dd600640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87d81129e0 0x7f87d819e6e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f87d0005e30 tx=0x7f87d0003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:13.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87d5600640 1 -- 172.21.15.107:0/280409471 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0034040 con 0x7f87d81129e0 2024-09-19T04:34:13.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87d5600640 1 -- 172.21.15.107:0/280409471 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d002f940 con 0x7f87d81129e0 2024-09-19T04:34:13.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87de000640 1 -- 172.21.15.107:0/280409471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d819f1e0 con 0x7f87d81129e0 2024-09-19T04:34:13.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87d5600640 1 -- 172.21.15.107:0/280409471 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d00045f0 con 0x7f87d81129e0 2024-09-19T04:34:13.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.557+0000 7f87de000640 1 -- 172.21.15.107:0/280409471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d81ab3c0 con 0x7f87d81129e0 2024-09-19T04:34:13.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.558+0000 7f87de000640 1 -- 172.21.15.107:0/280409471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d8075cd0 con 0x7f87d81129e0 2024-09-19T04:34:13.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.560+0000 7f87d5600640 1 -- 172.21.15.107:0/280409471 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f87d0037020 con 0x7f87d81129e0 2024-09-19T04:34:13.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.561+0000 7f87d5600640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f87c00a3430 0x7f87c00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:13.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.562+0000 7f87d5600640 1 -- 172.21.15.107:0/280409471 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f87d00f0930 con 0x7f87d81129e0 2024-09-19T04:34:13.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.562+0000 7f87dcc00640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f87c00a3430 0x7f87c00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:13.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.562+0000 7f87dcc00640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f87c00a3430 0x7f87c00a58f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009bd0 tx=0x7f87cc005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:13.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.566+0000 7f87d5600640 1 -- 172.21.15.107:0/280409471 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f87d00bd3e0 con 0x7f87d81129e0 2024-09-19T04:34:13.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.777+0000 7f87de000640 1 -- 172.21.15.107:0/280409471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87d81ab590 con 0x7f87d81129e0 2024-09-19T04:34:13.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.778+0000 7f87d5600640 1 -- 172.21.15.107:0/280409471 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+12009 (secure 0 0 0) 0x7f87d00bcce0 con 0x7f87d81129e0 2024-09-19T04:34:13.780 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:34:13.780 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":55,"btime":"2024-09-19T04:34:01:946200+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:34:01.717351+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:34:13.781 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 55 2024-09-19T04:34:13.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.780+0000 7f87de000640 1 -- 172.21.15.107:0/280409471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f87c00a3430 msgr2=0x7f87c00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:13.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.780+0000 7f87de000640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f87c00a3430 0x7f87c00a58f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009bd0 tx=0x7f87cc005f90 comp rx=0 tx=0).stop 2024-09-19T04:34:13.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.781+0000 7f87de000640 1 -- 172.21.15.107:0/280409471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87d81129e0 msgr2=0x7f87d819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:13.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.781+0000 7f87de000640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87d81129e0 0x7f87d819e6e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f87d0005e30 tx=0x7f87d0003b80 comp rx=0 tx=0).stop 2024-09-19T04:34:13.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.781+0000 7f87de000640 1 -- 172.21.15.107:0/280409471 shutdown_connections 2024-09-19T04:34:13.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.781+0000 7f87de000640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f87c00a3430 0x7f87c00a58f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:13.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.781+0000 7f87de000640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87d81129e0 0x7f87d819e6e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:13.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.781+0000 7f87de000640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87d8074eb0 0x7f87d81a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:13.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.781+0000 7f87de000640 1 --2- 172.21.15.107:0/280409471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87d8074590 0x7f87d81a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:13.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.781+0000 7f87de000640 1 -- 172.21.15.107:0/280409471 >> 172.21.15.107:0/280409471 conn(0x7f87d80fd710 msgr2=0x7f87d80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:13.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.781+0000 7f87de000640 1 -- 172.21.15.107:0/280409471 shutdown_connections 2024-09-19T04:34:13.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:13.781+0000 7f87de000640 1 -- 172.21.15.107:0/280409471 wait complete. 2024-09-19T04:34:14.312 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:13 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/280409471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:34:14.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:13 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/280409471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:34:14.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:13 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/280409471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:34:14.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T04:34:15.002 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:15.348 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:14 smithi107 ceph-mon[30659]: pgmap v436: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T04:34:15.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:14 smithi115 ceph-mon[35477]: pgmap v436: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T04:34:15.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:14 smithi145 ceph-mon[35096]: pgmap v436: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-09-19T04:34:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.624+0000 7fa261c00640 1 -- 172.21.15.107:0/1737673783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c10e260 msgr2=0x7fa25c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.624+0000 7fa261c00640 1 --2- 172.21.15.107:0/1737673783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c10e260 0x7fa25c110650 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa254009be0 tx=0x7fa25402a750 comp rx=0 tx=0).stop 2024-09-19T04:34:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.624+0000 7fa261c00640 1 -- 172.21.15.107:0/1737673783 shutdown_connections 2024-09-19T04:34:15.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.624+0000 7fa261c00640 1 --2- 172.21.15.107:0/1737673783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c10e260 0x7fa25c110650 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.624+0000 7fa261c00640 1 --2- 172.21.15.107:0/1737673783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa25c074ee0 0x7fa25c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.624+0000 7fa261c00640 1 --2- 172.21.15.107:0/1737673783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c0745c0 0x7fa25c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.624+0000 7fa261c00640 1 -- 172.21.15.107:0/1737673783 >> 172.21.15.107:0/1737673783 conn(0x7fa25c0fd740 msgr2=0x7fa25c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:15.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.624+0000 7fa261c00640 1 -- 172.21.15.107:0/1737673783 shutdown_connections 2024-09-19T04:34:15.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.624+0000 7fa261c00640 1 -- 172.21.15.107:0/1737673783 wait complete. 2024-09-19T04:34:15.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.625+0000 7fa261c00640 1 Processor -- start 2024-09-19T04:34:15.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.625+0000 7fa261c00640 1 -- start start 2024-09-19T04:34:15.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa261c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa25c0745c0 0x7fa25c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa261c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c074ee0 0x7fa25c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa261c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c10e260 0x7fa25c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa261c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c113f20 con 0x7fa25c074ee0 2024-09-19T04:34:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa261c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c113da0 con 0x7fa25c10e260 2024-09-19T04:34:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa261c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c1140a0 con 0x7fa25c0745c0 2024-09-19T04:34:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa253e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c074ee0 0x7fa25c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa25be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa25c0745c0 0x7fa25c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa253e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c074ee0 0x7fa25c19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54836/0 (socket says 172.21.15.107:54836) 2024-09-19T04:34:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa253e00640 1 -- 172.21.15.107:0/252380946 learned_addr learned my addr 172.21.15.107:0/252380946 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa261200640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c10e260 0x7fa25c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa25be00640 1 -- 172.21.15.107:0/252380946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c10e260 msgr2=0x7fa25c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa25be00640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c10e260 0x7fa25c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa25be00640 1 -- 172.21.15.107:0/252380946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c074ee0 msgr2=0x7fa25c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa25be00640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c074ee0 0x7fa25c19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa25be00640 1 -- 172.21.15.107:0/252380946 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c19f3d0 con 0x7fa25c0745c0 2024-09-19T04:34:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.626+0000 7fa253e00640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c074ee0 0x7fa25c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.627+0000 7fa25be00640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa25c0745c0 0x7fa25c19a420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa24c005fd0 tx=0x7fa24c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.627+0000 7fa259600640 1 -- 172.21.15.107:0/252380946 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00cd00 con 0x7fa25c0745c0 2024-09-19T04:34:15.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.627+0000 7fa261c00640 1 -- 172.21.15.107:0/252380946 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25c19f6c0 con 0x7fa25c0745c0 2024-09-19T04:34:15.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.627+0000 7fa259600640 1 -- 172.21.15.107:0/252380946 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24c0035f0 con 0x7fa25c0745c0 2024-09-19T04:34:15.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.627+0000 7fa259600640 1 -- 172.21.15.107:0/252380946 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c010670 con 0x7fa25c0745c0 2024-09-19T04:34:15.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.627+0000 7fa261c00640 1 -- 172.21.15.107:0/252380946 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25c1a6fa0 con 0x7fa25c0745c0 2024-09-19T04:34:15.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.631+0000 7fa265cab740 1 Processor -- start 2024-09-19T04:34:15.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.631+0000 7fa265cab740 1 -- start start 2024-09-19T04:34:15.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa265cab740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 0x55b423f61220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa265cab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f6bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa265cab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f6c020 0x55b423f6e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa265cab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b423ec73e0 con 0x55b423f6c020 2024-09-19T04:34:15.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa265cab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b423e6ed80 con 0x55b423f61760 2024-09-19T04:34:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa265cab740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b423ec33a0 con 0x55b423f60e40 2024-09-19T04:34:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa259600640 1 -- 172.21.15.107:0/252380946 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fa24c0037a0 con 0x7fa25c0745c0 2024-09-19T04:34:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa261200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 0x55b423f61220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa253e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f6c020 0x55b423f6e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa261200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 0x55b423f61220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55588/0 (socket says 172.21.15.107:55588) 2024-09-19T04:34:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa261200640 1 -- 172.21.15.107:0/1778006381 learned_addr learned my addr 172.21.15.107:0/1778006381 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa25be00640 1 --2- 172.21.15.107:0/1778006381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f6bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa253e00640 1 -- 172.21.15.107:0/1778006381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 msgr2=0x55b423f61220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:15.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa253e00640 1 --2- 172.21.15.107:0/1778006381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 0x55b423f61220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa253e00640 1 -- 172.21.15.107:0/1778006381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 msgr2=0x55b423f6bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:15.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.632+0000 7fa253e00640 1 --2- 172.21.15.107:0/1778006381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f6bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.633+0000 7fa253e00640 1 -- 172.21.15.107:0/1778006381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b423e8dae0 con 0x55b423f6c020 2024-09-19T04:34:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.633+0000 7fa253e00640 1 --2- 172.21.15.107:0/1778006381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f6c020 0x55b423f6e410 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa25c10f5f0 tx=0x7fa2440118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251da9e5e8843635 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.633+0000 7fa251600640 1 -- 172.21.15.107:0/1778006381 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24401e070 con 0x55b423f6c020 2024-09-19T04:34:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.633+0000 7fa251600640 1 -- 172.21.15.107:0/1778006381 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2440035f0 con 0x55b423f6c020 2024-09-19T04:34:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.633+0000 7fa251600640 1 -- 172.21.15.107:0/1778006381 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24400f800 con 0x55b423f6c020 2024-09-19T04:34:15.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa259600640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa2400a33b0 0x7fa2400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa259600640 1 -- 172.21.15.107:0/252380946 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fa24c0cff30 con 0x7fa25c0745c0 2024-09-19T04:34:15.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa265cab740 1 -- 172.21.15.107:0/1778006381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f6c020 msgr2=0x55b423f6e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:15.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa265cab740 1 --2- 172.21.15.107:0/1778006381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f6c020 0x55b423f6e410 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa25c10f5f0 tx=0x7fa2440118d0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa261200640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa2400a33b0 0x7fa2400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa265cab740 1 -- 172.21.15.107:0/1778006381 shutdown_connections 2024-09-19T04:34:15.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa265cab740 1 --2- 172.21.15.107:0/1778006381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f6c020 0x55b423f6e410 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa265cab740 1 --2- 172.21.15.107:0/1778006381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f6bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa265cab740 1 --2- 172.21.15.107:0/1778006381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 0x55b423f61220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa265cab740 1 -- 172.21.15.107:0/1778006381 >> 172.21.15.107:0/1778006381 conn(0x55b423f5d3d0 msgr2=0x55b423f5d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:15.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa265cab740 1 -- 172.21.15.107:0/1778006381 shutdown_connections 2024-09-19T04:34:15.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.634+0000 7fa265cab740 1 -- 172.21.15.107:0/1778006381 wait complete. 2024-09-19T04:34:15.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.635+0000 7fa261200640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa2400a33b0 0x7fa2400a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa254009be0 tx=0x7fa254009840 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:15.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.635+0000 7fa265cab740 1 -- start start 2024-09-19T04:34:15.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.635+0000 7fa265cab740 1 Processor -- start 2024-09-19T04:34:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.635+0000 7fa265cab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f68f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.635+0000 7fa265cab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.635+0000 7fa265cab740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f6c020 0x55b423f699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa265cab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b423f6ebf0 con 0x55b423f60e40 2024-09-19T04:34:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa265cab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b423e820f0 con 0x55b423f61760 2024-09-19T04:34:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa265cab740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b423f6ed70 con 0x55b423f6c020 2024-09-19T04:34:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa265cab740 10 client.0 _ll_register_callbacks cb 0x55b423e69bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa25be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f68f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa25be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f68f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54858/0 (socket says 172.21.15.107:54858) 2024-09-19T04:34:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa25be00640 1 -- 172.21.15.107:0/2380006928 learned_addr learned my addr 172.21.15.107:0/2380006928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa253e00640 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa261200640 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f6c020 0x55b423f699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa25be00640 1 -- 172.21.15.107:0/2380006928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f6c020 msgr2=0x55b423f699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa25be00640 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f6c020 0x55b423f699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa25be00640 1 -- 172.21.15.107:0/2380006928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 msgr2=0x55b423f694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa25be00640 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.636+0000 7fa25be00640 1 -- 172.21.15.107:0/2380006928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b423c383c0 con 0x55b423f60e40 2024-09-19T04:34:15.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.637+0000 7fa25be00640 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f68f70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa24c09c160 tx=0x7fa24c098530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3302d035af5c4796 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:15.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.637+0000 7fa24a000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:34:15.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.637+0000 7fa24a000640 1 -- 172.21.15.107:0/2380006928 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c0cdac0 con 0x55b423f60e40 2024-09-19T04:34:15.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.637+0000 7fa24a000640 1 -- 172.21.15.107:0/2380006928 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fa24c0a0120 con 0x55b423f60e40 2024-09-19T04:34:15.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.637+0000 7fa265cab740 1 -- 172.21.15.107:0/2380006928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2440035f0 con 0x55b423f60e40 2024-09-19T04:34:15.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.637+0000 7fa24a000640 1 -- 172.21.15.107:0/2380006928 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa24c009510 con 0x55b423f60e40 2024-09-19T04:34:15.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.637+0000 7fa265cab740 10 client.17673 fetch_fsmap learned FSMap version 55 2024-09-19T04:34:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.637+0000 7fa265cab740 1 -- 172.21.15.107:0/2380006928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b423f7ad30 con 0x55b423f60e40 2024-09-19T04:34:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.637+0000 7fa24a000640 1 -- 172.21.15.107:0/2380006928 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2440035f0 con 0x55b423f60e40 2024-09-19T04:34:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.637+0000 7fa24a000640 1 -- 172.21.15.107:0/2380006928 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7fa24c0a49b0 con 0x55b423f60e40 2024-09-19T04:34:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.638+0000 7fa265cab740 10 client.17673 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:34:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.638+0000 7fa265cab740 10 client.17673 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:34:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.638+0000 7fa265cab740 20 client.17673 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.638+0000 7fa265cab740 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b423f7c1f0 0x55b423f7e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.638+0000 7fa265cab740 4 client.17673 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:34:15.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.638+0000 7fa265cab740 1 -- 172.21.15.107:0/2380006928 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b423ed4950 con 0x55b423f7c1f0 2024-09-19T04:34:15.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.638+0000 7fa253e00640 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b423f7c1f0 0x55b423f7e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.639+0000 7fa253e00640 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b423f7c1f0 0x55b423f7e690 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb627f0d917f6fcd server_cookie=7272b7fd4951a56b in_seq=0 out_seq=0 2024-09-19T04:34:15.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.639+0000 7fa24a000640 10 client.17673 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:34:15.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.639+0000 7fa24a000640 1 -- 172.21.15.107:0/2380006928 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b423ed4950 con 0x55b423f7c1f0 2024-09-19T04:34:15.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.639+0000 7fa24a000640 10 client.17673 handle_command_reply: tid=0 2024-09-19T04:34:15.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.639+0000 7fa265cab740 1 client.17673 shutdown 2024-09-19T04:34:15.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 1 -- 172.21.15.107:0/2380006928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 msgr2=0x55b423f68f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:15.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f68f70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa24c09c160 tx=0x7fa24c098530 comp rx=0 tx=0).stop 2024-09-19T04:34:15.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 1 -- 172.21.15.107:0/2380006928 shutdown_connections 2024-09-19T04:34:15.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b423f7c1f0 0x55b423f7e690 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f6c020 0x55b423f699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 1 --2- 172.21.15.107:0/2380006928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f68f70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 1 -- 172.21.15.107:0/2380006928 >> 172.21.15.107:0/2380006928 conn(0x55b423f5d3d0 msgr2=0x55b423f630b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:15.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa24a000640 0 client.17673 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:34:15.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 1 -- 172.21.15.107:0/2380006928 shutdown_connections 2024-09-19T04:34:15.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 1 -- 172.21.15.107:0/2380006928 wait complete. 2024-09-19T04:34:15.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.640+0000 7fa265cab740 20 client.17673 trim_cache size 0 max 16384 2024-09-19T04:34:15.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.654+0000 7fa265cab740 1 Processor -- start 2024-09-19T04:34:15.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.654+0000 7fa265cab740 1 -- start start 2024-09-19T04:34:15.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.654+0000 7fa265cab740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 0x55b423f68890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.654+0000 7fa265cab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f61760 0x55b423f83f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.654+0000 7fa265cab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f6c020 0x55b423f844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.654+0000 7fa265cab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c1af8c0 con 0x55b423f61760 2024-09-19T04:34:15.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.654+0000 7fa265cab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c1b1580 con 0x55b423f6c020 2024-09-19T04:34:15.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.654+0000 7fa265cab740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c1b0050 con 0x55b423f60e40 2024-09-19T04:34:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.654+0000 7fa253e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f61760 0x55b423f83f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa253e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f61760 0x55b423f83f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54864/0 (socket says 172.21.15.107:54864) 2024-09-19T04:34:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa253e00640 1 -- 172.21.15.107:0/839741882 learned_addr learned my addr 172.21.15.107:0/839741882 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa25be00640 1 --2- 172.21.15.107:0/839741882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 0x55b423f68890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa261200640 1 --2- 172.21.15.107:0/839741882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f6c020 0x55b423f844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa253e00640 1 -- 172.21.15.107:0/839741882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 msgr2=0x55b423f68890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa253e00640 1 --2- 172.21.15.107:0/839741882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 0x55b423f68890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa253e00640 1 -- 172.21.15.107:0/839741882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f6c020 msgr2=0x55b423f844a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa253e00640 1 --2- 172.21.15.107:0/839741882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f6c020 0x55b423f844a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa253e00640 1 -- 172.21.15.107:0/839741882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b423f60c70 con 0x55b423f61760 2024-09-19T04:34:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa261200640 1 --2- 172.21.15.107:0/839741882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f6c020 0x55b423f844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:34:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa253e00640 1 --2- 172.21.15.107:0/839741882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f61760 0x55b423f83f60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa244009f40 tx=0x7fa244003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d298c1580b1b53f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.655+0000 7fa250c00640 1 -- 172.21.15.107:0/839741882 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24401e040 con 0x55b423f61760 2024-09-19T04:34:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa250c00640 1 -- 172.21.15.107:0/839741882 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24402a070 con 0x55b423f61760 2024-09-19T04:34:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa250c00640 1 -- 172.21.15.107:0/839741882 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244004280 con 0x55b423f61760 2024-09-19T04:34:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa265cab740 1 -- 172.21.15.107:0/839741882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f61760 msgr2=0x55b423f83f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa265cab740 1 --2- 172.21.15.107:0/839741882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f61760 0x55b423f83f60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa244009f40 tx=0x7fa244003c10 comp rx=0 tx=0).stop 2024-09-19T04:34:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa265cab740 1 -- 172.21.15.107:0/839741882 shutdown_connections 2024-09-19T04:34:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa265cab740 1 --2- 172.21.15.107:0/839741882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f6c020 0x55b423f844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa265cab740 1 --2- 172.21.15.107:0/839741882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f61760 0x55b423f83f60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa265cab740 1 --2- 172.21.15.107:0/839741882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f60e40 0x55b423f68890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa265cab740 1 -- 172.21.15.107:0/839741882 >> 172.21.15.107:0/839741882 conn(0x55b423f5d3d0 msgr2=0x55b423f6aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa265cab740 1 -- 172.21.15.107:0/839741882 shutdown_connections 2024-09-19T04:34:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.656+0000 7fa265cab740 1 -- 172.21.15.107:0/839741882 wait complete. 2024-09-19T04:34:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.657+0000 7fa265cab740 1 -- start start 2024-09-19T04:34:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.657+0000 7fa265cab740 1 Processor -- start 2024-09-19T04:34:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.657+0000 7fa265cab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.657+0000 7fa265cab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f74800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.657+0000 7fa265cab740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f6c020 0x55b423f65990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.657+0000 7fa265cab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c1b0e80 con 0x55b423f60e40 2024-09-19T04:34:15.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.657+0000 7fa265cab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c1b0750 con 0x55b423f61760 2024-09-19T04:34:15.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.657+0000 7fa265cab740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c1ad8c0 con 0x55b423f6c020 2024-09-19T04:34:15.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.657+0000 7fa265cab740 10 client.0 _ll_register_callbacks cb 0x7fa24c003dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:15.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa25be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa261200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f6c020 0x55b423f65990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa25be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54880/0 (socket says 172.21.15.107:54880) 2024-09-19T04:34:15.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa25be00640 1 -- 172.21.15.107:0/611524783 learned_addr learned my addr 172.21.15.107:0/611524783 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:15.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa253e00640 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f74800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa25be00640 1 -- 172.21.15.107:0/611524783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f6c020 msgr2=0x55b423f65990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:15.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa25be00640 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f6c020 0x55b423f65990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa25be00640 1 -- 172.21.15.107:0/611524783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 msgr2=0x55b423f74800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:15.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa25be00640 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f74800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa25be00640 1 -- 172.21.15.107:0/611524783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b423f5cc80 con 0x55b423f60e40 2024-09-19T04:34:15.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa25be00640 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f742c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009230 tx=0x7fa24c0cdaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f358cdcdc8ffb34c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:15.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.658+0000 7fa24a000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:34:15.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa24a000640 1 -- 172.21.15.107:0/611524783 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c09bd10 con 0x55b423f60e40 2024-09-19T04:34:15.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa24a000640 1 -- 172.21.15.107:0/611524783 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fa24c0d3250 con 0x55b423f60e40 2024-09-19T04:34:15.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa265cab740 1 -- 172.21.15.107:0/611524783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24402a070 con 0x55b423f60e40 2024-09-19T04:34:15.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa24a000640 1 -- 172.21.15.107:0/611524783 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa24c09c8d0 con 0x55b423f60e40 2024-09-19T04:34:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa265cab740 10 client.17679 fetch_fsmap learned FSMap version 55 2024-09-19T04:34:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa265cab740 1 -- 172.21.15.107:0/611524783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b423fe6910 con 0x55b423f60e40 2024-09-19T04:34:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa24a000640 1 -- 172.21.15.107:0/611524783 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24402a070 con 0x55b423f60e40 2024-09-19T04:34:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa24a000640 1 -- 172.21.15.107:0/611524783 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7fa24c09b670 con 0x55b423f60e40 2024-09-19T04:34:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa265cab740 10 client.17679 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:34:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa265cab740 10 client.17679 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T04:34:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa265cab740 20 client.17679 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa265cab740 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b423fe6f20 0x55b423fe9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa265cab740 4 client.17679 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:34:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.659+0000 7fa265cab740 1 -- 172.21.15.107:0/611524783 --> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b423ecee60 con 0x55b423fe6f20 2024-09-19T04:34:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.660+0000 7fa253e00640 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b423fe6f20 0x55b423fe9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.660+0000 7fa253e00640 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b423fe6f20 0x55b423fe9340 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3e4af4115c9044c1 server_cookie=b5583c8a927f26bd in_seq=0 out_seq=0 2024-09-19T04:34:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.660+0000 7fa24a000640 10 client.17679 ms_handle_connect on v2:172.21.15.145:6834/4237714429 2024-09-19T04:34:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.660+0000 7fa24a000640 1 -- 172.21.15.107:0/611524783 <== mds.4 v2:172.21.15.145:6834/4237714429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b423ecee60 con 0x55b423fe6f20 2024-09-19T04:34:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.660+0000 7fa24a000640 10 client.17679 handle_command_reply: tid=0 2024-09-19T04:34:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 client.17679 shutdown 2024-09-19T04:34:15.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 -- 172.21.15.107:0/611524783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 msgr2=0x55b423f742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:15.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f742c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009230 tx=0x7fa24c0cdaf0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 -- 172.21.15.107:0/611524783 shutdown_connections 2024-09-19T04:34:15.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.145:6834/4237714429,v1:172.21.15.145:6835/4237714429] conn(0x55b423fe6f20 0x55b423fe9340 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b423f6c020 0x55b423f65990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b423f61760 0x55b423f74800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 --2- 172.21.15.107:0/611524783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b423f60e40 0x55b423f742c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 -- 172.21.15.107:0/611524783 >> 172.21.15.107:0/611524783 conn(0x55b423f5d3d0 msgr2=0x55b423f68f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:15.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa24a000640 0 client.17679 ms_handle_reset on v2:172.21.15.145:6834/4237714429 2024-09-19T04:34:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 -- 172.21.15.107:0/611524783 shutdown_connections 2024-09-19T04:34:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 1 -- 172.21.15.107:0/611524783 wait complete. 2024-09-19T04:34:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.661+0000 7fa265cab740 20 client.17679 trim_cache size 0 max 16384 2024-09-19T04:34:15.671 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:34:15.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 -- 172.21.15.107:0/252380946 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa2400a33b0 msgr2=0x7fa2400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:15.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa2400a33b0 0x7fa2400a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa254009be0 tx=0x7fa254009840 comp rx=0 tx=0).stop 2024-09-19T04:34:15.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 -- 172.21.15.107:0/252380946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa25c0745c0 msgr2=0x7fa25c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa25c0745c0 0x7fa25c19a420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa24c005fd0 tx=0x7fa24c00ee50 comp rx=0 tx=0).stop 2024-09-19T04:34:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 -- 172.21.15.107:0/252380946 shutdown_connections 2024-09-19T04:34:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fa2400a33b0 0x7fa2400a5870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c10e260 0x7fa25c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c074ee0 0x7fa25c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 --2- 172.21.15.107:0/252380946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa25c0745c0 0x7fa25c19a420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 -- 172.21.15.107:0/252380946 >> 172.21.15.107:0/252380946 conn(0x7fa25c0fd740 msgr2=0x7fa25c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 -- 172.21.15.107:0/252380946 shutdown_connections 2024-09-19T04:34:15.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:15.662+0000 7fa261c00640 1 -- 172.21.15.107:0/252380946 wait complete. 2024-09-19T04:34:16.449 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T04:34:16.831 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:17.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:16 smithi107 ceph-mon[30659]: pgmap v437: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 97 KiB/s wr, 27 op/s 2024-09-19T04:34:17.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.393+0000 7fdfa5e00640 1 -- 172.21.15.107:0/957063678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa0106f10 msgr2=0x7fdfa010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:17.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.393+0000 7fdfa5e00640 1 --2- 172.21.15.107:0/957063678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa0106f10 0x7fdfa010ea90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009ca0 tx=0x7fdf9802a8d0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.393+0000 7fdfa5e00640 1 -- 172.21.15.107:0/957063678 shutdown_connections 2024-09-19T04:34:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.393+0000 7fdfa5e00640 1 --2- 172.21.15.107:0/957063678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa0106f10 0x7fdfa010ea90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.393+0000 7fdfa5e00640 1 --2- 172.21.15.107:0/957063678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfa0102790 0x7fdfa01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.393+0000 7fdfa5e00640 1 --2- 172.21.15.107:0/957063678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfa0101de0 0x7fdfa01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.393+0000 7fdfa5e00640 1 -- 172.21.15.107:0/957063678 >> 172.21.15.107:0/957063678 conn(0x7fdfa00fd8c0 msgr2=0x7fdfa00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.393+0000 7fdfa5e00640 1 -- 172.21.15.107:0/957063678 shutdown_connections 2024-09-19T04:34:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.393+0000 7fdfa5e00640 1 -- 172.21.15.107:0/957063678 wait complete. 2024-09-19T04:34:17.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.394+0000 7fdfa5e00640 1 Processor -- start 2024-09-19T04:34:17.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.394+0000 7fdfa5e00640 1 -- start start 2024-09-19T04:34:17.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdfa5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfa0101de0 0x7fdfa00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdfa5e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfa0102790 0x7fdfa0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdfa5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa0106f10 0x7fdfa0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdfa5e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0112000 con 0x7fdfa0106f10 2024-09-19T04:34:17.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdfa5e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0111e80 con 0x7fdfa0102790 2024-09-19T04:34:17.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdfa5e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0112180 con 0x7fdfa0101de0 2024-09-19T04:34:17.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdf9f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa0106f10 0x7fdfa0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdf9e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfa0102790 0x7fdfa0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdf9f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa0106f10 0x7fdfa0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54908/0 (socket says 172.21.15.107:54908) 2024-09-19T04:34:17.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdf9f400640 1 -- 172.21.15.107:0/1046074249 learned_addr learned my addr 172.21.15.107:0/1046074249 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:17.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdf9e000640 1 -- 172.21.15.107:0/1046074249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfa0101de0 msgr2=0x7fdfa00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:17.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdf9e000640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfa0101de0 0x7fdfa00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdf9e000640 1 -- 172.21.15.107:0/1046074249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa0106f10 msgr2=0x7fdfa0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:17.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdf9e000640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa0106f10 0x7fdfa0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdf9e000640 1 -- 172.21.15.107:0/1046074249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa01a6920 con 0x7fdfa0102790 2024-09-19T04:34:17.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.395+0000 7fdf9f400640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa0106f10 0x7fdfa0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:17.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.396+0000 7fdf9e000640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfa0102790 0x7fdfa0074a30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009fd0 tx=0x7fdf8c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:17.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.396+0000 7fdfa4a00640 1 -- 172.21.15.107:0/1046074249 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c015020 con 0x7fdfa0102790 2024-09-19T04:34:17.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.396+0000 7fdfa5e00640 1 -- 172.21.15.107:0/1046074249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa01a6c10 con 0x7fdfa0102790 2024-09-19T04:34:17.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.396+0000 7fdfa5e00640 1 -- 172.21.15.107:0/1046074249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa01a7150 con 0x7fdfa0102790 2024-09-19T04:34:17.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.396+0000 7fdfa4a00640 1 -- 172.21.15.107:0/1046074249 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8c011d10 con 0x7fdfa0102790 2024-09-19T04:34:17.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.396+0000 7fdfa4a00640 1 -- 172.21.15.107:0/1046074249 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0193b0 con 0x7fdfa0102790 2024-09-19T04:34:17.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.399+0000 7fdfa888d740 1 Processor -- start 2024-09-19T04:34:17.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.399+0000 7fdfa888d740 1 -- start start 2024-09-19T04:34:17.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f646050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f646590 0x557a3f650910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f650e50 0x557a3f653240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdfa888d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557a3f5ac3e0 con 0x557a3f645c70 2024-09-19T04:34:17.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdfa888d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557a3f553d80 con 0x557a3f650e50 2024-09-19T04:34:17.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdfa888d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557a3f5a83a0 con 0x557a3f646590 2024-09-19T04:34:17.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdf9f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f646050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdf9f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f646050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54912/0 (socket says 172.21.15.107:54912) 2024-09-19T04:34:17.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdf9f400640 1 -- 172.21.15.107:0/1706652558 learned_addr learned my addr 172.21.15.107:0/1706652558 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:17.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdfa4a00640 1 -- 172.21.15.107:0/1046074249 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fdf8c018020 con 0x7fdfa0102790 2024-09-19T04:34:17.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/1706652558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f646590 0x557a3f650910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdf9f400640 1 -- 172.21.15.107:0/1706652558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f646590 msgr2=0x557a3f650910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:17.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdf9f400640 1 --2- 172.21.15.107:0/1706652558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f646590 0x557a3f650910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdf9f400640 1 -- 172.21.15.107:0/1706652558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f650e50 msgr2=0x557a3f653240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:17.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdf9f400640 1 --2- 172.21.15.107:0/1706652558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f650e50 0x557a3f653240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdf9f400640 1 -- 172.21.15.107:0/1706652558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3f572ae0 con 0x557a3f645c70 2024-09-19T04:34:17.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.400+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/1706652558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f646590 0x557a3f650910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:34:17.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.401+0000 7fdf9f400640 1 --2- 172.21.15.107:0/1706652558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f646050 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009960 tx=0x7fdf98009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a267ef8a21c3695 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:17.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.401+0000 7fdf8be00640 1 -- 172.21.15.107:0/1706652558 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98003b70 con 0x557a3f645c70 2024-09-19T04:34:17.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.401+0000 7fdf8be00640 1 -- 172.21.15.107:0/1706652558 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf98036040 con 0x557a3f645c70 2024-09-19T04:34:17.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.401+0000 7fdf8be00640 1 -- 172.21.15.107:0/1706652558 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98004630 con 0x557a3f645c70 2024-09-19T04:34:17.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa4a00640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdf840a3430 0x7fdf840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa888d740 1 -- 172.21.15.107:0/1706652558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 msgr2=0x557a3f646050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:17.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa888d740 1 --2- 172.21.15.107:0/1706652558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f646050 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009960 tx=0x7fdf98009990 comp rx=0 tx=0).stop 2024-09-19T04:34:17.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa4a00640 1 -- 172.21.15.107:0/1046074249 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fdf8c0cf4f0 con 0x7fdfa0102790 2024-09-19T04:34:17.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdf840a3430 0x7fdf840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa888d740 1 -- 172.21.15.107:0/1706652558 shutdown_connections 2024-09-19T04:34:17.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa888d740 1 --2- 172.21.15.107:0/1706652558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f650e50 0x557a3f653240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa888d740 1 --2- 172.21.15.107:0/1706652558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f646590 0x557a3f650910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa888d740 1 --2- 172.21.15.107:0/1706652558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f646050 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa888d740 1 -- 172.21.15.107:0/1706652558 >> 172.21.15.107:0/1706652558 conn(0x557a3f642200 msgr2=0x557a3f6425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:17.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa888d740 1 -- 172.21.15.107:0/1706652558 shutdown_connections 2024-09-19T04:34:17.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.402+0000 7fdfa888d740 1 -- 172.21.15.107:0/1706652558 wait complete. 2024-09-19T04:34:17.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.403+0000 7fdfa888d740 1 -- start start 2024-09-19T04:34:17.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.403+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdf840a3430 0x7fdf840a58f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x557a3f643060 tx=0x7fdf90009530 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:17.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.403+0000 7fdfa888d740 1 Processor -- start 2024-09-19T04:34:17.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.403+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f645c70 0x557a3f64dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.403+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f64e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.403+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f650e50 0x557a3f64e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.403+0000 7fdfa888d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557a3f653a20 con 0x557a3f646590 2024-09-19T04:34:17.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.403+0000 7fdfa888d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557a3f5670f0 con 0x557a3f645c70 2024-09-19T04:34:17.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.403+0000 7fdfa888d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557a3f653ba0 con 0x557a3f650e50 2024-09-19T04:34:17.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdfa888d740 10 client.0 _ll_register_callbacks cb 0x557a3f54ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:17.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f64e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f645c70 0x557a3f64dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f650e50 0x557a3f64e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f64e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54924/0 (socket says 172.21.15.107:54924) 2024-09-19T04:34:17.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9e000640 1 -- 172.21.15.107:0/921132402 learned_addr learned my addr 172.21.15.107:0/921132402 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:17.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9e000640 1 -- 172.21.15.107:0/921132402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f650e50 msgr2=0x557a3f64e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:17.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9e000640 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f650e50 0x557a3f64e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9e000640 1 -- 172.21.15.107:0/921132402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f645c70 msgr2=0x557a3f64dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:17.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9e000640 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f645c70 0x557a3f64dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9e000640 1 -- 172.21.15.107:0/921132402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3f31d3c0 con 0x557a3f646590 2024-09-19T04:34:17.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.404+0000 7fdf9f400640 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f650e50 0x557a3f64e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:17.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.405+0000 7fdf9e000640 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f64e250 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0ccdd0 tx=0x7fdf8c09b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adb059db7308b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:17.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.405+0000 7fdf88c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:34:17.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.405+0000 7fdf88c00640 1 -- 172.21.15.107:0/921132402 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c09c000 con 0x557a3f646590 2024-09-19T04:34:17.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.405+0000 7fdf88c00640 1 -- 172.21.15.107:0/921132402 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fdf8c0cf020 con 0x557a3f646590 2024-09-19T04:34:17.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.405+0000 7fdfa888d740 1 -- 172.21.15.107:0/921132402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf98036040 con 0x557a3f646590 2024-09-19T04:34:17.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.405+0000 7fdf88c00640 1 -- 172.21.15.107:0/921132402 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c09b900 con 0x557a3f646590 2024-09-19T04:34:17.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.405+0000 7fdfa888d740 10 client.17691 fetch_fsmap learned FSMap version 55 2024-09-19T04:34:17.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.405+0000 7fdfa888d740 1 -- 172.21.15.107:0/921132402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557a3f65fab0 con 0x557a3f646590 2024-09-19T04:34:17.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.405+0000 7fdf88c00640 1 -- 172.21.15.107:0/921132402 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf98036040 con 0x557a3f646590 2024-09-19T04:34:17.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.406+0000 7fdf88c00640 1 -- 172.21.15.107:0/921132402 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7fdf8c09ad50 con 0x557a3f646590 2024-09-19T04:34:17.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.406+0000 7fdfa888d740 10 client.17691 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:34:17.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.406+0000 7fdfa888d740 10 client.17691 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:34:17.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.406+0000 7fdfa888d740 20 client.17691 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:17.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.406+0000 7fdfa888d740 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557a3f661010 0x557a3f663420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.406+0000 7fdfa888d740 4 client.17691 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:34:17.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.406+0000 7fdfa888d740 1 -- 172.21.15.107:0/921132402 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a3f5b9950 con 0x557a3f661010 2024-09-19T04:34:17.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.406+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557a3f661010 0x557a3f663420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.407+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557a3f661010 0x557a3f663420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97511650bbde4d2c server_cookie=367167324497673e in_seq=0 out_seq=0 2024-09-19T04:34:17.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.407+0000 7fdf88c00640 10 client.17691 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:34:17.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.407+0000 7fdf88c00640 1 -- 172.21.15.107:0/921132402 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a3f5b9950 con 0x557a3f661010 2024-09-19T04:34:17.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.407+0000 7fdf88c00640 10 client.17691 handle_command_reply: tid=0 2024-09-19T04:34:17.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 client.17691 shutdown 2024-09-19T04:34:17.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 -- 172.21.15.107:0/921132402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 msgr2=0x557a3f64e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:17.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f64e250 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0ccdd0 tx=0x7fdf8c09b790 comp rx=0 tx=0).stop 2024-09-19T04:34:17.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 -- 172.21.15.107:0/921132402 shutdown_connections 2024-09-19T04:34:17.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557a3f661010 0x557a3f663420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f650e50 0x557a3f64e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f64e250 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 --2- 172.21.15.107:0/921132402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f645c70 0x557a3f64dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 -- 172.21.15.107:0/921132402 >> 172.21.15.107:0/921132402 conn(0x557a3f642200 msgr2=0x557a3f647ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:17.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdf88c00640 0 client.17691 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:34:17.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 -- 172.21.15.107:0/921132402 shutdown_connections 2024-09-19T04:34:17.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 1 -- 172.21.15.107:0/921132402 wait complete. 2024-09-19T04:34:17.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.408+0000 7fdfa888d740 20 client.17691 trim_cache size 0 max 16384 2024-09-19T04:34:17.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.421+0000 7fdfa888d740 1 Processor -- start 2024-09-19T04:34:17.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdfa888d740 1 -- start start 2024-09-19T04:34:17.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f6425e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f646590 0x557a3f668cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f650e50 0x557a3f669230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdfa888d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01af8d0 con 0x557a3f645c70 2024-09-19T04:34:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdfa888d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01b1590 con 0x557a3f646590 2024-09-19T04:34:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdfa888d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01b0060 con 0x557a3f650e50 2024-09-19T04:34:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdf9ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f6425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdf9ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f6425e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54932/0 (socket says 172.21.15.107:54932) 2024-09-19T04:34:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdf9ea00640 1 -- 172.21.15.107:0/1086785614 learned_addr learned my addr 172.21.15.107:0/1086785614 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdf9e000640 1 --2- 172.21.15.107:0/1086785614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f646590 0x557a3f668cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdf9ea00640 1 -- 172.21.15.107:0/1086785614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f650e50 msgr2=0x557a3f669230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/1086785614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f650e50 0x557a3f669230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdf9ea00640 1 -- 172.21.15.107:0/1086785614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f646590 msgr2=0x557a3f668cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/1086785614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f646590 0x557a3f668cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdf9ea00640 1 -- 172.21.15.107:0/1086785614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3f642030 con 0x557a3f645c70 2024-09-19T04:34:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.422+0000 7fdf9e000640 1 --2- 172.21.15.107:0/1086785614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f646590 0x557a3f668cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:34:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.423+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/1086785614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f6425e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdf90011ff0 tx=0x7fdf90014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697a3e530b3c316e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.423+0000 7fdf8b400640 1 -- 172.21.15.107:0/1086785614 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90003770 con 0x557a3f645c70 2024-09-19T04:34:17.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.423+0000 7fdf8b400640 1 -- 172.21.15.107:0/1086785614 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf90016070 con 0x557a3f645c70 2024-09-19T04:34:17.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.423+0000 7fdf8b400640 1 -- 172.21.15.107:0/1086785614 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90029a20 con 0x557a3f645c70 2024-09-19T04:34:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 -- 172.21.15.107:0/1086785614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 msgr2=0x557a3f6425e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 --2- 172.21.15.107:0/1086785614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f6425e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdf90011ff0 tx=0x7fdf90014000 comp rx=0 tx=0).stop 2024-09-19T04:34:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 -- 172.21.15.107:0/1086785614 shutdown_connections 2024-09-19T04:34:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 --2- 172.21.15.107:0/1086785614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f650e50 0x557a3f669230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 --2- 172.21.15.107:0/1086785614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f646590 0x557a3f668cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 --2- 172.21.15.107:0/1086785614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f645c70 0x557a3f6425e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 -- 172.21.15.107:0/1086785614 >> 172.21.15.107:0/1086785614 conn(0x557a3f642200 msgr2=0x557a3f64ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 -- 172.21.15.107:0/1086785614 shutdown_connections 2024-09-19T04:34:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 -- 172.21.15.107:0/1086785614 wait complete. 2024-09-19T04:34:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 -- start start 2024-09-19T04:34:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.424+0000 7fdfa888d740 1 Processor -- start 2024-09-19T04:34:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f645c70 0x557a3f659320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f6546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdfa888d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f650e50 0x557a3f654be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdfa888d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01b0e90 con 0x557a3f646590 2024-09-19T04:34:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdfa888d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01b0760 con 0x557a3f650e50 2024-09-19T04:34:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdfa888d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01ad8d0 con 0x557a3f645c70 2024-09-19T04:34:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdfa888d740 10 client.0 _ll_register_callbacks cb 0x7fdf8c01a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdf9e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f6546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdf9e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f6546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54944/0 (socket says 172.21.15.107:54944) 2024-09-19T04:34:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdf9e000640 1 -- 172.21.15.107:0/3363457422 learned_addr learned my addr 172.21.15.107:0/3363457422 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdf9f400640 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f650e50 0x557a3f654be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f645c70 0x557a3f659320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.425+0000 7fdf9e000640 1 -- 172.21.15.107:0/3363457422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f645c70 msgr2=0x557a3f659320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdf9e000640 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f645c70 0x557a3f659320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdf9e000640 1 -- 172.21.15.107:0/3363457422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f650e50 msgr2=0x557a3f654be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdf9e000640 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f650e50 0x557a3f654be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdf9e000640 1 -- 172.21.15.107:0/3363457422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3f63d770 con 0x557a3f646590 2024-09-19T04:34:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdf9e000640 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f6546a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0cce20 tx=0x7fdf8c0981a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2072305fcffebb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdf88c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:34:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdf88c00640 1 -- 172.21.15.107:0/3363457422 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c07f060 con 0x557a3f646590 2024-09-19T04:34:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdf88c00640 1 -- 172.21.15.107:0/3363457422 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fdf8c011830 con 0x557a3f646590 2024-09-19T04:34:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdfa888d740 1 -- 172.21.15.107:0/3363457422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf90016070 con 0x557a3f646590 2024-09-19T04:34:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdf88c00640 1 -- 172.21.15.107:0/3363457422 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c097ba0 con 0x557a3f646590 2024-09-19T04:34:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdfa888d740 10 client.17697 fetch_fsmap learned FSMap version 55 2024-09-19T04:34:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.426+0000 7fdfa888d740 1 -- 172.21.15.107:0/3363457422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557a3f6432b0 con 0x557a3f646590 2024-09-19T04:34:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.427+0000 7fdf88c00640 1 -- 172.21.15.107:0/3363457422 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf90016070 con 0x557a3f646590 2024-09-19T04:34:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.427+0000 7fdf88c00640 1 -- 172.21.15.107:0/3363457422 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7fdf8c09a7c0 con 0x557a3f646590 2024-09-19T04:34:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.427+0000 7fdfa888d740 10 client.17697 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:34:17.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.427+0000 7fdfa888d740 10 client.17697 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T04:34:17.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.427+0000 7fdfa888d740 20 client.17697 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:17.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.427+0000 7fdfa888d740 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557a3f6cbc50 0x557a3f6ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:17.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.427+0000 7fdfa888d740 4 client.17697 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:34:17.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.427+0000 7fdfa888d740 1 -- 172.21.15.107:0/3363457422 --> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a3f642db0 con 0x557a3f6cbc50 2024-09-19T04:34:17.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.427+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557a3f6cbc50 0x557a3f6ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:17.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdf9ea00640 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557a3f6cbc50 0x557a3f6ce070 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf322f0612950ec1 server_cookie=397e97b3e639b44e in_seq=0 out_seq=0 2024-09-19T04:34:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdf88c00640 10 client.17697 ms_handle_connect on v2:172.21.15.145:6836/1756017901 2024-09-19T04:34:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdf88c00640 1 -- 172.21.15.107:0/3363457422 <== mds.3 v2:172.21.15.145:6836/1756017901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a3f642db0 con 0x557a3f6cbc50 2024-09-19T04:34:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdf88c00640 10 client.17697 handle_command_reply: tid=0 2024-09-19T04:34:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdfa888d740 1 client.17697 shutdown 2024-09-19T04:34:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdfa888d740 1 -- 172.21.15.107:0/3363457422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 msgr2=0x557a3f6546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdfa888d740 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f6546a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0cce20 tx=0x7fdf8c0981a0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdfa888d740 1 -- 172.21.15.107:0/3363457422 shutdown_connections 2024-09-19T04:34:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdfa888d740 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.145:6836/1756017901,v1:172.21.15.145:6837/1756017901] conn(0x557a3f6cbc50 0x557a3f6ce070 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdfa888d740 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a3f650e50 0x557a3f654be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdfa888d740 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a3f646590 0x557a3f6546a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdfa888d740 1 --2- 172.21.15.107:0/3363457422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557a3f645c70 0x557a3f659320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdfa888d740 1 -- 172.21.15.107:0/3363457422 >> 172.21.15.107:0/3363457422 conn(0x557a3f642200 msgr2=0x557a3f64f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.428+0000 7fdf88c00640 0 client.17697 ms_handle_reset on v2:172.21.15.145:6836/1756017901 2024-09-19T04:34:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa888d740 1 -- 172.21.15.107:0/3363457422 shutdown_connections 2024-09-19T04:34:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa888d740 1 -- 172.21.15.107:0/3363457422 wait complete. 2024-09-19T04:34:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa888d740 20 client.17697 trim_cache size 0 max 16384 2024-09-19T04:34:17.439 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:34:17.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 -- 172.21.15.107:0/1046074249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdf840a3430 msgr2=0x7fdf840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:17.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdf840a3430 0x7fdf840a58f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x557a3f643060 tx=0x7fdf90009530 comp rx=0 tx=0).stop 2024-09-19T04:34:17.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 -- 172.21.15.107:0/1046074249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfa0102790 msgr2=0x7fdfa0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:17.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfa0102790 0x7fdfa0074a30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009fd0 tx=0x7fdf8c00b870 comp rx=0 tx=0).stop 2024-09-19T04:34:17.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 -- 172.21.15.107:0/1046074249 shutdown_connections 2024-09-19T04:34:17.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fdf840a3430 0x7fdf840a58f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa0106f10 0x7fdfa0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfa0102790 0x7fdfa0074a30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 --2- 172.21.15.107:0/1046074249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfa0101de0 0x7fdfa00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:17.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 -- 172.21.15.107:0/1046074249 >> 172.21.15.107:0/1046074249 conn(0x7fdfa00fd8c0 msgr2=0x7fdfa00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:17.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.429+0000 7fdfa5e00640 1 -- 172.21.15.107:0/1046074249 shutdown_connections 2024-09-19T04:34:17.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:17.430+0000 7fdfa5e00640 1 -- 172.21.15.107:0/1046074249 wait complete. 2024-09-19T04:34:17.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:16 smithi115 ceph-mon[35477]: pgmap v437: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 97 KiB/s wr, 27 op/s 2024-09-19T04:34:17.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:16 smithi145 ceph-mon[35096]: pgmap v437: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 97 KiB/s wr, 27 op/s 2024-09-19T04:34:18.361 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T04:34:18.730 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:19.159 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:18 smithi107 ceph-mon[30659]: pgmap v438: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 90 KiB/s wr, 25 op/s 2024-09-19T04:34:19.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.270+0000 7f9805000640 1 -- 172.21.15.107:0/1902743481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9800109140 msgr2=0x7f9800110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:19.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.270+0000 7f9805000640 1 --2- 172.21.15.107:0/1902743481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9800109140 0x7f9800110cc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009ca0 tx=0x7f97f802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.270+0000 7f9805000640 1 -- 172.21.15.107:0/1902743481 shutdown_connections 2024-09-19T04:34:19.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.270+0000 7f9805000640 1 --2- 172.21.15.107:0/1902743481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9800109140 0x7f9800110cc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.270+0000 7f9805000640 1 --2- 172.21.15.107:0/1902743481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98001049c0 0x7f9800108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.270+0000 7f9805000640 1 --2- 172.21.15.107:0/1902743481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9800104010 0x7f98001043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.270+0000 7f9805000640 1 -- 172.21.15.107:0/1902743481 >> 172.21.15.107:0/1902743481 conn(0x7f98000ffa50 msgr2=0x7f9800101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:19.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.270+0000 7f9805000640 1 -- 172.21.15.107:0/1902743481 shutdown_connections 2024-09-19T04:34:19.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.270+0000 7f9805000640 1 -- 172.21.15.107:0/1902743481 wait complete. 2024-09-19T04:34:19.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.271+0000 7f9805000640 1 Processor -- start 2024-09-19T04:34:19.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.271+0000 7f9805000640 1 -- start start 2024-09-19T04:34:19.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f9805000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800104010 0x7f980019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f9805000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98001049c0 0x7f980019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f9805000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9800109140 0x7f980019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f9805000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9800114170 con 0x7f98001049c0 2024-09-19T04:34:19.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f9805000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9800113ff0 con 0x7f9800109140 2024-09-19T04:34:19.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f9805000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98001142f0 con 0x7f9800104010 2024-09-19T04:34:19.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f97fe800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9800109140 0x7f980019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f97fd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98001049c0 0x7f980019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f97fe800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9800109140 0x7f980019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:53674/0 (socket says 172.21.15.107:53674) 2024-09-19T04:34:19.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f97fe800640 1 -- 172.21.15.107:0/2181724600 learned_addr learned my addr 172.21.15.107:0/2181724600 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:19.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.272+0000 7f97fde00640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800104010 0x7f980019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f97fe800640 1 -- 172.21.15.107:0/2181724600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800104010 msgr2=0x7f980019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:19.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f97fe800640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800104010 0x7f980019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f97fe800640 1 -- 172.21.15.107:0/2181724600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98001049c0 msgr2=0x7f980019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:19.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f97fe800640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98001049c0 0x7f980019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f97fe800640 1 -- 172.21.15.107:0/2181724600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980019f600 con 0x7f9800109140 2024-09-19T04:34:19.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f97fd400640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98001049c0 0x7f980019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:19.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f97fe800640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9800109140 0x7f980019ef20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f97f8031a20 tx=0x7f97f8031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:19.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f97f6a00640 1 -- 172.21.15.107:0/2181724600 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8003b70 con 0x7f9800109140 2024-09-19T04:34:19.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f97f6a00640 1 -- 172.21.15.107:0/2181724600 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f8003d10 con 0x7f9800109140 2024-09-19T04:34:19.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f9805000640 1 -- 172.21.15.107:0/2181724600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980019f890 con 0x7f9800109140 2024-09-19T04:34:19.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f97f6a00640 1 -- 172.21.15.107:0/2181724600 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f802fc40 con 0x7f9800109140 2024-09-19T04:34:19.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.273+0000 7f9805000640 1 -- 172.21.15.107:0/2181724600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98001a71d0 con 0x7f9800109140 2024-09-19T04:34:19.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.276+0000 7f9807bfe740 1 Processor -- start 2024-09-19T04:34:19.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.277+0000 7f97f6a00640 1 -- 172.21.15.107:0/2181724600 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f97f8004050 con 0x7f9800109140 2024-09-19T04:34:19.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.277+0000 7f9807bfe740 1 -- start start 2024-09-19T04:34:19.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.277+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c4fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.277+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4fa590 0x55f35c504910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.277+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c507240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.277+0000 7f9807bfe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f35c4603e0 con 0x55f35c4f9c70 2024-09-19T04:34:19.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.277+0000 7f9807bfe740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f35c407d80 con 0x55f35c4fa590 2024-09-19T04:34:19.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.277+0000 7f9807bfe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f35c45c3a0 con 0x55f35c504e50 2024-09-19T04:34:19.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c4fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c4fa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39012/0 (socket says 172.21.15.107:39012) 2024-09-19T04:34:19.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fd400640 1 -- 172.21.15.107:0/3846284714 learned_addr learned my addr 172.21.15.107:0/3846284714 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:19.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fde00640 1 --2- 172.21.15.107:0/3846284714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4fa590 0x55f35c504910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fd400640 1 --2- 172.21.15.107:0/3846284714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c507240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fde00640 1 -- 172.21.15.107:0/3846284714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 msgr2=0x55f35c507240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:19.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fde00640 1 --2- 172.21.15.107:0/3846284714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c507240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fde00640 1 -- 172.21.15.107:0/3846284714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 msgr2=0x55f35c4fa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:19.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fde00640 1 --2- 172.21.15.107:0/3846284714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c4fa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fde00640 1 -- 172.21.15.107:0/3846284714 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f35c426ae0 con 0x55f35c4fa590 2024-09-19T04:34:19.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fd400640 1 --2- 172.21.15.107:0/3846284714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c4fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:19.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97fde00640 1 --2- 172.21.15.107:0/3846284714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4fa590 0x55f35c504910 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009c20 tx=0x7f97f0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b05de74b0893eba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:19.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.278+0000 7f97f6a00640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97e40a33b0 0x7f97e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.279+0000 7f97f6a00640 1 -- 172.21.15.107:0/2181724600 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f97f80f0d40 con 0x7f9800109140 2024-09-19T04:34:19.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.279+0000 7f97eb400640 1 -- 172.21.15.107:0/3846284714 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0004050 con 0x55f35c4fa590 2024-09-19T04:34:19.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.279+0000 7f97fd400640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97e40a33b0 0x7f97e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.279+0000 7f97eb400640 1 -- 172.21.15.107:0/3846284714 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f0005500 con 0x55f35c4fa590 2024-09-19T04:34:19.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.279+0000 7f97eb400640 1 -- 172.21.15.107:0/3846284714 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f00035f0 con 0x55f35c4fa590 2024-09-19T04:34:19.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f9807bfe740 1 -- 172.21.15.107:0/3846284714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4fa590 msgr2=0x55f35c504910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:19.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f97fd400640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97e40a33b0 0x7f97e40a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97ec005f60 tx=0x7f97ec015040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:19.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f9807bfe740 1 --2- 172.21.15.107:0/3846284714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4fa590 0x55f35c504910 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009c20 tx=0x7f97f0011810 comp rx=0 tx=0).stop 2024-09-19T04:34:19.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f9807bfe740 1 -- 172.21.15.107:0/3846284714 shutdown_connections 2024-09-19T04:34:19.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f9807bfe740 1 --2- 172.21.15.107:0/3846284714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c507240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f9807bfe740 1 --2- 172.21.15.107:0/3846284714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4fa590 0x55f35c504910 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f9807bfe740 1 --2- 172.21.15.107:0/3846284714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c4fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f9807bfe740 1 -- 172.21.15.107:0/3846284714 >> 172.21.15.107:0/3846284714 conn(0x55f35c4f6200 msgr2=0x55f35c4f65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:19.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f9807bfe740 1 -- 172.21.15.107:0/3846284714 shutdown_connections 2024-09-19T04:34:19.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f9807bfe740 1 -- 172.21.15.107:0/3846284714 wait complete. 2024-09-19T04:34:19.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.280+0000 7f9807bfe740 1 -- start start 2024-09-19T04:34:19.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f9807bfe740 1 Processor -- start 2024-09-19T04:34:19.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c501c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c4fa590 0x55f35c5021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c504e50 0x55f35c502710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f9807bfe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f35c507a20 con 0x55f35c4f9c70 2024-09-19T04:34:19.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f9807bfe740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f35c41b0f0 con 0x55f35c504e50 2024-09-19T04:34:19.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f9807bfe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f35c507ba0 con 0x55f35c4fa590 2024-09-19T04:34:19.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f9807bfe740 10 client.0 _ll_register_callbacks cb 0x55f35c402bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:19.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f97fde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c501c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f97fde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c501c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39028/0 (socket says 172.21.15.107:39028) 2024-09-19T04:34:19.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f97fde00640 1 -- 172.21.15.107:0/1043926321 learned_addr learned my addr 172.21.15.107:0/1043926321 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:19.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f97fd400640 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c4fa590 0x55f35c5021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.281+0000 7f97fe800640 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c504e50 0x55f35c502710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.282+0000 7f97fde00640 1 -- 172.21.15.107:0/1043926321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c4fa590 msgr2=0x55f35c5021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:19.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.282+0000 7f97fde00640 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c4fa590 0x55f35c5021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.282+0000 7f97fde00640 1 -- 172.21.15.107:0/1043926321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c504e50 msgr2=0x55f35c502710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:19.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.282+0000 7f97fde00640 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c504e50 0x55f35c502710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.282+0000 7f97fde00640 1 -- 172.21.15.107:0/1043926321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f35c1d13c0 con 0x55f35c4f9c70 2024-09-19T04:34:19.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.282+0000 7f97fde00640 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c501c90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f97f0000c00 tx=0x7f97f0004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193b02befd76e3be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:19.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.282+0000 7f97dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:34:19.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.282+0000 7f97dfe00640 1 -- 172.21.15.107:0/1043926321 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f00052d0 con 0x55f35c4f9c70 2024-09-19T04:34:19.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.282+0000 7f97dfe00640 1 -- 172.21.15.107:0/1043926321 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f97f00035f0 con 0x55f35c4f9c70 2024-09-19T04:34:19.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.282+0000 7f9807bfe740 1 -- 172.21.15.107:0/1043926321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f0005500 con 0x55f35c4f9c70 2024-09-19T04:34:19.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f97dfe00640 1 -- 172.21.15.107:0/1043926321 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f97f0033050 con 0x55f35c4f9c70 2024-09-19T04:34:19.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f9807bfe740 10 client.17709 fetch_fsmap learned FSMap version 55 2024-09-19T04:34:19.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f9807bfe740 1 -- 172.21.15.107:0/1043926321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f35c5139b0 con 0x55f35c4f9c70 2024-09-19T04:34:19.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f97dfe00640 1 -- 172.21.15.107:0/1043926321 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f0005500 con 0x55f35c4f9c70 2024-09-19T04:34:19.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f97dfe00640 1 -- 172.21.15.107:0/1043926321 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7f97f002e560 con 0x55f35c4f9c70 2024-09-19T04:34:19.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f9807bfe740 10 client.17709 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:34:19.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f9807bfe740 10 client.17709 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:34:19.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f9807bfe740 20 client.17709 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:19.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f9807bfe740 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55f35c514db0 0x55f35c5171a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f9807bfe740 4 client.17709 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:34:19.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.283+0000 7f9807bfe740 1 -- 172.21.15.107:0/1043926321 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f35c46d950 con 0x55f35c514db0 2024-09-19T04:34:19.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.284+0000 7f97fd400640 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55f35c514db0 0x55f35c5171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.284+0000 7f97fd400640 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55f35c514db0 0x55f35c5171a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a04d4e37e585dca server_cookie=1d40c58b19cfdc7c in_seq=0 out_seq=0 2024-09-19T04:34:19.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.284+0000 7f97dfe00640 10 client.17709 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:34:19.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f97dfe00640 1 -- 172.21.15.107:0/1043926321 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f35c46d950 con 0x55f35c514db0 2024-09-19T04:34:19.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f97dfe00640 10 client.17709 handle_command_reply: tid=0 2024-09-19T04:34:19.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f9807bfe740 1 client.17709 shutdown 2024-09-19T04:34:19.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f9807bfe740 1 -- 172.21.15.107:0/1043926321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 msgr2=0x55f35c501c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f9807bfe740 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c501c90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f97f0000c00 tx=0x7f97f0004ed0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f9807bfe740 1 -- 172.21.15.107:0/1043926321 shutdown_connections 2024-09-19T04:34:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f9807bfe740 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55f35c514db0 0x55f35c5171a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f9807bfe740 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c504e50 0x55f35c502710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f9807bfe740 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c4fa590 0x55f35c5021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f9807bfe740 1 --2- 172.21.15.107:0/1043926321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4f9c70 0x55f35c501c90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.285+0000 7f9807bfe740 1 -- 172.21.15.107:0/1043926321 >> 172.21.15.107:0/1043926321 conn(0x55f35c4f6200 msgr2=0x55f35c4fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:19.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.286+0000 7f97dfe00640 0 client.17709 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:34:19.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.286+0000 7f9807bfe740 1 -- 172.21.15.107:0/1043926321 shutdown_connections 2024-09-19T04:34:19.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.286+0000 7f9807bfe740 1 -- 172.21.15.107:0/1043926321 wait complete. 2024-09-19T04:34:19.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.286+0000 7f9807bfe740 20 client.17709 trim_cache size 0 max 16384 2024-09-19T04:34:19.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.299+0000 7f9807bfe740 1 Processor -- start 2024-09-19T04:34:19.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.299+0000 7f9807bfe740 1 -- start start 2024-09-19T04:34:19.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c500b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 0x55f35c51ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c51cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f9807bfe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98001afa30 con 0x55f35c4fa590 2024-09-19T04:34:19.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f9807bfe740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f98001b16f0 con 0x55f35c4f9c70 2024-09-19T04:34:19.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f9807bfe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98001b01c0 con 0x55f35c504e50 2024-09-19T04:34:19.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f97fd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 0x55f35c51ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f97fde00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c500b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f97fde00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c500b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:53694/0 (socket says 172.21.15.107:53694) 2024-09-19T04:34:19.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f97fde00640 1 -- 172.21.15.107:0/3994888061 learned_addr learned my addr 172.21.15.107:0/3994888061 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:19.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f97fe800640 1 --2- 172.21.15.107:0/3994888061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c51cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f97fde00640 1 -- 172.21.15.107:0/3994888061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 msgr2=0x55f35c51cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:19.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f97fde00640 1 --2- 172.21.15.107:0/3994888061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c51cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f97fde00640 1 -- 172.21.15.107:0/3994888061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 msgr2=0x55f35c51ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:19.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f97fde00640 1 --2- 172.21.15.107:0/3994888061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 0x55f35c51ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.300+0000 7f97fde00640 1 -- 172.21.15.107:0/3994888061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f35c4f6030 con 0x55f35c4f9c70 2024-09-19T04:34:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.301+0000 7f97fde00640 1 --2- 172.21.15.107:0/3994888061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c500b40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f97f002eac0 tx=0x7f97f0005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe747b1228a9e1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.301+0000 7f97eaa00640 1 -- 172.21.15.107:0/3994888061 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f002eb40 con 0x55f35c4f9c70 2024-09-19T04:34:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.301+0000 7f97eaa00640 1 -- 172.21.15.107:0/3994888061 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f002ece0 con 0x55f35c4f9c70 2024-09-19T04:34:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.301+0000 7f97eaa00640 1 -- 172.21.15.107:0/3994888061 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f00048c0 con 0x55f35c4f9c70 2024-09-19T04:34:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.301+0000 7f9807bfe740 1 -- 172.21.15.107:0/3994888061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 msgr2=0x55f35c500b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.301+0000 7f9807bfe740 1 --2- 172.21.15.107:0/3994888061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c500b40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f97f002eac0 tx=0x7f97f0005b90 comp rx=0 tx=0).stop 2024-09-19T04:34:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.302+0000 7f9807bfe740 1 -- 172.21.15.107:0/3994888061 shutdown_connections 2024-09-19T04:34:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.302+0000 7f9807bfe740 1 --2- 172.21.15.107:0/3994888061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c51cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.302+0000 7f9807bfe740 1 --2- 172.21.15.107:0/3994888061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 0x55f35c51ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.302+0000 7f9807bfe740 1 --2- 172.21.15.107:0/3994888061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c500b40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.302+0000 7f9807bfe740 1 -- 172.21.15.107:0/3994888061 >> 172.21.15.107:0/3994888061 conn(0x55f35c4f6200 msgr2=0x55f35c500430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:19.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.302+0000 7f9807bfe740 1 -- 172.21.15.107:0/3994888061 shutdown_connections 2024-09-19T04:34:19.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.302+0000 7f9807bfe740 1 -- 172.21.15.107:0/3994888061 wait complete. 2024-09-19T04:34:19.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.302+0000 7f9807bfe740 1 -- start start 2024-09-19T04:34:19.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.302+0000 7f9807bfe740 1 Processor -- start 2024-09-19T04:34:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c577d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 0x55f35c578290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f9807bfe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c5083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f9807bfe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98001b0ff0 con 0x55f35c4fa590 2024-09-19T04:34:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f9807bfe740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f98001b08c0 con 0x55f35c4f9c70 2024-09-19T04:34:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f9807bfe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98001ada30 con 0x55f35c504e50 2024-09-19T04:34:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f9807bfe740 10 client.0 _ll_register_callbacks cb 0x7f97ec009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 0x55f35c578290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fde00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c577d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fde00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c577d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:53700/0 (socket says 172.21.15.107:53700) 2024-09-19T04:34:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fde00640 1 -- 172.21.15.107:0/2524923875 learned_addr learned my addr 172.21.15.107:0/2524923875 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fe800640 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c5083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fde00640 1 -- 172.21.15.107:0/2524923875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 msgr2=0x55f35c5083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fde00640 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c5083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fde00640 1 -- 172.21.15.107:0/2524923875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 msgr2=0x55f35c578290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fde00640 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 0x55f35c578290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fde00640 1 -- 172.21.15.107:0/2524923875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f35c4f1770 con 0x55f35c4f9c70 2024-09-19T04:34:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.303+0000 7f97fd400640 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 0x55f35c578290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.304+0000 7f97fde00640 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c577d50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f97f002e950 tx=0x7f97f0011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=691b8cc37b006f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.304+0000 7f97dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:34:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.304+0000 7f97dfe00640 1 -- 172.21.15.107:0/2524923875 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f002eb40 con 0x55f35c4f9c70 2024-09-19T04:34:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.304+0000 7f97dfe00640 1 -- 172.21.15.107:0/2524923875 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f97f002e650 con 0x55f35c4f9c70 2024-09-19T04:34:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.304+0000 7f9807bfe740 1 -- 172.21.15.107:0/2524923875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f35c467e60 con 0x55f35c4f9c70 2024-09-19T04:34:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.304+0000 7f97dfe00640 1 -- 172.21.15.107:0/2524923875 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f97f00052d0 con 0x55f35c4f9c70 2024-09-19T04:34:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.304+0000 7f9807bfe740 10 client.27749 fetch_fsmap learned FSMap version 55 2024-09-19T04:34:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.304+0000 7f9807bfe740 1 -- 172.21.15.107:0/2524923875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f35c57f360 con 0x55f35c4f9c70 2024-09-19T04:34:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.305+0000 7f97dfe00640 1 -- 172.21.15.107:0/2524923875 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f35c467e60 con 0x55f35c4f9c70 2024-09-19T04:34:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.305+0000 7f97dfe00640 1 -- 172.21.15.107:0/2524923875 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7f97f0003950 con 0x55f35c4f9c70 2024-09-19T04:34:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.305+0000 7f9807bfe740 10 client.27749 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:34:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.305+0000 7f9807bfe740 10 client.27749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T04:34:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.305+0000 7f9807bfe740 20 client.27749 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.305+0000 7f9807bfe740 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55f35c57fa00 0x55f35c581e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.305+0000 7f9807bfe740 4 client.27749 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:34:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.305+0000 7f9807bfe740 1 -- 172.21.15.107:0/2524923875 --> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f35c3e4010 con 0x55f35c57fa00 2024-09-19T04:34:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.305+0000 7f97fd400640 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55f35c57fa00 0x55f35c581e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f97fd400640 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55f35c57fa00 0x55f35c581e20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc3efe8546770370 server_cookie=9dd011eb17531dff in_seq=0 out_seq=0 2024-09-19T04:34:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f97dfe00640 10 client.27749 ms_handle_connect on v2:172.21.15.115:6836/191835918 2024-09-19T04:34:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f97dfe00640 1 -- 172.21.15.107:0/2524923875 <== mds.2 v2:172.21.15.115:6836/191835918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f35c3e4010 con 0x55f35c57fa00 2024-09-19T04:34:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f97dfe00640 10 client.27749 handle_command_reply: tid=0 2024-09-19T04:34:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 client.27749 shutdown 2024-09-19T04:34:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 -- 172.21.15.107:0/2524923875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 msgr2=0x55f35c577d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c577d50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f97f002e950 tx=0x7f97f0011d90 comp rx=0 tx=0).stop 2024-09-19T04:34:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 -- 172.21.15.107:0/2524923875 shutdown_connections 2024-09-19T04:34:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.115:6836/191835918,v1:172.21.15.115:6837/191835918] conn(0x55f35c57fa00 0x55f35c581e20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f35c504e50 0x55f35c5083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f35c4fa590 0x55f35c578290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f97dfe00640 0 client.27749 ms_handle_reset on v2:172.21.15.115:6836/191835918 2024-09-19T04:34:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 --2- 172.21.15.107:0/2524923875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f35c4f9c70 0x55f35c577d50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 -- 172.21.15.107:0/2524923875 >> 172.21.15.107:0/2524923875 conn(0x55f35c4f6200 msgr2=0x55f35c4fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:19.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 -- 172.21.15.107:0/2524923875 shutdown_connections 2024-09-19T04:34:19.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 1 -- 172.21.15.107:0/2524923875 wait complete. 2024-09-19T04:34:19.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.306+0000 7f9807bfe740 20 client.27749 trim_cache size 0 max 16384 2024-09-19T04:34:19.319 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:34:19.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 -- 172.21.15.107:0/2181724600 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97e40a33b0 msgr2=0x7f97e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:19.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97e40a33b0 0x7f97e40a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97ec005f60 tx=0x7f97ec015040 comp rx=0 tx=0).stop 2024-09-19T04:34:19.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 -- 172.21.15.107:0/2181724600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9800109140 msgr2=0x7f980019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:19.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9800109140 0x7f980019ef20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f97f8031a20 tx=0x7f97f8031a50 comp rx=0 tx=0).stop 2024-09-19T04:34:19.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 -- 172.21.15.107:0/2181724600 shutdown_connections 2024-09-19T04:34:19.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f97e40a33b0 0x7f97e40a5870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9800109140 0x7f980019ef20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98001049c0 0x7f980019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 --2- 172.21.15.107:0/2181724600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800104010 0x7f980019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:19.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 -- 172.21.15.107:0/2181724600 >> 172.21.15.107:0/2181724600 conn(0x7f98000ffa50 msgr2=0x7f9800100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:19.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 -- 172.21.15.107:0/2181724600 shutdown_connections 2024-09-19T04:34:19.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:19.307+0000 7f9805000640 1 -- 172.21.15.107:0/2181724600 wait complete. 2024-09-19T04:34:19.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:18 smithi115 ceph-mon[35477]: pgmap v438: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 90 KiB/s wr, 25 op/s 2024-09-19T04:34:19.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:18 smithi145 ceph-mon[35096]: pgmap v438: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 90 KiB/s wr, 25 op/s 2024-09-19T04:34:20.211 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T04:34:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:21.155 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:20 smithi107 ceph-mon[30659]: pgmap v439: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 81 KiB/s wr, 23 op/s 2024-09-19T04:34:21.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:20 smithi145 ceph-mon[35096]: pgmap v439: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 81 KiB/s wr, 23 op/s 2024-09-19T04:34:21.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.380+0000 7f2b2a200640 1 -- 172.21.15.107:0/1494377482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b241036c0 msgr2=0x7f2b2410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:21.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.380+0000 7f2b2a200640 1 --2- 172.21.15.107:0/1494377482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b241036c0 0x7f2b2410bbb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009900 tx=0x7f2b1002a480 comp rx=0 tx=0).stop 2024-09-19T04:34:21.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.380+0000 7f2b2a200640 1 -- 172.21.15.107:0/1494377482 shutdown_connections 2024-09-19T04:34:21.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.380+0000 7f2b2a200640 1 --2- 172.21.15.107:0/1494377482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b2410c180 0x7f2b2410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.380+0000 7f2b2a200640 1 --2- 172.21.15.107:0/1494377482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b241036c0 0x7f2b2410bbb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.380+0000 7f2b2a200640 1 --2- 172.21.15.107:0/1494377482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b24102da0 0x7f2b24103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.380+0000 7f2b2a200640 1 -- 172.21.15.107:0/1494377482 >> 172.21.15.107:0/1494377482 conn(0x7f2b240fc7a0 msgr2=0x7f2b240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:21.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.380+0000 7f2b2a200640 1 -- 172.21.15.107:0/1494377482 shutdown_connections 2024-09-19T04:34:21.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.380+0000 7f2b2a200640 1 -- 172.21.15.107:0/1494377482 wait complete. 2024-09-19T04:34:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.381+0000 7f2b2a200640 1 Processor -- start 2024-09-19T04:34:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.381+0000 7f2b2a200640 1 -- start start 2024-09-19T04:34:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b2a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b24102da0 0x7f2b24100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b2a200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b241036c0 0x7f2b24100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b2a200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b2410c180 0x7f2b24101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b2a200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b24111f60 con 0x7f2b24102da0 2024-09-19T04:34:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b2a200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b24111de0 con 0x7f2b2410c180 2024-09-19T04:34:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b2a200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241120e0 con 0x7f2b241036c0 2024-09-19T04:34:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b23400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b24102da0 0x7f2b24100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b23400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b24102da0 0x7f2b24100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39040/0 (socket says 172.21.15.107:39040) 2024-09-19T04:34:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b23400640 1 -- 172.21.15.107:0/3126637248 learned_addr learned my addr 172.21.15.107:0/3126637248 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b22a00640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b241036c0 0x7f2b24100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.382+0000 7f2b23e00640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b2410c180 0x7f2b24101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b23400640 1 -- 172.21.15.107:0/3126637248 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b241036c0 msgr2=0x7f2b24100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:21.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b23400640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b241036c0 0x7f2b24100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b23400640 1 -- 172.21.15.107:0/3126637248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b2410c180 msgr2=0x7f2b24101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:21.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b23400640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b2410c180 0x7f2b24101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b23400640 1 -- 172.21.15.107:0/3126637248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b241a68f0 con 0x7f2b24102da0 2024-09-19T04:34:21.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b23e00640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b2410c180 0x7f2b24101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:34:21.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b22a00640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b241036c0 0x7f2b24100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:21.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b23400640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b24102da0 0x7f2b24100a00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400e9f0 tx=0x7f2b1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:21.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b1be00640 1 -- 172.21.15.107:0/3126637248 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400cd80 con 0x7f2b24102da0 2024-09-19T04:34:21.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b1be00640 1 -- 172.21.15.107:0/3126637248 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b14005500 con 0x7f2b24102da0 2024-09-19T04:34:21.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b2a200640 1 -- 172.21.15.107:0/3126637248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b241a6be0 con 0x7f2b24102da0 2024-09-19T04:34:21.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b1be00640 1 -- 172.21.15.107:0/3126637248 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14010690 con 0x7f2b24102da0 2024-09-19T04:34:21.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.383+0000 7f2b2a200640 1 -- 172.21.15.107:0/3126637248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b241a70a0 con 0x7f2b24102da0 2024-09-19T04:34:21.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.387+0000 7f2b2cc8e740 1 Processor -- start 2024-09-19T04:34:21.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b2cc8e740 1 -- start start 2024-09-19T04:34:21.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b1be00640 1 -- 172.21.15.107:0/3126637248 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f2b140046c0 con 0x7f2b24102da0 2024-09-19T04:34:21.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb6b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb6b590 0x55663eb75910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb75e50 0x55663eb78240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55663ead13e0 con 0x55663eb6ac70 2024-09-19T04:34:21.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55663ea78d80 con 0x55663eb6b590 2024-09-19T04:34:21.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55663eacd3a0 con 0x55663eb75e50 2024-09-19T04:34:21.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb6b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b23400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb6b590 0x55663eb75910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb6b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39048/0 (socket says 172.21.15.107:39048) 2024-09-19T04:34:21.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b23e00640 1 -- 172.21.15.107:0/84599304 learned_addr learned my addr 172.21.15.107:0/84599304 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:21.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.388+0000 7f2b22a00640 1 --2- 172.21.15.107:0/84599304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb75e50 0x55663eb78240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.389+0000 7f2b23e00640 1 -- 172.21.15.107:0/84599304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb75e50 msgr2=0x55663eb78240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:21.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.389+0000 7f2b23e00640 1 --2- 172.21.15.107:0/84599304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb75e50 0x55663eb78240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.389+0000 7f2b23e00640 1 -- 172.21.15.107:0/84599304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb6b590 msgr2=0x55663eb75910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:21.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.389+0000 7f2b23e00640 1 --2- 172.21.15.107:0/84599304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb6b590 0x55663eb75910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.389+0000 7f2b23e00640 1 -- 172.21.15.107:0/84599304 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55663ea97ae0 con 0x55663eb6ac70 2024-09-19T04:34:21.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.389+0000 7f2b23e00640 1 --2- 172.21.15.107:0/84599304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb6b050 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009fd0 tx=0x7f2b1c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710944eceaf4efb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:21.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.389+0000 7f2b0fe00640 1 -- 172.21.15.107:0/84599304 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c011700 con 0x55663eb6ac70 2024-09-19T04:34:21.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.389+0000 7f2b0fe00640 1 -- 172.21.15.107:0/84599304 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1c011d80 con 0x55663eb6ac70 2024-09-19T04:34:21.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.389+0000 7f2b0fe00640 1 -- 172.21.15.107:0/84599304 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00e3c0 con 0x55663eb6ac70 2024-09-19T04:34:21.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.390+0000 7f2b1be00640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2b080a33b0 0x7f2b080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.390+0000 7f2b1be00640 1 -- 172.21.15.107:0/3126637248 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f2b140d01d0 con 0x7f2b24102da0 2024-09-19T04:34:21.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.390+0000 7f2b23400640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2b080a33b0 0x7f2b080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.390+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/84599304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 msgr2=0x55663eb6b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:21.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.390+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/84599304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb6b050 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009fd0 tx=0x7f2b1c010860 comp rx=0 tx=0).stop 2024-09-19T04:34:21.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.390+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/84599304 shutdown_connections 2024-09-19T04:34:21.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.390+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/84599304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb75e50 0x55663eb78240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.390+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/84599304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb6b590 0x55663eb75910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.390+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/84599304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb6b050 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.390+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/84599304 >> 172.21.15.107:0/84599304 conn(0x55663eb67200 msgr2=0x55663eb675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:21.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.391+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/84599304 shutdown_connections 2024-09-19T04:34:21.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.391+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/84599304 wait complete. 2024-09-19T04:34:21.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.391+0000 7f2b23400640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2b080a33b0 0x7f2b080a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b140cd9f0 tx=0x7f2b14098040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:21.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.391+0000 7f2b2cc8e740 1 -- start start 2024-09-19T04:34:21.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.391+0000 7f2b2cc8e740 1 Processor -- start 2024-09-19T04:34:21.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6ac70 0x55663eb72cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663eb73230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb75e50 0x55663eb73770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55663eb78a70 con 0x55663eb6b590 2024-09-19T04:34:21.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55663ea8c0f0 con 0x55663eb75e50 2024-09-19T04:34:21.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55663eb78bf0 con 0x55663eb6ac70 2024-09-19T04:34:21.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b2cc8e740 10 client.0 _ll_register_callbacks cb 0x55663ea73bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:21.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b23e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6ac70 0x55663eb72cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b23e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6ac70 0x55663eb72cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46344/0 (socket says 172.21.15.107:46344) 2024-09-19T04:34:21.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b22a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663eb73230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b23e00640 1 -- 172.21.15.107:0/1290187227 learned_addr learned my addr 172.21.15.107:0/1290187227 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:21.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.392+0000 7f2b23e00640 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb75e50 0x55663eb73770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.393+0000 7f2b22a00640 1 -- 172.21.15.107:0/1290187227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6ac70 msgr2=0x55663eb72cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.393+0000 7f2b22a00640 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6ac70 0x55663eb72cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.393+0000 7f2b22a00640 1 -- 172.21.15.107:0/1290187227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb75e50 msgr2=0x55663eb73770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.393+0000 7f2b22a00640 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb75e50 0x55663eb73770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.393+0000 7f2b22a00640 1 -- 172.21.15.107:0/1290187227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55663e8423c0 con 0x55663eb6b590 2024-09-19T04:34:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.393+0000 7f2b22a00640 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663eb73230 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2b10005940 tx=0x7f2b10031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273c97f3d946326e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.393+0000 7f2b0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:34:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.393+0000 7f2b0cc00640 1 -- 172.21.15.107:0/1290187227 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10035040 con 0x55663eb6b590 2024-09-19T04:34:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.393+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1290187227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1c011d80 con 0x55663eb6b590 2024-09-19T04:34:21.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.393+0000 7f2b0cc00640 1 -- 172.21.15.107:0/1290187227 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f2b10004250 con 0x55663eb6b590 2024-09-19T04:34:21.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b0cc00640 1 -- 172.21.15.107:0/1290187227 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2b10004450 con 0x55663eb6b590 2024-09-19T04:34:21.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b2cc8e740 10 client.17727 fetch_fsmap learned FSMap version 55 2024-09-19T04:34:21.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1290187227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55663eb84ab0 con 0x55663eb6b590 2024-09-19T04:34:21.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b0cc00640 1 -- 172.21.15.107:0/1290187227 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1c011d80 con 0x55663eb6b590 2024-09-19T04:34:21.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b0cc00640 1 -- 172.21.15.107:0/1290187227 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7f2b10046750 con 0x55663eb6b590 2024-09-19T04:34:21.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b2cc8e740 10 client.17727 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:34:21.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b2cc8e740 10 client.17727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:34:21.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b2cc8e740 20 client.17727 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:21.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55663eb85f30 0x55663eb88350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b2cc8e740 4 client.17727 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:34:21.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.394+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1290187227 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55663eade950 con 0x55663eb85f30 2024-09-19T04:34:21.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.395+0000 7f2b23e00640 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55663eb85f30 0x55663eb88350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.395+0000 7f2b23e00640 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55663eb85f30 0x55663eb88350 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa347ecdb013e455 server_cookie=a366a5684252bcdc in_seq=0 out_seq=0 2024-09-19T04:34:21.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.395+0000 7f2b0cc00640 10 client.17727 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:34:21.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b0cc00640 1 -- 172.21.15.107:0/1290187227 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55663eade950 con 0x55663eb85f30 2024-09-19T04:34:21.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b0cc00640 10 client.17727 handle_command_reply: tid=0 2024-09-19T04:34:21.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 client.17727 shutdown 2024-09-19T04:34:21.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1290187227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 msgr2=0x55663eb73230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:21.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663eb73230 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2b10005940 tx=0x7f2b10031850 comp rx=0 tx=0).stop 2024-09-19T04:34:21.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1290187227 shutdown_connections 2024-09-19T04:34:21.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55663eb85f30 0x55663eb88350 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb75e50 0x55663eb73770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663eb73230 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/1290187227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6ac70 0x55663eb72cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b0cc00640 0 client.17727 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:34:21.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1290187227 >> 172.21.15.107:0/1290187227 conn(0x55663eb67200 msgr2=0x55663eb66380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:21.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1290187227 shutdown_connections 2024-09-19T04:34:21.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1290187227 wait complete. 2024-09-19T04:34:21.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.396+0000 7f2b2cc8e740 20 client.17727 trim_cache size 0 max 16384 2024-09-19T04:34:21.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.410+0000 7f2b2cc8e740 1 Processor -- start 2024-09-19T04:34:21.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.410+0000 7f2b2cc8e740 1 -- start start 2024-09-19T04:34:21.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6b590 0x55663eb8dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb75e50 0x55663eb8e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241afa30 con 0x55663eb6ac70 2024-09-19T04:34:21.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241b16f0 con 0x55663eb75e50 2024-09-19T04:34:21.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241b01c0 con 0x55663eb6b590 2024-09-19T04:34:21.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b22a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6b590 0x55663eb8dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb735c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39068/0 (socket says 172.21.15.107:39068) 2024-09-19T04:34:21.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b23e00640 1 -- 172.21.15.107:0/1178546712 learned_addr learned my addr 172.21.15.107:0/1178546712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:21.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b23e00640 1 --2- 172.21.15.107:0/1178546712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb75e50 0x55663eb8e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b22a00640 1 -- 172.21.15.107:0/1178546712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb75e50 msgr2=0x55663eb8e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:21.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b22a00640 1 --2- 172.21.15.107:0/1178546712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb75e50 0x55663eb8e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b22a00640 1 -- 172.21.15.107:0/1178546712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 msgr2=0x55663eb735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:21.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b22a00640 1 --2- 172.21.15.107:0/1178546712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b22a00640 1 -- 172.21.15.107:0/1178546712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55663eb62770 con 0x55663eb6b590 2024-09-19T04:34:21.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.411+0000 7f2b23e00640 1 --2- 172.21.15.107:0/1178546712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:34:21.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.412+0000 7f2b22a00640 1 --2- 172.21.15.107:0/1178546712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6b590 0x55663eb8dc70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009900 tx=0x7f2b10031b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=facfc89fd3bdb1a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:21.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.412+0000 7f2b0f400640 1 -- 172.21.15.107:0/1178546712 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10035070 con 0x55663eb6b590 2024-09-19T04:34:21.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.412+0000 7f2b0f400640 1 -- 172.21.15.107:0/1178546712 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b100468f0 con 0x55663eb6b590 2024-09-19T04:34:21.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.412+0000 7f2b0f400640 1 -- 172.21.15.107:0/1178546712 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10053070 con 0x55663eb6b590 2024-09-19T04:34:21.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.413+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1178546712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6b590 msgr2=0x55663eb8dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:21.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.413+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/1178546712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6b590 0x55663eb8dc70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009900 tx=0x7f2b10031b00 comp rx=0 tx=0).stop 2024-09-19T04:34:21.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.413+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1178546712 shutdown_connections 2024-09-19T04:34:21.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.413+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/1178546712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb75e50 0x55663eb8e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.413+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/1178546712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb6b590 0x55663eb8dc70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.413+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/1178546712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6ac70 0x55663eb735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.413+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1178546712 >> 172.21.15.107:0/1178546712 conn(0x55663eb67200 msgr2=0x55663eb74d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:21.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.413+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1178546712 shutdown_connections 2024-09-19T04:34:21.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.413+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/1178546712 wait complete. 2024-09-19T04:34:21.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.413+0000 7f2b2cc8e740 1 -- start start 2024-09-19T04:34:21.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b2cc8e740 1 Processor -- start 2024-09-19T04:34:21.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb6ac70 0x55663ebe8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663ebe9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b2cc8e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb75e50 0x55663eb79420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241b0ff0 con 0x55663eb6b590 2024-09-19T04:34:21.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241b08c0 con 0x55663eb6ac70 2024-09-19T04:34:21.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b2cc8e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241ada30 con 0x55663eb75e50 2024-09-19T04:34:21.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b2cc8e740 10 client.0 _ll_register_callbacks cb 0x7f2b10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:21.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b22a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663ebe9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b22a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663ebe9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39080/0 (socket says 172.21.15.107:39080) 2024-09-19T04:34:21.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b22a00640 1 -- 172.21.15.107:0/3123931802 learned_addr learned my addr 172.21.15.107:0/3123931802 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.414+0000 7f2b23e00640 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb75e50 0x55663eb79420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b23e00640 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb6ac70 0x55663ebe8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b22a00640 1 -- 172.21.15.107:0/3123931802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb75e50 msgr2=0x55663eb79420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b22a00640 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb75e50 0x55663eb79420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b22a00640 1 -- 172.21.15.107:0/3123931802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb6ac70 msgr2=0x55663ebe8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b22a00640 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb6ac70 0x55663ebe8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b22a00640 1 -- 172.21.15.107:0/3123931802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55663eb79b10 con 0x55663eb6b590 2024-09-19T04:34:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b22a00640 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663ebe9470 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009850 tx=0x7f2b10005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fd929ed15429a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:34:21.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b0cc00640 1 -- 172.21.15.107:0/3123931802 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1005b070 con 0x55663eb6b590 2024-09-19T04:34:21.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b0cc00640 1 -- 172.21.15.107:0/3123931802 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f2b10005260 con 0x55663eb6b590 2024-09-19T04:34:21.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.415+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/3123931802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b100468f0 con 0x55663eb6b590 2024-09-19T04:34:21.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b0cc00640 1 -- 172.21.15.107:0/3123931802 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2b10067050 con 0x55663eb6b590 2024-09-19T04:34:21.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b2cc8e740 10 client.17733 fetch_fsmap learned FSMap version 55 2024-09-19T04:34:21.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/3123931802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55663ebf0790 con 0x55663eb6b590 2024-09-19T04:34:21.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b0cc00640 1 -- 172.21.15.107:0/3123931802 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b100468f0 con 0x55663eb6b590 2024-09-19T04:34:21.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b0cc00640 1 -- 172.21.15.107:0/3123931802 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7f2b1003d590 con 0x55663eb6b590 2024-09-19T04:34:21.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b2cc8e740 10 client.17733 fetch_fsmap finished waiting for FSMap version 55 2024-09-19T04:34:21.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b2cc8e740 10 client.17733 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T04:34:21.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b2cc8e740 20 client.17733 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:21.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55663ebf0e20 0x55663ebf3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:21.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b2cc8e740 4 client.17733 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:34:21.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.416+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/3123931802 --> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55663ead8e60 con 0x55663ebf0e20 2024-09-19T04:34:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.417+0000 7f2b23e00640 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55663ebf0e20 0x55663ebf3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.417+0000 7f2b23e00640 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55663ebf0e20 0x55663ebf3240 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38bf271b38451df5 server_cookie=f4d23dfdda82b2af in_seq=0 out_seq=0 2024-09-19T04:34:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.417+0000 7f2b0cc00640 10 client.17733 ms_handle_connect on v2:172.21.15.145:6838/4220252383 2024-09-19T04:34:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.417+0000 7f2b0cc00640 1 -- 172.21.15.107:0/3123931802 <== mds.1 v2:172.21.15.145:6838/4220252383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55663ead8e60 con 0x55663ebf0e20 2024-09-19T04:34:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.417+0000 7f2b0cc00640 10 client.17733 handle_command_reply: tid=0 2024-09-19T04:34:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.417+0000 7f2b2cc8e740 1 client.17733 shutdown 2024-09-19T04:34:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.417+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/3123931802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 msgr2=0x55663ebe9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.417+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663ebe9470 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009850 tx=0x7f2b10005fd0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/3123931802 shutdown_connections 2024-09-19T04:34:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.145:6838/4220252383,v1:172.21.15.145:6839/4220252383] conn(0x55663ebf0e20 0x55663ebf3240 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55663eb75e50 0x55663eb79420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55663eb6b590 0x55663ebe9470 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b2cc8e740 1 --2- 172.21.15.107:0/3123931802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55663eb6ac70 0x55663ebe8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/3123931802 >> 172.21.15.107:0/3123931802 conn(0x55663eb67200 msgr2=0x55663eb65b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b0cc00640 0 client.17733 ms_handle_reset on v2:172.21.15.145:6838/4220252383 2024-09-19T04:34:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/3123931802 shutdown_connections 2024-09-19T04:34:21.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b2cc8e740 1 -- 172.21.15.107:0/3123931802 wait complete. 2024-09-19T04:34:21.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b2cc8e740 20 client.17733 trim_cache size 0 max 16384 2024-09-19T04:34:21.428 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:34:21.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.418+0000 7f2b2a200640 1 -- 172.21.15.107:0/3126637248 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2b080a33b0 msgr2=0x7f2b080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:21.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2b080a33b0 0x7f2b080a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b140cd9f0 tx=0x7f2b14098040 comp rx=0 tx=0).stop 2024-09-19T04:34:21.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 -- 172.21.15.107:0/3126637248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b24102da0 msgr2=0x7f2b24100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:21.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b24102da0 0x7f2b24100a00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400e9f0 tx=0x7f2b1400edf0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 -- 172.21.15.107:0/3126637248 shutdown_connections 2024-09-19T04:34:21.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f2b080a33b0 0x7f2b080a5870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b2410c180 0x7f2b24101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b241036c0 0x7f2b24100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 --2- 172.21.15.107:0/3126637248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b24102da0 0x7f2b24100a00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:21.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 -- 172.21.15.107:0/3126637248 >> 172.21.15.107:0/3126637248 conn(0x7f2b240fc7a0 msgr2=0x7f2b240fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:21.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 -- 172.21.15.107:0/3126637248 shutdown_connections 2024-09-19T04:34:21.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:21.419+0000 7f2b2a200640 1 -- 172.21.15.107:0/3126637248 wait complete. 2024-09-19T04:34:21.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:20 smithi115 ceph-mon[35477]: pgmap v439: 97 pgs: 97 active+clean; 449 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 81 KiB/s wr, 23 op/s 2024-09-19T04:34:22.422 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T04:34:22.799 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:23.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:22 smithi107 ceph-mon[30659]: pgmap v440: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 137 KiB/s wr, 30 op/s 2024-09-19T04:34:23.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:22 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:34:23.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.350+0000 7f3ba7200640 1 -- 172.21.15.107:0/2601238859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba810e260 msgr2=0x7f3ba8110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:23.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.350+0000 7f3ba7200640 1 --2- 172.21.15.107:0/2601238859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba810e260 0x7f3ba8110650 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009ca0 tx=0x7f3ba002a950 comp rx=0 tx=0).stop 2024-09-19T04:34:23.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.350+0000 7f3ba7200640 1 -- 172.21.15.107:0/2601238859 shutdown_connections 2024-09-19T04:34:23.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.350+0000 7f3ba7200640 1 --2- 172.21.15.107:0/2601238859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba810e260 0x7f3ba8110650 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.350+0000 7f3ba7200640 1 --2- 172.21.15.107:0/2601238859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ba8074ee0 0x7f3ba810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.350+0000 7f3ba7200640 1 --2- 172.21.15.107:0/2601238859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ba80745c0 0x7f3ba80749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.350+0000 7f3ba7200640 1 -- 172.21.15.107:0/2601238859 >> 172.21.15.107:0/2601238859 conn(0x7f3ba80fd6e0 msgr2=0x7f3ba80ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:23.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.350+0000 7f3ba7200640 1 -- 172.21.15.107:0/2601238859 shutdown_connections 2024-09-19T04:34:23.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.350+0000 7f3ba7200640 1 -- 172.21.15.107:0/2601238859 wait complete. 2024-09-19T04:34:23.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.351+0000 7f3ba7200640 1 Processor -- start 2024-09-19T04:34:23.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.351+0000 7f3ba7200640 1 -- start start 2024-09-19T04:34:23.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ba80745c0 0x7f3ba819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba7200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ba8074ee0 0x7f3ba819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba810e260 0x7f3ba819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba8113fb0 con 0x7f3ba810e260 2024-09-19T04:34:23.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba7200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba8113e30 con 0x7f3ba8074ee0 2024-09-19T04:34:23.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba8114130 con 0x7f3ba80745c0 2024-09-19T04:34:23.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba810e260 0x7f3ba819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba810e260 0x7f3ba819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39098/0 (socket says 172.21.15.107:39098) 2024-09-19T04:34:23.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba6800640 1 -- 172.21.15.107:0/3398309330 learned_addr learned my addr 172.21.15.107:0/3398309330 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:23.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba5400640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ba8074ee0 0x7f3ba819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ba80745c0 0x7f3ba819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba6800640 1 -- 172.21.15.107:0/3398309330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ba80745c0 msgr2=0x7f3ba819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:23.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba6800640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ba80745c0 0x7f3ba819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba6800640 1 -- 172.21.15.107:0/3398309330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ba8074ee0 msgr2=0x7f3ba819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:23.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba6800640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ba8074ee0 0x7f3ba819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba6800640 1 -- 172.21.15.107:0/3398309330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba819f3e0 con 0x7f3ba810e260 2024-09-19T04:34:23.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.352+0000 7f3ba5400640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ba8074ee0 0x7f3ba819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:23.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.353+0000 7f3ba6800640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba810e260 0x7f3ba819ed00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0004af0 tx=0x7f3ba002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:23.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.353+0000 7f3b9ea00640 1 -- 172.21.15.107:0/3398309330 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0034040 con 0x7f3ba810e260 2024-09-19T04:34:23.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.353+0000 7f3b9ea00640 1 -- 172.21.15.107:0/3398309330 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba0030920 con 0x7f3ba810e260 2024-09-19T04:34:23.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.353+0000 7f3ba7200640 1 -- 172.21.15.107:0/3398309330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba819f670 con 0x7f3ba810e260 2024-09-19T04:34:23.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.353+0000 7f3b9ea00640 1 -- 172.21.15.107:0/3398309330 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00046a0 con 0x7f3ba810e260 2024-09-19T04:34:23.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.353+0000 7f3ba7200640 1 -- 172.21.15.107:0/3398309330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba81a6fb0 con 0x7f3ba810e260 2024-09-19T04:34:23.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.356+0000 7f3b9ea00640 1 -- 172.21.15.107:0/3398309330 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f3ba0037020 con 0x7f3ba810e260 2024-09-19T04:34:23.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.356+0000 7f3bafa78740 1 Processor -- start 2024-09-19T04:34:23.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.356+0000 7f3bafa78740 1 -- start start 2024-09-19T04:34:23.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbde40 0x55fac1cbe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1cc8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1ccb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3bafa78740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fac1c243e0 con 0x55fac1cbe760 2024-09-19T04:34:23.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3bafa78740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fac1bcbd80 con 0x55fac1cc9020 2024-09-19T04:34:23.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3bafa78740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fac1c203a0 con 0x55fac1cbde40 2024-09-19T04:34:23.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1cc8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1cc8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39108/0 (socket says 172.21.15.107:39108) 2024-09-19T04:34:23.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3ba5e00640 1 -- 172.21.15.107:0/3967773869 learned_addr learned my addr 172.21.15.107:0/3967773869 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:23.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3ba5400640 1 --2- 172.21.15.107:0/3967773869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbde40 0x55fac1cbe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.357+0000 7f3ba5400640 1 --2- 172.21.15.107:0/3967773869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1ccb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3ba5e00640 1 -- 172.21.15.107:0/3967773869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbde40 msgr2=0x55fac1cbe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:23.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/3967773869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbde40 0x55fac1cbe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3ba5e00640 1 -- 172.21.15.107:0/3967773869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 msgr2=0x55fac1ccb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:23.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/3967773869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1ccb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3ba5e00640 1 -- 172.21.15.107:0/3967773869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac1beaae0 con 0x55fac1cbe760 2024-09-19T04:34:23.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3b9ea00640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3b8c0a33b0 0x7f3b8c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/3967773869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1cc8ae0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009d20 tx=0x7f3b98011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fa98a88c912109 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:23.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3b9ea00640 1 -- 172.21.15.107:0/3398309330 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f3ba00f0890 con 0x7f3ba810e260 2024-09-19T04:34:23.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3ba5400640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3b8c0a33b0 0x7f3b8c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3b93400640 1 -- 172.21.15.107:0/3967773869 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801e070 con 0x55fac1cbe760 2024-09-19T04:34:23.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3b93400640 1 -- 172.21.15.107:0/3967773869 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b98019b10 con 0x55fac1cbe760 2024-09-19T04:34:23.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.358+0000 7f3b93400640 1 -- 172.21.15.107:0/3967773869 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800ebb0 con 0x55fac1cbe760 2024-09-19T04:34:23.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.359+0000 7f3bafa78740 1 -- 172.21.15.107:0/3967773869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 msgr2=0x55fac1cc8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:23.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.359+0000 7f3bafa78740 1 --2- 172.21.15.107:0/3967773869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1cc8ae0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009d20 tx=0x7f3b98011810 comp rx=0 tx=0).stop 2024-09-19T04:34:23.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.359+0000 7f3ba5400640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3b8c0a33b0 0x7f3b8c0a5870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55fac1cbb750 tx=0x7f3b94005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:23.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.359+0000 7f3bafa78740 1 -- 172.21.15.107:0/3967773869 shutdown_connections 2024-09-19T04:34:23.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.359+0000 7f3bafa78740 1 --2- 172.21.15.107:0/3967773869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1ccb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.359+0000 7f3bafa78740 1 --2- 172.21.15.107:0/3967773869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1cc8ae0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.359+0000 7f3bafa78740 1 --2- 172.21.15.107:0/3967773869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbde40 0x55fac1cbe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.359+0000 7f3bafa78740 1 -- 172.21.15.107:0/3967773869 >> 172.21.15.107:0/3967773869 conn(0x55fac1cba3d0 msgr2=0x55fac1cba7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:23.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.359+0000 7f3bafa78740 1 -- 172.21.15.107:0/3967773869 shutdown_connections 2024-09-19T04:34:23.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.359+0000 7f3bafa78740 1 -- 172.21.15.107:0/3967773869 wait complete. 2024-09-19T04:34:23.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.360+0000 7f3bafa78740 1 -- start start 2024-09-19T04:34:23.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.360+0000 7f3bafa78740 1 Processor -- start 2024-09-19T04:34:23.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1cc5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbe760 0x55fac1cc6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1cc69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3bafa78740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fac1ccbbf0 con 0x55fac1cbde40 2024-09-19T04:34:23.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3bafa78740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fac1bdf0f0 con 0x55fac1cc9020 2024-09-19T04:34:23.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3bafa78740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fac1ccbd70 con 0x55fac1cbe760 2024-09-19T04:34:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3bafa78740 10 client.0 _ll_register_callbacks cb 0x55fac1bc6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1cc69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1cc5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1cc69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:53806/0 (socket says 172.21.15.107:53806) 2024-09-19T04:34:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbe760 0x55fac1cc6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba6800640 1 -- 172.21.15.107:0/1273658402 learned_addr learned my addr 172.21.15.107:0/1273658402 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba6800640 1 -- 172.21.15.107:0/1273658402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbe760 msgr2=0x55fac1cc6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba6800640 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbe760 0x55fac1cc6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba6800640 1 -- 172.21.15.107:0/1273658402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 msgr2=0x55fac1cc5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba6800640 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1cc5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba6800640 1 -- 172.21.15.107:0/1273658402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac19953c0 con 0x55fac1cc9020 2024-09-19T04:34:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.361+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1cc5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.362+0000 7f3ba6800640 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1cc69a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00b9610 tx=0x7f3ba00bc840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c2f534cfd565a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.362+0000 7f3b87e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-19T04:34:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.362+0000 7f3b87e00640 1 -- 172.21.15.107:0/1273658402 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba002c040 con 0x55fac1cc9020 2024-09-19T04:34:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.362+0000 7f3b87e00640 1 -- 172.21.15.107:0/1273658402 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f3ba00c5420 con 0x55fac1cc9020 2024-09-19T04:34:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.362+0000 7f3bafa78740 1 -- 172.21.15.107:0/1273658402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b98019b10 con 0x55fac1cc9020 2024-09-19T04:34:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.362+0000 7f3b87e00640 1 -- 172.21.15.107:0/1273658402 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f3ba00098d0 con 0x55fac1cc9020 2024-09-19T04:34:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.362+0000 7f3bafa78740 10 client.27779 fetch_fsmap learned FSMap version 56 2024-09-19T04:34:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.362+0000 7f3bafa78740 1 -- 172.21.15.107:0/1273658402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55fac1cd7c40 con 0x55fac1cc9020 2024-09-19T04:34:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.362+0000 7f3b87e00640 1 -- 172.21.15.107:0/1273658402 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b98019b10 con 0x55fac1cc9020 2024-09-19T04:34:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.362+0000 7f3b87e00640 1 -- 172.21.15.107:0/1273658402 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8418+0+0 (secure 0 0 0) 0x7f3ba004d030 con 0x55fac1cc9020 2024-09-19T04:34:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.363+0000 7f3bafa78740 10 client.27779 fetch_fsmap finished waiting for FSMap version 56 2024-09-19T04:34:23.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.363+0000 7f3bafa78740 10 client.27779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:34:23.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.363+0000 7f3bafa78740 20 client.27779 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:23.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.363+0000 7f3bafa78740 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fac1cd9270 0x55fac1cdb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.363+0000 7f3bafa78740 4 client.27779 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:34:23.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.363+0000 7f3bafa78740 1 -- 172.21.15.107:0/1273658402 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fac1c31950 con 0x55fac1cd9270 2024-09-19T04:34:23.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.363+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fac1cd9270 0x55fac1cdb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.363+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fac1cd9270 0x55fac1cdb710 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202c40b0f1e5a8d1 server_cookie=5568f19863f83dca in_seq=0 out_seq=0 2024-09-19T04:34:23.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.363+0000 7f3b87e00640 10 client.27779 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:34:23.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3b87e00640 1 -- 172.21.15.107:0/1273658402 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fac1c31950 con 0x55fac1cd9270 2024-09-19T04:34:23.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3b87e00640 10 client.27779 handle_command_reply: tid=0 2024-09-19T04:34:23.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3bafa78740 1 client.27779 shutdown 2024-09-19T04:34:23.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3bafa78740 1 -- 172.21.15.107:0/1273658402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 msgr2=0x55fac1cc69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:23.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3bafa78740 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1cc69a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00b9610 tx=0x7f3ba00bc840 comp rx=0 tx=0).stop 2024-09-19T04:34:23.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3bafa78740 1 -- 172.21.15.107:0/1273658402 shutdown_connections 2024-09-19T04:34:23.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3bafa78740 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fac1cd9270 0x55fac1cdb710 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3bafa78740 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1cc69a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3bafa78740 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbe760 0x55fac1cc6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3bafa78740 1 --2- 172.21.15.107:0/1273658402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1cc5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.364+0000 7f3bafa78740 1 -- 172.21.15.107:0/1273658402 >> 172.21.15.107:0/1273658402 conn(0x55fac1cba3d0 msgr2=0x55fac1cc00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:23.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.365+0000 7f3b87e00640 0 client.27779 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:34:23.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.365+0000 7f3bafa78740 1 -- 172.21.15.107:0/1273658402 shutdown_connections 2024-09-19T04:34:23.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.365+0000 7f3bafa78740 1 -- 172.21.15.107:0/1273658402 wait complete. 2024-09-19T04:34:23.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.365+0000 7f3bafa78740 20 client.27779 trim_cache size 0 max 16384 2024-09-19T04:34:23.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.378+0000 7f3bafa78740 1 Processor -- start 2024-09-19T04:34:23.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.378+0000 7f3bafa78740 1 -- start start 2024-09-19T04:34:23.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cbde40 0x55fac1cc57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1ce0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cc9020 0x55fac1ce1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3bafa78740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba81af7c0 con 0x55fac1cbe760 2024-09-19T04:34:23.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3bafa78740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba81b1510 con 0x55fac1cbde40 2024-09-19T04:34:23.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3bafa78740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba81aff50 con 0x55fac1cc9020 2024-09-19T04:34:23.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1ce0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cbde40 0x55fac1cc57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1ce0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39136/0 (socket says 172.21.15.107:39136) 2024-09-19T04:34:23.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba5400640 1 -- 172.21.15.107:0/830543523 learned_addr learned my addr 172.21.15.107:0/830543523 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:23.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba6800640 1 --2- 172.21.15.107:0/830543523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cc9020 0x55fac1ce1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba5e00640 1 -- 172.21.15.107:0/830543523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cc9020 msgr2=0x55fac1ce1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:23.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/830543523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cc9020 0x55fac1ce1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba5e00640 1 -- 172.21.15.107:0/830543523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 msgr2=0x55fac1ce0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:23.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/830543523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1ce0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba5e00640 1 -- 172.21.15.107:0/830543523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac1cba200 con 0x55fac1cbde40 2024-09-19T04:34:23.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.379+0000 7f3ba6800640 1 --2- 172.21.15.107:0/830543523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cc9020 0x55fac1ce1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:34:23.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.380+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/830543523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cbde40 0x55fac1cc57e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3b98011d10 tx=0x7f3b980051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46de18e122efa354 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:23.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.380+0000 7f3b92a00640 1 -- 172.21.15.107:0/830543523 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9802b070 con 0x55fac1cbde40 2024-09-19T04:34:23.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.380+0000 7f3b92a00640 1 -- 172.21.15.107:0/830543523 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9801f040 con 0x55fac1cbde40 2024-09-19T04:34:23.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.380+0000 7f3b92a00640 1 -- 172.21.15.107:0/830543523 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801e040 con 0x55fac1cbde40 2024-09-19T04:34:23.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.380+0000 7f3bafa78740 1 -- 172.21.15.107:0/830543523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cbde40 msgr2=0x55fac1cc57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:23.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.380+0000 7f3bafa78740 1 --2- 172.21.15.107:0/830543523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cbde40 0x55fac1cc57e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3b98011d10 tx=0x7f3b980051f0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.381+0000 7f3bafa78740 1 -- 172.21.15.107:0/830543523 shutdown_connections 2024-09-19T04:34:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.381+0000 7f3bafa78740 1 --2- 172.21.15.107:0/830543523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cc9020 0x55fac1ce1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.381+0000 7f3bafa78740 1 --2- 172.21.15.107:0/830543523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbe760 0x55fac1ce0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.381+0000 7f3bafa78740 1 --2- 172.21.15.107:0/830543523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cbde40 0x55fac1cc57e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.381+0000 7f3bafa78740 1 -- 172.21.15.107:0/830543523 >> 172.21.15.107:0/830543523 conn(0x55fac1cba3d0 msgr2=0x55fac1cc47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.381+0000 7f3bafa78740 1 -- 172.21.15.107:0/830543523 shutdown_connections 2024-09-19T04:34:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.381+0000 7f3bafa78740 1 -- 172.21.15.107:0/830543523 wait complete. 2024-09-19T04:34:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.381+0000 7f3bafa78740 1 -- start start 2024-09-19T04:34:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.381+0000 7f3bafa78740 1 Processor -- start 2024-09-19T04:34:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1ccbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbe760 0x55fac1ccc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3bafa78740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1ccca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3bafa78740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba81b0d80 con 0x55fac1cbde40 2024-09-19T04:34:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3bafa78740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba81b0650 con 0x55fac1cc9020 2024-09-19T04:34:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3bafa78740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba81ad7c0 con 0x55fac1cbe760 2024-09-19T04:34:23.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3bafa78740 10 client.0 _ll_register_callbacks cb 0x7f3ba0003b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:34:23.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1ccbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1ccbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39142/0 (socket says 172.21.15.107:39142) 2024-09-19T04:34:23.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1ccca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba5e00640 1 -- 172.21.15.107:0/3254267470 learned_addr learned my addr 172.21.15.107:0/3254267470 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:23.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba5400640 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbe760 0x55fac1ccc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba5e00640 1 -- 172.21.15.107:0/3254267470 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbe760 msgr2=0x55fac1ccc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:23.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbe760 0x55fac1ccc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba5e00640 1 -- 172.21.15.107:0/3254267470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 msgr2=0x55fac1ccca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:34:23.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1ccca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba5e00640 1 -- 172.21.15.107:0/3254267470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac1d41fa0 con 0x55fac1cbde40 2024-09-19T04:34:23.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.382+0000 7f3ba6800640 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1ccca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:23.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.383+0000 7f3ba5e00640 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1ccbf80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3b98005db0 tx=0x7f3b9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5467b3d0f6ff4a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:23.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.383+0000 7f3b87e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-19T04:34:23.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.383+0000 7f3b87e00640 1 -- 172.21.15.107:0/3254267470 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98037070 con 0x55fac1cbde40 2024-09-19T04:34:23.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.383+0000 7f3b87e00640 1 -- 172.21.15.107:0/3254267470 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f3b9800e860 con 0x55fac1cbde40 2024-09-19T04:34:23.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.383+0000 7f3bafa78740 1 -- 172.21.15.107:0/3254267470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b9801f040 con 0x55fac1cbde40 2024-09-19T04:34:23.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.383+0000 7f3b87e00640 1 -- 172.21.15.107:0/3254267470 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f3b98043050 con 0x55fac1cbde40 2024-09-19T04:34:23.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.383+0000 7f3bafa78740 10 client.17751 fetch_fsmap learned FSMap version 56 2024-09-19T04:34:23.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.383+0000 7f3bafa78740 1 -- 172.21.15.107:0/3254267470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55fac1d43220 con 0x55fac1cbde40 2024-09-19T04:34:23.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.384+0000 7f3b87e00640 1 -- 172.21.15.107:0/3254267470 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9801f040 con 0x55fac1cbde40 2024-09-19T04:34:23.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.384+0000 7f3b87e00640 1 -- 172.21.15.107:0/3254267470 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) ==== 8418+0+0 (secure 0 0 0) 0x7f3b98025040 con 0x55fac1cbde40 2024-09-19T04:34:23.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.384+0000 7f3bafa78740 10 client.17751 fetch_fsmap finished waiting for FSMap version 56 2024-09-19T04:34:23.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.384+0000 7f3bafa78740 10 client.17751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T04:34:23.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.384+0000 7f3bafa78740 20 client.17751 populate_metadata read hostname 'smithi107' 2024-09-19T04:34:23.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.384+0000 7f3bafa78740 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fac1d43790 0x55fac1d45bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:23.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.384+0000 7f3bafa78740 4 client.17751 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:34:23.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.384+0000 7f3bafa78740 1 -- 172.21.15.107:0/3254267470 --> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fac1c2be60 con 0x55fac1d43790 2024-09-19T04:34:23.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.384+0000 7f3ba5400640 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fac1d43790 0x55fac1d45bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:23.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3ba5400640 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fac1d43790 0x55fac1d45bb0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2cb8747d51af2c server_cookie=4e435e8798da008b in_seq=0 out_seq=0 2024-09-19T04:34:23.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3b87e00640 10 client.17751 ms_handle_connect on v2:172.21.15.115:6838/2907975805 2024-09-19T04:34:23.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3b87e00640 1 -- 172.21.15.107:0/3254267470 <== mds.0 v2:172.21.15.115:6838/2907975805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fac1c2be60 con 0x55fac1d43790 2024-09-19T04:34:23.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3b87e00640 10 client.17751 handle_command_reply: tid=0 2024-09-19T04:34:23.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3bafa78740 1 client.17751 shutdown 2024-09-19T04:34:23.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3bafa78740 1 -- 172.21.15.107:0/3254267470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 msgr2=0x55fac1ccbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:23.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3bafa78740 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1ccbf80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3b98005db0 tx=0x7f3b9800ef60 comp rx=0 tx=0).stop 2024-09-19T04:34:23.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3bafa78740 1 -- 172.21.15.107:0/3254267470 shutdown_connections 2024-09-19T04:34:23.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3bafa78740 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.115:6838/2907975805,v1:172.21.15.115:6839/2907975805] conn(0x55fac1d43790 0x55fac1d45bb0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3bafa78740 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fac1cc9020 0x55fac1ccca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.385+0000 7f3bafa78740 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fac1cbe760 0x55fac1ccc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.386+0000 7f3bafa78740 1 --2- 172.21.15.107:0/3254267470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fac1cbde40 0x55fac1ccbf80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.386+0000 7f3bafa78740 1 -- 172.21.15.107:0/3254267470 >> 172.21.15.107:0/3254267470 conn(0x55fac1cba3d0 msgr2=0x55fac1cc47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:23.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.386+0000 7f3b87e00640 0 client.17751 ms_handle_reset on v2:172.21.15.115:6838/2907975805 2024-09-19T04:34:23.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.386+0000 7f3bafa78740 1 -- 172.21.15.107:0/3254267470 shutdown_connections 2024-09-19T04:34:23.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.386+0000 7f3bafa78740 1 -- 172.21.15.107:0/3254267470 wait complete. 2024-09-19T04:34:23.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.386+0000 7f3bafa78740 20 client.17751 trim_cache size 0 max 16384 2024-09-19T04:34:23.396 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-09-19T04:34:23.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 -- 172.21.15.107:0/3398309330 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3b8c0a33b0 msgr2=0x7f3b8c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:23.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3b8c0a33b0 0x7f3b8c0a5870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55fac1cbb750 tx=0x7f3b94005f90 comp rx=0 tx=0).stop 2024-09-19T04:34:23.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 -- 172.21.15.107:0/3398309330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba810e260 msgr2=0x7f3ba819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:23.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba810e260 0x7f3ba819ed00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0004af0 tx=0x7f3ba002f3a0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 -- 172.21.15.107:0/3398309330 shutdown_connections 2024-09-19T04:34:23.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f3b8c0a33b0 0x7f3b8c0a5870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba810e260 0x7f3ba819ed00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ba8074ee0 0x7f3ba819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 --2- 172.21.15.107:0/3398309330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ba80745c0 0x7f3ba819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:23.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 -- 172.21.15.107:0/3398309330 >> 172.21.15.107:0/3398309330 conn(0x7f3ba80fd6e0 msgr2=0x7f3ba80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:23.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 -- 172.21.15.107:0/3398309330 shutdown_connections 2024-09-19T04:34:23.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:23.387+0000 7f3ba7200640 1 -- 172.21.15.107:0/3398309330 wait complete. 2024-09-19T04:34:23.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:22 smithi115 ceph-mon[35477]: pgmap v440: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 137 KiB/s wr, 30 op/s 2024-09-19T04:34:23.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:22 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:34:23.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:22 smithi145 ceph-mon[35096]: pgmap v440: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 137 KiB/s wr, 30 op/s 2024-09-19T04:34:23.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:22 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:34:24.201 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T04:34:24.207 INFO:tasks.fwd_scrub:done joining 2024-09-19T04:34:24.207 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T04:34:24.218 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:34:24.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.374+0000 7f72e7200640 1 -- 172.21.15.107:0/951801069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e01129d0 msgr2=0x7f72e0114dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:24.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.374+0000 7f72e7200640 1 --2- 172.21.15.107:0/951801069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e01129d0 0x7f72e0114dc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009ca0 tx=0x7f72d802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.374+0000 7f72e7200640 1 -- 172.21.15.107:0/951801069 shutdown_connections 2024-09-19T04:34:24.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.374+0000 7f72e7200640 1 --2- 172.21.15.107:0/951801069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e01129d0 0x7f72e0114dc0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.374+0000 7f72e7200640 1 --2- 172.21.15.107:0/951801069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72e0077040 0x7f72e0112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.374+0000 7f72e7200640 1 --2- 172.21.15.107:0/951801069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e0076720 0x7f72e0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.374+0000 7f72e7200640 1 -- 172.21.15.107:0/951801069 >> 172.21.15.107:0/951801069 conn(0x7f72e00ffa50 msgr2=0x7f72e0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:24.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.374+0000 7f72e7200640 1 -- 172.21.15.107:0/951801069 shutdown_connections 2024-09-19T04:34:24.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.374+0000 7f72e7200640 1 -- 172.21.15.107:0/951801069 wait complete. 2024-09-19T04:34:24.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.375+0000 7f72e7200640 1 Processor -- start 2024-09-19T04:34:24.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.375+0000 7f72e7200640 1 -- start start 2024-09-19T04:34:24.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.375+0000 7f72e7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0076720 0x7f72e01a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72e7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72e0077040 0x7f72e01a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72e7200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e01129d0 0x7f72e019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72e7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72e0118580 con 0x7f72e0076720 2024-09-19T04:34:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72e7200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f72e0118400 con 0x7f72e01129d0 2024-09-19T04:34:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72e7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f72e0118700 con 0x7f72e0077040 2024-09-19T04:34:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0076720 0x7f72e01a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0076720 0x7f72e01a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39166/0 (socket says 172.21.15.107:39166) 2024-09-19T04:34:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72dfe00640 1 -- 172.21.15.107:0/253585175 learned_addr learned my addr 172.21.15.107:0/253585175 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72df400640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72e0077040 0x7f72e01a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72e5200640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e01129d0 0x7f72e019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72dfe00640 1 -- 172.21.15.107:0/253585175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72e0077040 msgr2=0x7f72e01a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72dfe00640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72e0077040 0x7f72e01a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72dfe00640 1 -- 172.21.15.107:0/253585175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e01129d0 msgr2=0x7f72e019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72dfe00640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e01129d0 0x7f72e019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.376+0000 7f72dfe00640 1 -- 172.21.15.107:0/253585175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e019efe0 con 0x7f72e0076720 2024-09-19T04:34:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.377+0000 7f72dfe00640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0076720 0x7f72e01a46b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f72d000e9f0 tx=0x7f72d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.377+0000 7f72dcc00640 1 -- 172.21.15.107:0/253585175 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d000cd80 con 0x7f72e0076720 2024-09-19T04:34:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.377+0000 7f72dcc00640 1 -- 172.21.15.107:0/253585175 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d0010910 con 0x7f72e0076720 2024-09-19T04:34:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.377+0000 7f72e7200640 1 -- 172.21.15.107:0/253585175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e019f2d0 con 0x7f72e0076720 2024-09-19T04:34:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.377+0000 7f72dcc00640 1 -- 172.21.15.107:0/253585175 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0018990 con 0x7f72e0076720 2024-09-19T04:34:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.377+0000 7f72e7200640 1 -- 172.21.15.107:0/253585175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e01ab570 con 0x7f72e0076720 2024-09-19T04:34:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.378+0000 7f72e7200640 1 -- 172.21.15.107:0/253585175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e0076b00 con 0x7f72e0076720 2024-09-19T04:34:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.380+0000 7f72dcc00640 1 -- 172.21.15.107:0/253585175 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f72d0010ac0 con 0x7f72e0076720 2024-09-19T04:34:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.381+0000 7f72dcc00640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f72c40a3440 0x7f72c40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.381+0000 7f72dcc00640 1 -- 172.21.15.107:0/253585175 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f72d00cffd0 con 0x7f72e0076720 2024-09-19T04:34:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.382+0000 7f72df400640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f72c40a3440 0x7f72c40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.382+0000 7f72df400640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f72c40a3440 0x7f72c40a5900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f72e0101f20 tx=0x7f72cc005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:24.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.386+0000 7f72dcc00640 1 -- 172.21.15.107:0/253585175 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f72d009c320 con 0x7f72e0076720 2024-09-19T04:34:24.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.566+0000 7f72e7200640 1 -- 172.21.15.107:0/253585175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72e01ab800 con 0x7f72e0076720 2024-09-19T04:34:24.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.568+0000 7f72dcc00640 1 -- 172.21.15.107:0/253585175 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23211 (secure 0 0 0) 0x7f72d00a0620 con 0x7f72e0076720 2024-09-19T04:34:24.570 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:34:24.570 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":100,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:34:06.674424+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6800/3991722475":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/3866113757":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/1307215535":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/3489252098":"2024-09-20T04:20:19.150743+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/3991722475":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/3889998535":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:34:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.570+0000 7f72e7200640 1 -- 172.21.15.107:0/253585175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f72c40a3440 msgr2=0x7f72c40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.570+0000 7f72e7200640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f72c40a3440 0x7f72c40a5900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f72e0101f20 tx=0x7f72cc005f90 comp rx=0 tx=0).stop 2024-09-19T04:34:24.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.570+0000 7f72e7200640 1 -- 172.21.15.107:0/253585175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0076720 msgr2=0x7f72e01a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:24.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.570+0000 7f72e7200640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0076720 0x7f72e01a46b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f72d000e9f0 tx=0x7f72d000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.570+0000 7f72e7200640 1 -- 172.21.15.107:0/253585175 shutdown_connections 2024-09-19T04:34:24.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.570+0000 7f72e7200640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f72c40a3440 0x7f72c40a5900 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.570+0000 7f72e7200640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e01129d0 0x7f72e019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.570+0000 7f72e7200640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72e0077040 0x7f72e01a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.570+0000 7f72e7200640 1 --2- 172.21.15.107:0/253585175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0076720 0x7f72e01a46b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.570+0000 7f72e7200640 1 -- 172.21.15.107:0/253585175 >> 172.21.15.107:0/253585175 conn(0x7f72e00ffa50 msgr2=0x7f72e0114420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:24.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.571+0000 7f72e7200640 1 -- 172.21.15.107:0/253585175 shutdown_connections 2024-09-19T04:34:24.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.571+0000 7f72e7200640 1 -- 172.21.15.107:0/253585175 wait complete. 2024-09-19T04:34:24.589 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:13:18.308041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T04:18:09.031386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T04:18:11.455271+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:34:24.590 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:34:24.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.748+0000 7fee65600640 1 -- 172.21.15.107:0/3079146817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee60103f10 msgr2=0x7fee601042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:24.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.748+0000 7fee65600640 1 --2- 172.21.15.107:0/3079146817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee60103f10 0x7fee601042f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fee50009900 tx=0x7fee5002a3f0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.748+0000 7fee65600640 1 -- 172.21.15.107:0/3079146817 shutdown_connections 2024-09-19T04:34:24.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.748+0000 7fee65600640 1 --2- 172.21.15.107:0/3079146817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee60109060 0x7fee60110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.748+0000 7fee65600640 1 --2- 172.21.15.107:0/3079146817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee601048c0 0x7fee60108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.748+0000 7fee65600640 1 --2- 172.21.15.107:0/3079146817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee60103f10 0x7fee601042f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.748+0000 7fee65600640 1 -- 172.21.15.107:0/3079146817 >> 172.21.15.107:0/3079146817 conn(0x7fee600ff980 msgr2=0x7fee60101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.748+0000 7fee65600640 1 -- 172.21.15.107:0/3079146817 shutdown_connections 2024-09-19T04:34:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.748+0000 7fee65600640 1 -- 172.21.15.107:0/3079146817 wait complete. 2024-09-19T04:34:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.749+0000 7fee65600640 1 Processor -- start 2024-09-19T04:34:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.749+0000 7fee65600640 1 -- start start 2024-09-19T04:34:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.749+0000 7fee65600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee60103f10 0x7fee6019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.749+0000 7fee65600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee60109060 0x7fee6019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.749+0000 7fee65600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6019ef50 0x7fee6019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.749+0000 7fee65600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee601140a0 con 0x7fee6019ef50 2024-09-19T04:34:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.749+0000 7fee65600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fee60113f20 con 0x7fee60109060 2024-09-19T04:34:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.749+0000 7fee65600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fee60114220 con 0x7fee60103f10 2024-09-19T04:34:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.749+0000 7fee5f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6019ef50 0x7fee6019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee5f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6019ef50 0x7fee6019f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39190/0 (socket says 172.21.15.107:39190) 2024-09-19T04:34:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee5f400640 1 -- 172.21.15.107:0/978949236 learned_addr learned my addr 172.21.15.107:0/978949236 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee5e000640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee60109060 0x7fee6019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee5ea00640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee60103f10 0x7fee6019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee5f400640 1 -- 172.21.15.107:0/978949236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee60103f10 msgr2=0x7fee6019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee5f400640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee60103f10 0x7fee6019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee5f400640 1 -- 172.21.15.107:0/978949236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee60109060 msgr2=0x7fee6019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee5f400640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee60109060 0x7fee6019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee5f400640 1 -- 172.21.15.107:0/978949236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6019fa40 con 0x7fee6019ef50 2024-09-19T04:34:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee5f400640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6019ef50 0x7fee6019f400 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fee5800eaa0 tx=0x7fee5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee57400640 1 -- 172.21.15.107:0/978949236 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5800cdb0 con 0x7fee6019ef50 2024-09-19T04:34:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee65600640 1 -- 172.21.15.107:0/978949236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6019fd30 con 0x7fee6019ef50 2024-09-19T04:34:24.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee57400640 1 -- 172.21.15.107:0/978949236 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee58005020 con 0x7fee6019ef50 2024-09-19T04:34:24.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.750+0000 7fee57400640 1 -- 172.21.15.107:0/978949236 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee580116d0 con 0x7fee6019ef50 2024-09-19T04:34:24.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.751+0000 7fee65600640 1 -- 172.21.15.107:0/978949236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee601a7520 con 0x7fee6019ef50 2024-09-19T04:34:24.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.752+0000 7fee65600640 1 -- 172.21.15.107:0/978949236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee601042f0 con 0x7fee6019ef50 2024-09-19T04:34:24.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.753+0000 7fee57400640 1 -- 172.21.15.107:0/978949236 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fee580054d0 con 0x7fee6019ef50 2024-09-19T04:34:24.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.755+0000 7fee57400640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fee440a33c0 0x7fee440a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:24.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.755+0000 7fee57400640 1 -- 172.21.15.107:0/978949236 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fee580cfeb0 con 0x7fee6019ef50 2024-09-19T04:34:24.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.755+0000 7fee5ea00640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fee440a33c0 0x7fee440a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:24.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.756+0000 7fee5ea00640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fee440a33c0 0x7fee440a5880 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fee50009230 tx=0x7fee500058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:24.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.759+0000 7fee57400640 1 -- 172.21.15.107:0/978949236 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fee58016030 con 0x7fee6019ef50 2024-09-19T04:34:24.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.931+0000 7fee65600640 1 -- 172.21.15.107:0/978949236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee601048c0 con 0x7fee6019ef50 2024-09-19T04:34:24.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.932+0000 7fee57400640 1 -- 172.21.15.107:0/978949236 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fee5809c200 con 0x7fee6019ef50 2024-09-19T04:34:24.934 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-09-19T04:34:24.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 -- 172.21.15.107:0/978949236 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fee440a33c0 msgr2=0x7fee440a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:24.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fee440a33c0 0x7fee440a5880 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fee50009230 tx=0x7fee500058d0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 -- 172.21.15.107:0/978949236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6019ef50 msgr2=0x7fee6019f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:24.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6019ef50 0x7fee6019f400 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fee5800eaa0 tx=0x7fee5800ef60 comp rx=0 tx=0).stop 2024-09-19T04:34:24.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 -- 172.21.15.107:0/978949236 shutdown_connections 2024-09-19T04:34:24.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fee440a33c0 0x7fee440a5880 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6019ef50 0x7fee6019f400 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee60109060 0x7fee6019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 --2- 172.21.15.107:0/978949236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee60103f10 0x7fee6019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:24.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 -- 172.21.15.107:0/978949236 >> 172.21.15.107:0/978949236 conn(0x7fee600ff980 msgr2=0x7fee60100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:24.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 -- 172.21.15.107:0/978949236 shutdown_connections 2024-09-19T04:34:24.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:24.934+0000 7fee65600640 1 -- 172.21.15.107:0/978949236 wait complete. 2024-09-19T04:34:24.952 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T04:34:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.108+0000 7f2100200640 1 -- 172.21.15.107:0/1347584436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8101a80 msgr2=0x7f20f8101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.108+0000 7f2100200640 1 --2- 172.21.15.107:0/1347584436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8101a80 0x7f20f8101ee0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009930 tx=0x7f20e402a420 comp rx=0 tx=0).stop 2024-09-19T04:34:25.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.108+0000 7f2100200640 1 -- 172.21.15.107:0/1347584436 shutdown_connections 2024-09-19T04:34:25.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.108+0000 7f2100200640 1 --2- 172.21.15.107:0/1347584436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20f8102420 0x7f20f810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.108+0000 7f2100200640 1 --2- 172.21.15.107:0/1347584436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8101a80 0x7f20f8101ee0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.108+0000 7f2100200640 1 --2- 172.21.15.107:0/1347584436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20f8107a90 0x7f20f8107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.108+0000 7f2100200640 1 -- 172.21.15.107:0/1347584436 >> 172.21.15.107:0/1347584436 conn(0x7f20f80fd7c0 msgr2=0x7f20f80ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:25.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.108+0000 7f2100200640 1 -- 172.21.15.107:0/1347584436 shutdown_connections 2024-09-19T04:34:25.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.109+0000 7f2100200640 1 -- 172.21.15.107:0/1347584436 wait complete. 2024-09-19T04:34:25.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.109+0000 7f2100200640 1 Processor -- start 2024-09-19T04:34:25.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.109+0000 7f2100200640 1 -- start start 2024-09-19T04:34:25.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f2100200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20f8101a80 0x7f20f819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f2100200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8102420 0x7f20f819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f2100200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20f8107a90 0x7f20f819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f2100200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8111f20 con 0x7f20f8102420 2024-09-19T04:34:25.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f2100200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8111da0 con 0x7f20f8101a80 2024-09-19T04:34:25.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f2100200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f20f81120a0 con 0x7f20f8107a90 2024-09-19T04:34:25.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8102420 0x7f20f819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fd800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20f8101a80 0x7f20f819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8102420 0x7f20f819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39198/0 (socket says 172.21.15.107:39198) 2024-09-19T04:34:25.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fce00640 1 -- 172.21.15.107:0/2990884044 learned_addr learned my addr 172.21.15.107:0/2990884044 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:25.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fe200640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20f8107a90 0x7f20f819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fd800640 1 -- 172.21.15.107:0/2990884044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20f8107a90 msgr2=0x7f20f819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fd800640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20f8107a90 0x7f20f819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fd800640 1 -- 172.21.15.107:0/2990884044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8102420 msgr2=0x7f20f819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fd800640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8102420 0x7f20f819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fd800640 1 -- 172.21.15.107:0/2990884044 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f819f620 con 0x7f20f8101a80 2024-09-19T04:34:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.110+0000 7f20fce00640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8102420 0x7f20f819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:25.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.111+0000 7f20fd800640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20f8101a80 0x7f20f819a630 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00e9f0 tx=0x7f20ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.111+0000 7f20f6000640 1 -- 172.21.15.107:0/2990884044 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00ccc0 con 0x7f20f8101a80 2024-09-19T04:34:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.111+0000 7f2100200640 1 -- 172.21.15.107:0/2990884044 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f819f910 con 0x7f20f8101a80 2024-09-19T04:34:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.111+0000 7f20f6000640 1 -- 172.21.15.107:0/2990884044 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20ec005020 con 0x7f20f8101a80 2024-09-19T04:34:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.111+0000 7f20f6000640 1 -- 172.21.15.107:0/2990884044 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0049e0 con 0x7f20f8101a80 2024-09-19T04:34:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.111+0000 7f2100200640 1 -- 172.21.15.107:0/2990884044 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f81a71a0 con 0x7f20f8101a80 2024-09-19T04:34:25.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.112+0000 7f2100200640 1 -- 172.21.15.107:0/2990884044 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f8101ee0 con 0x7f20f8101a80 2024-09-19T04:34:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.114+0000 7f20f6000640 1 -- 172.21.15.107:0/2990884044 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f20ec005560 con 0x7f20f8101a80 2024-09-19T04:34:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.115+0000 7f20f6000640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20e00a33c0 0x7f20e00a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.116+0000 7f20f6000640 1 -- 172.21.15.107:0/2990884044 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f20ec0cfc30 con 0x7f20f8101a80 2024-09-19T04:34:25.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.116+0000 7f20fce00640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20e00a33c0 0x7f20e00a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.116+0000 7f20fce00640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20e00a33c0 0x7f20e00a5880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20f80ffef0 tx=0x7f20e4031510 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:25.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.119+0000 7f20f6000640 1 -- 172.21.15.107:0/2990884044 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f20ec0051d0 con 0x7f20f8101a80 2024-09-19T04:34:25.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:25 smithi107 ceph-mon[30659]: pgmap v441: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 88 KiB/s wr, 15 op/s 2024-09-19T04:34:25.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:25 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/253585175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:34:25.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:25 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/978949236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:34:25.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.299+0000 7f2100200640 1 -- 172.21.15.107:0/2990884044 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20f81a73d0 con 0x7f20f8101a80 2024-09-19T04:34:25.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.300+0000 7f20f6000640 1 -- 172.21.15.107:0/2990884044 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f20ec09c5d0 con 0x7f20f8101a80 2024-09-19T04:34:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-09-19T04:34:25.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 -- 172.21.15.107:0/2990884044 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20e00a33c0 msgr2=0x7f20e00a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20e00a33c0 0x7f20e00a5880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20f80ffef0 tx=0x7f20e4031510 comp rx=0 tx=0).stop 2024-09-19T04:34:25.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 -- 172.21.15.107:0/2990884044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20f8101a80 msgr2=0x7f20f819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20f8101a80 0x7f20f819a630 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00e9f0 tx=0x7f20ec00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 -- 172.21.15.107:0/2990884044 shutdown_connections 2024-09-19T04:34:25.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f20e00a33c0 0x7f20e00a5880 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20f8107a90 0x7f20f819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8102420 0x7f20f819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 --2- 172.21.15.107:0/2990884044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20f8101a80 0x7f20f819a630 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 -- 172.21.15.107:0/2990884044 >> 172.21.15.107:0/2990884044 conn(0x7f20f80fd7c0 msgr2=0x7f20f810ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:25.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 -- 172.21.15.107:0/2990884044 shutdown_connections 2024-09-19T04:34:25.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.302+0000 7f2100200640 1 -- 172.21.15.107:0/2990884044 wait complete. 2024-09-19T04:34:25.319 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T04:34:25.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:25 smithi115 ceph-mon[35477]: pgmap v441: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 88 KiB/s wr, 15 op/s 2024-09-19T04:34:25.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:25 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/253585175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:34:25.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:25 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/978949236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:34:25.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:25 smithi145 ceph-mon[35096]: pgmap v441: 97 pgs: 97 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 88 KiB/s wr, 15 op/s 2024-09-19T04:34:25.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:25 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/253585175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:34:25.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:25 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/978949236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:34:25.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.477+0000 7f4cabe00640 1 -- 172.21.15.107:0/3875735993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cac1023a0 msgr2=0x7f4cac10e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.477+0000 7f4cabe00640 1 --2- 172.21.15.107:0/3875735993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cac1023a0 0x7f4cac10e8d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009930 tx=0x7f4ca402a420 comp rx=0 tx=0).stop 2024-09-19T04:34:25.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.477+0000 7f4cabe00640 1 -- 172.21.15.107:0/3875735993 shutdown_connections 2024-09-19T04:34:25.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.477+0000 7f4cabe00640 1 --2- 172.21.15.107:0/3875735993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cac1023a0 0x7f4cac10e8d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.477+0000 7f4cabe00640 1 --2- 172.21.15.107:0/3875735993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cac101a00 0x7f4cac101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.477+0000 7f4cabe00640 1 --2- 172.21.15.107:0/3875735993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cac107a10 0x7f4cac107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.477+0000 7f4cabe00640 1 -- 172.21.15.107:0/3875735993 >> 172.21.15.107:0/3875735993 conn(0x7f4cac0fd700 msgr2=0x7f4cac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.477+0000 7f4cabe00640 1 -- 172.21.15.107:0/3875735993 shutdown_connections 2024-09-19T04:34:25.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.477+0000 7f4cabe00640 1 -- 172.21.15.107:0/3875735993 wait complete. 2024-09-19T04:34:25.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.478+0000 7f4cabe00640 1 Processor -- start 2024-09-19T04:34:25.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.478+0000 7f4cabe00640 1 -- start start 2024-09-19T04:34:25.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.478+0000 7f4cabe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cac101a00 0x7f4cac19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.478+0000 7f4cabe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cac1023a0 0x7f4cac19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.478+0000 7f4cabe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cac107a10 0x7f4cac19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.478+0000 7f4cabe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac111e50 con 0x7f4cac101a00 2024-09-19T04:34:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.478+0000 7f4cabe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac111cd0 con 0x7f4cac107a10 2024-09-19T04:34:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.478+0000 7f4cabe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac111fd0 con 0x7f4cac1023a0 2024-09-19T04:34:25.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4caaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cac101a00 0x7f4cac19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4caaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cac101a00 0x7f4cac19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39212/0 (socket says 172.21.15.107:39212) 2024-09-19T04:34:25.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4caaa00640 1 -- 172.21.15.107:0/3674840075 learned_addr learned my addr 172.21.15.107:0/3674840075 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:25.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4caa000640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cac1023a0 0x7f4cac19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4cab400640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cac107a10 0x7f4cac19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4caaa00640 1 -- 172.21.15.107:0/3674840075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cac1023a0 msgr2=0x7f4cac19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4caaa00640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cac1023a0 0x7f4cac19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4caaa00640 1 -- 172.21.15.107:0/3674840075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cac107a10 msgr2=0x7f4cac19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4caaa00640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cac107a10 0x7f4cac19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4caaa00640 1 -- 172.21.15.107:0/3674840075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac19f4c0 con 0x7f4cac101a00 2024-09-19T04:34:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4cab400640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cac107a10 0x7f4cac19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:34:25.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4caaa00640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cac101a00 0x7f4cac19a4d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00ebc0 tx=0x7f4c9c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:25.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.479+0000 7f4ca3400640 1 -- 172.21.15.107:0/3674840075 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c010630 con 0x7f4cac101a00 2024-09-19T04:34:25.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.480+0000 7f4cabe00640 1 -- 172.21.15.107:0/3674840075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac19f7b0 con 0x7f4cac101a00 2024-09-19T04:34:25.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.480+0000 7f4ca3400640 1 -- 172.21.15.107:0/3674840075 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c9c010cb0 con 0x7f4cac101a00 2024-09-19T04:34:25.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.480+0000 7f4ca3400640 1 -- 172.21.15.107:0/3674840075 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c018bd0 con 0x7f4cac101a00 2024-09-19T04:34:25.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.480+0000 7f4cabe00640 1 -- 172.21.15.107:0/3674840075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac1a7040 con 0x7f4cac101a00 2024-09-19T04:34:25.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.481+0000 7f4cabe00640 1 -- 172.21.15.107:0/3674840075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cac101e60 con 0x7f4cac101a00 2024-09-19T04:34:25.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.483+0000 7f4ca3400640 1 -- 172.21.15.107:0/3674840075 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f4c9c0107d0 con 0x7f4cac101a00 2024-09-19T04:34:25.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.484+0000 7f4ca3400640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c900a33c0 0x7f4c900a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.484+0000 7f4ca3400640 1 -- 172.21.15.107:0/3674840075 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f4c9c0d0370 con 0x7f4cac101a00 2024-09-19T04:34:25.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.484+0000 7f4caa000640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c900a33c0 0x7f4c900a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.485+0000 7f4caa000640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c900a33c0 0x7f4c900a5880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4cac10e570 tx=0x7f4c98005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:25.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.488+0000 7f4ca3400640 1 -- 172.21.15.107:0/3674840075 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4c9c09cd10 con 0x7f4cac101a00 2024-09-19T04:34:25.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.665+0000 7f4cabe00640 1 -- 172.21.15.107:0/3674840075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4cac1a7210 con 0x7f4cac101a00 2024-09-19T04:34:25.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.665+0000 7f4ca3400640 1 -- 172.21.15.107:0/3674840075 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f4c9c09c610 con 0x7f4cac101a00 2024-09-19T04:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-09-19T04:34:25.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.667+0000 7f4cabe00640 1 -- 172.21.15.107:0/3674840075 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c900a33c0 msgr2=0x7f4c900a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.667+0000 7f4cabe00640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c900a33c0 0x7f4c900a5880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4cac10e570 tx=0x7f4c98005f90 comp rx=0 tx=0).stop 2024-09-19T04:34:25.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.667+0000 7f4cabe00640 1 -- 172.21.15.107:0/3674840075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cac101a00 msgr2=0x7f4cac19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.667+0000 7f4cabe00640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cac101a00 0x7f4cac19a4d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00ebc0 tx=0x7f4c9c00c6a0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.668+0000 7f4cabe00640 1 -- 172.21.15.107:0/3674840075 shutdown_connections 2024-09-19T04:34:25.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.668+0000 7f4cabe00640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f4c900a33c0 0x7f4c900a5880 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.668+0000 7f4cabe00640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cac107a10 0x7f4cac19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.668+0000 7f4cabe00640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cac1023a0 0x7f4cac19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.668+0000 7f4cabe00640 1 --2- 172.21.15.107:0/3674840075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cac101a00 0x7f4cac19a4d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.668+0000 7f4cabe00640 1 -- 172.21.15.107:0/3674840075 >> 172.21.15.107:0/3674840075 conn(0x7f4cac0fd700 msgr2=0x7f4cac0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:25.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.668+0000 7f4cabe00640 1 -- 172.21.15.107:0/3674840075 shutdown_connections 2024-09-19T04:34:25.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.668+0000 7f4cabe00640 1 -- 172.21.15.107:0/3674840075 wait complete. 2024-09-19T04:34:25.686 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T04:34:25.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.839+0000 7fb196000640 1 -- 172.21.15.107:0/2161247575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb190104470 msgr2=0x7fb1901109a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.839+0000 7fb196000640 1 --2- 172.21.15.107:0/2161247575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb190104470 0x7fb1901109a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb188009be0 tx=0x7fb18802a6d0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.839+0000 7fb196000640 1 -- 172.21.15.107:0/2161247575 shutdown_connections 2024-09-19T04:34:25.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.839+0000 7fb196000640 1 --2- 172.21.15.107:0/2161247575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb190104470 0x7fb1901109a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.839+0000 7fb196000640 1 --2- 172.21.15.107:0/2161247575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb190103ad0 0x7fb190103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.839+0000 7fb196000640 1 --2- 172.21.15.107:0/2161247575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb190109b70 0x7fb190109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.839+0000 7fb196000640 1 -- 172.21.15.107:0/2161247575 >> 172.21.15.107:0/2161247575 conn(0x7fb1900ff7d0 msgr2=0x7fb190101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.839+0000 7fb196000640 1 -- 172.21.15.107:0/2161247575 shutdown_connections 2024-09-19T04:34:25.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.839+0000 7fb196000640 1 -- 172.21.15.107:0/2161247575 wait complete. 2024-09-19T04:34:25.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.840+0000 7fb196000640 1 Processor -- start 2024-09-19T04:34:25.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.840+0000 7fb196000640 1 -- start start 2024-09-19T04:34:25.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.840+0000 7fb196000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb190103ad0 0x7fb19019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.840+0000 7fb196000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb190104470 0x7fb19019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.840+0000 7fb196000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb190109b70 0x7fb19019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.840+0000 7fb196000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb190113ed0 con 0x7fb190103ad0 2024-09-19T04:34:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.840+0000 7fb196000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb190113d50 con 0x7fb190104470 2024-09-19T04:34:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.840+0000 7fb196000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb190114050 con 0x7fb190109b70 2024-09-19T04:34:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.840+0000 7fb194c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb190103ad0 0x7fb19019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb194c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb190103ad0 0x7fb19019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39236/0 (socket says 172.21.15.107:39236) 2024-09-19T04:34:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb194c00640 1 -- 172.21.15.107:0/4116789313 learned_addr learned my addr 172.21.15.107:0/4116789313 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb18fe00640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb190104470 0x7fb19019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb195600640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb190109b70 0x7fb19019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb194c00640 1 -- 172.21.15.107:0/4116789313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb190109b70 msgr2=0x7fb19019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb194c00640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb190109b70 0x7fb19019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb194c00640 1 -- 172.21.15.107:0/4116789313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb190104470 msgr2=0x7fb19019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb194c00640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb190104470 0x7fb19019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb194c00640 1 -- 172.21.15.107:0/4116789313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19019f390 con 0x7fb190103ad0 2024-09-19T04:34:25.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb18fe00640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb190104470 0x7fb19019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:25.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb194c00640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb190103ad0 0x7fb19019a3a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb18400e9f0 tx=0x7fb18400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:25.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.841+0000 7fb18d600640 1 -- 172.21.15.107:0/4116789313 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18400ccc0 con 0x7fb190103ad0 2024-09-19T04:34:25.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.842+0000 7fb18d600640 1 -- 172.21.15.107:0/4116789313 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18400ce60 con 0x7fb190103ad0 2024-09-19T04:34:25.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.842+0000 7fb196000640 1 -- 172.21.15.107:0/4116789313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19019f680 con 0x7fb190103ad0 2024-09-19T04:34:25.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.842+0000 7fb18d600640 1 -- 172.21.15.107:0/4116789313 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184010690 con 0x7fb190103ad0 2024-09-19T04:34:25.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.842+0000 7fb196000640 1 -- 172.21.15.107:0/4116789313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1901a6f10 con 0x7fb190103ad0 2024-09-19T04:34:25.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.843+0000 7fb196000640 1 -- 172.21.15.107:0/4116789313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb190103f30 con 0x7fb190103ad0 2024-09-19T04:34:25.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.845+0000 7fb18d600640 1 -- 172.21.15.107:0/4116789313 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fb1840046c0 con 0x7fb190103ad0 2024-09-19T04:34:25.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.846+0000 7fb18d600640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb1780a3440 0x7fb1780a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.846+0000 7fb18d600640 1 -- 172.21.15.107:0/4116789313 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fb1840cfff0 con 0x7fb190103ad0 2024-09-19T04:34:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.846+0000 7fb18fe00640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb1780a3440 0x7fb1780a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.847+0000 7fb18fe00640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb1780a3440 0x7fb1780a5900 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb1901091a0 tx=0x7fb17c009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:25.850+0000 7fb18d600640 1 -- 172.21.15.107:0/4116789313 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb18409c9f0 con 0x7fb190103ad0 2024-09-19T04:34:26.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.142+0000 7fb196000640 1 -- 172.21.15.107:0/4116789313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fb19019b6f0 con 0x7fb190103ad0 2024-09-19T04:34:26.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.158+0000 7fb18d600640 1 -- 172.21.15.107:0/4116789313 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+742879 (secure 0 0 0) 0x7fb18409c2f0 con 0x7fb190103ad0 2024-09-19T04:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 27, 2024-09-19T04:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout: "active_gid": 15114, 2024-09-19T04:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout: "active_name": "x", 2024-09-19T04:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addrs": { 2024-09-19T04:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-19T04:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-19T04:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-09-19T04:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1724794486 2024-09-19T04:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-19T04:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-09-19T04:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1724794486 2024-09-19T04:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addr": "172.21.15.107:6801/1724794486", 2024-09-19T04:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout: "active_change": "2024-09-19T04:20:19.151027+0000", 2024-09-19T04:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T04:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout: "available": true, 2024-09-19T04:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout: "standbys": [ 2024-09-19T04:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 25120, 2024-09-19T04:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "z", 2024-09-19T04:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540701547738038271, 2024-09-19T04:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-09-19T04:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-09-19T04:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.181 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.181 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.181 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.181 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.181 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.181 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.182 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.182 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.182 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.182 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.182 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.182 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.182 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.183 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.183 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.183 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.183 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.183 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.183 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.183 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.183 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.184 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.184 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.184 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.184 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.184 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.184 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.184 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.185 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.185 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.185 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.185 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.185 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.185 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.185 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.186 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.186 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.186 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.186 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.186 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.186 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.186 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.186 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.186 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.187 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.187 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-09-19T04:34:26.187 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-09-19T04:34:26.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.187 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.187 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.188 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.188 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.188 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.188 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:34:26.188 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.188 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.188 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.189 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-09-19T04:34:26.189 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-09-19T04:34:26.189 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.189 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.189 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.189 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-09-19T04:34:26.189 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.190 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.190 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.190 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-09-19T04:34:26.190 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.190 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.190 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.190 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-09-19T04:34:26.191 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-09-19T04:34:26.191 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.191 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.191 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.191 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.191 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.191 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.192 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.192 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-09-19T04:34:26.192 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.192 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.192 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.192 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-09-19T04:34:26.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-09-19T04:34:26.193 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.193 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.193 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.193 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.193 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.194 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-09-19T04:34:26.194 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.194 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.194 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.194 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-09-19T04:34:26.194 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-09-19T04:34:26.194 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.194 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.195 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.195 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-09-19T04:34:26.195 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.195 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.195 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.195 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-09-19T04:34:26.195 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.195 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.196 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.196 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-09-19T04:34:26.196 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-09-19T04:34:26.196 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.196 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.196 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.196 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.197 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.197 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.197 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.197 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:34:26.197 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.197 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.198 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.198 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-09-19T04:34:26.198 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-09-19T04:34:26.198 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.198 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.198 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.198 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.199 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.199 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.199 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.199 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:34:26.199 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.199 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.199 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.200 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-09-19T04:34:26.200 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-09-19T04:34:26.200 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.200 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.200 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.200 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.200 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.200 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.201 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.201 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-09-19T04:34:26.201 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.201 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.201 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.201 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.201 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-09-19T04:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-09-19T04:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-09-19T04:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.204 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.204 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.204 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-09-19T04:34:26.204 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-09-19T04:34:26.204 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.204 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.204 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.205 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-09-19T04:34:26.205 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.205 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.205 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.205 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:34:26.205 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:34:26.205 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.205 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-09-19T04:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-09-19T04:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.207 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:26.207 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-09-19T04:34:26.207 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-09-19T04:34:26.207 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.207 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:34:26.207 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:34:26.207 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.208 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.208 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-09-19T04:34:26.208 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:34:26.208 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:26.208 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.208 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.208 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-09-19T04:34:26.208 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-09-19T04:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-09-19T04:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-09-19T04:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:34:26.211 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:34:26.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.211 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.211 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-09-19T04:34:26.211 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-09-19T04:34:26.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:26.211 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.212 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.212 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-09-19T04:34:26.212 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-09-19T04:34:26.212 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-09-19T04:34:26.212 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.212 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:34:26.212 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:34:26.212 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.213 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.213 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-09-19T04:34:26.213 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-09-19T04:34:26.213 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.213 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.213 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-09-19T04:34:26.214 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.214 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.214 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.214 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:34:26.214 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:34:26.214 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.214 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.215 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-09-19T04:34:26.215 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-09-19T04:34:26.215 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:26.215 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.215 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.215 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:26.215 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-09-19T04:34:26.216 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-09-19T04:34:26.216 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.216 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:34:26.216 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:34:26.216 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.216 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.217 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.217 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.217 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.217 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.217 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.217 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.218 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.218 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.218 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.218 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.218 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.218 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.218 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.218 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.218 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.219 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.219 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.219 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.219 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.219 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.220 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.220 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.220 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.220 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.220 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.220 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.220 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-09-19T04:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-09-19T04:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-09-19T04:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-09-19T04:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-09-19T04:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-09-19T04:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-09-19T04:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-09-19T04:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-09-19T04:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-09-19T04:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-09-19T04:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-09-19T04:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-09-19T04:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-09-19T04:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-09-19T04:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-09-19T04:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-09-19T04:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-09-19T04:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-09-19T04:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-09-19T04:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-09-19T04:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-09-19T04:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-09-19T04:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-09-19T04:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-09-19T04:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-09-19T04:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-09-19T04:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-09-19T04:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-09-19T04:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-09-19T04:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-09-19T04:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-09-19T04:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-09-19T04:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-09-19T04:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-09-19T04:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-09-19T04:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-09-19T04:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-09-19T04:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-09-19T04:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-09-19T04:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-09-19T04:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-09-19T04:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-09-19T04:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-09-19T04:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-09-19T04:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-09-19T04:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-09-19T04:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-09-19T04:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-09-19T04:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-09-19T04:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-09-19T04:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-09-19T04:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-09-19T04:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-09-19T04:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-09-19T04:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-09-19T04:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-09-19T04:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-09-19T04:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-09-19T04:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-09-19T04:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-09-19T04:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-09-19T04:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-09-19T04:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-09-19T04:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-09-19T04:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-09-19T04:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-09-19T04:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-09-19T04:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-09-19T04:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-09-19T04:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-09-19T04:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-09-19T04:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-09-19T04:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_samba": { 2024-09-19T04:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_samba", 2024-09-19T04:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-09-19T04:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-09-19T04:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-09-19T04:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-09-19T04:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-09-19T04:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-09-19T04:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-09-19T04:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-09-19T04:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-09-19T04:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-09-19T04:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-09-19T04:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-09-19T04:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-09-19T04:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-09-19T04:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-09-19T04:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "grafana_dashboards_path": { 2024-09-19T04:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-09-19T04:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-09-19T04:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-09-19T04:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-09-19T04:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-09-19T04:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-09-19T04:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-09-19T04:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-09-19T04:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-09-19T04:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-09-19T04:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-09-19T04:34:26.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.319 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.319 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:26.319 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.319 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.319 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.319 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:34:26.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.320 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.320 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-09-19T04:34:26.320 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-09-19T04:34:26.320 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.320 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.320 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.321 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:26.321 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.321 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.321 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.321 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:34:26.321 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.321 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.321 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.322 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-09-19T04:34:26.322 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-09-19T04:34:26.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.322 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.322 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.322 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.322 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.322 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.323 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.323 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-09-19T04:34:26.323 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.323 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.323 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.323 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-09-19T04:34:26.323 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-09-19T04:34:26.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.324 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.324 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-09-19T04:34:26.324 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.324 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.324 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.325 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-09-19T04:34:26.325 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-09-19T04:34:26.325 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.325 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:34:26.325 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.326 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.326 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-09-19T04:34:26.326 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-09-19T04:34:26.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.326 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.326 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-09-19T04:34:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:34:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-09-19T04:34:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:34:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:34:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.329 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.329 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.329 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:34:26.329 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.329 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.329 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.329 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-09-19T04:34:26.330 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-09-19T04:34:26.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.330 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.330 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.330 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.330 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.331 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.331 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:34:26.331 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.331 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.331 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-09-19T04:34:26.332 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-09-19T04:34:26.332 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.332 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.332 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.332 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.332 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.332 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.333 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.333 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:34:26.333 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.333 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.333 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.333 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-09-19T04:34:26.333 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-09-19T04:34:26.334 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.334 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.334 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.334 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.334 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.334 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.334 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.334 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:34:26.335 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.335 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.335 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-09-19T04:34:26.335 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-09-19T04:34:26.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.336 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.336 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.336 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.336 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.336 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.336 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.336 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:34:26.336 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.337 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.337 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.337 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-09-19T04:34:26.337 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:34:26.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.337 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.338 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.338 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.338 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.338 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.338 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:34:26.338 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.338 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.339 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.339 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-09-19T04:34:26.339 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-09-19T04:34:26.339 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.339 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.339 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.339 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-09-19T04:34:26.340 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.340 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.340 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.340 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:34:26.340 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.340 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.340 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.341 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-09-19T04:34:26.341 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-09-19T04:34:26.341 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.341 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.341 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.341 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.341 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.342 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.342 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.342 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:34:26.342 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.342 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.342 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.342 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-09-19T04:34:26.343 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-09-19T04:34:26.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.343 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.343 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.343 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.343 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.343 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.344 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:34:26.344 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.344 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.344 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.344 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-09-19T04:34:26.344 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-09-19T04:34:26.345 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.345 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.345 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.345 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.345 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.345 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.345 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.345 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:34:26.346 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.346 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.346 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.346 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-09-19T04:34:26.346 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:34:26.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.347 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.347 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.347 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.347 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.347 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.347 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.347 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:34:26.347 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.348 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.348 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.348 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-09-19T04:34:26.348 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:34:26.348 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.348 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.348 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.349 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.349 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.349 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.349 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.349 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:34:26.349 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.349 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.349 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.350 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-09-19T04:34:26.350 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:34:26.350 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.350 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.350 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.350 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.350 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.350 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.351 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.351 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:34:26.351 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.351 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.351 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.351 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.351 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.352 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-09-19T04:34:26.352 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.352 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.352 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.352 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.352 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.353 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.353 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.353 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.353 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.353 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.353 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.353 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.353 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.354 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.354 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.354 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.354 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.354 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.354 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.354 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.355 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.355 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.355 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.355 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.355 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.355 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.355 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.355 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.356 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.356 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.356 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.356 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.356 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.356 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.356 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.357 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.357 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.357 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.357 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.357 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.357 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.357 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.357 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.358 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.358 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.358 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.358 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.358 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.358 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.358 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.358 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.359 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.359 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.359 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.359 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.359 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.359 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.359 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.360 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.360 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.360 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.360 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.360 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.360 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.360 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.360 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.360 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.361 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.361 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-09-19T04:34:26.361 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-09-19T04:34:26.361 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.361 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.361 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.361 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-09-19T04:34:26.362 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.362 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.362 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.362 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:34:26.362 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.362 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.362 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.363 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-09-19T04:34:26.363 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-09-19T04:34:26.363 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.363 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.363 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.363 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-09-19T04:34:26.363 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.364 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.364 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.364 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:34:26.364 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.364 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.364 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.365 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-09-19T04:34:26.365 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.365 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.365 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.365 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:34:26.365 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:34:26.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.366 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.366 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.366 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:26.366 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.366 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.366 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.366 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.366 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.367 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.367 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.367 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:34:26.367 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:34:26.367 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.367 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.368 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.368 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.368 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.368 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.368 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.368 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.368 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.369 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.369 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.369 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:34:26.369 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:34:26.369 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.369 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.369 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.370 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.370 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.370 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.370 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.370 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.370 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.370 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.370 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.371 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:34:26.371 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:34:26.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.371 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.371 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.371 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.371 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.371 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.371 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.372 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.372 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.372 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.372 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.372 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:34:26.372 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:34:26.373 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.373 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.373 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.373 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.373 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.373 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.373 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.374 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.374 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.374 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.374 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.374 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:34:26.374 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:34:26.374 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.375 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.375 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.375 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.375 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.375 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.375 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.375 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.375 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.376 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.376 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.376 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:34:26.376 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:34:26.376 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.376 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.377 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.377 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.377 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.377 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.377 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.377 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.377 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.378 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.378 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.378 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:34:26.378 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:34:26.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.378 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.378 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.378 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.379 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.379 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.379 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.379 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.379 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.379 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.379 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.380 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:34:26.380 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:34:26.380 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.380 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.380 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.380 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.380 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.380 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.380 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.381 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.381 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.381 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.381 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.381 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:34:26.381 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:34:26.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.382 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.382 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.382 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.382 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.382 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.382 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.382 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.382 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.383 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.383 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.383 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:34:26.383 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:34:26.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.383 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.383 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.384 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.384 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.384 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.384 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.384 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.384 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.384 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.385 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.385 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:34:26.385 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:34:26.385 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.385 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.385 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.385 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.386 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.386 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.386 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.386 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.386 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.386 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.386 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.387 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:34:26.387 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:34:26.387 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.387 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.387 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.387 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.387 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.387 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.388 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.388 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.388 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.388 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.388 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.388 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:34:26.389 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:34:26.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.389 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.389 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.389 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.389 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.389 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.389 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.390 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.390 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.390 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.390 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:34:26.390 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:34:26.390 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.390 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.391 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.391 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-09-19T04:34:26.391 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.391 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.391 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.391 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.391 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.391 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.392 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.392 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:34:26.392 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:34:26.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.392 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.392 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.392 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.393 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.393 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.393 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.393 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.393 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.393 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.393 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.394 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-09-19T04:34:26.394 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:34:26.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.394 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.394 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.394 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.395 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.395 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.395 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.395 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.395 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.395 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.395 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.396 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:34:26.396 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:34:26.396 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.396 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.396 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.396 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-09-19T04:34:26.396 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.397 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.397 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.397 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.397 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.397 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.397 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.397 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:34:26.398 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:34:26.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.398 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.398 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.398 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.398 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.398 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.399 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.399 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.399 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.399 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.399 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:34:26.399 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:34:26.400 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.400 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.400 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.400 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.400 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.400 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.400 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.400 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.401 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.401 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.401 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.401 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:34:26.401 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:34:26.401 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.401 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.402 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.402 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.402 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.402 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.402 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.402 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.402 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.403 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.403 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:34:26.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:34:26.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.403 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.403 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.404 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.404 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.404 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.404 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.404 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.404 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.404 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.405 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:34:26.405 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:34:26.405 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.405 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.405 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.405 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.405 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.405 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.406 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.406 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.406 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.406 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.406 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.406 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:34:26.406 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:34:26.407 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.407 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.407 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.407 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.407 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.407 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.407 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.407 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.408 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.408 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.408 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.408 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:34:26.408 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:34:26.408 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.409 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.409 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.409 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.409 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.409 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.409 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.409 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.409 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.410 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.410 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.410 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:34:26.410 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:34:26.410 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.410 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.410 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.411 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.411 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.411 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.411 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.411 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.411 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.411 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.412 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.412 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:34:26.412 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:34:26.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.412 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.412 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.412 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.413 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.413 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.413 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.413 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.413 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.413 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.413 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:34:26.414 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:34:26.414 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.414 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.414 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.414 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.414 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.414 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.414 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.415 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.415 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.415 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.415 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.415 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:34:26.415 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:34:26.415 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.416 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.416 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.416 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.416 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.416 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.416 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.416 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.416 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.416 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.417 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.417 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:34:26.417 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:34:26.417 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.417 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.417 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.418 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.418 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.418 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.418 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.418 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.418 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.418 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.418 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:34:26.419 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:34:26.419 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.419 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.419 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.419 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.419 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.419 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.419 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.420 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.420 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.420 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.420 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.420 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:34:26.420 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:34:26.420 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.421 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.421 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.421 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.421 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.421 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.421 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.421 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.421 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.422 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.422 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.422 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:34:26.422 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:34:26.422 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.422 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.422 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.423 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:34:26.423 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.423 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.423 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.423 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.423 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.423 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.423 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.424 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:34:26.424 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:34:26.424 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.424 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.424 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.424 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:26.424 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.425 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.425 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.425 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.425 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.425 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.425 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.426 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:34:26.426 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:34:26.426 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.426 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.426 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.426 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-09-19T04:34:26.426 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.426 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.427 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.427 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.427 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.427 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.427 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.427 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:34:26.427 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:34:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-09-19T04:34:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.429 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.429 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.429 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.429 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.429 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:34:26.429 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:34:26.429 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.430 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.430 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.430 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.430 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.430 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.430 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.430 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.430 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.431 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.431 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.431 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:34:26.431 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:34:26.431 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.431 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.432 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.432 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-09-19T04:34:26.432 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.432 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.432 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.432 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.432 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.432 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.433 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.433 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:34:26.433 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:34:26.433 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.433 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.433 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.433 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.434 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.434 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.434 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.434 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.434 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.434 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.434 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.435 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:34:26.435 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:34:26.435 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.435 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.435 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.435 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.436 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.436 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.436 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.436 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.436 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.436 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.436 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.437 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:34:26.437 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:34:26.437 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.437 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.437 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.437 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:26.437 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.437 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.438 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.438 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.438 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.438 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.438 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.438 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:34:26.439 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:34:26.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.439 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.439 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.439 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:26.439 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.439 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.439 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.440 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.440 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.440 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.440 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.440 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:34:26.440 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:34:26.440 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.441 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.441 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.441 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:26.441 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.441 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.441 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.441 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.442 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.442 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.442 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.442 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-09-19T04:34:26.442 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-09-19T04:34:26.442 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.442 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.443 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.443 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.443 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.443 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.443 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.443 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.443 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.443 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.444 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.444 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-09-19T04:34:26.444 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-09-19T04:34:26.444 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.444 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.444 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.445 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.445 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.445 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.445 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.445 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.445 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.445 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.446 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.446 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-09-19T04:34:26.446 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-09-19T04:34:26.446 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.446 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.446 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.446 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.447 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.447 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.447 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.447 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:34:26.447 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.447 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.447 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.448 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-09-19T04:34:26.448 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-09-19T04:34:26.448 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.448 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.448 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.448 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-09-19T04:34:26.449 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.449 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.449 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.449 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.449 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.449 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.449 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.450 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-09-19T04:34:26.450 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-09-19T04:34:26.450 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.450 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.450 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.450 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.450 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.451 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.451 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.451 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.451 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.451 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.451 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.451 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.452 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.452 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.452 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.452 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.452 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.452 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.452 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.452 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.453 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.453 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.453 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.453 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.453 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.453 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.453 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.453 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.454 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.454 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.454 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.454 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.454 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.454 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.455 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.455 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.455 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.455 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.455 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.455 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.455 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.455 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.456 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.456 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.456 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.456 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.456 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.456 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.457 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.457 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.457 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.457 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.457 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.457 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.457 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.458 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.458 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.458 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.458 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.458 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.458 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.458 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.458 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.459 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.459 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.459 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.459 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.459 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.459 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.460 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.460 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.460 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.460 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.460 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.460 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.461 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-09-19T04:34:26.461 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-09-19T04:34:26.461 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.461 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.461 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.461 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.461 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.462 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.462 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.462 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-09-19T04:34:26.462 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.462 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.462 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.462 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:34:26.463 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:34:26.463 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.463 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.463 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.463 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.463 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.463 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.464 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.464 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.464 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.464 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.464 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-09-19T04:34:26.464 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-09-19T04:34:26.465 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.465 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.465 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.465 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-09-19T04:34:26.465 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.465 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.465 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.465 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.466 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.466 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.466 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.466 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-09-19T04:34:26.466 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-09-19T04:34:26.466 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.466 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.466 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.467 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-09-19T04:34:26.467 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.467 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.467 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.467 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.467 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.467 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.467 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.468 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-09-19T04:34:26.468 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-09-19T04:34:26.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.468 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.468 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.468 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.468 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.469 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.469 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.469 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.469 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.469 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.469 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.469 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-09-19T04:34:26.470 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-09-19T04:34:26.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.470 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.470 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.470 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-09-19T04:34:26.470 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.470 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.470 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.470 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.471 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.471 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.471 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.471 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-09-19T04:34:26.471 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-09-19T04:34:26.471 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.472 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.472 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-09-19T04:34:26.472 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.472 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.472 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.472 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.472 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-09-19T04:34:26.473 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.473 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.473 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.473 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.473 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.473 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-09-19T04:34:26.473 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-09-19T04:34:26.473 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.474 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.474 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.474 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-09-19T04:34:26.474 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-09-19T04:34:26.474 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-09-19T04:34:26.474 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.474 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-09-19T04:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-09-19T04:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.476 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.476 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.476 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.476 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.476 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.476 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.476 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.476 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.477 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.477 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.477 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.477 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-09-19T04:34:26.477 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.478 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.478 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.478 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-09-19T04:34:26.478 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-09-19T04:34:26.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.478 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.478 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.478 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.479 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.479 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.479 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.479 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-09-19T04:34:26.479 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.479 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.479 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.480 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.480 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.480 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.480 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.480 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.480 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.480 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.481 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.481 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.481 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.481 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.481 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.481 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.481 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.481 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.482 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.482 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.482 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.482 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.482 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.482 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.483 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.483 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.483 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.483 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.483 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.483 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.483 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.483 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.484 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.484 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.484 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.484 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.484 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.484 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.484 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.485 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.485 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.485 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.485 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.485 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.485 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.485 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.486 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.486 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.486 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.486 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.486 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.486 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.486 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.486 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.487 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.487 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.487 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.487 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.487 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.487 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.487 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.488 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.488 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.488 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.488 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.488 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.488 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.488 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-09-19T04:34:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-09-19T04:34:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-09-19T04:34:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.490 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.490 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.490 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:34:26.490 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.490 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.490 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.490 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-09-19T04:34:26.491 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-09-19T04:34:26.491 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.491 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.491 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.491 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-09-19T04:34:26.491 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.491 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.491 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.492 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:34:26.492 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.492 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.492 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.492 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-09-19T04:34:26.492 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-09-19T04:34:26.492 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.493 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.493 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-09-19T04:34:26.493 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.493 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.493 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.493 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:34:26.494 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.494 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.494 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.494 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-09-19T04:34:26.494 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-09-19T04:34:26.494 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.494 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.495 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.495 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-09-19T04:34:26.495 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.495 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.495 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.495 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:34:26.496 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.496 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.496 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.496 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-09-19T04:34:26.496 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-09-19T04:34:26.496 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.496 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.497 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.497 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.497 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.497 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.497 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.497 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:34:26.497 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.498 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.498 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.498 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:26.498 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:26.498 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.498 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.498 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.499 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:26.499 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.499 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.499 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.499 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:34:26.499 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.499 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.500 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.500 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-09-19T04:34:26.500 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-09-19T04:34:26.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.500 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.500 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.500 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-09-19T04:34:26.501 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.501 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.501 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.501 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:34:26.501 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.501 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.501 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.502 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.502 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.502 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-09-19T04:34:26.502 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.502 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.502 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.503 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.503 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.503 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.503 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.503 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.503 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.503 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.503 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.504 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.504 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.504 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.504 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.504 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.504 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.504 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.504 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.505 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.505 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.505 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.505 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.505 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.505 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.506 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.506 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.506 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.506 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.506 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.506 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.506 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.506 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.507 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.507 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.507 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.507 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.507 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.507 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.507 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.508 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.508 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.508 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.508 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.508 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.508 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.508 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.508 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.509 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.509 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.509 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.509 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.509 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.509 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.509 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.510 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.510 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.510 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.510 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.510 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.511 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.511 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.511 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.511 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.511 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.511 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.511 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.512 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-09-19T04:34:26.512 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-09-19T04:34:26.512 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.512 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.512 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.512 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-09-19T04:34:26.512 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.513 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.513 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.513 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.513 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.513 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.513 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.513 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-09-19T04:34:26.514 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-09-19T04:34:26.514 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.514 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.514 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.514 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-09-19T04:34:26.514 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.514 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.514 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.515 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.515 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.515 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.515 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.515 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:26.515 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:26.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.516 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.516 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.516 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:26.516 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.516 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.516 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.516 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.516 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.517 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.517 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.517 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.517 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-09-19T04:34:26.517 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-09-19T04:34:26.518 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:34:26.518 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.518 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-09-19T04:34:26.518 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-09-19T04:34:26.518 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.518 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.518 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.519 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-09-19T04:34:26.519 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.519 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.519 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.519 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:34:26.519 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.519 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.519 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.520 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-09-19T04:34:26.520 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-09-19T04:34:26.520 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.520 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.520 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-09-19T04:34:26.521 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.521 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.521 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.521 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:34:26.521 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.521 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.522 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-09-19T04:34:26.522 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-09-19T04:34:26.522 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.522 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.522 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.522 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.522 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.523 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.523 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.523 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:34:26.523 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.523 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.523 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.523 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:26.524 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:26.524 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.524 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.524 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.524 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-09-19T04:34:26.524 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-09-19T04:34:26.524 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.525 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.525 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:34:26.525 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.525 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.525 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.525 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.526 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.526 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.526 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.526 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.526 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.526 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.526 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.527 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.527 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.527 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.527 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.527 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.527 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.527 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.527 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.528 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.528 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.528 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.528 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.528 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.528 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.529 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.529 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.529 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.529 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.529 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.529 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.529 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.529 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.530 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.530 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.530 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.530 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.530 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.530 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.531 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.531 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.531 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.531 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.531 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.531 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.531 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.531 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.532 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.532 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.532 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.532 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.532 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.532 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.532 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.533 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.533 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.533 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.533 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.533 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.533 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.534 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.534 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.534 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.534 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.534 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.534 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.534 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.535 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-09-19T04:34:26.535 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-09-19T04:34:26.535 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.535 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.535 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.535 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.535 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.535 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.536 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.536 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:34:26.536 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.536 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.536 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.536 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-09-19T04:34:26.537 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-09-19T04:34:26.537 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.537 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.537 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.537 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-09-19T04:34:26.537 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.537 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.537 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.538 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-09-19T04:34:26.538 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.538 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.538 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-09-19T04:34:26.538 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-09-19T04:34:26.538 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.538 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.539 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.539 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-09-19T04:34:26.539 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.539 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.539 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.539 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:34:26.539 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.539 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.540 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.540 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-09-19T04:34:26.540 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-09-19T04:34:26.540 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.540 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.540 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.540 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:26.541 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:26.541 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-09-19T04:34:26.541 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.541 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:34:26.541 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.541 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.542 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-09-19T04:34:26.542 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-09-19T04:34:26.542 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.542 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.542 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.542 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.542 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.542 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.543 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.543 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:34:26.543 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.543 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.543 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-09-19T04:34:26.544 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-09-19T04:34:26.544 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.544 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.544 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.544 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-09-19T04:34:26.544 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.544 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.544 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.545 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:34:26.545 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.545 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.545 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.545 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.545 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.546 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-09-19T04:34:26.546 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.546 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.546 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.546 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.546 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.547 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.547 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.547 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.547 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.547 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.547 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.547 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.547 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.547 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.548 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.548 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.548 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.548 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.548 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.548 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.548 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.548 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.549 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.549 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.549 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.549 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.549 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.549 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.550 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.550 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.550 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.550 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.550 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.550 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.550 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.551 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.551 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.551 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.551 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.551 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.551 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.552 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.552 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.552 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.552 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.552 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.552 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.552 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.552 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.553 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.553 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.553 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.553 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.553 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.553 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.554 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.554 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.554 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.554 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.554 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.554 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.554 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.554 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.555 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.555 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.555 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.555 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.555 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.555 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.556 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-09-19T04:34:26.556 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.556 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.556 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.556 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.556 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.556 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.557 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.557 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.557 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.557 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.557 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.557 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.557 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.558 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.558 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.558 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.558 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.558 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.558 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.558 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.558 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.559 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.559 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.559 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.559 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.559 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.560 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.560 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.560 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.560 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.560 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.560 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.560 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.560 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.561 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.561 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.561 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.561 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.561 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.561 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.561 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.562 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.562 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.562 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.562 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.562 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.562 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.562 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.562 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.563 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.563 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.563 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.563 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.563 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.563 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.563 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.564 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.564 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.564 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.564 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.564 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.564 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.564 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.564 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.565 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.565 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.565 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.565 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.565 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.565 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-09-19T04:34:26.566 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.566 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.566 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.566 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-09-19T04:34:26.566 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:34:26.566 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.566 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.566 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.567 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-09-19T04:34:26.567 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.567 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.567 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.567 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:34:26.567 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.567 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.567 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.568 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-09-19T04:34:26.568 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-09-19T04:34:26.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.568 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.568 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:26.568 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-09-19T04:34:26.569 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.569 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.569 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:34:26.569 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.569 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.569 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.569 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.570 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.570 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.570 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.570 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.570 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.570 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.570 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.570 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.570 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.571 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.571 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.571 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.571 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.571 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.571 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.571 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.571 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.572 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.572 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.572 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.572 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.572 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.572 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.572 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.572 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.573 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.573 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.573 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.573 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.573 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.573 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.573 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.574 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.574 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.574 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.574 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.574 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.574 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.574 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.574 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.575 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.575 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.575 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.575 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.575 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.575 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.575 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.575 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.575 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.576 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.576 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.576 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.576 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.576 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.577 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.577 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.577 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.577 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.577 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.577 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.577 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.578 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.578 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.578 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.578 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-09-19T04:34:26.578 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.579 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.579 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.579 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-09-19T04:34:26.579 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-09-19T04:34:26.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.579 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.579 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.579 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-09-19T04:34:26.579 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.580 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.580 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.580 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:34:26.580 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:34:26.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.580 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.580 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.581 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.581 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.581 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.581 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.581 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.581 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.581 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.581 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.582 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.582 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.582 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.582 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.582 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.582 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.582 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.582 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.582 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.583 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.583 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.583 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.583 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.583 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.583 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.584 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.584 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.584 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.584 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.584 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.584 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.585 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.585 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.585 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.585 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.585 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.586 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.586 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.586 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.586 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.586 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.586 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.586 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.586 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.588 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.588 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.588 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.588 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.588 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.589 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.589 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.589 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.589 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.589 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.589 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.590 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-09-19T04:34:26.590 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-09-19T04:34:26.590 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.590 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.590 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.590 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.590 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.590 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.591 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.591 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:34:26.591 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:34:26.591 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.591 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.591 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-09-19T04:34:26.591 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-09-19T04:34:26.592 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.592 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.592 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.592 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-09-19T04:34:26.592 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.592 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.592 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.593 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:34:26.593 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.593 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.593 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-09-19T04:34:26.593 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-09-19T04:34:26.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.594 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.594 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.594 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-09-19T04:34:26.594 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.594 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.594 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.594 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:34:26.594 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.594 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.595 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.595 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-09-19T04:34:26.595 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-09-19T04:34:26.595 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.595 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.595 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.595 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.595 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.596 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.596 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.596 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:34:26.596 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.596 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.596 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.596 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-09-19T04:34:26.597 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-09-19T04:34:26.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.597 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.597 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.597 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-09-19T04:34:26.597 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.597 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.597 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.598 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:34:26.598 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:34:26.598 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.598 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.598 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.598 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.599 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-09-19T04:34:26.599 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.599 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.599 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.599 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.599 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.599 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.599 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.601 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.601 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.601 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.601 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.601 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.601 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.601 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.601 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.602 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.602 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.602 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.602 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.602 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.602 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.603 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.603 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.603 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.603 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.603 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.603 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.603 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.604 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.604 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.604 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.604 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.604 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.604 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.604 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.605 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.605 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.605 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.605 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.605 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.605 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.605 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.605 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.606 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.606 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.606 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.606 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.606 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.606 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.606 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.606 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.607 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.607 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.607 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.607 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.607 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.607 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.607 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.608 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.608 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.608 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-09-19T04:34:26.609 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.609 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.609 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.609 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.612 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.612 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.612 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.612 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.613 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.613 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.613 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.613 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.613 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.613 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.613 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.613 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.614 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.614 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.614 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.614 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.614 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.614 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.614 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.615 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.615 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.615 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.615 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.615 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.615 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.615 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.616 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.616 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.616 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.616 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.616 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.616 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.616 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.617 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.617 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.617 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.617 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.617 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.617 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.617 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.617 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.618 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.618 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.618 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.618 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.618 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.618 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.618 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.618 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.619 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.619 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.619 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.619 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.619 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.619 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.619 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.620 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.620 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.620 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.620 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.620 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.620 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.620 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.621 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.621 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.621 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.621 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.621 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-09-19T04:34:26.622 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.622 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.622 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.622 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.622 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.622 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.622 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.622 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.623 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.623 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.623 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.623 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.623 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.623 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.623 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.623 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.624 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.624 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.624 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.624 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.624 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.624 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.624 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.625 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.625 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.625 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.625 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.625 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.625 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.626 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.626 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.626 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.626 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.626 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.626 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.626 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.627 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.627 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.627 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.627 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.627 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.627 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.628 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.628 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.628 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.628 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.628 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.628 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.628 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.628 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.629 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.629 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.629 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.629 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.629 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.629 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.630 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.630 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.630 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.630 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.630 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.630 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.630 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.630 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.630 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.631 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.631 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.631 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.631 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-09-19T04:34:26.631 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.631 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.632 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.632 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-09-19T04:34:26.632 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-09-19T04:34:26.632 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.632 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.632 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.632 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.632 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.633 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.633 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.633 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:34:26.633 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.633 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.633 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.633 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.634 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.634 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.634 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.634 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.634 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.634 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.634 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.635 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.635 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.635 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.635 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.635 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.635 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.635 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.635 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.636 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.636 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.636 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.636 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.636 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.636 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.636 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.637 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.637 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.637 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.637 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.637 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.638 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.638 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.638 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.638 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.638 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.639 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.639 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.639 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.639 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.639 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.639 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.639 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.639 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.640 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.640 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.640 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.640 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.640 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.640 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.640 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.640 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.641 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.641 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.641 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.641 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.641 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.641 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.641 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.641 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.642 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.642 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.642 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.642 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.642 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-09-19T04:34:26.642 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-09-19T04:34:26.642 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.643 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.643 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.643 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.643 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.643 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.643 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.643 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-09-19T04:34:26.643 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-09-19T04:34:26.644 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-09-19T04:34:26.644 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.644 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-09-19T04:34:26.644 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.644 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.644 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.644 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.645 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.645 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-09-19T04:34:26.645 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.645 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.645 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.646 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.646 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.646 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.646 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.646 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.646 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.646 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.647 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.647 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.647 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.647 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.647 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.647 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.647 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.647 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.648 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.648 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.648 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.648 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.648 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.648 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.648 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.649 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.649 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.649 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.649 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.649 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.649 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.649 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.649 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.650 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.650 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.650 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.650 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.650 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.650 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.650 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.650 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.651 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.651 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.651 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.651 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.651 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.651 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.651 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.652 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.652 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.652 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.652 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.652 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.652 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.652 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.653 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.653 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.653 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.653 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.653 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.653 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.653 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.653 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.654 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.654 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.654 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.654 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.654 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.654 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.654 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.655 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-09-19T04:34:26.655 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.655 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.655 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.655 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.655 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.655 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.656 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.656 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.656 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.656 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.656 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.656 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.656 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.656 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.657 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.657 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.657 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.657 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.657 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.657 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.657 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.658 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.658 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.658 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.658 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.658 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.658 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.659 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.659 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.659 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.659 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.659 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.659 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.659 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.659 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.660 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.660 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.660 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.660 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.660 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.660 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.660 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.661 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.661 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.661 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.661 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.661 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.661 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.661 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.662 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.662 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.662 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.662 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.662 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.662 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.662 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.663 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.663 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.663 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.663 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.663 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.663 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.663 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.663 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.664 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.664 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.664 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.664 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.664 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.664 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.664 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.665 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-09-19T04:34:26.665 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.665 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.665 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.665 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.665 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.666 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.666 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.666 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.666 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.666 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.666 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.666 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.666 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.666 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.667 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.667 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.667 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.667 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.667 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.667 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.667 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.667 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.668 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.668 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.668 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.668 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.668 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.668 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.668 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.669 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.669 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.669 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.669 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.669 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.669 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.669 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.670 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.670 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.670 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.670 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.670 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.670 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.671 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.671 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.671 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.671 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.671 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.671 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.671 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.671 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.671 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.672 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.672 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.672 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.672 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.672 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.672 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.673 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.673 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.673 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.673 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.673 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.673 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.673 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.674 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.674 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.674 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.674 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.674 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:26.674 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:26.674 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.674 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.675 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.675 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:26.675 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.675 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.675 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.675 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.675 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.675 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.676 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.676 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-09-19T04:34:26.676 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-09-19T04:34:26.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:26.676 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.676 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-09-19T04:34:26.677 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-09-19T04:34:26.677 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.677 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.677 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-09-19T04:34:26.677 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:34:26.677 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.677 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.678 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.678 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.678 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-09-19T04:34:26.678 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.678 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.678 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.679 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-09-19T04:34:26.679 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:34:26.679 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.679 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.679 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.679 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.679 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.680 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.680 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.680 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:34:26.680 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.680 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.680 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.680 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-09-19T04:34:26.681 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-09-19T04:34:26.681 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.681 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.681 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.681 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.681 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.681 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.682 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.682 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.682 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.682 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.682 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.682 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.682 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.683 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.683 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.683 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.683 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.683 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.683 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.683 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.683 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.684 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.684 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.684 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.684 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.684 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.684 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.684 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.684 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.685 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.685 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.685 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.685 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.685 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.686 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.686 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.686 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.686 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.686 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.686 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.686 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.686 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.687 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.687 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.687 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.687 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.687 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.687 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.688 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.688 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.688 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.688 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.688 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.688 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.688 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.688 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.689 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.689 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.689 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.689 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.689 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.689 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.689 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.690 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.690 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.690 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.690 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.690 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.690 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.690 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.691 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.691 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.691 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.691 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.691 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.691 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.691 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-09-19T04:34:26.692 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-09-19T04:34:26.692 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.692 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.692 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.692 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-09-19T04:34:26.692 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.692 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.692 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.693 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:34:26.693 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.693 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.693 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.693 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:26.693 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:26.694 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.694 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.694 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.694 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:26.694 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.694 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.694 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.694 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:34:26.695 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.695 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.695 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.695 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.695 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.695 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-09-19T04:34:26.696 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.696 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.696 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.696 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-09-19T04:34:26.696 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-09-19T04:34:26.696 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.696 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.696 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.697 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.697 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.697 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.697 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.697 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.697 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.697 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.697 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.698 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-09-19T04:34:26.698 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-09-19T04:34:26.698 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.698 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.698 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.698 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.698 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.698 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.699 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.699 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:34:26.699 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:34:26.699 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.699 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.699 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.699 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.699 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.700 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.700 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.700 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.700 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.700 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.700 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.700 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.700 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.701 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.701 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.701 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.701 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.701 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.701 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.701 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.701 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.702 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.702 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.702 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.702 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.702 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.702 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.702 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.703 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.703 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.703 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.703 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.703 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.703 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.703 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.703 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.704 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.704 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.704 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.704 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.704 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.704 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.705 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.705 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.705 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.705 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.705 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.705 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.705 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.705 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.706 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.706 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.706 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.706 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.706 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.706 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.706 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.707 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.707 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.707 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.707 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.707 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.707 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.707 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.707 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.708 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.708 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-09-19T04:34:26.708 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-09-19T04:34:26.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.708 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.708 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.709 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.709 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.709 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.709 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.709 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.709 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.710 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:34:26.710 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:34:26.710 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.710 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.710 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.710 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-09-19T04:34:26.710 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.710 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.711 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.711 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.711 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.711 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.711 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.711 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-09-19T04:34:26.711 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-09-19T04:34:26.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:26.712 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.712 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-09-19T04:34:26.712 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.712 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.712 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.713 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.713 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.713 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.713 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.713 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-09-19T04:34:26.713 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-09-19T04:34:26.713 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.714 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.714 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.714 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-09-19T04:34:26.714 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.714 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.714 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.714 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:34:26.714 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.715 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.715 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.715 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-09-19T04:34:26.715 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-09-19T04:34:26.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.715 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.715 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.715 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-09-19T04:34:26.716 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.716 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.716 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.716 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:34:26.716 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.716 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.716 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.717 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-09-19T04:34:26.717 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-09-19T04:34:26.717 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.717 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.717 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.717 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-09-19T04:34:26.717 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.718 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.718 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.718 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.718 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.718 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.718 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.718 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-09-19T04:34:26.719 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-09-19T04:34:26.719 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.719 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.719 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.719 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-09-19T04:34:26.719 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.719 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.719 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.720 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-09-19T04:34:26.720 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-09-19T04:34:26.720 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.720 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.720 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.720 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.720 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.721 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-09-19T04:34:26.721 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-09-19T04:34:26.721 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.721 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.721 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.721 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-09-19T04:34:26.721 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-09-19T04:34:26.722 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-09-19T04:34:26.722 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.722 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.722 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.722 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.722 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.722 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.722 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.723 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-09-19T04:34:26.723 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.723 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.723 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.723 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.723 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.724 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.724 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.724 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.724 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.724 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.724 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.724 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.725 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.725 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.725 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.725 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.725 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.725 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.725 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.725 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.726 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.726 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.726 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.726 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.726 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.726 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.727 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.727 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.727 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.727 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.727 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.727 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.727 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.727 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.728 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.728 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.728 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.728 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.728 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.729 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.729 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.729 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.729 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.729 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.729 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.729 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.729 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.730 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.730 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.730 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.730 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.730 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.730 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.731 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.731 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.731 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.731 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.731 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.731 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.731 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.731 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.731 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.732 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.732 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.732 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.732 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-09-19T04:34:26.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:34:26.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.732 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.733 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.733 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:26.733 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.733 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.733 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.733 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.733 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.733 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.734 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.734 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:34:26.734 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:34:26.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.734 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.734 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.734 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.735 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.735 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.735 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.735 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.735 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.735 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-09-19T04:34:26.735 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-09-19T04:34:26.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.736 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.736 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.736 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.736 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.736 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.736 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.736 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.737 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.737 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.737 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.737 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.737 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.738 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-09-19T04:34:26.738 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.738 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.738 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.738 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-09-19T04:34:26.738 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-09-19T04:34:26.738 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.738 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.739 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.739 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.739 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.739 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.739 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.739 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.739 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.739 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.739 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.740 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-09-19T04:34:26.740 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-09-19T04:34:26.740 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.740 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.740 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.740 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.740 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.740 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.741 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.741 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.741 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.741 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.742 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.742 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.742 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.742 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.742 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.742 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.742 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.743 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.743 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.743 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.743 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.743 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.743 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.743 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.743 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.744 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.744 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.744 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.745 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.745 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.745 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.746 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.746 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.746 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.746 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.746 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.746 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.747 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.747 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.748 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.748 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.748 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.748 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.748 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.748 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.748 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.748 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.749 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.749 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.749 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.749 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.749 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.749 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.750 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.750 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.750 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.750 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.750 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.750 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.750 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.751 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-09-19T04:34:26.751 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-09-19T04:34:26.751 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.751 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.751 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.751 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.752 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.752 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.752 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.752 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.752 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.752 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.752 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-09-19T04:34:26.753 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-09-19T04:34:26.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.753 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.753 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.753 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.754 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.754 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.754 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.754 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.754 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.754 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.754 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.755 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-09-19T04:34:26.755 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.755 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.755 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.755 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.755 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.755 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.756 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.756 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.756 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.756 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.756 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.756 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.756 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.756 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.757 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.757 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.757 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.757 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.757 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.757 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.757 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.758 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.758 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.758 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.758 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.758 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.758 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.758 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.759 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.759 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.759 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.759 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.759 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.760 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.760 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.760 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.760 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.760 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.760 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.760 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.761 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.761 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.761 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.761 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.761 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.761 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.761 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.761 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.762 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.762 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.762 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.762 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.762 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.763 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.763 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.763 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.763 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.763 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.763 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.764 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.764 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.764 INFO:teuthology.orchestra.run.smithi107.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:34:26.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:34:26.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.764 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.764 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.764 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:26.765 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.765 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.765 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:34:26.765 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.765 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.765 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.766 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.766 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-09-19T04:34:26.766 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.766 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.766 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.767 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.767 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.767 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.767 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.767 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.767 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.767 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.767 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.768 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.768 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.768 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.768 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.768 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.768 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.768 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.769 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.769 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.769 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.770 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.770 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.770 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.770 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.770 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.770 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.771 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.771 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.771 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.771 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.771 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.771 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.771 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.772 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.772 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.772 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.772 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.772 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.773 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.773 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.773 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.773 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.773 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.773 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.773 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.773 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.774 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.774 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.774 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.774 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.774 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.774 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.774 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.775 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.775 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.775 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.775 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.775 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.775 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.775 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.775 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.775 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.776 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-09-19T04:34:26.776 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-09-19T04:34:26.776 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.776 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.776 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.776 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-09-19T04:34:26.776 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.777 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.777 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.777 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:34:26.777 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.777 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.777 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.777 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.778 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.778 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-09-19T04:34:26.778 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.778 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.778 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.779 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.779 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.779 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.779 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.779 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.779 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.780 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.780 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.780 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.780 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.780 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.780 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.780 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.780 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.781 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.781 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.781 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.781 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.781 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.782 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.782 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.782 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.782 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.782 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.782 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.782 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.782 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.783 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.783 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.783 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.783 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.783 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.783 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.783 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.784 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.784 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.784 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.784 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.784 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.784 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.784 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.785 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.785 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.785 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.785 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.785 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.785 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.785 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.785 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.786 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.786 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.786 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.786 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.786 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.786 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.787 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.787 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.787 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.787 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.787 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.787 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.787 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.787 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.788 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-09-19T04:34:26.788 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-09-19T04:34:26.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.788 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.788 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.788 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.788 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.789 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.789 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.789 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.789 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.789 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.789 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.790 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-09-19T04:34:26.790 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-09-19T04:34:26.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.790 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.790 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.790 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-09-19T04:34:26.790 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.790 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.791 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.791 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.791 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.791 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.791 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.791 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-09-19T04:34:26.791 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-09-19T04:34:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-09-19T04:34:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-09-19T04:34:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.795 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.795 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.795 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.795 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-09-19T04:34:26.795 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-09-19T04:34:26.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:26.795 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-09-19T04:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-09-19T04:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-09-19T04:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-09-19T04:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-09-19T04:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-09-19T04:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-09-19T04:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-09-19T04:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-09-19T04:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-09-19T04:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-09-19T04:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-09-19T04:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-09-19T04:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-09-19T04:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-09-19T04:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-09-19T04:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-09-19T04:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-09-19T04:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-09-19T04:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-09-19T04:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-09-19T04:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-09-19T04:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-09-19T04:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-09-19T04:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-09-19T04:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-09-19T04:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-09-19T04:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-09-19T04:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-09-19T04:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-09-19T04:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-09-19T04:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-09-19T04:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-09-19T04:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-09-19T04:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-09-19T04:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-09-19T04:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-09-19T04:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-09-19T04:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-09-19T04:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-09-19T04:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-09-19T04:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-09-19T04:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-09-19T04:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-09-19T04:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-09-19T04:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-09-19T04:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-09-19T04:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-09-19T04:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-09-19T04:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-09-19T04:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-09-19T04:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-09-19T04:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-09-19T04:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-09-19T04:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-09-19T04:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-09-19T04:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-09-19T04:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-09-19T04:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-09-19T04:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-09-19T04:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-09-19T04:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-09-19T04:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.905 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.905 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.905 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.905 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.905 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.905 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.906 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.906 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.906 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.906 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.906 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.906 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.906 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.907 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.907 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.907 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.907 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.907 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.907 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.907 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.907 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.908 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.908 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.908 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.908 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.908 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.908 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.909 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.909 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.909 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.909 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.909 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.909 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.910 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.910 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.910 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.910 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.910 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.911 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.911 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.911 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.911 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.911 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.911 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.911 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.911 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.912 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.912 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.912 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.912 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.912 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.913 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.913 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.913 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.913 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.914 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.914 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-09-19T04:34:26.914 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-09-19T04:34:26.914 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.914 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.914 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.915 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-09-19T04:34:26.915 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.915 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.915 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:34:26.915 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.915 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.915 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.916 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-09-19T04:34:26.916 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-09-19T04:34:26.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.916 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.916 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.916 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.916 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.916 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.917 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-09-19T04:34:26.917 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.917 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.917 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.917 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-09-19T04:34:26.917 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:34:26.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.918 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.918 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.918 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:26.918 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.918 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.918 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.919 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:34:26.919 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.919 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.919 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.919 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:34:26.919 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:34:26.919 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.920 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.920 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.920 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.920 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.920 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.920 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.920 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:34:26.920 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.921 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.921 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.921 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.921 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.921 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-09-19T04:34:26.921 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.922 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.922 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.922 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-09-19T04:34:26.922 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-09-19T04:34:26.922 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:26.922 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.923 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-09-19T04:34:26.923 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.923 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.923 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.923 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.923 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.923 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.923 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.924 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-09-19T04:34:26.924 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-09-19T04:34:26.924 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.924 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.924 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.924 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.925 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.925 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.925 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.925 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.925 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.925 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.925 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.926 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:26.926 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:26.926 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.926 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.926 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.926 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:26.926 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.926 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.927 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.927 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.927 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.927 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.927 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.927 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.928 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.928 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.928 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.929 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.929 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.929 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.929 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.929 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.929 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.929 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.929 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.929 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.930 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.930 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.930 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.930 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.931 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.931 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.931 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.931 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.931 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.931 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.931 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.932 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.932 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.932 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.932 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.932 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.932 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.933 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.933 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.933 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.933 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.933 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.933 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.933 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.933 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.933 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.934 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.934 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.934 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.934 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.934 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.934 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.935 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.935 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.935 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.935 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.935 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.935 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.935 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.936 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.936 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-09-19T04:34:26.936 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-09-19T04:34:26.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.936 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.936 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.936 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.937 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.937 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.937 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.937 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.937 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.937 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.938 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-09-19T04:34:26.938 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-09-19T04:34:26.938 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.938 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.938 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.938 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-09-19T04:34:26.938 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.938 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.939 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.939 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.939 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.939 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.939 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.939 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-09-19T04:34:26.939 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-09-19T04:34:26.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.940 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:26.940 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:34:26.940 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.940 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.940 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.941 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.941 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.941 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.941 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.941 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.941 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.941 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:26.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.942 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 25124, 2024-09-19T04:34:26.942 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "y", 2024-09-19T04:34:26.942 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540701547738038271, 2024-09-19T04:34:26.942 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-09-19T04:34:26.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.942 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-09-19T04:34:26.943 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.943 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.943 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.943 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:26.943 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:26.943 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.943 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.943 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.944 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:26.944 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.944 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.944 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.944 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:34:26.944 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.944 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.944 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.945 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.945 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.945 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.945 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.945 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.945 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.945 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.946 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.946 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.946 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.946 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.946 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.946 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.946 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.946 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.947 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.947 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.947 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.947 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.947 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.947 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.948 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.948 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.948 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.948 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.948 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.948 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.948 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.948 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.949 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.949 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.949 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.949 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.949 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.950 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.950 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.950 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.950 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.950 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.950 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.951 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.951 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.951 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.951 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.951 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.951 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.951 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.951 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.952 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.952 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.952 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.952 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.953 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.953 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.953 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.953 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.953 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.954 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.954 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-09-19T04:34:26.954 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-09-19T04:34:26.954 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.954 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.954 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.954 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.955 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.955 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.955 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.955 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:34:26.955 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.955 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.955 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.956 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-09-19T04:34:26.956 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-09-19T04:34:26.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.956 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.956 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-09-19T04:34:26.956 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.957 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.957 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.957 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-09-19T04:34:26.957 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.957 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.957 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.957 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-09-19T04:34:26.958 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-09-19T04:34:26.958 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.958 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.958 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.958 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.958 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.958 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.958 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.959 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-09-19T04:34:26.959 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.959 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.959 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.959 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-09-19T04:34:26.959 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-09-19T04:34:26.959 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.960 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.960 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.960 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.960 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.960 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.960 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-09-19T04:34:26.961 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.961 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.961 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.961 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-09-19T04:34:26.961 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-09-19T04:34:26.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:26.961 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.962 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-09-19T04:34:26.962 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.962 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.962 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.962 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-09-19T04:34:26.962 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.962 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.963 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.963 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-09-19T04:34:26.963 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-09-19T04:34:26.963 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.963 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.963 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.964 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.964 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.964 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.964 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.964 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:34:26.964 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.964 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.964 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.965 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-09-19T04:34:26.965 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-09-19T04:34:26.965 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.965 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.965 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.966 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.966 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.966 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.966 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:34:26.966 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.966 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.966 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.967 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-09-19T04:34:26.967 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-09-19T04:34:26.967 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.967 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.967 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.967 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.967 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.967 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.967 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.968 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-09-19T04:34:26.968 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.968 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:26.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:26.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-09-19T04:34:26.969 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:26.969 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:26.969 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:26.969 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-09-19T04:34:26.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-09-19T04:34:26.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.969 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.970 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:26.970 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.970 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.970 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.970 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:34:26.970 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.971 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.971 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-09-19T04:34:26.971 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-09-19T04:34:26.971 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.971 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.971 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-09-19T04:34:26.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.972 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.972 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.972 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:34:26.972 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:34:26.972 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.973 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.973 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-09-19T04:34:26.973 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-09-19T04:34:26.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:26.973 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.973 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.973 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:26.974 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-09-19T04:34:26.974 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-09-19T04:34:26.974 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.974 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:34:26.974 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:34:26.974 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.974 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.975 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-09-19T04:34:26.975 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:34:26.975 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:26.975 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.975 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.975 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-09-19T04:34:26.976 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:26.976 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-09-19T04:34:26.976 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.976 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:34:26.976 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.976 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.976 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.977 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-09-19T04:34:26.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-09-19T04:34:26.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.977 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:34:26.977 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.978 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.978 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.978 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:34:26.978 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:34:26.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.978 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.978 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-09-19T04:34:26.979 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-09-19T04:34:26.979 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:26.979 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.979 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-09-19T04:34:26.979 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-09-19T04:34:26.979 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-09-19T04:34:26.980 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.980 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:34:26.980 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:34:26.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.980 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.980 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-09-19T04:34:26.980 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-09-19T04:34:26.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.981 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.981 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.981 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-09-19T04:34:26.981 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.981 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.981 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:34:26.982 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:34:26.982 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.982 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.982 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-09-19T04:34:26.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-09-19T04:34:26.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:26.982 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-09-19T04:34:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-09-19T04:34:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:34:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:34:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.986 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.986 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.986 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.986 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.986 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.986 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:26.987 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:26.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.987 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.987 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.987 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.987 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.987 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.988 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.988 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.988 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.988 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:26.988 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:26.989 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.989 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.989 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.989 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:26.989 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.989 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.989 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.989 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:26.990 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:26.990 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:26.990 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:26.990 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:26.990 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:26.990 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.990 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.990 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.991 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.991 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:26.991 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:26.991 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:26.991 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.992 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:26.992 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.992 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.992 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.992 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:26.992 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.993 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.993 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.993 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-09-19T04:34:26.993 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-09-19T04:34:26.993 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:26.993 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.993 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:26.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.994 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.994 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.994 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:34:26.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.994 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.994 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.995 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-09-19T04:34:26.995 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-09-19T04:34:26.995 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.995 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.995 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.995 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-09-19T04:34:26.995 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.996 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.996 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:26.996 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-09-19T04:34:26.996 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-09-19T04:34:26.996 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-09-19T04:34:26.996 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-09-19T04:34:26.996 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-09-19T04:34:26.996 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:26.997 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-09-19T04:34:26.997 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.997 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.997 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.997 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-09-19T04:34:26.997 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-09-19T04:34:26.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:26.998 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.998 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:26.998 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:26.998 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:26.998 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:26.998 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:26.998 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:34:26.998 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:26.999 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:26.999 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:26.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:26.999 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:26.999 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:26.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:26.999 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:26.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.000 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:27.000 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.000 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.000 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.000 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:34:27.000 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.000 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.001 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-09-19T04:34:27.001 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-09-19T04:34:27.001 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.001 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.001 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.001 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:27.002 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:27.002 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.002 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:34:27.002 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:34:27.002 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.002 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-09-19T04:34:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:34:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.004 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.004 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:34:27.004 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.004 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.004 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.004 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.004 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-09-19T04:34:27.005 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.005 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.005 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.005 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-09-19T04:34:27.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-09-19T04:34:27.006 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:27.006 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.006 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.006 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-09-19T04:34:27.006 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.006 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.006 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.007 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:34:27.007 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.007 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.007 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.007 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-09-19T04:34:27.007 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-09-19T04:34:27.007 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.008 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.008 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-09-19T04:34:27.008 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.008 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.008 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:34:27.008 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.009 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-09-19T04:34:27.009 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-09-19T04:34:27.009 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.009 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.009 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.010 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-09-19T04:34:27.010 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.010 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.010 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.010 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:34:27.010 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.010 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.011 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.011 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-09-19T04:34:27.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-09-19T04:34:27.011 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.011 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.011 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.011 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.011 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.012 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.012 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:34:27.012 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:34:27.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.012 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.013 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-09-19T04:34:27.013 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-09-19T04:34:27.013 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.013 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.013 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.013 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:27.013 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.013 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.014 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.014 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:34:27.014 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.014 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.014 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.014 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:34:27.015 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:34:27.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:27.015 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.015 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.015 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-09-19T04:34:27.015 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.015 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.015 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.016 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:34:27.016 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.016 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.016 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.016 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-09-19T04:34:27.016 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-09-19T04:34:27.017 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.017 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.017 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.017 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.017 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.017 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.017 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.017 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-09-19T04:34:27.018 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-09-19T04:34:27.018 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-09-19T04:34:27.018 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.018 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:34:27.018 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.018 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.019 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-09-19T04:34:27.019 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-09-19T04:34:27.019 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.019 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.019 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.019 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.019 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.020 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.020 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.020 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:34:27.020 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.020 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.020 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-09-19T04:34:27.021 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-09-19T04:34:27.021 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.021 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.021 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.021 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.024 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.024 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.024 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.024 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:34:27.025 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.025 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.025 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.025 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-09-19T04:34:27.025 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-09-19T04:34:27.025 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.026 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.026 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.026 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.026 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.026 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.026 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.026 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:34:27.027 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.027 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.027 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.027 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-09-19T04:34:27.027 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-09-19T04:34:27.027 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.027 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.027 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.028 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:34:27.028 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.028 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.028 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.028 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:27.028 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.028 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.028 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.029 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-09-19T04:34:27.029 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-09-19T04:34:27.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.029 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.029 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:34:27.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.030 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.030 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:34:27.030 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.030 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.030 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.031 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-09-19T04:34:27.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:34:27.031 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.031 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.031 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.031 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:34:27.031 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.031 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.032 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-09-19T04:34:27.032 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.032 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.032 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-09-19T04:34:27.032 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-09-19T04:34:27.033 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.033 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.033 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.033 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:34:27.033 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.033 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.033 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.033 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:27.034 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.034 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.034 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-09-19T04:34:27.034 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-09-19T04:34:27.034 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.035 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.035 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:34:27.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.035 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.035 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.035 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-09-19T04:34:27.035 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.036 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.036 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.036 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-09-19T04:34:27.036 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:34:27.036 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.036 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.036 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.037 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:34:27.037 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.037 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.037 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.037 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:34:27.037 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.037 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.038 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.038 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-09-19T04:34:27.038 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:34:27.038 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.038 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.038 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.038 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:34:27.039 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.039 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.039 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.039 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:34:27.039 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.039 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.039 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-09-19T04:34:27.040 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:34:27.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:34:27.040 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.040 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.040 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.041 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-09-19T04:34:27.041 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.041 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.041 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.041 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-09-19T04:34:27.041 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-09-19T04:34:27.042 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.042 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.042 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.042 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:34:27.042 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.042 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.042 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.042 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-09-19T04:34:27.043 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.043 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.043 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-09-19T04:34:27.043 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-09-19T04:34:27.043 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.044 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.044 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.044 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:34:27.044 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.044 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.044 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.044 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-09-19T04:34:27.044 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.045 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.045 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.045 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-09-19T04:34:27.045 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-09-19T04:34:27.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.045 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.046 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:34:27.046 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.046 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.046 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.046 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:27.046 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.046 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.046 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.047 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-09-19T04:34:27.047 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-09-19T04:34:27.047 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.047 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.047 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.047 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:34:27.047 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.048 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.048 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.048 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-09-19T04:34:27.048 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.048 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.048 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.048 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-09-19T04:34:27.049 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-09-19T04:34:27.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.049 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.049 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:34:27.049 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.049 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.049 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.050 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:27.050 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.050 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.050 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.050 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-09-19T04:34:27.050 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-09-19T04:34:27.050 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.051 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.051 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:34:27.051 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.051 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.051 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.051 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-09-19T04:34:27.051 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.052 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.052 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.052 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_samba": { 2024-09-19T04:34:27.052 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_samba", 2024-09-19T04:34:27.052 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.052 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.053 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.053 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:34:27.053 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.053 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.053 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.053 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:34:27.053 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.053 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.054 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.054 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-09-19T04:34:27.054 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:34:27.054 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.054 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.054 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.054 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:34:27.054 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.055 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.055 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.055 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:34:27.055 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.055 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.055 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.056 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-09-19T04:34:27.056 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-09-19T04:34:27.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.056 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.056 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.057 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.057 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.057 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.057 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:34:27.057 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.057 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.058 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-09-19T04:34:27.058 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:34:27.058 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.058 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.058 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.058 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:27.058 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.059 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.059 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.059 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:34:27.059 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.059 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.059 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.059 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:34:27.060 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:34:27.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.060 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.060 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-09-19T04:34:27.060 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.060 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.060 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.061 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:34:27.061 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.061 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.061 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.061 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-09-19T04:34:27.061 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-09-19T04:34:27.062 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.062 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.062 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.062 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-09-19T04:34:27.062 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.062 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.062 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.062 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:34:27.063 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.063 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.063 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.063 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-09-19T04:34:27.063 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-09-19T04:34:27.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.063 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.064 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-09-19T04:34:27.064 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.064 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.064 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.064 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:34:27.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.064 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.065 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-09-19T04:34:27.065 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-09-19T04:34:27.065 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.065 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.065 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.065 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:34:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-09-19T04:34:27.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-09-19T04:34:27.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.067 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.067 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:27.067 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.067 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.067 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.068 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:34:27.068 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.068 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.068 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.068 INFO:teuthology.orchestra.run.smithi107.stdout: "grafana_dashboards_path": { 2024-09-19T04:34:27.068 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:34:27.069 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.069 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.069 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.069 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:34:27.069 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.069 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.069 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.069 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:34:27.070 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.070 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-09-19T04:34:27.070 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-09-19T04:34:27.070 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.070 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.071 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:27.071 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.071 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.071 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.071 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:34:27.071 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.071 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.072 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.072 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-09-19T04:34:27.072 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-09-19T04:34:27.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.072 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.072 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.072 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.073 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.073 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.073 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.073 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:34:27.073 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.073 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.074 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-09-19T04:34:27.074 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-09-19T04:34:27.074 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.074 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.074 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.074 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.074 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.074 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.074 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.075 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:34:27.075 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.075 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.075 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.075 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.076 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.076 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.076 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.076 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.076 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.076 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.076 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.076 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.077 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.077 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.077 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.077 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.077 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.077 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.077 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.078 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.078 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-09-19T04:34:27.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-09-19T04:34:27.078 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.078 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.078 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.078 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.079 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.079 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.079 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.079 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:34:27.079 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.079 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.079 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.080 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.080 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.080 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.080 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.080 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.080 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.080 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.081 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.081 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:34:27.081 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.081 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.081 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.082 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.082 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.082 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.082 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.082 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.082 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.082 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.083 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.083 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.083 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.083 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.083 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.083 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.083 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.083 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.084 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.084 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.084 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.084 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.084 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.084 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.085 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.085 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.085 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.085 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.085 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.085 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.086 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.086 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.086 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:34:27.086 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:34:27.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.086 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.087 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.087 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.087 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.087 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.087 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:34:27.087 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.088 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.088 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.088 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:34:27.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:34:27.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.088 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.088 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.089 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-09-19T04:34:27.089 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.089 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:34:27.089 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.089 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.090 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.090 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-09-19T04:34:27.090 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-09-19T04:34:27.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.090 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.090 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.091 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.091 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.091 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.091 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:34:27.091 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.091 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.091 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.092 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-09-19T04:34:27.092 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-09-19T04:34:27.092 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.092 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.092 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.092 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.092 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.092 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.093 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.093 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:34:27.093 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.093 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.093 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.093 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-09-19T04:34:27.094 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-09-19T04:34:27.094 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.094 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.094 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.094 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.094 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.094 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.095 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-09-19T04:34:27.095 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.095 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.095 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.095 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-09-19T04:34:27.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-09-19T04:34:27.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.096 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.096 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-09-19T04:34:27.096 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.096 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.096 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.096 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-09-19T04:34:27.097 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-09-19T04:34:27.097 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.097 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:34:27.097 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.097 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.097 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.097 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-09-19T04:34:27.098 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-09-19T04:34:27.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.098 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.098 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-09-19T04:34:27.098 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.098 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.098 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.099 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:34:27.099 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.099 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.099 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.099 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-09-19T04:34:27.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:34:27.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.100 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.100 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.100 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:34:27.100 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.100 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.100 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.100 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:34:27.100 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.101 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.101 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.101 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-09-19T04:34:27.101 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-09-19T04:34:27.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.101 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.102 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.102 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.102 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.102 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.102 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.102 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:34:27.102 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.103 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.103 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.103 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-09-19T04:34:27.103 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-09-19T04:34:27.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.103 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.104 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.104 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.104 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.104 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.104 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:34:27.104 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.104 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.105 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-09-19T04:34:27.105 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-09-19T04:34:27.105 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.105 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.105 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.105 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.106 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.106 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.106 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.106 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:34:27.106 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.106 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.106 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.107 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-09-19T04:34:27.107 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-09-19T04:34:27.107 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.107 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.107 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.107 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.107 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.108 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.108 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:34:27.108 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.108 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.108 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-09-19T04:34:27.108 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:34:27.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.109 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.109 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.109 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.109 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.109 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.109 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.110 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:34:27.110 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.110 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.110 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.110 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-09-19T04:34:27.110 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-09-19T04:34:27.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.111 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.111 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-09-19T04:34:27.111 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.111 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.111 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.111 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:34:27.111 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.112 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.112 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-09-19T04:34:27.112 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-09-19T04:34:27.112 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.112 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.112 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.113 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.113 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.113 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.113 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.113 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:34:27.113 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.113 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.113 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.114 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-09-19T04:34:27.114 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-09-19T04:34:27.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.114 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.114 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.114 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.114 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.115 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.115 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.115 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:34:27.115 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.115 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.115 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-09-19T04:34:27.116 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-09-19T04:34:27.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.116 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.116 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.116 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.116 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.116 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.116 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:34:27.117 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.117 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.117 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.117 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-09-19T04:34:27.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:34:27.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.117 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.118 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.118 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.118 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.118 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:34:27.118 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.118 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.118 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.119 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-09-19T04:34:27.119 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:34:27.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.119 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.119 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.120 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.120 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.120 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.120 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:34:27.120 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.120 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.121 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-09-19T04:34:27.121 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:34:27.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.121 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.121 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.121 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.121 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.121 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.122 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.122 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:34:27.122 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.122 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.122 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.122 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.122 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-09-19T04:34:27.123 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.123 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.123 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.123 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.124 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.124 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.124 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.124 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.124 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.124 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.124 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.125 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.125 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.125 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.125 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.125 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.125 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.125 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.126 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.126 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.126 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.126 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.126 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.126 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.127 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.127 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.127 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.127 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.128 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.128 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.128 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.128 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.128 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.128 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.129 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.129 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.129 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.129 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.129 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.129 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.129 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.130 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.130 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.130 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.130 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.130 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.130 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.131 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.131 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.131 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.131 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.131 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.131 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.131 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.132 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.132 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.132 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.132 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.132 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.132 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-09-19T04:34:27.133 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-09-19T04:34:27.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.133 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.133 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.133 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-09-19T04:34:27.137 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.137 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.137 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.137 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:34:27.137 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.138 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.138 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.138 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-09-19T04:34:27.138 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-09-19T04:34:27.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.138 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.138 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.139 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-09-19T04:34:27.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.139 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.139 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.139 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:34:27.139 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.139 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.140 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.140 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.140 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.140 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-09-19T04:34:27.140 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.140 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.141 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.141 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:34:27.141 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:34:27.141 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.141 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.141 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.141 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.141 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.142 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.142 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.142 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.142 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.142 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.142 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.143 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:34:27.143 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:34:27.143 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.143 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.143 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.143 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.143 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.143 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.143 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.144 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.144 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.144 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.144 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.144 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:34:27.144 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:34:27.145 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.145 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.145 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.145 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.145 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.145 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.145 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.145 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.146 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.146 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.146 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.146 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:34:27.146 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:34:27.146 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.147 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.147 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.147 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.147 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.147 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.147 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.147 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.147 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.148 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.148 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.148 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:34:27.148 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:34:27.148 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.148 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.149 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.149 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.149 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.149 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.149 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.149 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.150 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.150 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.150 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.150 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:34:27.150 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:34:27.150 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.150 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.151 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.151 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.151 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.151 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.151 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.151 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.151 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.152 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.152 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.152 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:34:27.152 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:34:27.152 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.152 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.152 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.153 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.153 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.153 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.153 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.153 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.153 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.153 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.154 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.154 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:34:27.154 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:34:27.154 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.154 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.154 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.154 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.155 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.155 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.155 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.155 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.155 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.155 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.155 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.156 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:34:27.156 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:34:27.156 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.156 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.156 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.156 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.156 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.156 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.157 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.157 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.157 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.157 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.157 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.157 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:34:27.157 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:34:27.158 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.158 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.158 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.158 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.158 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.158 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.158 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.158 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.159 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.159 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.159 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.159 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:34:27.159 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:34:27.159 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.159 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.160 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.160 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.160 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.160 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.160 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.160 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.160 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.161 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.161 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.161 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:34:27.161 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:34:27.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.161 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.162 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.162 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.162 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.162 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.162 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.162 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.162 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.162 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.163 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.163 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:34:27.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:34:27.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.163 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.163 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.163 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.164 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.164 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.164 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.164 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.164 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.164 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.164 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.165 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:34:27.165 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:34:27.165 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.165 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.165 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.165 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.165 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.165 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.166 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.166 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.166 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.166 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.166 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.166 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:34:27.166 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:34:27.167 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.167 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.167 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-09-19T04:34:27.167 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.167 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.167 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.167 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.167 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.168 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.168 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.168 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:34:27.168 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:34:27.168 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.168 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.168 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.169 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.169 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.169 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.169 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.169 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.169 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.169 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.169 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.170 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-09-19T04:34:27.170 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:34:27.170 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.170 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.170 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.170 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.170 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.171 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.171 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.171 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.171 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.171 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.171 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.171 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:34:27.172 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:34:27.172 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.172 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.172 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.172 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-09-19T04:34:27.172 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.172 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.172 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.173 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.173 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.173 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.173 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.173 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:34:27.173 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:34:27.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.174 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.174 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.174 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.174 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.174 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.174 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.174 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.174 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.175 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.175 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.175 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:34:27.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:34:27.175 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.175 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.175 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.176 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.176 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.176 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.176 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.176 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.176 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.176 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.177 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:34:27.177 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:34:27.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.177 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.177 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.177 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.177 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.177 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.178 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.178 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.178 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.178 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:34:27.178 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:34:27.178 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.179 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.179 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.179 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.179 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.179 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.179 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.179 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.179 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.180 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.180 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.180 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:34:27.180 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:34:27.180 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.180 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.181 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.181 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.181 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.181 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.181 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.181 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.181 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.182 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.182 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.182 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:34:27.182 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:34:27.182 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.182 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.182 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.182 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.183 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.183 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.183 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.183 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.183 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.183 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.184 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:34:27.184 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:34:27.184 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.184 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.184 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.184 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.184 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.184 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.185 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.185 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.185 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.185 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.185 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.185 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:34:27.186 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:34:27.186 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.186 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.186 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.186 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.186 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.186 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.186 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.187 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.187 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.187 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.187 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.187 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:34:27.187 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:34:27.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.188 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.188 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.188 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.188 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.188 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.188 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.188 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.188 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.189 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.189 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.189 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:34:27.189 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:34:27.189 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.189 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.190 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.190 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.190 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.190 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.190 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.190 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.190 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.190 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.191 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.191 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:34:27.191 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:34:27.191 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.191 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.191 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.191 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.192 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.192 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.192 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.192 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.192 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.192 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.192 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.193 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:34:27.193 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:34:27.193 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.193 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.193 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.193 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.194 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.194 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.194 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.194 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.194 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.194 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:34:27.194 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:34:27.194 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.195 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.195 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.195 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.195 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.195 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.195 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.195 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.195 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.195 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.196 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.196 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:34:27.196 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:34:27.196 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.196 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.196 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.196 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.197 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.197 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.197 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.197 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.197 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.197 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.197 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.198 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:34:27.198 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:34:27.198 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.198 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.198 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.198 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:34:27.198 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.199 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.199 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.199 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.199 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.199 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.199 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.199 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:34:27.200 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:34:27.200 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.200 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.200 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.200 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.200 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.200 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.200 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.201 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.201 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.201 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.201 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.201 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:34:27.201 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:34:27.202 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.202 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.202 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.202 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-09-19T04:34:27.202 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.202 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.202 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.202 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.202 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.203 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.203 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.203 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:34:27.203 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:34:27.203 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.203 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.203 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.204 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-09-19T04:34:27.204 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.204 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.204 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.204 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.204 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.204 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.205 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.205 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:34:27.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:34:27.205 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.205 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.205 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.206 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.206 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.206 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.206 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.206 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.206 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.206 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.206 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.207 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:34:27.207 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:34:27.207 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.207 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.207 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.207 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-09-19T04:34:27.208 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.208 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.208 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.208 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.208 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.208 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.208 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.209 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:34:27.209 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:34:27.209 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.209 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.209 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.209 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.209 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.209 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.210 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.210 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.210 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.210 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.210 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.210 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:34:27.211 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:34:27.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.211 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.211 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.211 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.211 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.211 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.212 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.212 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.212 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.212 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.212 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:34:27.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:34:27.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.213 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.213 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.213 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:27.213 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.213 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.213 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.213 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.214 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.214 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.214 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:34:27.214 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:34:27.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.214 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.215 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.215 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.215 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.215 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.215 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.215 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.215 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.215 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:34:27.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:34:27.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.216 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.216 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.216 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:27.216 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.216 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.217 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.217 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.217 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.217 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.217 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.217 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-09-19T04:34:27.217 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-09-19T04:34:27.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.218 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.218 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.218 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.218 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.218 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.218 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.218 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.218 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.218 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.219 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.219 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-09-19T04:34:27.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-09-19T04:34:27.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.219 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.220 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.220 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.220 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.220 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.220 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.220 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.220 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.221 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-09-19T04:34:27.221 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-09-19T04:34:27.221 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.221 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.221 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.221 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.222 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.222 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.222 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:34:27.222 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.222 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.222 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.222 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-09-19T04:34:27.223 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-09-19T04:34:27.223 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.223 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.223 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.223 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-09-19T04:34:27.223 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.223 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.223 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.224 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.224 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.224 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.224 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-09-19T04:34:27.224 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-09-19T04:34:27.224 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.225 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.225 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.225 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.225 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.225 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.225 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.225 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.225 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.226 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.226 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.226 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.226 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.226 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.226 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.226 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.227 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.227 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.227 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.227 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.227 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.228 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.228 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.228 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.228 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.228 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.228 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.229 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.229 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.229 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.229 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.229 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.229 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.229 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.230 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.230 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.230 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.230 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.230 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.230 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.231 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.231 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.231 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.231 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.231 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.231 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.231 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.231 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.232 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.232 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.232 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.232 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.232 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.232 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.232 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.232 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.233 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.233 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.233 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.233 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.233 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.233 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.234 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.234 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.234 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.234 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.234 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.234 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.235 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-09-19T04:34:27.235 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-09-19T04:34:27.235 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.235 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.235 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.235 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.235 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.236 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.236 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.236 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-09-19T04:34:27.236 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.236 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.236 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.236 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:34:27.237 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:34:27.237 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.237 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.237 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.237 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.237 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.237 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.237 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.238 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.238 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.238 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.238 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.238 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-09-19T04:34:27.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-09-19T04:34:27.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.238 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.239 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-09-19T04:34:27.239 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.239 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.239 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.239 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.239 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.239 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.240 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-09-19T04:34:27.240 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-09-19T04:34:27.240 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.240 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.240 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.240 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-09-19T04:34:27.241 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.241 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.241 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.241 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.241 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.241 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.241 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.242 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-09-19T04:34:27.242 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-09-19T04:34:27.242 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.242 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.242 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.242 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.242 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.242 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.243 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.243 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.243 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.243 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.243 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.243 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-09-19T04:34:27.243 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-09-19T04:34:27.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.244 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.244 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-09-19T04:34:27.244 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.244 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.244 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.245 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.245 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.245 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.245 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.245 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-09-19T04:34:27.245 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-09-19T04:34:27.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.246 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.246 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.246 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-09-19T04:34:27.246 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.246 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.246 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.246 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.246 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-09-19T04:34:27.247 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.247 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.247 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.247 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.247 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.247 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-09-19T04:34:27.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-09-19T04:34:27.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.248 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.248 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.248 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-09-19T04:34:27.248 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-09-19T04:34:27.248 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-09-19T04:34:27.249 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.249 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.249 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.249 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.249 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.249 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-09-19T04:34:27.249 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-09-19T04:34:27.250 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.250 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.250 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.250 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.250 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.250 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.250 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.250 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.251 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.251 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.251 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-09-19T04:34:27.252 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.252 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.252 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.252 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-09-19T04:34:27.252 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-09-19T04:34:27.252 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.252 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.253 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.253 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.253 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.253 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-09-19T04:34:27.253 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.253 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.253 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.254 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.254 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.254 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.254 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.254 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.254 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.255 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.255 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.255 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.255 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.255 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.255 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.255 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.255 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.256 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.256 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.256 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.256 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.256 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.256 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.256 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.257 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.257 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.257 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.257 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.257 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.257 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.257 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.258 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.258 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.258 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.258 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.258 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.259 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.259 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.259 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.259 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.259 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.259 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.259 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.259 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.259 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.260 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.260 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.260 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.260 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.260 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.260 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.260 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.261 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.261 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.261 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.261 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.261 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.261 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.261 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.261 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.262 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.262 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.262 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.262 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.262 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-09-19T04:34:27.262 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-09-19T04:34:27.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.263 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.263 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.263 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-09-19T04:34:27.263 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.263 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.263 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.263 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:34:27.264 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.264 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.264 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.264 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-09-19T04:34:27.264 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-09-19T04:34:27.264 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.264 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.265 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-09-19T04:34:27.265 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.265 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.265 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.265 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:34:27.265 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.266 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.266 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-09-19T04:34:27.266 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-09-19T04:34:27.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.266 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.266 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.267 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-09-19T04:34:27.267 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.267 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.267 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.267 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:34:27.267 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.267 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.267 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.268 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-09-19T04:34:27.268 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-09-19T04:34:27.268 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.268 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.268 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.268 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-09-19T04:34:27.269 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.269 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.269 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.269 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:34:27.269 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.269 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.269 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.270 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-09-19T04:34:27.270 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-09-19T04:34:27.270 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.270 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.270 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.270 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.270 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.270 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.271 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.271 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:34:27.271 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.271 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.271 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.271 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:27.271 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:27.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.272 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.272 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.272 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:27.272 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.272 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.272 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.272 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:34:27.273 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.273 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.273 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.273 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-09-19T04:34:27.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-09-19T04:34:27.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.274 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.274 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-09-19T04:34:27.274 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.274 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.274 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.274 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:34:27.274 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.274 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.275 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.275 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.275 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.275 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-09-19T04:34:27.275 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.275 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.276 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.276 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.276 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.276 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.276 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.276 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.276 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.277 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.277 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.277 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.277 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.277 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.277 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.277 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.278 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.278 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.278 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.278 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.278 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.278 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.278 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.279 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.279 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.279 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.279 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.279 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.279 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.280 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.280 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.280 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.280 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.280 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.280 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.280 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.281 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.281 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.281 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.281 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.281 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.281 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.281 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.282 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.282 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.282 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.282 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.282 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.282 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.282 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.283 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.283 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.283 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.283 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.284 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.284 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.284 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.284 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.284 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.285 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-09-19T04:34:27.285 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-09-19T04:34:27.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.285 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.285 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.285 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-09-19T04:34:27.285 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.285 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.286 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.286 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.286 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.286 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.286 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-09-19T04:34:27.286 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-09-19T04:34:27.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.287 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.287 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-09-19T04:34:27.287 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.287 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.287 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.287 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.288 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.288 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.288 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:27.288 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:27.288 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.289 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.289 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:27.289 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.289 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.289 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.289 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.289 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.290 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.290 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.290 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.290 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-09-19T04:34:27.290 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-09-19T04:34:27.291 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:34:27.291 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.291 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-09-19T04:34:27.291 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-09-19T04:34:27.291 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.291 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.291 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.291 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-09-19T04:34:27.292 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.292 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.292 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.292 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:34:27.292 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.292 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.293 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-09-19T04:34:27.293 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-09-19T04:34:27.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.293 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.293 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-09-19T04:34:27.293 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.294 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.294 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.294 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:34:27.294 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.294 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.294 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.294 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-09-19T04:34:27.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-09-19T04:34:27.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.295 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.295 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.295 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.295 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.296 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.296 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:34:27.296 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.296 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.296 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.296 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:27.296 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:27.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.297 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.297 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.297 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-09-19T04:34:27.297 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-09-19T04:34:27.297 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.297 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.297 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:34:27.298 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.298 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.298 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.298 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.298 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.299 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.299 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.299 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.299 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.299 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.299 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.299 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.299 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.300 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.300 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.300 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.300 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.300 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.300 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.300 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.301 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.301 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.301 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.301 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.301 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.302 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.302 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.302 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.302 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.302 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.302 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.302 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.303 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.303 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.303 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.303 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.303 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.303 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.303 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.304 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.304 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.304 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.304 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.304 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.304 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.304 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.304 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.305 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.305 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.305 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.305 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.305 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.305 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.305 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.306 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.306 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.306 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.306 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.306 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.306 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.306 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.307 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.307 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-09-19T04:34:27.307 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-09-19T04:34:27.307 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.307 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.307 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.307 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.308 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.308 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.308 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.308 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:34:27.308 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.308 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.308 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.309 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-09-19T04:34:27.309 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-09-19T04:34:27.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.309 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.309 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.309 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-09-19T04:34:27.309 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.310 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.310 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.310 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-09-19T04:34:27.310 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.310 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.310 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.310 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-09-19T04:34:27.311 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-09-19T04:34:27.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.311 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.311 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-09-19T04:34:27.311 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.311 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.311 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.311 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:34:27.312 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.312 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.312 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.312 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-09-19T04:34:27.312 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-09-19T04:34:27.312 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.312 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.313 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:27.313 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:27.313 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-09-19T04:34:27.313 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.313 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:34:27.313 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.313 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.314 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.314 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-09-19T04:34:27.314 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-09-19T04:34:27.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.314 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.314 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.314 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.315 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.315 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.315 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.315 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:34:27.315 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.315 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.315 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.316 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-09-19T04:34:27.316 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-09-19T04:34:27.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.316 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.316 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.316 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-09-19T04:34:27.317 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.317 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.317 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.317 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:34:27.317 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.317 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.317 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.318 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-09-19T04:34:27.318 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.318 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.318 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.318 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.318 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.319 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.319 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.319 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.319 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.319 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.319 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.320 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.320 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.320 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.320 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.320 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.320 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.320 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.321 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.321 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.321 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.321 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.321 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.322 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.322 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.322 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.322 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.322 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.322 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.322 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.322 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.323 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.323 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.323 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.323 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.323 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.323 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.323 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.324 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.324 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.324 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.324 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.324 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.324 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.324 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.324 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.325 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.325 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.325 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.325 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.325 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.325 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.326 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.326 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.326 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.326 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.326 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.326 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.326 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.327 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.327 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.327 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.327 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.327 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.327 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-09-19T04:34:27.328 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.328 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.328 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.328 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.328 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.329 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.329 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.329 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.329 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.329 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.329 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.329 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.330 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.330 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.330 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.330 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.330 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.330 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.330 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.330 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.331 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.331 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.331 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.331 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.331 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.331 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.331 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.332 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.332 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.332 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.332 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.332 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.332 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.332 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.333 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.333 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.333 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.333 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.333 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.333 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.334 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.334 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.334 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.334 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.334 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.334 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.334 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.334 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.335 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.335 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.335 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.335 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.335 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.336 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.336 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.336 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.336 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.336 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.336 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.336 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.336 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.337 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.337 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.337 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-09-19T04:34:27.337 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.338 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.338 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.338 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-09-19T04:34:27.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:34:27.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.338 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.338 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-09-19T04:34:27.339 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.339 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.339 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.339 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:34:27.339 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.339 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.339 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.340 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-09-19T04:34:27.340 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-09-19T04:34:27.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.340 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.340 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.340 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.340 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-09-19T04:34:27.340 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.341 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.341 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:34:27.341 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.341 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.341 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.341 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.342 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.342 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.342 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.342 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.342 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.342 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.342 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.342 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.342 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.343 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.343 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.343 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.343 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.343 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.343 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.343 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.343 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.344 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.344 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.344 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.344 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.344 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.345 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.345 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.345 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.345 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.345 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.345 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.345 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.345 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.346 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.346 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.346 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.346 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.346 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.347 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.347 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.347 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.347 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.347 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.347 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.347 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.347 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.348 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.348 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.348 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.348 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.348 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.348 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.348 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.349 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.349 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.349 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.349 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.349 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.349 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.350 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.350 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.350 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.350 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.350 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.350 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.350 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-09-19T04:34:27.351 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.351 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.351 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.351 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-09-19T04:34:27.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-09-19T04:34:27.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.352 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.352 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.352 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-09-19T04:34:27.352 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.352 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.352 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.352 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:34:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:34:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.353 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.354 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.355 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.355 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.355 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.355 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.355 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.355 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.355 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.356 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.356 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.356 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.356 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.356 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.356 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.356 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.357 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.357 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.357 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.357 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.357 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.357 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.357 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.358 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.358 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.358 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.358 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.358 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.358 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.358 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.358 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.359 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.359 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.359 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.359 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.359 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.359 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.359 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.359 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.360 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.360 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.360 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.360 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.360 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.360 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.361 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.361 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.361 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.361 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.361 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.361 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.361 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.362 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.362 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-09-19T04:34:27.362 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-09-19T04:34:27.362 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.362 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.362 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.362 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.363 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.363 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.363 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.363 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:34:27.363 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:34:27.363 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.363 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.364 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-09-19T04:34:27.364 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-09-19T04:34:27.364 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.364 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.364 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.365 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-09-19T04:34:27.365 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.365 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.365 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.365 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:34:27.365 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.365 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.366 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-09-19T04:34:27.366 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-09-19T04:34:27.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.366 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.366 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.366 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-09-19T04:34:27.366 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.366 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.367 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.367 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:34:27.367 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.367 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.367 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.367 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-09-19T04:34:27.368 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-09-19T04:34:27.368 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.368 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.368 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.368 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.368 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.368 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.368 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.369 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:34:27.369 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.369 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.369 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.369 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-09-19T04:34:27.369 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-09-19T04:34:27.370 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.370 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.370 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.370 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-09-19T04:34:27.370 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.370 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.370 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.370 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:34:27.371 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:34:27.371 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.371 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.371 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.371 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.371 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-09-19T04:34:27.372 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.372 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.372 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.372 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.372 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.372 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.372 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.372 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.373 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.373 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.373 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.373 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.373 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.373 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.373 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.373 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.374 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.374 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.374 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.374 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.374 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.374 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.374 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.375 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.375 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.375 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.375 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.375 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.375 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.376 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.376 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.376 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.376 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.376 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.376 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.376 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.377 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.377 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.377 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.377 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.377 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.377 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.377 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.378 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.378 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.378 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.378 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.378 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.378 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.378 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.378 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.379 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.379 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.379 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.379 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.379 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.379 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.379 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.380 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.380 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.380 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.380 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.380 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.380 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.380 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.380 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.381 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.381 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.381 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.381 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.381 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.381 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-09-19T04:34:27.382 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.382 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.382 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.382 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.382 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.382 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.382 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.382 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.383 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.383 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.383 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.383 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.383 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.383 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.383 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.384 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.384 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.384 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.384 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.384 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.384 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.384 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.384 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.385 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.385 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.385 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.385 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.385 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.385 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.385 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.386 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.386 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.386 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.386 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.386 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.386 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.386 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.386 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.387 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.387 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.387 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.387 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.387 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.387 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.387 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.387 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.387 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.388 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.388 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.388 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.388 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.388 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.388 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.388 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.388 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.389 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.389 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.389 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.389 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.390 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.390 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.390 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.390 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.390 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.390 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.390 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.390 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.391 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.391 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.391 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-09-19T04:34:27.391 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.391 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.391 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.392 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.392 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.392 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.392 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.392 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.392 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.393 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.393 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.393 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.393 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.393 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.393 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.393 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.393 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.394 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.394 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.394 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.394 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.394 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.394 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.394 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.395 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.395 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.395 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.395 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.395 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.395 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.395 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.395 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.396 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.396 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.396 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.396 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.396 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.396 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.397 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.397 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.397 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.397 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.397 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.397 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.397 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.397 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.398 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.398 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.398 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.398 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.398 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.398 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.398 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.398 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.399 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.399 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.399 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.399 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.399 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.399 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.399 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.400 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.400 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.400 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.400 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.400 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.400 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.400 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.400 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.401 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-09-19T04:34:27.401 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.401 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.401 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.401 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-09-19T04:34:27.402 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-09-19T04:34:27.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.402 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.402 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.402 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.402 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.402 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.403 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:34:27.403 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.403 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.403 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.403 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.404 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.404 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.404 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.404 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.404 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.404 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.404 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.405 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.405 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.405 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.405 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.405 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.405 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.405 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.405 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.406 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.406 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.406 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.406 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.406 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.406 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.406 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.407 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.407 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.407 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.407 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.407 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.407 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.407 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.407 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.408 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.408 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.408 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.408 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.408 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.408 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.408 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.408 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.409 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.409 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.409 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.409 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.409 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.409 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.409 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.409 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.410 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.410 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.410 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.410 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.410 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.410 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.410 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.410 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.411 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.411 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.411 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.411 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.411 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.411 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.411 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.412 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.412 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.412 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-09-19T04:34:27.412 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-09-19T04:34:27.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.412 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.413 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.413 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.413 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.413 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.413 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-09-19T04:34:27.413 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-09-19T04:34:27.413 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-09-19T04:34:27.413 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.414 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-09-19T04:34:27.414 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.414 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.414 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.414 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.414 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.415 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-09-19T04:34:27.415 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.415 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.415 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.415 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.415 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.415 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.416 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.416 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.416 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.416 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.416 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.416 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.416 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.416 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.417 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.417 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.417 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.417 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.417 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.417 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.417 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.418 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.418 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.418 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.418 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.418 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.418 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.419 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.419 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.419 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.419 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.419 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.419 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.419 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.420 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.420 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.420 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.420 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.420 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.420 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.420 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.421 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.421 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.421 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.421 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.421 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.421 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.421 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.421 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.422 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.422 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.422 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.422 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.422 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.422 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.423 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.423 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.423 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.423 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.423 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.423 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.423 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.423 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.424 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.424 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.424 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.424 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.424 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.424 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.424 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-09-19T04:34:27.425 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.425 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.425 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.425 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.426 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.426 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.426 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.426 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.426 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.426 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.426 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.426 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.427 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.427 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.427 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.427 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.427 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.427 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.427 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.427 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.427 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.428 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.428 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.428 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.428 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.428 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.428 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.428 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.428 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.429 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.429 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.429 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.429 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.429 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.429 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.432 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.432 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.432 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.432 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.433 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.433 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.433 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.433 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.433 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.433 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.433 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.433 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.434 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.434 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.434 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.434 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.434 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.434 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.434 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.434 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.435 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.435 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.435 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.435 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.435 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.435 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.435 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.436 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.436 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.436 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.436 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.436 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.436 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.436 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.436 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.437 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-09-19T04:34:27.437 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.437 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.437 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.437 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.437 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.438 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.438 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.438 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.438 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.438 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.438 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.438 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.439 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.439 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.439 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.439 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.439 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.439 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.439 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.439 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.440 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.440 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.440 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.440 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.440 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.440 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.440 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.441 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.441 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.441 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.441 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.441 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.441 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.441 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.441 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.442 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.442 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.442 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.442 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.442 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.442 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.443 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.443 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.443 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.443 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.443 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.443 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.443 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.443 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.444 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.444 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.444 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.444 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.444 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.444 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.444 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.445 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.445 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.445 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.445 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.445 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.445 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.445 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.446 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.446 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.446 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.446 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.446 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.446 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.446 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:27.447 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:27.447 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.447 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.447 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.447 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:27.447 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.447 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.447 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.448 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.448 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.448 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.448 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.448 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-09-19T04:34:27.448 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-09-19T04:34:27.448 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:27.449 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.449 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.449 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-09-19T04:34:27.449 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-09-19T04:34:27.449 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.449 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.449 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-09-19T04:34:27.449 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:34:27.450 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.450 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.450 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.450 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.450 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-09-19T04:34:27.450 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.451 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.451 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.451 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-09-19T04:34:27.451 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:34:27.451 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.451 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.451 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.452 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.452 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.452 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.452 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.452 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:34:27.452 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.452 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.452 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.453 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-09-19T04:34:27.453 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-09-19T04:34:27.453 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.453 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.453 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.453 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.454 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.454 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.454 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.454 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.454 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.454 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.454 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.455 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.455 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.455 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.455 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.455 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.455 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.455 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.455 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.456 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.456 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.456 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.456 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.456 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.456 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.456 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.457 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.457 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.457 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.457 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.457 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.457 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.457 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.458 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.458 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.458 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.458 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.458 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.458 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.458 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.458 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.459 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.459 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.459 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.459 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.459 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.459 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.460 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.460 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.460 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.460 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.460 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.460 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.460 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.461 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.461 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.461 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.461 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.461 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.461 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.461 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.461 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.462 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.462 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.462 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.462 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.462 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.462 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.462 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.463 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.463 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.463 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.463 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.463 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.463 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.463 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.464 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-09-19T04:34:27.464 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-09-19T04:34:27.464 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.464 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.464 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.464 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-09-19T04:34:27.464 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.464 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.464 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.465 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:34:27.465 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.465 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.465 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.465 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:27.465 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:27.466 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.466 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.466 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.466 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:27.466 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.466 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.466 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.466 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:34:27.467 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.467 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.467 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.467 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.467 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.467 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-09-19T04:34:27.468 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.468 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.468 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.468 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-09-19T04:34:27.468 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-09-19T04:34:27.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.468 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.469 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.469 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.469 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.469 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.469 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.469 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.469 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.469 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.469 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.470 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-09-19T04:34:27.470 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-09-19T04:34:27.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.470 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.470 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.470 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:34:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:34:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.473 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.473 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.473 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.473 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.473 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.473 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.473 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.473 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.475 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.475 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.475 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.475 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.475 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.475 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.475 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.475 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.476 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.476 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.476 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.476 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.476 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.476 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.477 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.477 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.477 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.477 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.477 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.477 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.477 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.477 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.478 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.478 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.478 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.478 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.478 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.478 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.478 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.479 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.479 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.479 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.479 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.479 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.479 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.479 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.480 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.480 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.480 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.480 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.480 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.480 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.480 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.481 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-09-19T04:34:27.481 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-09-19T04:34:27.481 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.481 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.481 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.481 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.481 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.481 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.482 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.482 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.482 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.482 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.482 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.482 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:34:27.482 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:34:27.483 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.483 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.483 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.483 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-09-19T04:34:27.483 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.483 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.483 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.483 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.484 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.484 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.484 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-09-19T04:34:27.484 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-09-19T04:34:27.484 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:27.484 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.484 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.485 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-09-19T04:34:27.485 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.485 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.485 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.485 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.485 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.485 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.486 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.486 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-09-19T04:34:27.486 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-09-19T04:34:27.486 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.486 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.486 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.486 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-09-19T04:34:27.487 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.487 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.487 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.487 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:34:27.487 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.487 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.487 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.487 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-09-19T04:34:27.488 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-09-19T04:34:27.488 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.488 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.488 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.488 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-09-19T04:34:27.488 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.488 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.489 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.489 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:34:27.489 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.489 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.489 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.489 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-09-19T04:34:27.489 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-09-19T04:34:27.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.490 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.490 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.490 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-09-19T04:34:27.490 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.490 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.490 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.490 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.491 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.491 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.491 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.491 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-09-19T04:34:27.491 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-09-19T04:34:27.491 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.491 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.492 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.492 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-09-19T04:34:27.492 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.492 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.492 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.492 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-09-19T04:34:27.492 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-09-19T04:34:27.493 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.493 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.493 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.493 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.493 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.493 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-09-19T04:34:27.493 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-09-19T04:34:27.494 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.494 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.494 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.494 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-09-19T04:34:27.494 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-09-19T04:34:27.494 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-09-19T04:34:27.494 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.494 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.495 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.495 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.495 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.495 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-09-19T04:34:27.496 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.496 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.496 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.496 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.496 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.496 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.496 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.496 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.496 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.497 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.497 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.497 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.497 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.497 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.497 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.497 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.497 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.498 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.498 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.498 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.498 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.498 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.498 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.498 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.498 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.499 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.499 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.499 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.499 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.499 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.499 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.500 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.500 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.500 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.500 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.500 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.501 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.501 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.501 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.501 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.501 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.501 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.501 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.501 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.502 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.502 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.502 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.502 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.502 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.502 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.502 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.502 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.503 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.503 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.503 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.503 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.503 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.503 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.503 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.504 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.504 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.504 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.504 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.504 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.504 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.504 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.505 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-09-19T04:34:27.505 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:34:27.505 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.505 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.505 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.505 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.505 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.505 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.506 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.506 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.506 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.506 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.506 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.506 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:34:27.506 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:34:27.507 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.507 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.507 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.507 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.507 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.507 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.507 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.507 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.508 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.508 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.508 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.508 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-09-19T04:34:27.508 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-09-19T04:34:27.508 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.508 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.509 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.509 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.509 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.509 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.509 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.509 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.509 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.509 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.510 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.510 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.510 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.510 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-09-19T04:34:27.510 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.510 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.511 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.511 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-09-19T04:34:27.511 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-09-19T04:34:27.511 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.511 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.511 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.511 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.511 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.512 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.512 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.512 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.512 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.512 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.512 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-09-19T04:34:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-09-19T04:34:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.514 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.514 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.514 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.514 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.514 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.514 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.514 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.515 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.515 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.515 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.515 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.515 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.515 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.515 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.516 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.516 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.516 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.516 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.516 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.516 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.516 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.516 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.517 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.517 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.517 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.517 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.517 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.518 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.518 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.518 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.518 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.518 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.518 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.518 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.518 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.518 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.519 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.519 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.519 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.519 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.519 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.519 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.519 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.520 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.520 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.520 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.520 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.520 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.520 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.520 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.520 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.521 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.521 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.521 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.521 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.521 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.521 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.522 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.522 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.522 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.522 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.522 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.522 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.522 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.522 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.523 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.523 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.523 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.523 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.523 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-09-19T04:34:27.523 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-09-19T04:34:27.523 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.524 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.524 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.524 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.524 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.524 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.524 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.524 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.524 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.525 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.525 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.525 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-09-19T04:34:27.525 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-09-19T04:34:27.525 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.525 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.525 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.526 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.526 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.526 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.526 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.526 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.526 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.526 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.527 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.527 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.527 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.527 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-09-19T04:34:27.527 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.527 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.528 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.528 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.528 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.528 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.528 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.528 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.528 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.528 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.529 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.529 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.529 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.529 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.529 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.529 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.529 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.530 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.530 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.530 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.530 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.530 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.530 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.530 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.531 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.531 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.531 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.531 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.531 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.531 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.531 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.531 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.531 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.531 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.532 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.532 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.532 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.532 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.532 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.532 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.533 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.533 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.533 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.533 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.533 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.533 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.533 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.533 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.534 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.534 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.534 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.534 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.534 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.534 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.534 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.534 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.535 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.535 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.535 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.535 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.535 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.535 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.535 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.535 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.535 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.536 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.536 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.536 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.536 INFO:teuthology.orchestra.run.smithi107.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:34:27.536 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:34:27.536 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.537 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.537 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.537 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:27.537 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.537 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.537 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.537 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:34:27.537 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.538 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.538 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-09-19T04:34:27.538 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.538 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.539 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.539 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.539 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.539 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.539 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.539 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.539 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.539 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.540 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.540 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.540 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.540 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.540 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.540 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.540 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.540 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.540 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.541 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.541 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.541 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.541 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.541 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.541 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.542 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.542 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.542 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.542 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.542 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.542 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.542 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.542 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.543 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.543 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.543 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.543 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.543 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.544 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.544 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.544 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.544 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.544 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.544 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.544 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.544 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.544 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.545 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.545 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.545 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.545 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.545 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.545 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.545 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.546 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.546 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.546 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.546 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.546 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.547 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.547 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.547 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.547 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.547 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.547 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.548 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-09-19T04:34:27.548 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-09-19T04:34:27.548 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.548 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.548 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.548 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-09-19T04:34:27.548 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.548 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.549 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.549 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:34:27.549 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.549 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.549 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.549 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.549 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.550 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-09-19T04:34:27.550 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.550 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.550 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.550 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.550 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.550 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.550 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.551 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.551 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.551 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.551 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.551 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.551 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.551 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.551 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.552 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.552 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.552 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.552 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.552 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.552 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.553 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.553 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.553 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.553 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.553 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.553 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.553 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.554 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.554 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.554 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.554 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.554 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.554 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.554 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.555 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.555 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.555 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.555 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.555 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.555 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.555 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.555 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.556 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.556 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.556 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.556 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.556 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.556 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.556 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.556 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.557 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.557 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.557 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.557 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.557 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.557 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.558 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.558 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.558 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.558 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.558 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.558 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.558 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.559 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.559 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.559 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.559 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.559 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-09-19T04:34:27.559 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-09-19T04:34:27.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.559 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.560 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.560 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.560 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.560 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.560 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.560 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.560 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.561 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.561 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.561 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-09-19T04:34:27.561 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-09-19T04:34:27.561 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.561 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.561 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.562 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-09-19T04:34:27.562 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.562 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.562 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.562 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.562 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.562 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.562 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.563 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-09-19T04:34:27.563 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-09-19T04:34:27.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.563 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.563 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.564 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.564 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.564 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.564 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.564 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.564 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.565 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-09-19T04:34:27.565 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-09-19T04:34:27.565 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.565 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.565 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.565 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.565 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.566 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.566 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.566 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.566 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.566 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.566 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.566 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-09-19T04:34:27.567 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-09-19T04:34:27.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:27.567 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.567 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.567 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.567 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.567 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.568 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.568 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.568 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.568 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.568 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-09-19T04:34:27.568 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-09-19T04:34:27.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.569 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.569 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.569 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.569 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.569 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.569 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.569 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.570 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.570 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.570 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-09-19T04:34:27.570 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-09-19T04:34:27.570 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.570 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.570 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.571 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.571 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.571 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.571 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.571 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.571 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.571 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.572 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-09-19T04:34:27.572 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-09-19T04:34:27.572 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.572 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.572 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.572 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.573 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.573 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.573 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.573 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.573 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.573 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.573 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-09-19T04:34:27.574 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-09-19T04:34:27.574 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.574 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.574 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.574 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.574 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:27.574 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-09-19T04:34:27.574 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.575 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.575 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.575 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.575 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.575 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-09-19T04:34:27.575 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-09-19T04:34:27.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.576 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.576 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.576 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.576 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.576 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.576 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.576 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.577 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.577 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.577 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.577 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-09-19T04:34:27.577 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-09-19T04:34:27.577 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.577 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.578 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.578 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.578 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.578 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.578 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.578 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.578 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.578 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.579 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-09-19T04:34:27.579 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-09-19T04:34:27.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.579 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.579 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.579 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.579 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.579 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.580 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.580 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.580 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.580 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.581 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-09-19T04:34:27.581 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.581 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.581 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.581 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-09-19T04:34:27.581 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-09-19T04:34:27.581 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.581 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.582 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.582 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.582 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.582 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.582 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.582 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:34:27.582 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.583 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.583 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.583 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-09-19T04:34:27.583 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-09-19T04:34:27.583 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.583 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.584 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.584 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.584 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.584 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.584 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:34:27.584 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.584 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.585 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.585 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.585 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.585 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.585 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.585 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.585 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.586 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.586 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.586 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.586 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.586 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.586 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.586 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.586 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.587 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.587 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.587 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.587 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.587 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.587 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.587 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.588 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.588 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.588 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.588 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.588 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.588 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.588 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.589 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.589 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.589 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.589 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.589 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.589 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.589 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.590 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.590 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.590 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.590 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.590 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.590 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.590 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.590 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.591 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.591 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.591 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.591 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.591 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.591 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.591 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.591 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.592 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.592 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.592 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.592 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.592 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.592 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.592 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.592 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.593 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.593 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.593 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-09-19T04:34:27.594 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.594 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.594 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.594 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.594 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.594 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.594 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.594 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.594 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.595 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.595 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.595 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.595 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.595 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.595 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.595 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.595 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.595 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.595 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.596 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.596 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.596 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.596 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.596 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.596 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.597 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.597 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.597 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.597 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.597 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.597 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.597 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.597 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.598 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.598 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.598 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.598 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.598 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.598 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.599 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.599 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.599 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.599 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.599 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.599 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.599 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.599 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.600 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.600 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.600 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.600 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.600 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.600 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.601 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.601 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.601 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.601 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.601 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.601 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.601 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.601 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.602 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.602 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.602 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.602 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.602 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.602 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.602 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-09-19T04:34:27.603 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.603 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.603 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.603 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.603 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.603 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.604 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.604 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.604 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.604 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.604 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.604 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.604 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.604 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.605 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.605 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.605 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.605 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.605 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.605 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.605 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.605 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.606 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.606 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.606 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.606 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.606 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.606 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.606 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.606 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.607 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.607 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.607 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.607 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.607 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.607 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.607 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.608 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.608 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.608 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.608 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.608 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.608 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.608 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.608 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.609 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.609 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.609 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.609 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.609 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.609 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.609 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.609 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.610 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.610 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.610 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.610 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.610 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.610 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.610 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.610 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.611 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.611 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.611 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.611 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.611 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.611 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.611 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.611 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.612 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.612 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-09-19T04:34:27.612 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.612 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.612 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.612 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-09-19T04:34:27.613 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-09-19T04:34:27.613 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.613 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.613 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.613 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:34:27.613 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.613 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.613 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.614 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.614 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.614 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.614 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.614 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:27.614 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:27.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.614 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.615 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.615 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-09-19T04:34:27.615 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.615 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.615 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.615 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.615 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.615 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.615 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.616 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.616 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.616 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.616 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.616 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.616 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.616 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.617 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.617 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.617 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.617 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.617 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.617 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.617 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.617 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.618 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.618 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.618 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.618 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.618 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.618 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.618 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.618 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.619 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.619 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.619 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.619 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.619 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.619 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.619 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.619 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.620 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.620 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.620 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.620 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.620 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.620 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.620 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.621 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.621 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.621 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.621 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.621 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.621 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.621 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.621 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.622 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.622 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.622 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.622 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.622 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.622 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.622 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.623 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.623 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.623 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.623 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.623 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.623 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.623 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.623 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.624 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.624 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.624 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.624 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.624 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.624 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.624 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.625 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-09-19T04:34:27.625 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.625 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.625 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.625 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-09-19T04:34:27.626 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-09-19T04:34:27.626 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.626 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.626 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.626 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.626 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.626 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.626 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.627 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:34:27.627 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.627 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.627 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.627 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-09-19T04:34:27.627 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-09-19T04:34:27.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.628 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.628 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.628 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.628 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.628 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.628 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.628 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:34:27.628 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.629 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.629 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.629 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-09-19T04:34:27.629 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-09-19T04:34:27.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.629 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.629 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.630 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.630 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.630 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.630 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.630 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:34:27.630 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.630 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.630 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.631 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-09-19T04:34:27.631 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-09-19T04:34:27.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.631 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.631 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.631 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.631 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.632 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.632 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.632 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:34:27.632 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.632 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.632 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.633 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-09-19T04:34:27.633 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-09-19T04:34:27.633 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.633 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.633 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.633 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.633 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.633 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.633 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.634 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:34:27.634 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.634 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.634 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.634 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-09-19T04:34:27.634 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-09-19T04:34:27.635 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.635 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.635 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.635 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.635 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.635 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.635 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.635 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.635 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.636 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.636 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.636 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-09-19T04:34:27.636 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-09-19T04:34:27.636 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.636 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.636 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.637 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.637 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.637 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.637 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.637 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.637 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.638 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-09-19T04:34:27.638 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-09-19T04:34:27.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.638 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.640 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:34:27.640 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.640 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.640 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.641 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.641 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.641 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.641 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-09-19T04:34:27.641 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-09-19T04:34:27.641 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.642 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.642 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.642 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.642 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.642 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.642 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.642 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.642 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.643 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.643 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.643 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:27.643 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:27.643 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.643 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.643 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.644 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-09-19T04:34:27.644 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-09-19T04:34:27.644 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.644 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.644 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.644 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.644 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.645 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.645 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-09-19T04:34:27.645 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-09-19T04:34:27.645 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.645 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.645 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.645 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-09-19T04:34:27.646 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.646 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.646 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.646 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.646 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.646 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.646 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.647 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-09-19T04:34:27.647 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-09-19T04:34:27.647 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.647 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.647 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.647 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.647 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.647 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.648 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.648 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.648 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.648 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.648 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.648 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-09-19T04:34:27.648 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-09-19T04:34:27.649 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.649 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.649 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.649 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.649 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.649 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.649 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.649 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.650 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.650 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.650 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.650 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.650 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.650 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.650 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.651 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.651 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.651 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.651 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.651 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.651 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.651 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.652 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.652 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.652 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.652 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.652 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.652 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.652 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.652 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.653 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.653 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.653 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.653 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.653 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.654 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.654 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.654 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.654 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.654 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.654 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.654 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.654 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.655 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.655 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.655 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.655 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.655 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.656 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.656 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.656 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.656 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.656 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.656 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.656 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.656 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.657 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.657 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.657 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.657 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.657 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.657 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.657 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.657 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.658 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.658 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.658 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.658 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.658 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.658 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.658 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.659 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.659 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.659 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.659 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.659 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.659 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.659 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-09-19T04:34:27.660 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-09-19T04:34:27.660 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.660 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.660 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.660 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.660 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.660 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.661 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.661 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.661 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.661 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.661 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.661 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-09-19T04:34:27.661 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-09-19T04:34:27.662 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.662 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.662 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.662 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.662 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.662 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.662 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.663 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.663 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.663 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.663 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.663 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-09-19T04:34:27.663 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-09-19T04:34:27.663 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.664 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.664 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.664 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:34:27.664 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.664 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.664 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.664 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.664 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.665 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.665 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.665 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.665 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.665 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-09-19T04:34:27.666 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.666 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.666 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.666 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.666 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.666 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.666 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.667 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.667 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.667 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.667 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.667 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.667 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.667 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.667 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.667 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.668 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.668 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.668 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.668 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.668 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.668 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.668 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.669 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.669 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.669 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.669 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.669 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.669 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.669 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.670 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.670 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.670 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.670 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.670 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.670 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.670 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.671 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.671 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.671 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.671 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.671 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.671 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.671 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.672 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.672 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.672 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.672 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.672 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.672 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.672 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.672 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.672 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.673 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.673 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.673 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.673 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.673 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.674 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.674 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.674 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.674 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.674 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.674 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.674 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.674 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.675 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.675 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.675 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-09-19T04:34:27.675 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.676 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.676 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.676 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.676 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.676 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.676 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.676 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.677 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.677 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.677 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.677 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.677 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.677 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.677 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.677 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.677 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.678 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.678 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.678 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.678 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.678 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.678 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.678 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.679 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.679 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.679 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.679 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.679 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.679 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.679 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.679 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.679 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.680 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.680 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.680 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.680 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.680 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.680 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.681 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.681 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.681 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.681 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.681 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.681 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.681 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.681 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.681 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.682 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.682 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.682 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.682 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.682 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.682 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.683 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.683 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.683 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.683 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.683 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.683 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.683 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.683 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.683 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.684 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.684 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.684 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.684 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.684 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-09-19T04:34:27.684 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-09-19T04:34:27.684 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.684 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.685 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.685 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-09-19T04:34:27.685 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.685 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.685 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.685 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:34:27.685 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.686 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.686 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.686 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-09-19T04:34:27.686 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-09-19T04:34:27.686 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.686 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.687 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.687 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.687 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.687 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.687 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.687 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-09-19T04:34:27.687 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.687 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.688 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.688 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-09-19T04:34:27.688 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:34:27.688 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.688 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.689 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:27.689 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.689 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.689 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.689 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:34:27.689 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.689 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.689 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.690 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:34:27.690 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:34:27.690 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.690 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.690 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.690 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.690 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.690 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.691 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.691 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:34:27.691 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.691 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.691 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.691 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.691 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.692 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-09-19T04:34:27.692 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.692 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.692 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.692 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-09-19T04:34:27.692 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-09-19T04:34:27.693 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:27.693 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.693 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-09-19T04:34:27.693 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.693 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.693 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.694 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.694 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.694 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.694 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.694 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-09-19T04:34:27.694 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-09-19T04:34:27.694 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.694 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.695 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.695 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.695 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.695 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.695 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.695 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.695 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.695 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.696 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.696 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:27.696 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:27.696 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.696 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.696 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.696 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:27.696 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.697 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.697 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.697 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.697 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.697 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.697 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.697 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.698 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.698 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.698 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.698 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.698 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.698 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.698 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.698 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.698 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.699 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.699 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.699 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.699 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.699 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.699 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.699 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.699 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.699 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.700 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.700 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.700 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.700 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.700 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.700 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.700 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.701 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.701 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.701 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.701 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.701 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.701 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.701 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.702 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.702 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.702 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.702 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.702 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.702 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.702 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.703 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.703 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.703 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.703 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.703 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.703 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.703 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.703 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.704 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.704 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.704 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.704 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.704 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.704 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.705 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.705 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.705 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.705 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.705 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.705 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.705 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.705 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.706 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.706 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.706 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.706 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-09-19T04:34:27.706 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-09-19T04:34:27.706 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.706 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.707 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.707 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.707 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.707 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.707 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.707 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.707 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.708 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.708 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.708 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-09-19T04:34:27.708 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-09-19T04:34:27.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.708 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.709 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-09-19T04:34:27.709 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.709 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.709 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.709 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.709 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.710 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.710 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-09-19T04:34:27.710 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-09-19T04:34:27.710 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.710 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.710 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.710 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:34:27.711 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.711 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.711 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.711 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.711 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.711 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.711 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.712 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:27.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.712 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.712 INFO:teuthology.orchestra.run.smithi107.stdout: "modules": [ 2024-09-19T04:34:27.712 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-09-19T04:34:27.713 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard", 2024-09-19T04:34:27.713 INFO:teuthology.orchestra.run.smithi107.stdout: "iostat", 2024-09-19T04:34:27.713 INFO:teuthology.orchestra.run.smithi107.stdout: "nfs", 2024-09-19T04:34:27.713 INFO:teuthology.orchestra.run.smithi107.stdout: "restful", 2024-09-19T04:34:27.713 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_schedule" 2024-09-19T04:34:27.713 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.713 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-09-19T04:34:27.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.714 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-09-19T04:34:27.714 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.714 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.714 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.714 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:27.714 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:27.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.715 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.715 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.715 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:27.715 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.715 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.715 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.715 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:34:27.716 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.716 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.716 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.716 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.716 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.716 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.716 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.717 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.717 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.717 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.717 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.717 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.717 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.717 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.717 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.718 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.718 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.718 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.718 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.718 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.718 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.718 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.718 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.719 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.719 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.719 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.719 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.719 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.719 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.719 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.719 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.720 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.720 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.720 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.720 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.720 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.720 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.720 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.721 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.721 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.721 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.721 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.721 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.721 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.721 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.721 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.722 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.722 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.722 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.722 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.722 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.722 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.722 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.722 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.723 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.723 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.723 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.723 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.723 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.723 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.724 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.724 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.724 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.724 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.724 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.724 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.724 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.724 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.725 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-09-19T04:34:27.725 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-09-19T04:34:27.725 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.725 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.725 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.725 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.725 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.725 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.726 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.726 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:34:27.726 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.726 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.726 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.726 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-09-19T04:34:27.726 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-09-19T04:34:27.727 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.727 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.727 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.727 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-09-19T04:34:27.727 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.727 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.727 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.727 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-09-19T04:34:27.728 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.728 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.728 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.728 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-09-19T04:34:27.728 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-09-19T04:34:27.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.728 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.729 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.729 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.729 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.729 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.729 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.729 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-09-19T04:34:27.729 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.729 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.729 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.730 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-09-19T04:34:27.730 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-09-19T04:34:27.730 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.730 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.730 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.730 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.730 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.731 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.731 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.731 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-09-19T04:34:27.731 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.731 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.731 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.732 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-09-19T04:34:27.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-09-19T04:34:27.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.732 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.732 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-09-19T04:34:27.732 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.732 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.733 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.733 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-09-19T04:34:27.733 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.733 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.733 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.733 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-09-19T04:34:27.733 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-09-19T04:34:27.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.734 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.734 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.734 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.734 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.734 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.734 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:34:27.735 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.735 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.735 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-09-19T04:34:27.735 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-09-19T04:34:27.735 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.735 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.735 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.736 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.736 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.736 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.736 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.736 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:34:27.736 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.736 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.736 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.737 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-09-19T04:34:27.737 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-09-19T04:34:27.737 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.737 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.737 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.737 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.737 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.738 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.738 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.738 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-09-19T04:34:27.738 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.738 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.738 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.738 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.738 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-09-19T04:34:27.739 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.739 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.739 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.739 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-09-19T04:34:27.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-09-19T04:34:27.739 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.740 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.740 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.740 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.740 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.740 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.740 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:34:27.740 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.740 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.741 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.741 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-09-19T04:34:27.741 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-09-19T04:34:27.741 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.741 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.741 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.741 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-09-19T04:34:27.742 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.742 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.742 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.742 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:34:27.742 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:34:27.742 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.742 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.742 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-09-19T04:34:27.743 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-09-19T04:34:27.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:27.743 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.743 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.743 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:27.743 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-09-19T04:34:27.743 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-09-19T04:34:27.744 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.744 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:34:27.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:34:27.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.744 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-09-19T04:34:27.744 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:34:27.744 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:27.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.745 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-09-19T04:34:27.745 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:27.745 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-09-19T04:34:27.745 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.745 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:34:27.745 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.746 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.746 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-09-19T04:34:27.746 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-09-19T04:34:27.746 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.746 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.746 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:34:27.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.747 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.747 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:34:27.747 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:34:27.747 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.747 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.748 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-09-19T04:34:27.748 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-09-19T04:34:27.748 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:27.748 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.748 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.748 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-09-19T04:34:27.748 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-09-19T04:34:27.749 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-09-19T04:34:27.749 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.749 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:34:27.749 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:34:27.749 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.749 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.750 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-09-19T04:34:27.750 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-09-19T04:34:27.750 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.750 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.750 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.750 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-09-19T04:34:27.750 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.750 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.751 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.751 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:34:27.751 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:34:27.751 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.751 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.751 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-09-19T04:34:27.751 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-09-19T04:34:27.751 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:27.752 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.752 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.752 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:27.752 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-09-19T04:34:27.752 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-09-19T04:34:27.752 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.752 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:34:27.752 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:34:27.753 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.753 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.753 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.753 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.754 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.754 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.754 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.754 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.754 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.754 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.754 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.754 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.755 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.755 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.755 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.755 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.755 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.755 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.755 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.756 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.756 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.756 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.756 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.756 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.756 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.756 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.757 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.757 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.757 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.757 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.757 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.758 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.758 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.758 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.758 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.758 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.758 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.758 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.758 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.761 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.761 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.761 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.761 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.761 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.761 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.761 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.761 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.762 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-09-19T04:34:27.762 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-09-19T04:34:27.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:27.762 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.762 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.762 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:27.762 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.763 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:34:27.763 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.763 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.763 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.764 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-09-19T04:34:27.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-09-19T04:34:27.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.764 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.764 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.764 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-09-19T04:34:27.764 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.764 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.765 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.765 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-09-19T04:34:27.765 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-09-19T04:34:27.765 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-09-19T04:34:27.765 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-09-19T04:34:27.765 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-09-19T04:34:27.765 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-09-19T04:34:27.766 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.766 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.766 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.766 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-09-19T04:34:27.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-09-19T04:34:27.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.767 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.767 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.767 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.767 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.767 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.767 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.767 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:34:27.767 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.768 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.768 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.768 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:27.768 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:27.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.768 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.769 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:27.769 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.769 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.769 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:34:27.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.770 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.770 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-09-19T04:34:27.770 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-09-19T04:34:27.770 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.770 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.771 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:27.771 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:27.771 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.771 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.771 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:34:27.771 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:34:27.771 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.771 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.772 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-09-19T04:34:27.772 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:34:27.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:27.772 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.772 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.773 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.773 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.773 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.773 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:34:27.773 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.773 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.773 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.774 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.774 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-09-19T04:34:27.774 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.774 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.774 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.775 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-09-19T04:34:27.775 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-09-19T04:34:27.775 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:27.775 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.775 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.775 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-09-19T04:34:27.775 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.775 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.776 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.776 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:34:27.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.776 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.776 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.776 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-09-19T04:34:27.776 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-09-19T04:34:27.777 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.777 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.777 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.777 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-09-19T04:34:27.777 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.777 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.777 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.777 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:34:27.778 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.778 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.778 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.778 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-09-19T04:34:27.778 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-09-19T04:34:27.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.778 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.779 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-09-19T04:34:27.779 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.779 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.779 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.779 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:34:27.779 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.780 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.780 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-09-19T04:34:27.780 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-09-19T04:34:27.780 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.780 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.780 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.780 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.781 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.781 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.781 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.781 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:34:27.781 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:34:27.781 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.781 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.782 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-09-19T04:34:27.782 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-09-19T04:34:27.782 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.782 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.782 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.782 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:27.782 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.782 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.782 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.783 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:34:27.783 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.783 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.783 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.783 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:34:27.783 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:34:27.783 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:27.784 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.784 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.784 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-09-19T04:34:27.784 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.784 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.784 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.784 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:34:27.784 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.784 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-09-19T04:34:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-09-19T04:34:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.786 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.786 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.786 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-09-19T04:34:27.786 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-09-19T04:34:27.786 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-09-19T04:34:27.786 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.786 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:34:27.786 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.787 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.787 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.787 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-09-19T04:34:27.787 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-09-19T04:34:27.787 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.787 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.787 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.787 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.788 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.788 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.788 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.788 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:34:27.788 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.788 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.788 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.789 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-09-19T04:34:27.789 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-09-19T04:34:27.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.789 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.789 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.789 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.789 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.789 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.790 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.790 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:34:27.790 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.790 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.790 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.790 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-09-19T04:34:27.790 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-09-19T04:34:27.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.791 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.791 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.791 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.791 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.791 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.791 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.791 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:34:27.792 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.792 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.792 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.792 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-09-19T04:34:27.792 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-09-19T04:34:27.792 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.793 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.793 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.793 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:34:27.793 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.793 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.793 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.793 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:27.794 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.794 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.794 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.794 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-09-19T04:34:27.794 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-09-19T04:34:27.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.794 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.795 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:34:27.795 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.795 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.795 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.795 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:34:27.795 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.796 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.796 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.796 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-09-19T04:34:27.796 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:34:27.796 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.796 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.796 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.797 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:34:27.797 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.797 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.797 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.797 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-09-19T04:34:27.797 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.797 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.797 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.798 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-09-19T04:34:27.798 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-09-19T04:34:27.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.798 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.798 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.798 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:34:27.798 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.799 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.799 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.799 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:27.799 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.799 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.799 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.799 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-09-19T04:34:27.800 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-09-19T04:34:27.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.800 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.800 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:34:27.800 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.800 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.801 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.801 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-09-19T04:34:27.801 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.801 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.801 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.801 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-09-19T04:34:27.801 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:34:27.802 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.802 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.802 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.802 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:34:27.802 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.802 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.802 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.802 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:34:27.803 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.803 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.803 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.803 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-09-19T04:34:27.803 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:34:27.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.803 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.804 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:34:27.804 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.804 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.804 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.804 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:34:27.804 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.804 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.804 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.805 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-09-19T04:34:27.805 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:34:27.805 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.805 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.805 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.805 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:34:27.806 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.806 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.806 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.806 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-09-19T04:34:27.806 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.806 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.806 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.807 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-09-19T04:34:27.807 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-09-19T04:34:27.807 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.807 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.807 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.807 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:34:27.807 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.807 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.808 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.808 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-09-19T04:34:27.808 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.808 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.808 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.808 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-09-19T04:34:27.809 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-09-19T04:34:27.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.809 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.809 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:34:27.809 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.809 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.809 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.810 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-09-19T04:34:27.810 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.810 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.810 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.810 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-09-19T04:34:27.810 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-09-19T04:34:27.811 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.811 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.811 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.811 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:34:27.811 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.811 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.811 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.811 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:27.812 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.812 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.812 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-09-19T04:34:27.812 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-09-19T04:34:27.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.812 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.813 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.813 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:34:27.813 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.813 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.813 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.813 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-09-19T04:34:27.813 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.814 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.814 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-09-19T04:34:27.814 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-09-19T04:34:27.814 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.814 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.814 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.814 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:34:27.815 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.815 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.815 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.815 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-09-19T04:34:27.815 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.815 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.815 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.816 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-09-19T04:34:27.816 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-09-19T04:34:27.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.816 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.816 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:34:27.816 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.817 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.817 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.817 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-09-19T04:34:27.817 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.817 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.817 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.817 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_samba": { 2024-09-19T04:34:27.818 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_samba", 2024-09-19T04:34:27.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.818 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.818 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:34:27.818 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.818 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.818 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.819 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:34:27.819 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.819 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.819 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.819 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-09-19T04:34:27.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:34:27.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.820 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.820 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:34:27.820 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.820 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.820 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.820 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:34:27.820 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.821 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.821 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-09-19T04:34:27.821 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-09-19T04:34:27.821 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.821 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.821 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.822 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.822 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.822 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.822 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.822 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:34:27.822 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.822 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.823 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.823 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-09-19T04:34:27.823 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:34:27.823 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.823 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.823 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.823 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:27.823 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.824 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.824 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.824 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:34:27.824 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.824 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.824 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.824 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:34:27.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:34:27.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.825 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.825 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-09-19T04:34:27.825 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.825 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.825 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.825 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:34:27.826 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.826 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.826 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-09-19T04:34:27.826 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-09-19T04:34:27.826 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.826 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.827 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.827 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-09-19T04:34:27.827 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.827 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.827 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.827 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:34:27.827 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.827 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.827 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.828 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-09-19T04:34:27.828 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-09-19T04:34:27.828 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.828 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.828 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.828 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-09-19T04:34:27.828 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.829 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.829 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.829 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:34:27.829 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.829 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.829 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.830 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-09-19T04:34:27.830 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-09-19T04:34:27.830 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.830 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.830 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.830 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.830 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.830 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.831 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:34:27.831 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.831 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.831 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.831 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-09-19T04:34:27.831 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-09-19T04:34:27.831 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.832 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.832 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.832 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:27.832 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.832 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.832 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.832 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:34:27.832 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.833 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.833 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.833 INFO:teuthology.orchestra.run.smithi107.stdout: "grafana_dashboards_path": { 2024-09-19T04:34:27.833 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:34:27.833 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.833 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.834 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:34:27.834 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.834 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.834 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.834 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:34:27.834 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.834 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.835 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-09-19T04:34:27.835 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-09-19T04:34:27.835 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.835 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.835 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.835 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:27.836 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.836 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.836 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.836 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:34:27.836 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.836 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.836 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.837 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-09-19T04:34:27.837 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-09-19T04:34:27.837 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.837 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.837 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.837 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.837 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:34:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-09-19T04:34:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-09-19T04:34:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:34:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.842 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.842 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.842 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.842 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.842 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.842 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.842 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.842 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-09-19T04:34:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-09-19T04:34:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:34:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.845 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.845 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.845 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.845 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.845 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.845 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.845 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:34:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.847 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.847 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.847 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.847 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.847 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.847 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.847 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.847 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.848 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.848 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.848 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.848 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.848 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.848 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.848 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.849 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.849 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.849 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.849 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.849 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.849 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.849 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.850 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.850 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.850 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.850 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.850 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.850 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.850 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.851 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.851 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.851 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:34:27.851 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:34:27.851 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.851 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.852 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.852 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.852 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.852 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.852 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.852 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:34:27.852 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.852 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.852 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.853 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:34:27.853 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:34:27.853 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.853 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.853 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.853 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-09-19T04:34:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:34:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.855 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-09-19T04:34:27.855 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-09-19T04:34:27.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.855 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.855 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.855 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.855 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.855 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.856 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:34:27.856 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.856 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.856 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.856 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-09-19T04:34:27.856 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-09-19T04:34:27.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.856 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.857 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.857 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.857 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.857 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.857 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.857 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:34:27.857 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.857 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.858 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.858 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-09-19T04:34:27.858 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-09-19T04:34:27.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.858 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.858 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.859 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.859 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.859 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.859 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.859 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-09-19T04:34:27.859 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.859 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.859 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.860 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-09-19T04:34:27.860 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-09-19T04:34:27.860 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.860 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.860 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.860 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-09-19T04:34:27.860 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.861 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.861 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.861 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-09-19T04:34:27.861 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-09-19T04:34:27.861 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.861 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:34:27.861 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.861 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.862 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.862 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-09-19T04:34:27.862 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-09-19T04:34:27.862 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.862 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.862 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.862 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-09-19T04:34:27.863 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.863 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.863 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.863 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:34:27.863 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.863 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.863 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.864 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-09-19T04:34:27.864 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:34:27.864 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.864 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.864 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.864 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:34:27.864 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.865 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.865 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:34:27.865 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.865 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.865 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.865 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-09-19T04:34:27.865 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-09-19T04:34:27.865 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.866 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.866 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.866 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.866 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.866 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.866 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.866 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:34:27.866 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.867 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.867 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-09-19T04:34:27.867 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-09-19T04:34:27.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.867 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.868 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.868 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.868 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.868 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.868 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:34:27.868 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.869 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-09-19T04:34:27.869 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-09-19T04:34:27.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.869 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.869 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.869 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.869 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.870 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.870 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.870 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:34:27.870 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.870 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.870 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.870 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-09-19T04:34:27.870 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-09-19T04:34:27.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.871 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.871 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.871 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.871 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.871 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.871 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.871 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:34:27.871 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.872 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.872 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.872 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-09-19T04:34:27.872 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:34:27.872 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.872 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.873 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.873 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.873 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.873 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.873 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:34:27.873 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.873 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.874 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.874 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-09-19T04:34:27.874 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-09-19T04:34:27.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.874 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.874 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.874 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-09-19T04:34:27.875 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.875 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.875 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.875 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:34:27.875 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.875 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.876 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-09-19T04:34:27.876 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-09-19T04:34:27.876 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.876 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.876 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.876 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.876 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.876 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.876 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.877 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:34:27.877 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.877 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.877 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.877 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-09-19T04:34:27.877 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-09-19T04:34:27.877 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.877 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.878 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.878 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.878 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.878 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.878 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.878 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:34:27.878 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.879 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.879 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-09-19T04:34:27.879 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-09-19T04:34:27.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.879 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.879 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.880 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.880 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.880 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:34:27.880 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.880 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.880 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-09-19T04:34:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:34:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:34:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-09-19T04:34:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:34:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:34:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-09-19T04:34:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:34:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:34:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-09-19T04:34:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-09-19T04:34:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-09-19T04:34:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-09-19T04:34:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:34:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-09-19T04:34:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-09-19T04:34:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-09-19T04:34:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:34:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-09-19T04:34:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:34:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:34:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:34:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:34:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:34:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:34:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:34:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:34:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:34:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:34:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:34:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:34:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:34:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:34:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:34:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:34:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:34:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:34:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:34:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:34:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:34:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:34:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:34:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:34:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:34:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:34:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:34:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:34:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:34:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:34:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-09-19T04:34:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:34:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:34:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-09-19T04:34:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:34:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:34:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:34:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-09-19T04:34:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:34:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:34:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:34:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:34:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:34:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:34:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:34:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:34:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:34:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:34:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:34:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:34:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:34:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:34:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:34:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:34:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:34:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:34:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:34:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:34:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:34:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:34:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:34:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:34:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:34:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:34:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:34:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:34:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:34:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:34:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:34:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:34:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:34:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:34:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:34:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-09-19T04:34:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:34:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:34:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-09-19T04:34:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:34:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:34:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:34:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:34:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-09-19T04:34:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:34:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:34:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:34:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:34:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.969 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.969 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.969 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.969 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.969 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.969 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.969 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.969 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.971 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.971 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.971 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.971 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.971 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.971 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:34:27.972 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:34:27.972 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.972 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:27.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.972 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.972 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.973 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.973 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.973 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.973 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:34:27.973 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:34:27.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.974 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.974 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:27.974 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.974 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.974 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.974 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.974 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.975 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.975 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.975 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-09-19T04:34:27.975 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-09-19T04:34:27.975 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.975 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.975 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.975 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.976 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.976 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.976 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.976 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.976 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.976 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.976 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.977 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-09-19T04:34:27.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-09-19T04:34:27.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.977 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.977 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.977 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.977 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.978 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.978 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.978 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.978 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-09-19T04:34:27.978 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-09-19T04:34:27.978 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.978 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.979 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.979 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.979 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.979 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.979 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:34:27.979 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.979 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.980 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.980 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-09-19T04:34:27.980 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-09-19T04:34:27.980 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.980 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.980 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.980 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-09-19T04:34:27.981 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.981 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.981 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.981 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.981 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.981 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.981 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-09-19T04:34:27.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-09-19T04:34:27.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.982 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.982 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.982 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.982 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.982 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.983 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.983 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.983 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.983 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:27.983 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:27.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.984 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.984 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.984 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.984 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.984 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.984 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.985 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.985 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.985 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.985 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.985 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.985 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.985 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.985 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.986 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:27.986 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:27.986 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.986 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.986 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.988 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:27.988 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:27.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.988 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.988 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:27.988 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.988 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.990 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.990 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.990 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:27.990 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:27.990 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.990 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.990 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:27.991 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.991 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.991 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.991 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.991 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.991 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.991 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.992 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-09-19T04:34:27.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-09-19T04:34:27.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.992 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.992 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:27.992 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.992 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.992 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.993 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-09-19T04:34:27.993 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.993 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.993 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.993 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:34:27.993 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:34:27.993 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:27.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.994 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.994 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.994 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.995 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.995 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-09-19T04:34:27.995 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-09-19T04:34:27.995 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:27.995 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.995 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.996 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-09-19T04:34:27.996 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.996 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.996 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.996 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.996 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.996 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.996 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.997 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-09-19T04:34:27.997 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-09-19T04:34:27.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:27.997 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.997 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-09-19T04:34:27.997 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.997 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.998 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.998 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:27.998 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:27.998 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:27.998 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:27.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:27.998 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-09-19T04:34:27.999 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-09-19T04:34:27.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:27.999 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:27.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:27.999 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:27.999 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:27.999 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:27.999 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:27.999 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.000 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.000 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.000 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-09-19T04:34:28.000 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-09-19T04:34:28.000 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.000 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.001 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.001 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-09-19T04:34:28.001 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.001 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.001 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.001 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.001 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.001 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.002 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.002 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-09-19T04:34:28.002 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-09-19T04:34:28.002 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.002 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.002 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.002 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-09-19T04:34:28.002 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.003 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.003 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.003 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.003 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-09-19T04:34:28.003 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.003 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.003 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.004 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.004 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-09-19T04:34:28.004 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-09-19T04:34:28.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.004 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-09-19T04:34:28.004 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-09-19T04:34:28.005 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-09-19T04:34:28.005 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.005 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.005 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.005 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.005 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.005 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-09-19T04:34:28.006 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-09-19T04:34:28.006 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.006 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.006 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.006 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.006 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.006 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.006 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.007 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.007 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.007 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.007 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.008 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-09-19T04:34:28.008 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.008 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.008 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-09-19T04:34:28.008 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-09-19T04:34:28.008 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.008 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.009 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.009 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.009 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.009 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.009 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.009 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-09-19T04:34:28.009 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.010 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.010 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.010 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.010 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.010 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.010 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.010 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.011 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.011 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.011 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.011 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.011 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.011 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.011 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.011 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.012 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.012 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.012 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.012 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.012 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.012 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.012 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.013 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.013 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.013 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.013 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.013 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.013 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.013 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.014 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.014 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.014 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.014 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.014 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.014 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.014 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.014 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.015 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.015 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.015 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.015 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.015 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.015 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.015 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.016 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.016 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.016 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.016 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.016 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.016 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.017 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.017 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.017 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.017 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.017 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.017 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.018 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.018 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.018 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.018 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.018 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.018 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.018 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.019 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.019 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-09-19T04:34:28.019 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-09-19T04:34:28.019 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:28.019 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.019 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.019 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-09-19T04:34:28.020 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.020 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.020 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.020 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:34:28.020 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.020 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.020 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-09-19T04:34:28.021 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-09-19T04:34:28.021 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.021 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.021 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.021 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-09-19T04:34:28.021 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.021 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.021 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.021 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:34:28.022 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.022 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.022 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.022 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-09-19T04:34:28.022 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-09-19T04:34:28.022 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:28.022 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.022 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.023 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-09-19T04:34:28.023 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.023 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.023 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.023 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:34:28.023 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.024 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-09-19T04:34:28.024 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-09-19T04:34:28.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:28.024 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.024 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-09-19T04:34:28.024 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.025 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.025 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.025 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:34:28.025 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.025 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.025 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.025 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-09-19T04:34:28.026 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-09-19T04:34:28.026 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.026 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.026 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.026 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.026 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.026 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.027 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.027 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:34:28.027 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.027 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.027 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.027 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:28.027 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:28.028 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:28.028 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.028 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:28.028 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.028 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.028 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.028 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:34:28.029 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.029 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.029 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.029 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-09-19T04:34:28.029 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-09-19T04:34:28.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:28.029 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.030 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.030 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-09-19T04:34:28.030 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.030 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.030 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:34:28.030 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.030 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.030 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.031 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.031 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-09-19T04:34:28.031 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.031 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.031 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.031 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.032 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.032 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.032 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.032 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.032 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.032 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.032 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.033 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.033 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.033 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.033 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.033 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.033 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.034 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.034 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.034 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.034 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.034 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.034 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.035 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.035 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.035 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.035 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.035 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.036 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.036 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.036 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.036 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.036 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.037 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.037 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.037 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.037 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.037 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.037 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.037 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.038 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.038 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.038 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.038 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.038 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.038 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.038 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.039 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.039 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.039 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.039 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.039 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.039 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.040 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.040 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.040 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.040 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.041 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.041 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.041 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.041 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-09-19T04:34:28.041 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-09-19T04:34:28.041 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.041 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.042 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.042 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-09-19T04:34:28.042 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.042 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.042 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.042 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.042 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.043 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.043 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-09-19T04:34:28.043 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-09-19T04:34:28.043 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.043 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.044 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.044 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-09-19T04:34:28.044 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.044 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.044 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.044 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.044 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.044 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.045 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.045 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:28.045 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:28.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.045 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.045 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-09-19T04:34:28.046 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.046 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.046 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.046 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.046 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.046 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.046 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.046 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.047 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.047 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-09-19T04:34:28.047 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-09-19T04:34:28.047 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:34:28.047 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.047 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-09-19T04:34:28.048 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-09-19T04:34:28.048 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.048 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.048 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.048 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-09-19T04:34:28.048 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.048 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.048 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.049 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:34:28.049 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.049 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.052 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.052 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-09-19T04:34:28.053 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-09-19T04:34:28.053 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.053 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.053 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.053 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-09-19T04:34:28.053 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.053 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.054 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.054 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:34:28.054 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.054 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.054 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.054 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-09-19T04:34:28.054 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-09-19T04:34:28.055 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.055 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.055 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.055 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.055 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.055 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.055 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.055 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:34:28.056 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.056 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.056 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.056 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:28.056 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:28.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:28.056 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.057 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.057 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-09-19T04:34:28.057 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-09-19T04:34:28.057 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.057 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.057 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:34:28.057 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.058 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.058 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.058 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.058 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.058 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.058 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.059 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.059 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.059 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.059 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.059 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.059 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.059 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.059 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.060 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.060 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.060 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.060 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.060 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.060 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.061 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.061 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.061 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.061 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.061 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.061 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.061 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.061 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.062 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.062 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.062 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.062 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.062 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.062 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.063 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.063 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.063 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.063 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.063 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.063 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.064 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.064 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.064 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.064 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.064 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.064 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.064 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.065 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.065 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.065 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.065 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.065 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.065 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.066 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.066 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.066 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.066 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.066 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.066 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.067 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-09-19T04:34:28.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-09-19T04:34:28.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.067 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.067 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.067 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.067 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.068 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.068 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:34:28.068 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.068 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.068 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.068 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-09-19T04:34:28.069 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-09-19T04:34:28.069 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.069 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.069 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.069 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-09-19T04:34:28.069 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.069 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.070 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.070 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-09-19T04:34:28.070 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.070 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-09-19T04:34:28.070 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-09-19T04:34:28.071 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.071 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.071 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-09-19T04:34:28.071 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.071 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.071 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.071 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:34:28.072 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.072 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.072 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.072 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-09-19T04:34:28.072 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-09-19T04:34:28.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.072 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.073 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:28.073 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:28.073 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-09-19T04:34:28.073 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.073 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:34:28.073 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.074 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.074 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-09-19T04:34:28.074 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-09-19T04:34:28.074 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.074 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.074 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.074 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.074 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.075 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.075 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.075 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:34:28.075 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.075 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.075 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.075 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-09-19T04:34:28.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-09-19T04:34:28.076 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.076 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.076 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.076 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-09-19T04:34:28.076 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.076 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.076 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.076 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:34:28.077 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.077 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.077 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.077 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.077 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-09-19T04:34:28.077 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.078 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.078 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.078 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.078 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.078 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.078 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.079 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.079 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.079 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.079 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.079 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.079 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.079 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.079 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.080 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.080 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.080 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.080 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.080 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.080 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.080 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.081 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.081 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.081 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.081 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.081 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.081 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.082 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.082 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.082 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.082 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.082 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.082 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.083 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.083 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.083 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.083 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.083 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.083 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.083 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.083 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.084 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.084 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.084 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.084 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.084 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.084 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.084 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.084 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.085 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.085 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.085 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.085 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.085 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.086 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.086 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.086 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.086 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.086 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.086 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.087 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.087 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-09-19T04:34:28.087 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.087 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.088 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.088 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.088 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.088 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.088 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.088 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.089 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.089 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.089 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.089 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.089 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.089 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.089 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.090 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.090 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.090 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.090 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.090 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.090 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.091 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.091 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.091 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.091 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.091 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.091 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.091 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.091 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.091 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.092 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.092 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.092 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.092 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.092 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.092 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.093 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.093 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.093 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.093 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.093 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.093 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.093 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.093 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.094 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.094 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.094 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.094 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.094 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.094 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.094 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.095 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.095 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.095 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.095 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.095 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.096 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.096 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.096 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.096 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.096 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.096 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.097 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-09-19T04:34:28.097 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.097 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.097 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.097 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-09-19T04:34:28.097 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:34:28.097 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.098 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.098 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-09-19T04:34:28.098 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.098 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.098 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.098 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:34:28.098 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.099 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.099 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.099 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-09-19T04:34:28.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-09-19T04:34:28.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.099 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.100 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.100 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:28.100 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-09-19T04:34:28.100 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.100 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.100 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:34:28.100 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.101 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.101 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.101 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.101 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.101 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.101 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.102 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.102 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.102 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.102 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.102 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.102 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.102 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.102 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.103 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.103 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.103 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.103 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.103 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.103 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.103 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.104 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.104 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.104 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.104 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.104 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.104 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.104 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.104 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.105 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.105 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.105 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.105 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.105 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.105 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.105 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.106 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.106 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.106 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.106 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.106 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.106 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.106 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.106 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.107 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.107 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.107 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.107 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.107 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.107 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.107 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.107 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.108 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.108 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.108 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.108 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.108 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.108 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.109 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.109 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.109 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.109 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.109 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.109 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.109 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.109 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.110 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.110 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.110 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-09-19T04:34:28.110 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.110 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.110 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.111 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-09-19T04:34:28.111 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-09-19T04:34:28.111 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.111 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.111 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-09-19T04:34:28.111 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.111 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.112 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.112 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:34:28.112 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:34:28.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.112 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.112 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.112 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.113 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.113 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.113 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.113 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.113 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.113 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.114 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.114 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.114 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.114 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.114 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.114 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.114 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.114 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.115 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.115 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.115 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.115 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.115 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.115 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.116 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.116 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.116 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.116 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.116 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.116 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.117 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.117 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.117 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.117 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.117 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.118 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.118 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.118 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.118 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.118 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.118 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.119 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.119 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.119 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.119 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.119 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.119 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.119 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.120 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.120 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.120 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.120 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.120 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.120 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.120 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.120 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.121 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.121 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.121 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.121 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.121 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.121 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-09-19T04:34:28.122 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-09-19T04:34:28.122 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.122 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.122 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.122 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.122 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.122 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.122 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.123 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:34:28.123 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:34:28.123 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.123 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.123 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-09-19T04:34:28.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-09-19T04:34:28.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.124 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.124 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-09-19T04:34:28.124 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.124 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.124 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.124 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:34:28.124 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.125 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.125 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.125 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-09-19T04:34:28.125 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-09-19T04:34:28.125 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.125 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.126 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-09-19T04:34:28.126 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.126 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.126 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.126 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:34:28.126 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.126 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.127 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-09-19T04:34:28.127 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-09-19T04:34:28.127 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.127 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.127 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.128 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.128 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.128 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.128 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:34:28.128 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.128 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.129 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-09-19T04:34:28.129 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-09-19T04:34:28.129 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.129 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.129 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-09-19T04:34:28.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.129 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.130 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.130 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:34:28.130 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:34:28.130 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.130 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.130 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.130 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-09-19T04:34:28.131 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.131 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.131 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.131 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.132 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.132 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.132 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.132 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.132 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.133 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.133 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.133 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.133 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.133 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.133 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.133 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.133 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.133 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.134 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.134 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.134 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.134 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.134 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.136 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.136 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.136 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.136 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.136 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.136 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.137 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.137 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.137 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.137 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.137 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.137 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.137 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.137 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.138 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.138 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.138 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.138 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.138 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.138 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.138 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.139 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.139 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.139 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.139 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.139 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.139 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.140 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.140 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.140 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.140 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.140 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.140 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.140 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.141 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-09-19T04:34:28.141 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.141 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.141 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.141 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.141 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.141 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.142 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.142 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.142 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.142 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.142 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.142 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.142 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.143 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.143 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.143 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.143 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.143 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.143 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.143 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.144 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.144 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.144 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.144 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.144 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.144 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.144 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.145 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.145 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.145 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.145 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.145 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.145 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.145 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.145 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.146 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.146 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.146 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.146 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.146 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.146 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.147 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.147 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.147 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.147 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.147 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.147 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.147 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.148 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.148 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.148 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.148 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.148 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.148 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.148 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.149 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.149 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.149 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.149 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.149 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.149 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.150 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.150 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.150 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.150 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.150 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.150 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.150 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.150 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.151 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.151 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-09-19T04:34:28.151 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.151 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.151 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.151 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.151 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.152 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.152 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.152 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.152 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.152 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.152 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.152 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.152 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.153 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.153 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.153 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.153 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.153 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.153 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.153 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.153 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.154 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.154 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.154 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.154 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.154 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.154 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.156 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.156 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.156 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.156 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.156 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.156 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.157 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.157 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.157 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.157 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.157 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.157 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.157 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.157 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.158 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.158 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.158 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.158 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.158 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.158 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.158 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.159 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.159 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.159 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.159 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.159 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.159 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.159 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.160 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.160 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.160 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.160 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.160 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.160 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.160 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.160 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.161 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-09-19T04:34:28.161 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.161 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.161 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.161 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-09-19T04:34:28.161 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-09-19T04:34:28.162 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.162 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.162 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.162 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.162 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.162 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.162 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.163 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:34:28.163 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.163 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.163 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.163 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.164 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.164 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.164 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.164 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.164 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.164 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.164 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.164 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.165 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.165 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.165 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.165 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.165 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.165 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.165 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.166 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.166 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.166 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.166 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.166 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.166 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.166 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.166 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.167 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.167 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.167 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.167 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.167 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.167 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.167 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.168 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.168 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.168 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.168 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.168 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.168 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.168 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.168 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.169 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.169 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.169 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.169 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.169 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.169 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.169 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.170 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.170 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.170 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.170 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.170 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.170 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.171 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.171 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.171 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.171 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.171 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.171 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.171 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.171 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.172 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.172 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.172 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.172 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.172 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-09-19T04:34:28.172 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-09-19T04:34:28.172 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.173 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.173 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.173 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.173 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.173 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.173 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.173 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-09-19T04:34:28.174 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-09-19T04:34:28.174 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-09-19T04:34:28.174 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.174 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-09-19T04:34:28.174 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.174 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.174 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.175 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.175 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-09-19T04:34:28.175 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.175 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.175 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.176 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.176 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.176 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.176 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.176 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.176 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.176 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.176 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.177 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.177 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.177 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.177 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.177 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.177 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.177 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.177 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.177 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.178 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.178 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.178 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.178 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.178 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.178 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.178 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.179 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.179 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.179 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.179 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.179 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.179 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.179 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.179 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.180 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.180 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.180 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.180 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.180 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.180 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.180 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.180 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.181 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.181 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.181 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.181 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.181 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.181 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.181 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.181 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.182 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.182 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.182 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.182 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.182 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.182 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.182 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.182 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.183 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.183 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.183 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.183 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.183 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.183 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.183 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.184 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.184 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.184 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.184 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-09-19T04:34:28.184 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.184 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.185 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.185 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.185 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.185 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.185 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.185 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.185 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.185 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.186 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.186 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.186 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.186 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.186 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.186 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.186 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.187 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.187 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.187 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.187 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.187 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.187 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.187 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.188 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.188 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.188 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.188 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.188 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.188 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.188 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.188 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.189 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.189 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.189 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.189 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.189 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.189 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.189 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.190 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.190 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.190 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.190 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.190 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.190 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.190 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.190 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.191 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.191 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.191 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.191 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.191 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.191 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.191 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.191 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.192 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.192 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.192 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.192 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.192 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.192 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.193 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.193 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.193 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.193 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.193 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.193 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.194 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.194 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-09-19T04:34:28.194 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.194 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.194 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.194 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.195 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.195 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.195 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.195 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.195 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.195 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.195 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.195 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.196 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.196 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.196 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.196 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.196 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.196 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.196 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.197 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.197 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.197 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.197 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.197 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.197 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.197 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.198 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.198 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.198 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.198 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.198 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.198 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.198 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.198 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.199 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.199 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.199 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.199 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.199 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.199 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.199 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.200 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.200 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.200 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.200 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.200 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.200 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.200 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.200 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.201 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.201 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.201 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.201 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.201 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.201 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.201 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.202 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.202 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.202 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.202 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.202 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.202 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.202 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.203 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.203 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.203 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.203 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.203 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.203 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.203 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:28.204 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:28.204 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:28.204 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.204 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.204 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:28.204 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.204 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.204 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.205 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.205 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.205 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.205 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.205 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-09-19T04:34:28.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-09-19T04:34:28.206 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:28.206 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.206 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.206 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-09-19T04:34:28.206 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-09-19T04:34:28.206 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.206 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.206 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-09-19T04:34:28.207 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:34:28.207 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.207 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.207 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.207 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.207 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-09-19T04:34:28.207 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.208 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.208 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.208 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-09-19T04:34:28.208 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:34:28.208 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.208 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.208 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.209 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.209 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.209 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.209 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.209 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:34:28.209 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.209 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.209 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.210 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-09-19T04:34:28.210 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-09-19T04:34:28.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.210 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.210 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.210 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.210 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.211 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.211 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.211 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.211 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.211 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.211 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.212 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.212 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.212 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.212 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.212 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.213 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.213 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.213 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.213 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.213 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.213 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.213 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.214 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.214 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.214 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.214 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.214 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.214 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.215 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.215 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.215 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.215 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.215 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.215 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.215 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.215 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.215 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.216 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.216 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.216 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.216 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.216 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.217 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.217 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.217 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.217 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.217 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.217 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.217 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.218 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.218 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.218 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.218 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.218 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.218 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.218 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.219 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.219 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.219 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.219 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.220 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.220 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.220 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.220 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.220 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.220 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-09-19T04:34:28.220 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-09-19T04:34:28.221 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.221 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.221 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-09-19T04:34:28.221 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.221 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.221 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.221 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:34:28.222 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.222 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.222 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.222 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-09-19T04:34:28.222 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-09-19T04:34:28.222 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:28.222 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.223 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.223 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:28.223 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.223 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.223 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.223 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:34:28.223 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.223 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.223 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.224 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.224 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.224 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-09-19T04:34:28.224 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.224 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.224 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.225 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-09-19T04:34:28.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-09-19T04:34:28.225 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.225 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.225 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.225 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.225 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.225 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.225 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.226 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.226 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.226 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.226 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.226 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-09-19T04:34:28.226 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-09-19T04:34:28.226 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.227 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.227 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.227 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:34:28.227 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:34:28.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.228 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.228 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.228 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.228 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.228 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.228 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.229 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.229 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.229 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.229 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.229 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.229 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.230 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.230 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.230 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.230 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.230 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.230 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.230 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.231 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.231 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.231 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.231 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.231 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.231 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.231 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.231 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.232 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.232 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.232 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.232 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.232 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.232 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.232 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.233 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.233 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.233 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.233 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.233 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.233 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.234 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.234 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.234 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.234 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.234 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.234 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.234 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.234 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.235 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.235 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.235 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.235 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.235 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.235 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.235 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.236 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.236 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.236 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.236 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.236 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.236 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.236 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.237 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.237 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.237 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-09-19T04:34:28.237 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-09-19T04:34:28.237 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.237 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.237 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.238 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.238 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.238 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.238 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.238 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.238 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.238 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.238 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.239 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:34:28.239 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:34:28.239 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.239 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.239 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-09-19T04:34:28.239 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.240 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.240 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.240 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.240 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.240 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.240 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.240 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-09-19T04:34:28.240 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-09-19T04:34:28.241 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:28.241 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.241 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.241 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-09-19T04:34:28.241 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.241 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.241 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.242 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.242 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.242 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-09-19T04:34:28.242 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-09-19T04:34:28.242 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.243 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.243 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.243 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-09-19T04:34:28.243 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.243 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.243 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.243 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:34:28.243 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.244 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.244 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.244 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-09-19T04:34:28.244 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-09-19T04:34:28.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.244 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.245 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-09-19T04:34:28.245 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.245 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.245 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.245 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:34:28.245 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.245 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.246 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.246 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-09-19T04:34:28.246 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-09-19T04:34:28.246 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.246 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.246 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.246 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-09-19T04:34:28.247 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.247 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.247 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.247 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.247 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.247 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.247 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.248 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-09-19T04:34:28.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-09-19T04:34:28.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.248 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.248 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.248 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-09-19T04:34:28.248 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.249 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.249 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.249 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-09-19T04:34:28.249 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-09-19T04:34:28.249 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.249 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.249 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.249 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.249 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.250 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-09-19T04:34:28.250 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-09-19T04:34:28.250 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.250 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.250 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.250 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-09-19T04:34:28.250 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-09-19T04:34:28.251 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-09-19T04:34:28.251 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.251 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.251 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.251 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.251 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.252 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-09-19T04:34:28.252 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.252 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.252 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.252 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.252 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.252 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.253 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.253 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.253 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.253 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.253 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.253 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.254 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.254 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.254 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.254 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.254 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.254 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.254 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.254 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.255 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.255 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.255 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.255 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.255 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.256 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.256 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.256 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.256 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.256 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.256 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.261 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.261 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-09-19T04:34:28.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:34:28.261 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.261 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.261 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.261 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-09-19T04:34:28.262 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.262 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.262 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.262 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.262 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.262 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.262 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.263 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:34:28.263 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:34:28.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.263 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.263 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.263 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.263 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.264 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.264 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.264 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.264 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.264 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.264 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.264 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-09-19T04:34:28.265 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-09-19T04:34:28.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.265 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.265 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.265 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.265 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.266 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.266 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.266 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.266 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.266 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.266 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.267 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-09-19T04:34:28.267 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.267 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.267 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.267 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-09-19T04:34:28.267 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-09-19T04:34:28.267 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.268 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.268 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.268 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.268 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.268 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.268 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.268 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.268 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.269 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.269 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.269 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-09-19T04:34:28.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-09-19T04:34:28.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.269 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.270 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.270 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.270 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.270 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.270 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.270 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.270 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.270 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.271 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.271 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.271 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.271 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.271 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.271 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.272 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.272 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.272 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.272 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.272 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.272 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.272 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.272 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-09-19T04:34:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-09-19T04:34:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-09-19T04:34:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-09-19T04:34:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-09-19T04:34:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:34:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:34:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-09-19T04:34:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:34:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-09-19T04:34:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.296 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.296 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.296 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.296 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.296 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.296 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.296 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.296 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.297 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.297 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.297 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.297 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.297 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.297 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.298 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.298 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.298 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.298 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.298 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.298 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.298 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.299 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.299 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.299 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.299 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.300 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.300 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.300 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.300 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.300 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.300 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.300 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.300 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.301 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.301 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.301 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.301 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.301 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.302 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.302 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.302 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.302 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.302 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.302 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.302 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.302 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.302 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.303 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.303 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.303 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.303 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-09-19T04:34:28.303 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-09-19T04:34:28.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.304 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.304 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.304 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-09-19T04:34:28.304 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.304 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.304 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.304 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:34:28.304 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.304 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.305 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.305 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-09-19T04:34:28.305 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.306 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.306 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.306 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.306 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.306 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.306 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.307 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.307 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.307 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.307 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.307 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.307 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.307 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.308 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.308 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.308 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.308 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.308 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.308 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.308 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.309 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.309 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.309 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.309 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.309 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.309 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.309 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.310 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.310 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.310 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.310 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.310 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.310 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.311 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.311 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.311 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.311 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.311 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.311 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.312 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.312 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.312 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.312 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.312 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.312 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.312 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.313 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.313 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.313 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.313 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.313 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.313 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.314 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.314 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.314 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.314 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.314 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.314 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.314 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.315 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.315 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.315 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.315 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-09-19T04:34:28.315 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-09-19T04:34:28.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.316 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.316 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.316 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.316 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.316 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.316 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.316 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.316 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.317 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.317 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.317 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-09-19T04:34:28.317 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-09-19T04:34:28.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.317 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.318 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-09-19T04:34:28.318 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.318 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.318 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.318 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.318 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.318 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.319 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.319 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-09-19T04:34:28.319 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-09-19T04:34:28.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.319 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.319 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.320 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.320 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.320 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.320 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.320 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.321 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-09-19T04:34:28.321 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-09-19T04:34:28.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.321 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.321 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.321 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.321 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.322 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.322 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.322 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.322 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.322 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.322 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.322 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-09-19T04:34:28.323 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-09-19T04:34:28.323 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-09-19T04:34:28.323 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.323 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.323 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.323 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.323 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.323 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.324 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.324 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.324 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.324 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.324 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-09-19T04:34:28.324 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-09-19T04:34:28.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.324 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.326 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.326 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-09-19T04:34:28.326 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-09-19T04:34:28.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.326 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.326 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.326 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.327 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.327 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.327 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.327 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.327 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.327 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.327 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.328 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-09-19T04:34:28.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-09-19T04:34:28.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.328 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.328 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.328 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.328 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.329 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.329 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.329 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.329 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.329 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.329 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.329 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-09-19T04:34:28.330 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-09-19T04:34:28.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.330 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.330 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.330 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-09-19T04:34:28.330 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-09-19T04:34:28.330 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.331 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.331 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.331 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.331 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-09-19T04:34:28.331 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-09-19T04:34:28.332 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.332 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.332 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.332 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.332 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.332 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.332 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.332 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.333 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.333 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.333 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.333 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-09-19T04:34:28.333 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-09-19T04:34:28.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.334 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.334 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.334 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.334 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.334 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.334 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.334 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.334 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.335 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.335 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-09-19T04:34:28.335 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-09-19T04:34:28.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.335 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.335 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.336 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.336 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.336 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.336 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.336 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.336 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.336 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-09-19T04:34:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.338 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-09-19T04:34:28.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-09-19T04:34:28.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.338 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.338 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.338 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.338 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.339 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.339 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:34:28.339 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.339 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.339 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.339 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-09-19T04:34:28.339 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-09-19T04:34:28.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.340 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.340 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.340 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.340 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.340 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.340 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.341 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:34:28.341 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.341 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.341 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.341 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.341 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.341 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.344 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.344 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.344 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.344 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.344 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.345 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.345 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.345 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.345 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.345 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.345 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.345 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.346 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.346 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.346 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.346 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.346 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.346 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.348 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.348 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.348 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.348 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.348 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.348 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.349 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.349 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.349 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.349 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.349 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.349 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.349 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.350 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.350 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.350 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.350 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.350 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.350 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.350 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-09-19T04:34:28.351 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.351 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.351 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.351 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.352 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.352 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.352 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.352 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.352 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.352 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.352 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.352 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.352 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.353 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.353 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.353 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.353 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.353 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.353 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.353 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.354 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.354 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.354 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.354 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.354 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.354 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.355 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.355 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.355 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.355 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.355 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.355 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.355 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.356 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.356 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.356 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.356 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.356 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.356 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.356 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.357 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.357 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.357 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.357 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.357 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.357 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.357 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.357 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.358 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.358 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.358 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.358 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.358 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.358 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.358 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.359 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.359 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.359 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.359 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.359 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.359 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.359 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.359 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.360 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.360 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.360 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.360 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.360 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-09-19T04:34:28.360 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.361 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.361 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.361 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.361 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.361 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.361 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.361 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.361 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.362 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.362 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.362 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.362 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.362 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.362 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.362 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.362 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.363 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.363 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.363 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.363 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.363 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.363 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.363 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.364 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.364 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.364 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.364 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.364 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.364 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.364 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.364 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.365 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.365 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.365 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.365 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.365 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.365 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.366 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.366 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.366 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.366 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.366 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.366 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.366 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.366 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.367 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.367 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.367 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.367 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.367 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.367 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.367 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.367 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.368 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.368 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.368 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.368 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.368 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.368 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.368 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.369 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.369 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.369 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.369 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.369 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.369 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.369 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.369 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.369 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.370 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-09-19T04:34:28.370 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.370 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.370 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.370 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-09-19T04:34:28.370 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-09-19T04:34:28.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.371 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.371 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.371 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:34:28.371 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.371 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.371 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.371 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.372 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.372 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.372 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.372 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:28.372 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:28.372 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:28.372 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.373 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.373 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-09-19T04:34:28.373 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.373 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.373 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.373 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.373 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.373 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.374 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.374 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.374 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.374 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.374 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.374 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.374 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.375 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.375 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.375 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.375 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.375 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.375 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.375 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.375 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.375 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.376 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.376 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.376 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.376 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.376 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.376 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.377 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.377 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.377 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.377 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.377 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.377 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.377 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.377 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.378 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.378 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.378 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.378 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.378 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.378 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.379 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.379 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.379 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.379 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.379 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.379 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.379 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.379 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.380 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.380 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.380 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.380 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.380 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.380 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.380 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.380 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.381 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.381 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.381 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.381 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.381 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.381 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.382 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.382 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.382 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.382 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.382 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.382 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.382 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.383 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.383 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-09-19T04:34:28.383 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.383 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.383 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.383 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-09-19T04:34:28.384 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-09-19T04:34:28.384 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.384 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.384 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.384 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.384 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.384 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.384 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.385 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:34:28.385 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.385 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.385 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.385 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-09-19T04:34:28.385 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-09-19T04:34:28.385 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.386 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.386 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.386 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.386 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.386 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.386 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.386 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:34:28.386 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.387 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.387 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.387 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-09-19T04:34:28.387 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-09-19T04:34:28.387 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.387 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.387 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.388 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.388 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.388 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.388 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.388 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:34:28.388 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.388 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.388 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.389 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-09-19T04:34:28.389 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-09-19T04:34:28.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.389 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.389 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.389 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.390 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.390 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.390 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:34:28.390 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.390 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.390 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.390 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-09-19T04:34:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-09-19T04:34:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.392 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:34:28.392 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.392 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.392 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.392 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-09-19T04:34:28.392 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-09-19T04:34:28.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.394 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.394 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.394 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-09-19T04:34:28.394 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-09-19T04:34:28.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.394 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.394 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.395 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.395 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.395 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.395 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.395 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.395 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.395 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.396 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.396 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-09-19T04:34:28.396 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-09-19T04:34:28.396 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.396 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.396 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:34:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-09-19T04:34:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-09-19T04:34:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:28.400 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.400 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.400 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.400 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-09-19T04:34:28.400 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-09-19T04:34:28.400 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.400 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-09-19T04:34:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-09-19T04:34:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.402 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.402 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-09-19T04:34:28.402 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.402 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.402 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.402 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-09-19T04:34:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-09-19T04:34:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.404 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.404 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.404 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.404 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.404 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.404 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.404 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.405 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.405 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.405 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-09-19T04:34:28.405 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-09-19T04:34:28.405 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.405 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.405 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.406 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.406 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.406 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.406 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.406 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.406 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.406 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.406 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.407 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.407 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.407 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.407 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.407 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.407 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.407 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.408 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.408 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.408 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.408 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.408 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.408 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.408 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.408 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.409 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.409 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.409 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.409 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.409 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.409 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.409 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.410 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.410 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.410 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.410 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.410 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.410 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.410 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.410 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.411 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.411 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.411 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.411 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.411 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.411 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.411 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.411 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.412 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.412 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.412 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.412 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.412 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.412 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.412 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.412 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.413 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.413 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.413 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.413 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.413 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.413 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.413 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.414 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.414 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.414 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.414 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.414 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.414 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.414 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.415 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.415 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.415 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.415 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.415 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.415 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.415 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-09-19T04:34:28.416 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-09-19T04:34:28.416 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.416 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.416 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.416 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.416 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.416 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.416 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.417 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.417 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.417 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.417 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.417 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-09-19T04:34:28.417 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-09-19T04:34:28.417 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-09-19T04:34:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-09-19T04:34:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:34:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.421 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.421 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.421 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-09-19T04:34:28.421 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.421 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-09-19T04:34:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-09-19T04:34:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-09-19T04:34:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-09-19T04:34:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:34:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-09-19T04:34:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-09-19T04:34:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-09-19T04:34:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-09-19T04:34:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:34:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-09-19T04:34:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:34:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:34:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:34:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-09-19T04:34:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:34:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-09-19T04:34:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-09-19T04:34:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-09-19T04:34:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-09-19T04:34:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-09-19T04:34:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-09-19T04:34:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-09-19T04:34:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-09-19T04:34:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-09-19T04:34:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-09-19T04:34:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-09-19T04:34:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-09-19T04:34:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-09-19T04:34:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-09-19T04:34:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-09-19T04:34:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-09-19T04:34:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-09-19T04:34:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-09-19T04:34:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-09-19T04:34:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-09-19T04:34:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-09-19T04:34:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-09-19T04:34:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-09-19T04:34:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-09-19T04:34:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-09-19T04:34:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-09-19T04:34:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-09-19T04:34:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-09-19T04:34:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-09-19T04:34:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-09-19T04:34:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-09-19T04:34:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-19T04:34:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-09-19T04:34:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-09-19T04:34:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-09-19T04:34:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-09-19T04:34:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-09-19T04:34:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-09-19T04:34:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-09-19T04:34:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-09-19T04:34:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-09-19T04:34:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-09-19T04:34:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-09-19T04:34:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-09-19T04:34:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-09-19T04:34:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:34:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-09-19T04:34:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-09-19T04:34:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-09-19T04:34:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-09-19T04:34:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-09-19T04:34:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-09-19T04:34:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-09-19T04:34:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: "services": { 2024-09-19T04:34:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard": "https://172.21.15.107:8443/" 2024-09-19T04:34:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "always_on_modules": { 2024-09-19T04:34:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "octopus": [ 2024-09-19T04:34:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-09-19T04:34:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-09-19T04:34:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-09-19T04:34:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-09-19T04:34:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-09-19T04:34:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-09-19T04:34:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-09-19T04:34:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-09-19T04:34:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-09-19T04:34:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-09-19T04:34:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "pacific": [ 2024-09-19T04:34:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-09-19T04:34:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-09-19T04:34:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-09-19T04:34:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-09-19T04:34:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-09-19T04:34:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-09-19T04:34:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-09-19T04:34:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-09-19T04:34:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-09-19T04:34:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-09-19T04:34:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "quincy": [ 2024-09-19T04:34:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-09-19T04:34:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-09-19T04:34:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-09-19T04:34:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-09-19T04:34:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-09-19T04:34:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-09-19T04:34:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-09-19T04:34:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-09-19T04:34:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-09-19T04:34:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-09-19T04:34:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "reef": [ 2024-09-19T04:34:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-09-19T04:34:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-09-19T04:34:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-09-19T04:34:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-09-19T04:34:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-09-19T04:34:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-09-19T04:34:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-09-19T04:34:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-09-19T04:34:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-09-19T04:34:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-09-19T04:34:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-19T04:34:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "squid": [ 2024-09-19T04:34:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-09-19T04:34:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-09-19T04:34:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-09-19T04:34:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-09-19T04:34:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-09-19T04:34:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-09-19T04:34:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-09-19T04:34:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-09-19T04:34:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-09-19T04:34:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-09-19T04:34:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "last_failure_osd_epoch": 84, 2024-09-19T04:34:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "active_clients": [ 2024-09-19T04:34:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-09-19T04:34:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-19T04:34:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-19T04:34:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-09-19T04:34:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2518211116 2024-09-19T04:34:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "libcephsqlite", 2024-09-19T04:34:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-19T04:34:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-19T04:34:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-09-19T04:34:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 370029072 2024-09-19T04:34:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-09-19T04:34:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-19T04:34:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-19T04:34:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-09-19T04:34:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 449045805 2024-09-19T04:34:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-09-19T04:34:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-19T04:34:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-19T04:34:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-09-19T04:34:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 45985619 2024-09-19T04:34:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-09-19T04:34:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-19T04:34:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-19T04:34:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-09-19T04:34:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1300617104 2024-09-19T04:34:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-09-19T04:34:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-19T04:34:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-19T04:34:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-19T04:34:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-09-19T04:34:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3690114252 2024-09-19T04:34:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-19T04:34:28.484 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:34:28.486 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:26 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2990884044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:34:28.486 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:26 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3674840075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:34:28.486 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:27 smithi145 ceph-mon[35096]: pgmap v442: 97 pgs: 97 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 13 MiB/s wr, 63 op/s 2024-09-19T04:34:28.486 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:27 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/4116789313' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:34:28.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:26 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2990884044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:34:28.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:26 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3674840075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:34:28.487 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:27 smithi115 ceph-mon[35477]: pgmap v442: 97 pgs: 97 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 13 MiB/s wr, 63 op/s 2024-09-19T04:34:28.487 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:27 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/4116789313' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:34:28.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.170+0000 7fb196000640 1 -- 172.21.15.107:0/4116789313 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb1780a3440 msgr2=0x7fb1780a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:28.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.171+0000 7fb196000640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb1780a3440 0x7fb1780a5900 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb1901091a0 tx=0x7fb17c009260 comp rx=0 tx=0).stop 2024-09-19T04:34:28.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.171+0000 7fb196000640 1 -- 172.21.15.107:0/4116789313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb190103ad0 msgr2=0x7fb19019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:28.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.171+0000 7fb196000640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb190103ad0 0x7fb19019a3a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb18400e9f0 tx=0x7fb18400edf0 comp rx=0 tx=0).stop 2024-09-19T04:34:28.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.172+0000 7fb196000640 1 -- 172.21.15.107:0/4116789313 shutdown_connections 2024-09-19T04:34:28.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.172+0000 7fb196000640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb1780a3440 0x7fb1780a5900 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:28.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.172+0000 7fb196000640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb190109b70 0x7fb19019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:28.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.172+0000 7fb196000640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb190104470 0x7fb19019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:28.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.172+0000 7fb196000640 1 --2- 172.21.15.107:0/4116789313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb190103ad0 0x7fb19019a3a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:28.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.172+0000 7fb196000640 1 -- 172.21.15.107:0/4116789313 >> 172.21.15.107:0/4116789313 conn(0x7fb1900ff7d0 msgr2=0x7fb190100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:28.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.172+0000 7fb196000640 1 -- 172.21.15.107:0/4116789313 shutdown_connections 2024-09-19T04:34:28.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:26.172+0000 7fb196000640 1 -- 172.21.15.107:0/4116789313 wait complete. 2024-09-19T04:34:28.496 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T04:34:28.496 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T04:34:28.497 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:26 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2990884044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:34:28.497 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:26 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3674840075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:34:28.497 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:27 smithi107 ceph-mon[30659]: pgmap v442: 97 pgs: 97 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 13 MiB/s wr, 63 op/s 2024-09-19T04:34:28.497 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:27 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/4116789313' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:34:28.880 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:29.106 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:29 smithi107 ceph-mon[30659]: pgmap v443: 97 pgs: 97 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 13 MiB/s wr, 55 op/s 2024-09-19T04:34:29.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:29 smithi115 ceph-mon[35477]: pgmap v443: 97 pgs: 97 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 13 MiB/s wr, 55 op/s 2024-09-19T04:34:29.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:29 smithi145 ceph-mon[35096]: pgmap v443: 97 pgs: 97 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 13 MiB/s wr, 55 op/s 2024-09-19T04:34:29.757 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:34:29.757 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:29.758 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1666, 2024-09-19T04:34:29.758 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1668, 2024-09-19T04:34:29.758 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 8894727, 2024-09-19T04:34:29.758 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 529789, 2024-09-19T04:34:29.758 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 7, 2024-09-19T04:34:29.758 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-09-19T04:34:29.758 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.192953663, 2024-09-19T04:34:29.758 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.065990425, 2024-09-19T04:34:29.759 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.059122441, 2024-09-19T04:34:29.759 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.047379783, 2024-09-19T04:34:29.759 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:29.759 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1666, 2024-09-19T04:34:29.759 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.078539791, 2024-09-19T04:34:29.759 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000047142 2024-09-19T04:34:29.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.759 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:29.760 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.760 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.760 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.760 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:34:29.760 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:34:29.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.760 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:29.761 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1160, 2024-09-19T04:34:29.761 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1358, 2024-09-19T04:34:29.761 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 11764876, 2024-09-19T04:34:29.761 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 887488, 2024-09-19T04:34:29.761 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 8, 2024-09-19T04:34:29.761 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 6, 2024-09-19T04:34:29.761 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.151832222, 2024-09-19T04:34:29.762 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.048743070, 2024-09-19T04:34:29.762 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.058048365, 2024-09-19T04:34:29.762 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.031667240, 2024-09-19T04:34:29.762 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:29.762 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1356, 2024-09-19T04:34:29.762 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.081752819, 2024-09-19T04:34:29.762 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000060289 2024-09-19T04:34:29.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.763 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:29.763 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.763 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.763 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.763 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T04:34:29.763 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 520448 2024-09-19T04:34:29.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.764 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:29.764 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 686, 2024-09-19T04:34:29.764 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 645, 2024-09-19T04:34:29.764 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 6248750, 2024-09-19T04:34:29.764 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 280648, 2024-09-19T04:34:29.764 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-09-19T04:34:29.764 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-09-19T04:34:29.765 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.107490538, 2024-09-19T04:34:29.765 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.044192699, 2024-09-19T04:34:29.765 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.069689807, 2024-09-19T04:34:29.765 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.008283546, 2024-09-19T04:34:29.765 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:29.765 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 644, 2024-09-19T04:34:29.765 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.033518476, 2024-09-19T04:34:29.765 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000052047 2024-09-19T04:34:29.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.766 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:29.766 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.766 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.766 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.766 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1964096, 2024-09-19T04:34:29.767 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 167440 2024-09-19T04:34:29.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.767 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-19T04:34:29.767 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-09-19T04:34:29.767 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-19T04:34:29.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.767 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-09-19T04:34:29.767 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-19T04:34:29.768 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-19T04:34:29.768 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 5342, 2024-09-19T04:34:29.768 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.142063259, 2024-09-19T04:34:29.768 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000026593 2024-09-19T04:34:29.768 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:29.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.768 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-09-19T04:34:29.768 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-19T04:34:29.769 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-19T04:34:29.769 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.769 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.769 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.769 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:29.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.769 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-09-19T04:34:29.769 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-09-19T04:34:29.770 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-09-19T04:34:29.770 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-09-19T04:34:29.770 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.770 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.770 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.770 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-09-19T04:34:29.770 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-09-19T04:34:29.771 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-09-19T04:34:29.771 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-09-19T04:34:29.771 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-09-19T04:34:29.771 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-09-19T04:34:29.771 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-09-19T04:34:29.771 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 2395, 2024-09-19T04:34:29.771 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 4, 2024-09-19T04:34:29.771 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 2391, 2024-09-19T04:34:29.772 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-09-19T04:34:29.772 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 2390, 2024-09-19T04:34:29.772 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-09-19T04:34:29.772 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-09-19T04:34:29.772 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-09-19T04:34:29.772 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 5, 2024-09-19T04:34:29.772 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-09-19T04:34:29.772 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-09-19T04:34:29.773 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-09-19T04:34:29.773 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-09-19T04:34:29.773 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:34:29.773 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-09-19T04:34:29.773 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-09-19T04:34:29.773 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-09-19T04:34:29.773 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-09-19T04:34:29.774 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-09-19T04:34:29.774 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-09-19T04:34:29.774 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-09-19T04:34:29.774 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-09-19T04:34:29.774 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-09-19T04:34:29.774 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:34:29.774 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-09-19T04:34:29.774 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 0, 2024-09-19T04:34:29.775 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 0, 2024-09-19T04:34:29.775 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-09-19T04:34:29.775 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:34:29.775 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-09-19T04:34:29.775 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:34:29.775 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:34:29.775 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:34:29.775 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:34:29.775 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:34:29.776 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-09-19T04:34:29.776 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:34:29.776 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:34:29.776 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:34:29.776 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:34:29.776 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:34:29.776 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:34:29.777 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-09-19T04:34:29.777 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:34:29.777 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-09-19T04:34:29.777 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-09-19T04:34:29.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.777 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-09-19T04:34:29.777 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-09-19T04:34:29.777 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-09-19T04:34:29.778 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-09-19T04:34:29.778 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-09-19T04:34:29.778 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-09-19T04:34:29.778 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-09-19T04:34:29.778 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-09-19T04:34:29.778 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:34:29.778 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-09-19T04:34:29.778 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-09-19T04:34:29.779 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-09-19T04:34:29.779 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-09-19T04:34:29.779 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-09-19T04:34:29.779 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:34:29.779 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:34:29.779 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-09-19T04:34:29.779 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-09-19T04:34:29.779 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:34:29.780 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:34:29.780 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:34:29.780 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-09-19T04:34:29.780 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-09-19T04:34:29.780 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:34:29.780 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-09-19T04:34:29.780 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-09-19T04:34:29.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.781 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-09-19T04:34:29.781 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-09-19T04:34:29.781 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-09-19T04:34:29.781 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-09-19T04:34:29.781 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-09-19T04:34:29.781 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 16335, 2024-09-19T04:34:29.781 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-09-19T04:34:29.782 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-09-19T04:34:29.782 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-09-19T04:34:29.782 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-09-19T04:34:29.782 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-09-19T04:34:29.782 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 20, 2024-09-19T04:34:29.782 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 3, 2024-09-19T04:34:29.782 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-09-19T04:34:29.782 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-09-19T04:34:29.783 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-09-19T04:34:29.783 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 28245752, 2024-09-19T04:34:29.783 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 28241774, 2024-09-19T04:34:29.783 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-09-19T04:34:29.783 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.783 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.783 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.784 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 16335 2024-09-19T04:34:29.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.784 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-09-19T04:34:29.784 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 336, 2024-09-19T04:34:29.784 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 2357, 2024-09-19T04:34:29.784 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 2021, 2024-09-19T04:34:29.784 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 44, 2024-09-19T04:34:29.784 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 47, 2024-09-19T04:34:29.785 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 3, 2024-09-19T04:34:29.785 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 2395, 2024-09-19T04:34:29.785 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 2533, 2024-09-19T04:34:29.785 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 138, 2024-09-19T04:34:29.785 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-09-19T04:34:29.785 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-09-19T04:34:29.785 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-09-19T04:34:29.785 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 51772, 2024-09-19T04:34:29.786 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 215328 2024-09-19T04:34:29.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.786 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-09-19T04:34:29.786 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-09-19T04:34:29.786 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-09-19T04:34:29.786 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-09-19T04:34:29.786 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-09-19T04:34:29.786 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-09-19T04:34:29.787 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-09-19T04:34:29.787 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.787 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.787 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.787 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-09-19T04:34:29.787 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.788 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.788 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.788 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-09-19T04:34:29.788 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.788 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.788 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.789 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-09-19T04:34:29.789 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.789 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.789 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.789 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-09-19T04:34:29.790 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.790 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.790 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.790 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-09-19T04:34:29.790 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.790 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.791 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.791 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-09-19T04:34:29.791 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.791 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.791 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.791 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-09-19T04:34:29.792 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.792 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.792 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.792 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-09-19T04:34:29.792 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.792 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.793 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.793 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-09-19T04:34:29.793 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.793 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.793 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.793 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-09-19T04:34:29.794 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.794 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.794 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.794 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-09-19T04:34:29.795 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.795 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.795 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.795 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-09-19T04:34:29.795 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.796 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.796 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.796 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-09-19T04:34:29.796 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.796 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.796 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.797 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-09-19T04:34:29.797 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.797 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.797 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.798 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-09-19T04:34:29.798 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.798 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.798 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.798 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-09-19T04:34:29.798 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.799 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.799 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.799 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-09-19T04:34:29.799 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.799 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.799 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.800 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-09-19T04:34:29.800 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.800 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.800 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.800 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-09-19T04:34:29.800 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.801 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.801 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.801 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-09-19T04:34:29.801 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.801 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.801 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.802 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-09-19T04:34:29.802 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.802 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.802 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.802 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-09-19T04:34:29.802 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.803 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.803 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.803 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-09-19T04:34:29.803 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.803 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.803 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.804 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-09-19T04:34:29.804 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.804 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.804 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.804 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-09-19T04:34:29.804 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.804 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.805 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.805 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-09-19T04:34:29.805 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.805 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.805 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.805 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-09-19T04:34:29.806 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.806 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.806 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.806 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-09-19T04:34:29.806 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.806 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.806 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.807 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:34:29.807 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:34:29.807 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-09-19T04:34:29.807 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.807 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.807 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.807 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:29.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.808 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-09-19T04:34:29.808 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 2, 2024-09-19T04:34:29.808 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 4, 2024-09-19T04:34:29.808 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 2, 2024-09-19T04:34:29.808 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 3, 2024-09-19T04:34:29.808 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-09-19T04:34:29.809 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-09-19T04:34:29.809 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-09-19T04:34:29.809 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 2550, 2024-09-19T04:34:29.809 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-09-19T04:34:29.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.809 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-19T04:34:29.809 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:29.809 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:29.810 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:29.810 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:29.810 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:29.810 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:29.810 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:29.810 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:29.810 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:29.810 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:29.811 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:29.811 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:29.811 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:29.811 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:29.811 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:29.811 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:29.811 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:29.811 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:29.812 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:29.812 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:29.812 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:29.812 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:29.812 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:29.812 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:29.812 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:29.812 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:29.813 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:29.813 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:29.813 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:29.813 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:29.813 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:29.814 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-19T04:34:29.814 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:29.814 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:29.814 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:29.814 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:29.814 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 228713, 2024-09-19T04:34:29.814 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 413, 2024-09-19T04:34:29.814 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:34:29.815 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-19T04:34:29.815 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-19T04:34:29.815 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-19T04:34:29.815 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:29.815 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:29.815 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:29.815 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:29.815 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8032, 2024-09-19T04:34:29.816 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 177, 2024-09-19T04:34:29.816 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:29.816 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:29.816 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:29.816 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-19T04:34:29.816 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 2587436, 2024-09-19T04:34:29.816 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 18129, 2024-09-19T04:34:29.816 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:29.816 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-19T04:34:29.817 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:29.817 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-19T04:34:29.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.817 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-09-19T04:34:29.817 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-09-19T04:34:29.817 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-09-19T04:34:29.817 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 3176, 2024-09-19T04:34:29.817 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 21, 2024-09-19T04:34:29.818 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 3, 2024-09-19T04:34:29.818 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 3176, 2024-09-19T04:34:29.818 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-09-19T04:34:29.818 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3176, 2024-09-19T04:34:29.818 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2.388960815, 2024-09-19T04:34:29.818 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000752191 2024-09-19T04:34:29.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.818 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-09-19T04:34:29.819 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-09-19T04:34:29.819 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3176, 2024-09-19T04:34:29.819 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3177 2024-09-19T04:34:29.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.819 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 3176, 2024-09-19T04:34:29.819 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 3176, 2024-09-19T04:34:29.819 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-09-19T04:34:29.820 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-09-19T04:34:29.820 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-09-19T04:34:29.820 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 1827, 2024-09-19T04:34:29.820 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-09-19T04:34:29.820 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 1347, 2024-09-19T04:34:29.820 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-09-19T04:34:29.820 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-09-19T04:34:29.821 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-09-19T04:34:29.821 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-09-19T04:34:29.821 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-09-19T04:34:29.821 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-09-19T04:34:29.821 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-09-19T04:34:29.821 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-09-19T04:34:29.821 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-09-19T04:34:29.821 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-09-19T04:34:29.822 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 1, 2024-09-19T04:34:29.822 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-09-19T04:34:29.822 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:34:29.822 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-09-19T04:34:29.822 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:34:29.822 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-09-19T04:34:29.822 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-09-19T04:34:29.823 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-09-19T04:34:29.823 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:34:29.823 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-09-19T04:34:29.823 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:34:29.823 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-09-19T04:34:29.823 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-09-19T04:34:29.823 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-09-19T04:34:29.823 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-09-19T04:34:29.824 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-09-19T04:34:29.824 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-09-19T04:34:29.824 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-09-19T04:34:29.824 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-09-19T04:34:29.824 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-09-19T04:34:29.824 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-09-19T04:34:29.824 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-09-19T04:34:29.824 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-09-19T04:34:29.824 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-09-19T04:34:29.825 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-09-19T04:34:29.825 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-09-19T04:34:29.825 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-09-19T04:34:29.825 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-09-19T04:34:29.825 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 100, 2024-09-19T04:34:29.825 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-09-19T04:34:29.825 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 19, 2024-09-19T04:34:29.825 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 8, 2024-09-19T04:34:29.826 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 9, 2024-09-19T04:34:29.826 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 1, 2024-09-19T04:34:29.826 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-09-19T04:34:29.826 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-09-19T04:34:29.826 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-09-19T04:34:29.826 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-09-19T04:34:29.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.826 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-09-19T04:34:29.827 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-09-19T04:34:29.827 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:34:29.827 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-09-19T04:34:29.827 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-09-19T04:34:29.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.827 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-09-19T04:34:29.827 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-09-19T04:34:29.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.828 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-09-19T04:34:29.828 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-09-19T04:34:29.828 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:34:29.828 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-09-19T04:34:29.828 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-09-19T04:34:29.828 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-09-19T04:34:29.828 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-09-19T04:34:29.828 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-09-19T04:34:29.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.829 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:29.829 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:29.829 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-19T04:34:29.829 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:29.829 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 3508, 2024-09-19T04:34:29.829 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 26572830, 2024-09-19T04:34:29.830 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:29.830 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 3508, 2024-09-19T04:34:29.830 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-19T04:34:29.830 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-19T04:34:29.830 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3508, 2024-09-19T04:34:29.830 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 26572830, 2024-09-19T04:34:29.830 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:29.830 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.830 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.831 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.831 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:29.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.831 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-09-19T04:34:29.831 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:29.831 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-19T04:34:29.831 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:29.831 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-19T04:34:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-19T04:34:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3176, 2024-09-19T04:34:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 24056535, 2024-09-19T04:34:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3176, 2024-09-19T04:34:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 24056535, 2024-09-19T04:34:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-09-19T04:34:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:29.834 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-09-19T04:34:29.834 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:29.834 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-19T04:34:29.834 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-19T04:34:29.834 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:29.834 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:29.834 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3176, 2024-09-19T04:34:29.835 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 3176, 2024-09-19T04:34:29.835 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3176, 2024-09-19T04:34:29.835 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 3176, 2024-09-19T04:34:29.835 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:29.835 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.835 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.835 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.835 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:29.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.835 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:34:29.836 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:29.836 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-19T04:34:29.836 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:29.836 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-19T04:34:29.836 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-19T04:34:29.836 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:29.836 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:29.836 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-19T04:34:29.837 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-19T04:34:29.837 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-19T04:34:29.837 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-19T04:34:29.837 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:29.837 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.837 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.837 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:29.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:29.838 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x557985336be0": { 2024-09-19T04:34:29.838 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:29.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-19T04:34:29.838 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:29.838 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-19T04:34:29.838 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-19T04:34:29.838 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:29.839 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:29.839 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-19T04:34:29.839 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-19T04:34:29.839 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-19T04:34:29.839 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-19T04:34:29.839 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:29.839 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:29.839 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:29.840 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:29.840 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:29.840 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:29.840 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:34:31.069 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T04:34:31.069 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T04:34:31.069 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-19T04:34:31.070 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-19T04:34:31.070 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-19T04:34:31.070 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:34:31.070 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:34:31.070 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T04:34:31.070 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T04:34:31.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:31 smithi107 ceph-mon[30659]: pgmap v444: 97 pgs: 97 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 13 MiB/s wr, 55 op/s 2024-09-19T04:34:31.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:31 smithi115 ceph-mon[35477]: pgmap v444: 97 pgs: 97 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 13 MiB/s wr, 55 op/s 2024-09-19T04:34:31.447 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:31.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:31 smithi145 ceph-mon[35096]: pgmap v444: 97 pgs: 97 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 13 MiB/s wr, 55 op/s 2024-09-19T04:34:32.364 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:34:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-09-19T04:34:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-09-19T04:34:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-09-19T04:34:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-09-19T04:34:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-19T04:34:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-09-19T04:34:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.003372243, 2024-09-19T04:34:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T04:34:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000704995, 2024-09-19T04:34:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:34:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:34:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:34:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 287, 2024-09-19T04:34:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 273, 2024-09-19T04:34:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1745200, 2024-09-19T04:34:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 167552, 2024-09-19T04:34:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-09-19T04:34:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-09-19T04:34:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.063439327, 2024-09-19T04:34:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.026690161, 2024-09-19T04:34:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.052465646, 2024-09-19T04:34:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:34:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 273, 2024-09-19T04:34:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.024388800, 2024-09-19T04:34:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000089336 2024-09-19T04:34:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1745200, 2024-09-19T04:34:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 167552 2024-09-19T04:34:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2, 2024-09-19T04:34:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 207, 2024-09-19T04:34:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 320, 2024-09-19T04:34:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 193664, 2024-09-19T04:34:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-09-19T04:34:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-09-19T04:34:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.022635195, 2024-09-19T04:34:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.016296398, 2024-09-19T04:34:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.002835234, 2024-09-19T04:34:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:34:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 204, 2024-09-19T04:34:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.022409697, 2024-09-19T04:34:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000109851 2024-09-19T04:34:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:34:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 193664 2024-09-19T04:34:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-19T04:34:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-09-19T04:34:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-19T04:34:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-19T04:34:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-19T04:34:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 148977, 2024-09-19T04:34:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 30, 2024-09-19T04:34:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:34:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-19T04:34:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-19T04:34:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-19T04:34:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-09-19T04:34:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-09-19T04:34:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-19T04:34:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-09-19T04:34:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-09-19T04:34:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-19T04:34:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-19T04:34:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-19T04:34:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 285, 2024-09-19T04:34:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1702227, 2024-09-19T04:34:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 285, 2024-09-19T04:34:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-19T04:34:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-19T04:34:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 285, 2024-09-19T04:34:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1702227, 2024-09-19T04:34:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:32.383 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:34:33.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:33 smithi107 ceph-mon[30659]: pgmap v445: 97 pgs: 97 active+clean; 661 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 19 MiB/s wr, 132 op/s 2024-09-19T04:34:33.408 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T04:34:33.408 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T04:34:33.409 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-19T04:34:33.409 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-19T04:34:33.409 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-19T04:34:33.409 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-19T04:34:33.409 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-19T04:34:33.409 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T04:34:33.410 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T04:34:33.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:33 smithi115 ceph-mon[35477]: pgmap v445: 97 pgs: 97 active+clean; 661 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 19 MiB/s wr, 132 op/s 2024-09-19T04:34:33.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:33 smithi145 ceph-mon[35096]: pgmap v445: 97 pgs: 97 active+clean; 661 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 19 MiB/s wr, 132 op/s 2024-09-19T04:34:33.780 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:34.405 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:34:34.405 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:34.405 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1524, 2024-09-19T04:34:34.405 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1481, 2024-09-19T04:34:34.405 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 6542468, 2024-09-19T04:34:34.405 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 546952, 2024-09-19T04:34:34.406 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-09-19T04:34:34.406 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-09-19T04:34:34.406 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.216997313, 2024-09-19T04:34:34.406 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.095444293, 2024-09-19T04:34:34.406 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.091517175, 2024-09-19T04:34:34.406 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.032262461, 2024-09-19T04:34:34.406 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:34.407 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1481, 2024-09-19T04:34:34.407 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.088064069, 2024-09-19T04:34:34.407 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000059462 2024-09-19T04:34:34.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.407 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:34.407 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.407 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.407 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.408 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1967664, 2024-09-19T04:34:34.408 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 169168 2024-09-19T04:34:34.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.408 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:34.408 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1850, 2024-09-19T04:34:34.408 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2052, 2024-09-19T04:34:34.408 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 17250928, 2024-09-19T04:34:34.409 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 1111305, 2024-09-19T04:34:34.409 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 7, 2024-09-19T04:34:34.409 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-09-19T04:34:34.409 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.206721617, 2024-09-19T04:34:34.409 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.060517780, 2024-09-19T04:34:34.409 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.083748405, 2024-09-19T04:34:34.409 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.046885829, 2024-09-19T04:34:34.410 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:34.410 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2049, 2024-09-19T04:34:34.410 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.090115259, 2024-09-19T04:34:34.410 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000043980 2024-09-19T04:34:34.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.410 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:34.410 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.410 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.410 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.411 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:34:34.411 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 523584 2024-09-19T04:34:34.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.411 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:34.411 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 185, 2024-09-19T04:34:34.411 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 187, 2024-09-19T04:34:34.412 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 2528660, 2024-09-19T04:34:34.412 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 58842, 2024-09-19T04:34:34.412 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-09-19T04:34:34.412 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-09-19T04:34:34.412 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.025159665, 2024-09-19T04:34:34.412 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.005368842, 2024-09-19T04:34:34.412 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.012606427, 2024-09-19T04:34:34.412 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.005339110, 2024-09-19T04:34:34.413 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:34.413 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 185, 2024-09-19T04:34:34.413 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.009063991, 2024-09-19T04:34:34.413 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000048994 2024-09-19T04:34:34.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.413 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:34.413 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.413 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.414 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.414 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:34:34.414 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:34:34.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.414 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-19T04:34:34.414 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-09-19T04:34:34.414 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-19T04:34:34.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.415 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-09-19T04:34:34.415 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-19T04:34:34.415 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-19T04:34:34.415 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 5475, 2024-09-19T04:34:34.415 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.136553611, 2024-09-19T04:34:34.415 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000024941 2024-09-19T04:34:34.415 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:34.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.416 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-09-19T04:34:34.416 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-19T04:34:34.416 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-19T04:34:34.416 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.416 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.416 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.416 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:34.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.417 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-09-19T04:34:34.417 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-09-19T04:34:34.417 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-09-19T04:34:34.417 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-09-19T04:34:34.417 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.417 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.417 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.418 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-09-19T04:34:34.418 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-09-19T04:34:34.418 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-09-19T04:34:34.418 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-09-19T04:34:34.418 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-09-19T04:34:34.418 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-09-19T04:34:34.418 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-09-19T04:34:34.419 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 2844, 2024-09-19T04:34:34.419 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 9, 2024-09-19T04:34:34.419 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 2835, 2024-09-19T04:34:34.419 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-09-19T04:34:34.419 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 2832, 2024-09-19T04:34:34.419 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-09-19T04:34:34.419 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-09-19T04:34:34.419 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-09-19T04:34:34.419 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 6, 2024-09-19T04:34:34.420 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-09-19T04:34:34.420 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-09-19T04:34:34.420 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-09-19T04:34:34.420 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-09-19T04:34:34.420 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:34:34.420 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-09-19T04:34:34.420 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-09-19T04:34:34.420 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-09-19T04:34:34.421 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-09-19T04:34:34.421 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-09-19T04:34:34.421 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-09-19T04:34:34.421 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-09-19T04:34:34.421 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-09-19T04:34:34.421 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-09-19T04:34:34.421 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:34:34.421 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-09-19T04:34:34.422 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 0, 2024-09-19T04:34:34.422 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 0, 2024-09-19T04:34:34.422 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-09-19T04:34:34.422 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:34:34.422 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-09-19T04:34:34.422 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:34:34.422 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:34:34.422 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:34:34.423 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:34:34.423 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:34:34.423 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-09-19T04:34:34.423 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:34:34.423 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:34:34.423 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:34:34.423 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:34:34.423 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:34:34.424 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:34:34.424 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-09-19T04:34:34.424 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:34:34.424 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-09-19T04:34:34.424 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-09-19T04:34:34.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.424 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-09-19T04:34:34.424 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-09-19T04:34:34.425 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-09-19T04:34:34.425 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-09-19T04:34:34.425 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-09-19T04:34:34.425 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-09-19T04:34:34.425 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-09-19T04:34:34.425 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-09-19T04:34:34.425 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:34:34.426 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-09-19T04:34:34.426 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-09-19T04:34:34.426 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-09-19T04:34:34.426 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-09-19T04:34:34.426 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-09-19T04:34:34.426 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:34:34.426 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:34:34.426 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-09-19T04:34:34.427 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-09-19T04:34:34.427 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:34:34.427 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:34:34.427 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:34:34.427 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-09-19T04:34:34.427 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-09-19T04:34:34.427 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:34:34.427 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-09-19T04:34:34.428 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-09-19T04:34:34.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.428 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-09-19T04:34:34.428 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-09-19T04:34:34.428 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-09-19T04:34:34.428 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-09-19T04:34:34.428 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-09-19T04:34:34.429 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 14905, 2024-09-19T04:34:34.429 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-09-19T04:34:34.429 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-09-19T04:34:34.429 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-09-19T04:34:34.429 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-09-19T04:34:34.429 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-09-19T04:34:34.429 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 18, 2024-09-19T04:34:34.430 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 3, 2024-09-19T04:34:34.430 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-09-19T04:34:34.430 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-09-19T04:34:34.430 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-09-19T04:34:34.430 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 27647145, 2024-09-19T04:34:34.430 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 27634998, 2024-09-19T04:34:34.430 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-09-19T04:34:34.430 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.431 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.431 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.431 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 14905 2024-09-19T04:34:34.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.431 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-09-19T04:34:34.431 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 764, 2024-09-19T04:34:34.431 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 2890, 2024-09-19T04:34:34.432 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 2126, 2024-09-19T04:34:34.432 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 48, 2024-09-19T04:34:34.432 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 63, 2024-09-19T04:34:34.432 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 15, 2024-09-19T04:34:34.432 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 2844, 2024-09-19T04:34:34.432 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 3069, 2024-09-19T04:34:34.432 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 225, 2024-09-19T04:34:34.433 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-09-19T04:34:34.433 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-09-19T04:34:34.433 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-09-19T04:34:34.433 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 52844, 2024-09-19T04:34:34.433 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 215328 2024-09-19T04:34:34.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.433 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-09-19T04:34:34.434 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-09-19T04:34:34.434 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-09-19T04:34:34.434 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-09-19T04:34:34.434 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-09-19T04:34:34.434 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-09-19T04:34:34.434 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-09-19T04:34:34.434 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.434 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.435 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.435 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-09-19T04:34:34.435 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.435 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.435 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.435 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-09-19T04:34:34.436 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.436 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.436 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.436 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-09-19T04:34:34.436 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.436 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.437 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.437 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-09-19T04:34:34.437 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.437 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.437 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.438 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-09-19T04:34:34.438 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.438 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.438 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.438 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-09-19T04:34:34.439 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.439 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.439 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.439 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-09-19T04:34:34.439 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.439 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.439 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.440 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-09-19T04:34:34.440 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.440 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.440 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.440 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-09-19T04:34:34.440 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.441 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.441 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.441 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-09-19T04:34:34.441 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.441 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.441 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.442 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-09-19T04:34:34.442 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.443 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.443 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.443 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-09-19T04:34:34.443 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.443 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.443 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.444 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-09-19T04:34:34.444 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.444 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.444 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.444 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-09-19T04:34:34.445 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.445 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.445 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.445 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-09-19T04:34:34.445 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.445 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.445 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.446 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-09-19T04:34:34.446 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.446 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.446 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.446 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-09-19T04:34:34.447 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.447 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.447 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.447 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-09-19T04:34:34.447 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.447 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.448 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.448 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-09-19T04:34:34.448 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.448 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.448 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.448 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-09-19T04:34:34.449 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.449 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.449 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.449 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-09-19T04:34:34.449 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.449 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.449 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.450 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-09-19T04:34:34.450 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.450 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.450 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.450 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-09-19T04:34:34.450 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.451 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.451 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.451 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-09-19T04:34:34.451 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.451 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.451 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.452 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-09-19T04:34:34.452 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.452 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.452 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.452 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-09-19T04:34:34.452 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.453 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.453 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.453 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-09-19T04:34:34.453 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.453 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.453 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.454 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-09-19T04:34:34.454 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.454 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.454 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.454 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:34:34.454 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:34:34.454 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-09-19T04:34:34.455 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.455 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.455 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.455 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:34.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.455 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-09-19T04:34:34.455 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 2, 2024-09-19T04:34:34.455 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 3, 2024-09-19T04:34:34.456 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 1, 2024-09-19T04:34:34.456 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 3, 2024-09-19T04:34:34.456 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-09-19T04:34:34.456 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-09-19T04:34:34.456 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-09-19T04:34:34.456 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 2551, 2024-09-19T04:34:34.456 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-09-19T04:34:34.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.457 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-19T04:34:34.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:34.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:34.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:34.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:34.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:34.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:34.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:34.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:34.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:34.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:34.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:34.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:34.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:34.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:34.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:34.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:34.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:34.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:34.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:34.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:34.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:34.460 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:34.460 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:34.460 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:34.460 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:34.460 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:34.460 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:34.460 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:34.461 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:34.461 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:34.461 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:34.461 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-19T04:34:34.461 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:34.461 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:34.461 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:34.461 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:34.462 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 231217, 2024-09-19T04:34:34.462 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 967, 2024-09-19T04:34:34.462 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:34:34.462 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-19T04:34:34.462 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-19T04:34:34.462 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-19T04:34:34.462 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:34.463 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:34.463 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:34.463 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:34.463 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8032, 2024-09-19T04:34:34.463 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 177, 2024-09-19T04:34:34.463 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:34.463 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:34.464 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:34.464 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-19T04:34:34.464 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 4015853, 2024-09-19T04:34:34.464 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 31785, 2024-09-19T04:34:34.464 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:34.464 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-19T04:34:34.464 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:34.465 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-19T04:34:34.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.465 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-09-19T04:34:34.465 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-09-19T04:34:34.465 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-09-19T04:34:34.465 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 3221, 2024-09-19T04:34:34.465 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 21, 2024-09-19T04:34:34.466 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 3, 2024-09-19T04:34:34.466 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 3221, 2024-09-19T04:34:34.466 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-09-19T04:34:34.466 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3221, 2024-09-19T04:34:34.466 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2.190964052, 2024-09-19T04:34:34.466 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000680212 2024-09-19T04:34:34.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.466 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-09-19T04:34:34.467 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-09-19T04:34:34.467 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3221, 2024-09-19T04:34:34.467 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3222 2024-09-19T04:34:34.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.467 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 3221, 2024-09-19T04:34:34.467 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 3221, 2024-09-19T04:34:34.467 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-09-19T04:34:34.467 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-09-19T04:34:34.468 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-09-19T04:34:34.468 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 1819, 2024-09-19T04:34:34.468 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-09-19T04:34:34.468 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 1400, 2024-09-19T04:34:34.468 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-09-19T04:34:34.468 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-09-19T04:34:34.468 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-09-19T04:34:34.469 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-09-19T04:34:34.469 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-09-19T04:34:34.469 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-09-19T04:34:34.469 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-09-19T04:34:34.469 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-09-19T04:34:34.469 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-09-19T04:34:34.469 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-09-19T04:34:34.470 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 1, 2024-09-19T04:34:34.470 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-09-19T04:34:34.470 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:34:34.470 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-09-19T04:34:34.470 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:34:34.470 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-09-19T04:34:34.470 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-09-19T04:34:34.470 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-09-19T04:34:34.471 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:34:34.471 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-09-19T04:34:34.471 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:34:34.471 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-09-19T04:34:34.471 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-09-19T04:34:34.471 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-09-19T04:34:34.471 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-09-19T04:34:34.472 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-09-19T04:34:34.472 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-09-19T04:34:34.472 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-09-19T04:34:34.472 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-09-19T04:34:34.472 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-09-19T04:34:34.472 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-09-19T04:34:34.472 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-09-19T04:34:34.472 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-09-19T04:34:34.473 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-09-19T04:34:34.473 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-09-19T04:34:34.473 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-09-19T04:34:34.473 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-09-19T04:34:34.473 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-09-19T04:34:34.473 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 100, 2024-09-19T04:34:34.473 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-09-19T04:34:34.473 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 19, 2024-09-19T04:34:34.474 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 6, 2024-09-19T04:34:34.474 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 6, 2024-09-19T04:34:34.474 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-09-19T04:34:34.474 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-09-19T04:34:34.474 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-09-19T04:34:34.474 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-09-19T04:34:34.474 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-09-19T04:34:34.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.475 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-09-19T04:34:34.475 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-09-19T04:34:34.475 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:34:34.475 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-09-19T04:34:34.475 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-09-19T04:34:34.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.475 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-09-19T04:34:34.476 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-09-19T04:34:34.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.476 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-09-19T04:34:34.476 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-09-19T04:34:34.476 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:34:34.476 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-09-19T04:34:34.476 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-09-19T04:34:34.476 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-09-19T04:34:34.477 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-09-19T04:34:34.477 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-09-19T04:34:34.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.477 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:34.477 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:34.477 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-19T04:34:34.477 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:34.477 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 3555, 2024-09-19T04:34:34.478 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 25982199, 2024-09-19T04:34:34.478 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:34.478 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 3555, 2024-09-19T04:34:34.478 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-19T04:34:34.478 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-19T04:34:34.478 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3555, 2024-09-19T04:34:34.478 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 25982199, 2024-09-19T04:34:34.478 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:34.479 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.479 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.479 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.479 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:34.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.479 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-09-19T04:34:34.479 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:34.479 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-19T04:34:34.479 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:34.480 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-19T04:34:34.480 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-19T04:34:34.480 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:34.480 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:34.480 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3221, 2024-09-19T04:34:34.480 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 23458142, 2024-09-19T04:34:34.480 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3221, 2024-09-19T04:34:34.480 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 23458142, 2024-09-19T04:34:34.481 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:34.481 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.481 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.481 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.481 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:34.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.481 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-09-19T04:34:34.482 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:34.482 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-09-19T04:34:34.482 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:34.482 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-19T04:34:34.482 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-19T04:34:34.482 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:34.482 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:34.482 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3221, 2024-09-19T04:34:34.483 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 3221, 2024-09-19T04:34:34.483 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3221, 2024-09-19T04:34:34.483 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 3221, 2024-09-19T04:34:34.483 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:34.483 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.483 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.483 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.483 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:34.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.484 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:34:34.484 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:34.484 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-19T04:34:34.484 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:34.484 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-19T04:34:34.484 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-19T04:34:34.484 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:34.485 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:34.485 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-19T04:34:34.485 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-19T04:34:34.485 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-19T04:34:34.485 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-19T04:34:34.485 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:34.485 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.485 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.486 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.486 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:34.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:34.486 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x55acdd440be0": { 2024-09-19T04:34:34.486 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:34.486 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-19T04:34:34.486 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:34.487 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-19T04:34:34.487 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-19T04:34:34.487 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:34.487 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:34.487 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-19T04:34:34.487 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-19T04:34:34.487 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-19T04:34:34.487 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-19T04:34:34.488 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:34.488 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:34.488 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:34.488 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:34.488 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:34.488 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:34.488 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:34:35.252 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T04:34:35.253 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T04:34:35.253 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-19T04:34:35.253 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-19T04:34:35.253 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-19T04:34:35.253 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:34:35.253 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:34:35.253 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T04:34:35.254 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T04:34:35.305 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:35 smithi107 ceph-mon[30659]: pgmap v446: 97 pgs: 97 active+clean; 661 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 19 MiB/s wr, 124 op/s 2024-09-19T04:34:35.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:35 smithi115 ceph-mon[35477]: pgmap v446: 97 pgs: 97 active+clean; 661 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 19 MiB/s wr, 124 op/s 2024-09-19T04:34:35.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:35 smithi145 ceph-mon[35096]: pgmap v446: 97 pgs: 97 active+clean; 661 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 19 MiB/s wr, 124 op/s 2024-09-19T04:34:35.634 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:36.267 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-19T04:34:36.267 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:36.267 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 285, 2024-09-19T04:34:36.267 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 273, 2024-09-19T04:34:36.268 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1742720, 2024-09-19T04:34:36.268 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 168384, 2024-09-19T04:34:36.268 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-19T04:34:36.268 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-09-19T04:34:36.268 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.062557202, 2024-09-19T04:34:36.268 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.025285474, 2024-09-19T04:34:36.268 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.052098482, 2024-09-19T04:34:36.269 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:34:36.269 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:36.269 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 272, 2024-09-19T04:34:36.269 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.017469534, 2024-09-19T04:34:36.269 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000064226 2024-09-19T04:34:36.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.269 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:36.270 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:36.270 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:36.270 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:36.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.270 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1742720, 2024-09-19T04:34:36.270 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 168384 2024-09-19T04:34:36.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.270 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:36.271 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2, 2024-09-19T04:34:36.271 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 209, 2024-09-19T04:34:36.271 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 320, 2024-09-19T04:34:36.271 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 194016, 2024-09-19T04:34:36.271 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-09-19T04:34:36.271 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-09-19T04:34:36.271 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.024174384, 2024-09-19T04:34:36.271 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.016407117, 2024-09-19T04:34:36.271 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.004218433, 2024-09-19T04:34:36.272 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:34:36.272 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:36.272 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 206, 2024-09-19T04:34:36.272 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.026317778, 2024-09-19T04:34:36.272 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000127756 2024-09-19T04:34:36.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.272 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:36.272 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:36.273 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:36.273 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:36.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.273 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:34:36.273 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 194016 2024-09-19T04:34:36.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.273 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:36.273 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 4, 2024-09-19T04:34:36.274 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 4, 2024-09-19T04:34:36.274 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3024, 2024-09-19T04:34:36.274 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 448, 2024-09-19T04:34:36.274 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-09-19T04:34:36.274 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-09-19T04:34:36.274 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.003573197, 2024-09-19T04:34:36.274 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000067360, 2024-09-19T04:34:36.274 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.003454129, 2024-09-19T04:34:36.275 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:34:36.275 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:36.275 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-09-19T04:34:36.275 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.006035112, 2024-09-19T04:34:36.275 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002011704 2024-09-19T04:34:36.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.275 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:36.276 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:36.276 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:36.276 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:36.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.276 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T04:34:36.276 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T04:34:36.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.276 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-19T04:34:36.277 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-09-19T04:34:36.277 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-19T04:34:36.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.277 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-19T04:34:36.277 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:36.277 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:36.278 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:36.278 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:36.278 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:36.278 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:36.278 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:36.278 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:36.278 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:36.278 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:36.279 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:36.279 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:36.279 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:36.279 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:36.279 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:36.279 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:36.279 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:36.279 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:36.280 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:36.280 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:36.280 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:36.280 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:36.280 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:36.281 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:36.281 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:36.281 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:36.281 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:36.282 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:36.282 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:36.282 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:36.282 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:36.283 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-19T04:34:36.283 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:36.283 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:36.283 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:36.283 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:36.284 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 148977, 2024-09-19T04:34:36.284 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 30, 2024-09-19T04:34:36.284 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:34:36.284 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-19T04:34:36.285 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-19T04:34:36.285 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-19T04:34:36.285 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:36.285 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:36.286 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:36.286 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:36.286 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-09-19T04:34:36.286 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-09-19T04:34:36.287 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:36.287 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:36.287 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:36.288 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-19T04:34:36.288 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-09-19T04:34:36.288 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-09-19T04:34:36.288 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:36.288 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-19T04:34:36.289 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:36.289 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-19T04:34:36.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-19T04:34:36.289 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:36.289 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-19T04:34:36.289 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-19T04:34:36.289 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-19T04:34:36.289 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 287, 2024-09-19T04:34:36.290 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1702487, 2024-09-19T04:34:36.290 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:36.290 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 287, 2024-09-19T04:34:36.290 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-19T04:34:36.290 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-19T04:34:36.290 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 287, 2024-09-19T04:34:36.290 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1702487, 2024-09-19T04:34:36.290 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-19T04:34:36.291 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-19T04:34:36.291 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-19T04:34:36.291 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-19T04:34:36.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:36.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-19T04:34:36.291 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-19T04:34:36.991 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T04:34:36.991 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T04:34:36.991 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-19T04:34:36.991 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-19T04:34:36.991 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T04:34:36.991 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T04:34:36.991 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T04:34:36.991 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T04:34:36.992 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T04:34:37.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:37 smithi107 ceph-mon[30659]: pgmap v447: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 19 MiB/s wr, 180 op/s 2024-09-19T04:34:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:34:37.373 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:37 smithi115 ceph-mon[35477]: pgmap v447: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 19 MiB/s wr, 180 op/s 2024-09-19T04:34:37.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:37 smithi145 ceph-mon[35096]: pgmap v447: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 19 MiB/s wr, 180 op/s 2024-09-19T04:34:37.831 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-19T04:34:37.831 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:37.831 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 66364, 2024-09-19T04:34:37.831 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 103310, 2024-09-19T04:34:37.832 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 16877583, 2024-09-19T04:34:37.832 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 99547923, 2024-09-19T04:34:37.832 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 17, 2024-09-19T04:34:37.832 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-09-19T04:34:37.832 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 4.370301320, 2024-09-19T04:34:37.832 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 1.839172146, 2024-09-19T04:34:37.832 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 2.573903470, 2024-09-19T04:34:37.833 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.176878423, 2024-09-19T04:34:37.833 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:37.833 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 103306, 2024-09-19T04:34:37.833 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.758374368, 2024-09-19T04:34:37.833 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000017021 2024-09-19T04:34:37.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.833 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:37.833 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 153010, 2024-09-19T04:34:37.834 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.026540042, 2024-09-19T04:34:37.834 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000173 2024-09-19T04:34:37.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.834 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2110944, 2024-09-19T04:34:37.834 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 631088 2024-09-19T04:34:37.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.834 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:37.834 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 191621, 2024-09-19T04:34:37.835 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 237598, 2024-09-19T04:34:37.835 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 54432443, 2024-09-19T04:34:37.835 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 261827061, 2024-09-19T04:34:37.835 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 17, 2024-09-19T04:34:37.835 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2024-09-19T04:34:37.835 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 10.076550649, 2024-09-19T04:34:37.835 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 4.787843034, 2024-09-19T04:34:37.835 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 6.258689532, 2024-09-19T04:34:37.835 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.406410525, 2024-09-19T04:34:37.836 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:37.836 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 237599, 2024-09-19T04:34:37.836 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.941042959, 2024-09-19T04:34:37.836 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000016586 2024-09-19T04:34:37.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.836 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:37.836 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 288273, 2024-09-19T04:34:37.836 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.051625189, 2024-09-19T04:34:37.837 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000179 2024-09-19T04:34:37.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.837 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 432720, 2024-09-19T04:34:37.837 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 264496 2024-09-19T04:34:37.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.837 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:37.837 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 146204, 2024-09-19T04:34:37.837 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 209872, 2024-09-19T04:34:37.838 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 33154637, 2024-09-19T04:34:37.838 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 164158070, 2024-09-19T04:34:37.838 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 141, 2024-09-19T04:34:37.838 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 10, 2024-09-19T04:34:37.838 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 7.781962367, 2024-09-19T04:34:37.838 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 3.229951860, 2024-09-19T04:34:37.838 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 4.855514218, 2024-09-19T04:34:37.838 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.260207648, 2024-09-19T04:34:37.838 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:37.839 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 209870, 2024-09-19T04:34:37.839 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.532859789, 2024-09-19T04:34:37.839 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000016833 2024-09-19T04:34:37.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.839 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:37.839 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 332455, 2024-09-19T04:34:37.839 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.049478858, 2024-09-19T04:34:37.840 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000148 2024-09-19T04:34:37.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.840 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 42544, 2024-09-19T04:34:37.840 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 1136 2024-09-19T04:34:37.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.840 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-19T04:34:37.840 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-19T04:34:37.840 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-19T04:34:37.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.841 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-19T04:34:37.841 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-19T04:34:37.841 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-19T04:34:37.841 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 50182, 2024-09-19T04:34:37.841 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 25.871858976, 2024-09-19T04:34:37.841 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000515560 2024-09-19T04:34:37.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:37.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.842 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-19T04:34:37.842 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-19T04:34:37.842 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-19T04:34:37.842 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 9690, 2024-09-19T04:34:37.842 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.772436168, 2024-09-19T04:34:37.842 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000079714 2024-09-19T04:34:37.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:37.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.843 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-19T04:34:37.843 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 62684, 2024-09-19T04:34:37.843 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 57736, 2024-09-19T04:34:37.843 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-19T04:34:37.843 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 57736, 2024-09-19T04:34:37.843 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 598.851885292, 2024-09-19T04:34:37.843 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.010372244 2024-09-19T04:34:37.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.844 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-19T04:34:37.844 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 4948, 2024-09-19T04:34:37.844 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 4, 2024-09-19T04:34:37.844 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-19T04:34:37.844 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 56, 2024-09-19T04:34:37.844 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 5, 2024-09-19T04:34:37.845 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 3, 2024-09-19T04:34:37.845 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 469, 2024-09-19T04:34:37.845 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 11, 2024-09-19T04:34:37.845 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 17, 2024-09-19T04:34:37.846 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 441, 2024-09-19T04:34:37.846 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 452, 2024-09-19T04:34:37.846 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 3245, 2024-09-19T04:34:37.846 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 86, 2024-09-19T04:34:37.846 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 86, 2024-09-19T04:34:37.846 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 12, 2024-09-19T04:34:37.846 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 96634, 2024-09-19T04:34:37.846 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 70952, 2024-09-19T04:34:37.847 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 4941, 2024-09-19T04:34:37.847 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 182, 2024-09-19T04:34:37.847 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 1, 2024-09-19T04:34:37.847 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-19T04:34:37.847 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 338, 2024-09-19T04:34:37.847 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 32349, 2024-09-19T04:34:37.847 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 18, 2024-09-19T04:34:37.847 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 22, 2024-09-19T04:34:37.848 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 2327, 2024-09-19T04:34:37.848 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 11, 2024-09-19T04:34:37.848 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 617, 2024-09-19T04:34:37.848 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-19T04:34:37.848 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 696, 2024-09-19T04:34:37.848 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-19T04:34:37.848 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 656, 2024-09-19T04:34:37.848 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 181019030, 2024-09-19T04:34:37.849 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2024-09-19T04:34:37.849 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 696, 2024-09-19T04:34:37.849 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-19T04:34:37.849 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:34:37.849 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:34:37.849 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 330, 2024-09-19T04:34:37.849 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 17, 2024-09-19T04:34:37.850 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 313, 2024-09-19T04:34:37.850 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 366, 2024-09-19T04:34:37.850 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 14, 2024-09-19T04:34:37.850 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 17087, 2024-09-19T04:34:37.850 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 19315, 2024-09-19T04:34:37.850 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:34:37.851 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:34:37.851 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:34:37.851 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 32999, 2024-09-19T04:34:37.851 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 5464, 2024-09-19T04:34:37.851 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 2075, 2024-09-19T04:34:37.851 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 30175 2024-09-19T04:34:37.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.851 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-19T04:34:37.852 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 492, 2024-09-19T04:34:37.852 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-09-19T04:34:37.852 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-09-19T04:34:37.852 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 182, 2024-09-19T04:34:37.852 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 3848, 2024-09-19T04:34:37.852 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-09-19T04:34:37.852 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-19T04:34:37.852 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:34:37.853 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 4852, 2024-09-19T04:34:37.853 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 4852, 2024-09-19T04:34:37.853 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-19T04:34:37.853 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-19T04:34:37.853 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-19T04:34:37.853 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:34:37.853 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:34:37.854 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-19T04:34:37.854 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-19T04:34:37.854 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:34:37.854 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:34:37.854 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 17, 2024-09-19T04:34:37.854 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 23, 2024-09-19T04:34:37.854 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-19T04:34:37.854 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 8, 2024-09-19T04:34:37.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-19T04:34:37.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-19T04:34:37.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.855 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-19T04:34:37.855 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-19T04:34:37.855 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 43089, 2024-09-19T04:34:37.855 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-09-19T04:34:37.855 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-09-19T04:34:37.856 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 43089, 2024-09-19T04:34:37.856 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-19T04:34:37.856 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-09-19T04:34:37.856 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 86, 2024-09-19T04:34:37.856 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-09-19T04:34:37.856 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-09-19T04:34:37.856 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 86, 2024-09-19T04:34:37.857 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 5, 2024-09-19T04:34:37.857 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-19T04:34:37.857 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-09-19T04:34:37.857 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-09-19T04:34:37.857 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 305298086, 2024-09-19T04:34:37.857 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-09-19T04:34:37.857 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-19T04:34:37.857 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 25204, 2024-09-19T04:34:37.857 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 50.356750784, 2024-09-19T04:34:37.858 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001997966 2024-09-19T04:34:37.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.858 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-09-19T04:34:37.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.858 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-19T04:34:37.858 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 190, 2024-09-19T04:34:37.858 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 6992, 2024-09-19T04:34:37.859 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 6802, 2024-09-19T04:34:37.859 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 59, 2024-09-19T04:34:37.859 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 192, 2024-09-19T04:34:37.859 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 133, 2024-09-19T04:34:37.859 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 378, 2024-09-19T04:34:37.859 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 13907, 2024-09-19T04:34:37.859 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 13529, 2024-09-19T04:34:37.859 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 86, 2024-09-19T04:34:37.859 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 5798, 2024-09-19T04:34:37.860 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 5712, 2024-09-19T04:34:37.860 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 82060, 2024-09-19T04:34:37.860 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 231712 2024-09-19T04:34:37.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.860 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-19T04:34:37.860 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 89784, 2024-09-19T04:34:37.860 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 28207, 2024-09-19T04:34:37.861 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 62684, 2024-09-19T04:34:37.861 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 708, 2024-09-19T04:34:37.861 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 56712, 2024-09-19T04:34:37.861 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-19T04:34:37.861 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5478, 2024-09-19T04:34:37.861 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 22.219408892, 2024-09-19T04:34:37.861 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004056116 2024-09-19T04:34:37.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.862 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-19T04:34:37.862 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 17566, 2024-09-19T04:34:37.862 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 87.374184118, 2024-09-19T04:34:37.862 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004974051 2024-09-19T04:34:37.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.862 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-19T04:34:37.863 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.863 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.863 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.863 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-19T04:34:37.863 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.863 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.863 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.864 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-19T04:34:37.864 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 10858, 2024-09-19T04:34:37.864 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 215.689250623, 2024-09-19T04:34:37.864 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.019864546 2024-09-19T04:34:37.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.864 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-19T04:34:37.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.865 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.865 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-19T04:34:37.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.865 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.866 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-19T04:34:37.866 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.866 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.866 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.866 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-19T04:34:37.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.867 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.867 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-19T04:34:37.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.868 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.868 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.868 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-19T04:34:37.869 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 14, 2024-09-19T04:34:37.869 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.003211350, 2024-09-19T04:34:37.869 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000229382 2024-09-19T04:34:37.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.869 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-19T04:34:37.869 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 119, 2024-09-19T04:34:37.869 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.061411115, 2024-09-19T04:34:37.870 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.008919421 2024-09-19T04:34:37.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.870 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-19T04:34:37.870 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.870 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.870 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.871 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-19T04:34:37.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 14, 2024-09-19T04:34:37.871 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.770021818, 2024-09-19T04:34:37.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.197858701 2024-09-19T04:34:37.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.871 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-19T04:34:37.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 10, 2024-09-19T04:34:37.871 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.005863661, 2024-09-19T04:34:37.872 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000586366 2024-09-19T04:34:37.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.872 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-19T04:34:37.872 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 17731, 2024-09-19T04:34:37.872 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 42.680126887, 2024-09-19T04:34:37.872 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002407090 2024-09-19T04:34:37.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.872 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-19T04:34:37.873 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 961, 2024-09-19T04:34:37.873 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 202.151290687, 2024-09-19T04:34:37.873 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.210355141 2024-09-19T04:34:37.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.873 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-19T04:34:37.873 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.873 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.874 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.874 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-19T04:34:37.874 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.874 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.874 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.874 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-19T04:34:37.875 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-09-19T04:34:37.875 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.002701239, 2024-09-19T04:34:37.875 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002701239 2024-09-19T04:34:37.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.875 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-19T04:34:37.875 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 8, 2024-09-19T04:34:37.875 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 5.666145052, 2024-09-19T04:34:37.875 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.708268131 2024-09-19T04:34:37.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.876 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-19T04:34:37.876 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.876 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.876 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.876 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-19T04:34:37.877 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-09-19T04:34:37.877 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000703152, 2024-09-19T04:34:37.877 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000703152 2024-09-19T04:34:37.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.877 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-19T04:34:37.877 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.877 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.877 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.878 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-19T04:34:37.878 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 116, 2024-09-19T04:34:37.878 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.257775552, 2024-09-19T04:34:37.878 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002222203 2024-09-19T04:34:37.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.878 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-19T04:34:37.878 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.879 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.879 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.879 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-19T04:34:37.879 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7, 2024-09-19T04:34:37.879 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.018093821, 2024-09-19T04:34:37.879 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002584831 2024-09-19T04:34:37.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.879 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-19T04:34:37.880 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.880 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.880 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.880 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-19T04:34:37.880 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4852, 2024-09-19T04:34:37.880 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 18.951697325, 2024-09-19T04:34:37.881 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.003905955 2024-09-19T04:34:37.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.881 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:34:37.881 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:34:37.881 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-19T04:34:37.881 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.881 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.881 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:37.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.882 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-19T04:34:37.882 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-09-19T04:34:37.882 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 5, 2024-09-19T04:34:37.882 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 3, 2024-09-19T04:34:37.882 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2024-09-19T04:34:37.883 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-19T04:34:37.883 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 13438, 2024-09-19T04:34:37.883 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 6719, 2024-09-19T04:34:37.883 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 837, 2024-09-19T04:34:37.883 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-19T04:34:37.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.883 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-19T04:34:37.883 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:37.884 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:37.884 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:37.884 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:37.884 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:37.884 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:37.884 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:37.884 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:37.884 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:37.885 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:37.885 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:37.885 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:37.885 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:37.885 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:37.885 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:37.885 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:37.885 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:37.885 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:37.886 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:37.886 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:37.886 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:37.886 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:37.886 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:37.886 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:37.886 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:37.887 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:37.887 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:37.887 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:37.887 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:37.887 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:37.887 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:37.887 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-19T04:34:37.887 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:37.887 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:37.888 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:37.888 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:37.888 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 1112926, 2024-09-19T04:34:37.888 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 514, 2024-09-19T04:34:37.888 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:34:37.888 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-19T04:34:37.888 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-19T04:34:37.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-19T04:34:37.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:37.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:37.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:37.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:37.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 8032, 2024-09-19T04:34:37.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 177, 2024-09-19T04:34:37.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:37.890 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:37.890 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:37.890 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-19T04:34:37.890 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 1473668, 2024-09-19T04:34:37.890 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 25223, 2024-09-19T04:34:37.891 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:37.891 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-19T04:34:37.891 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:37.891 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-19T04:34:37.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.891 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-19T04:34:37.891 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-19T04:34:37.891 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-19T04:34:37.892 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 32928, 2024-09-19T04:34:37.892 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 302680357, 2024-09-19T04:34:37.892 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 37, 2024-09-19T04:34:37.892 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 32928, 2024-09-19T04:34:37.892 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-19T04:34:37.892 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 32928, 2024-09-19T04:34:37.892 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 81.888403740, 2024-09-19T04:34:37.892 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002486892 2024-09-19T04:34:37.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.893 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-19T04:34:37.893 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-19T04:34:37.893 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 32928, 2024-09-19T04:34:37.893 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 33573 2024-09-19T04:34:37.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.893 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 32928, 2024-09-19T04:34:37.893 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 1096, 2024-09-19T04:34:37.894 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 31832, 2024-09-19T04:34:37.894 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-19T04:34:37.894 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-19T04:34:37.894 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2024-09-19T04:34:37.894 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 11, 2024-09-19T04:34:37.894 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 376, 2024-09-19T04:34:37.894 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 25563, 2024-09-19T04:34:37.894 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 242, 2024-09-19T04:34:37.895 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-19T04:34:37.895 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-19T04:34:37.895 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-19T04:34:37.895 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 5, 2024-09-19T04:34:37.895 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 5459, 2024-09-19T04:34:37.895 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-19T04:34:37.895 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-19T04:34:37.895 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-19T04:34:37.895 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 713, 2024-09-19T04:34:37.896 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 22, 2024-09-19T04:34:37.896 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:34:37.896 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-19T04:34:37.896 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:34:37.896 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-19T04:34:37.896 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-19T04:34:37.896 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-19T04:34:37.896 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:34:37.897 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-19T04:34:37.897 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:34:37.897 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-09-19T04:34:37.897 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-19T04:34:37.897 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-19T04:34:37.897 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-19T04:34:37.897 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-19T04:34:37.898 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-19T04:34:37.898 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-19T04:34:37.898 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-19T04:34:37.898 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-19T04:34:37.898 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-19T04:34:37.898 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-19T04:34:37.898 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-19T04:34:37.898 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-19T04:34:37.899 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-19T04:34:37.899 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-19T04:34:37.899 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-19T04:34:37.899 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-19T04:34:37.899 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 100, 2024-09-19T04:34:37.899 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-19T04:34:37.899 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 19, 2024-09-19T04:34:37.899 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2024-09-19T04:34:37.900 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2024-09-19T04:34:37.900 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-09-19T04:34:37.900 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-19T04:34:37.900 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 967, 2024-09-19T04:34:37.900 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 11, 2024-09-19T04:34:37.900 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 204 2024-09-19T04:34:37.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.900 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-19T04:34:37.901 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-09-19T04:34:37.901 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:34:37.901 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-19T04:34:37.901 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-19T04:34:37.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.901 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-19T04:34:37.901 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-09-19T04:34:37.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.902 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-19T04:34:37.902 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-19T04:34:37.902 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T04:34:37.902 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-19T04:34:37.902 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 6, 2024-09-19T04:34:37.902 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 4852, 2024-09-19T04:34:37.902 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 4852, 2024-09-19T04:34:37.902 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-19T04:34:37.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.903 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:37.903 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:37.903 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-19T04:34:37.903 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:37.903 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 404185, 2024-09-19T04:34:37.903 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 69620623, 2024-09-19T04:34:37.904 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:37.904 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 404185, 2024-09-19T04:34:37.904 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:37.904 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:37.904 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 404185, 2024-09-19T04:34:37.904 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 69620623, 2024-09-19T04:34:37.904 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:37.905 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.905 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.905 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:37.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.905 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-19T04:34:37.905 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:37.905 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-19T04:34:37.906 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:37.906 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:37.906 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:37.906 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:37.906 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:37.906 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 32928, 2024-09-19T04:34:37.906 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 303171228, 2024-09-19T04:34:37.906 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 32928, 2024-09-19T04:34:37.907 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 303171228, 2024-09-19T04:34:37.907 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:37.907 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.907 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.907 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:37.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.908 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-19T04:34:37.908 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:37.908 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-19T04:34:37.908 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:37.908 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:37.908 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:37.908 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:37.908 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:37.909 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 32928, 2024-09-19T04:34:37.909 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 32928, 2024-09-19T04:34:37.909 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 32928, 2024-09-19T04:34:37.909 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 32928, 2024-09-19T04:34:37.909 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:37.909 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.909 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.910 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:37.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.910 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:34:37.910 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:37.910 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-19T04:34:37.910 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:37.910 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 4852, 2024-09-19T04:34:37.911 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 491028, 2024-09-19T04:34:37.911 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:37.911 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 4852, 2024-09-19T04:34:37.911 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:37.911 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:37.911 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 359, 2024-09-19T04:34:37.911 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 491028, 2024-09-19T04:34:37.911 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:37.912 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.912 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.912 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:37.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:37.912 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x5599a77ae960": { 2024-09-19T04:34:37.912 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:37.912 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-19T04:34:37.913 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:37.913 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 43089, 2024-09-19T04:34:37.913 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 301103782, 2024-09-19T04:34:37.913 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:37.913 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 43089, 2024-09-19T04:34:37.913 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:37.913 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:37.913 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 25204, 2024-09-19T04:34:37.914 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 301103782, 2024-09-19T04:34:37.914 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:37.914 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:37.914 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:37.914 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:37.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:37.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:37.914 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-19T04:34:38.749 INFO:tasks.check_counter:Daemon mds.b mds.exported=22 2024-09-19T04:34:38.749 INFO:tasks.check_counter:Daemon mds.b mds.imported=11 2024-09-19T04:34:38.749 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=492 2024-09-19T04:34:38.749 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-19T04:34:38.749 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-19T04:34:38.750 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=14 2024-09-19T04:34:38.750 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=8 2024-09-19T04:34:38.750 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T04:34:38.750 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T04:34:39.001 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:34:39.303 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:39 smithi115 ceph-mon[35477]: pgmap v448: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 6.6 MiB/s wr, 133 op/s 2024-09-19T04:34:39.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:39 smithi145 ceph-mon[35096]: pgmap v448: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 6.6 MiB/s wr, 133 op/s 2024-09-19T04:34:39.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:39 smithi107 ceph-mon[30659]: pgmap v448: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 6.6 MiB/s wr, 133 op/s 2024-09-19T04:34:39.545 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-19T04:34:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2190, 2024-09-19T04:34:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 2149, 2024-09-19T04:34:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 89712920, 2024-09-19T04:34:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 760862, 2024-09-19T04:34:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 7, 2024-09-19T04:34:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-09-19T04:34:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.247394748, 2024-09-19T04:34:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.071734156, 2024-09-19T04:34:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.136351679, 2024-09-19T04:34:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.043586201, 2024-09-19T04:34:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2148, 2024-09-19T04:34:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.078117498, 2024-09-19T04:34:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000036367 2024-09-19T04:34:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1966496, 2024-09-19T04:34:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 172240 2024-09-19T04:34:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 868, 2024-09-19T04:34:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1074, 2024-09-19T04:34:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 117662556, 2024-09-19T04:34:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 801358, 2024-09-19T04:34:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2024-09-19T04:34:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2024-09-19T04:34:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.129672834, 2024-09-19T04:34:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.023925276, 2024-09-19T04:34:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.080025752, 2024-09-19T04:34:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.018817071, 2024-09-19T04:34:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1071, 2024-09-19T04:34:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.040786353, 2024-09-19T04:34:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000038082 2024-09-19T04:34:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:34:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 525904 2024-09-19T04:34:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 942, 2024-09-19T04:34:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 942, 2024-09-19T04:34:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 96741148, 2024-09-19T04:34:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 298739, 2024-09-19T04:34:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 6, 2024-09-19T04:34:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-09-19T04:34:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.125954677, 2024-09-19T04:34:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.018202882, 2024-09-19T04:34:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.078631585, 2024-09-19T04:34:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.022842041, 2024-09-19T04:34:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 941, 2024-09-19T04:34:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.029497448, 2024-09-19T04:34:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000031346 2024-09-19T04:34:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T04:34:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T04:34:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-19T04:34:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-19T04:34:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-19T04:34:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-19T04:34:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-19T04:34:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-19T04:34:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6092, 2024-09-19T04:34:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.126234734, 2024-09-19T04:34:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000020721 2024-09-19T04:34:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-19T04:34:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-19T04:34:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-19T04:34:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-19T04:34:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2024-09-19T04:34:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2024-09-19T04:34:39.558 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-19T04:34:39.558 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.558 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.558 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.558 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-19T04:34:39.558 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-09-19T04:34:39.558 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-09-19T04:34:39.559 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-19T04:34:39.559 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2024-09-19T04:34:39.559 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-09-19T04:34:39.559 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-09-19T04:34:39.559 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 5090, 2024-09-19T04:34:39.559 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 0, 2024-09-19T04:34:39.559 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 5090, 2024-09-19T04:34:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-09-19T04:34:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 5090, 2024-09-19T04:34:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-09-19T04:34:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-09-19T04:34:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-09-19T04:34:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 11, 2024-09-19T04:34:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2024-09-19T04:34:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2024-09-19T04:34:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-09-19T04:34:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-09-19T04:34:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:34:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-19T04:34:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-09-19T04:34:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-09-19T04:34:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-09-19T04:34:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-09-19T04:34:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-09-19T04:34:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-09-19T04:34:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-09-19T04:34:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-19T04:34:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:34:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-19T04:34:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 656, 2024-09-19T04:34:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 181019030, 2024-09-19T04:34:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2024-09-19T04:34:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:34:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-19T04:34:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:34:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:34:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:34:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:34:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:34:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-09-19T04:34:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:34:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:34:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:34:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:34:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:34:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:34:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-09-19T04:34:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:34:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-09-19T04:34:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2024-09-19T04:34:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-19T04:34:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-09-19T04:34:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-09-19T04:34:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-09-19T04:34:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-09-19T04:34:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-09-19T04:34:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-09-19T04:34:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-19T04:34:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:34:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-09-19T04:34:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-09-19T04:34:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-19T04:34:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-19T04:34:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-19T04:34:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:34:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:34:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-19T04:34:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-19T04:34:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:34:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:34:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:34:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-09-19T04:34:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-19T04:34:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:34:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-19T04:34:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-19T04:34:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-19T04:34:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-19T04:34:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2024-09-19T04:34:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-09-19T04:34:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-09-19T04:34:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 43089, 2024-09-19T04:34:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-19T04:34:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-09-19T04:34:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2024-09-19T04:34:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-09-19T04:34:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-09-19T04:34:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 86, 2024-09-19T04:34:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 5, 2024-09-19T04:34:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-19T04:34:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-09-19T04:34:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-09-19T04:34:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 305298086, 2024-09-19T04:34:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 305287495, 2024-09-19T04:34:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-19T04:34:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 43089 2024-09-19T04:34:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-19T04:34:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 167, 2024-09-19T04:34:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 6672, 2024-09-19T04:34:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 6505, 2024-09-19T04:34:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 48, 2024-09-19T04:34:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 174, 2024-09-19T04:34:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 126, 2024-09-19T04:34:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 5090, 2024-09-19T04:34:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 7456, 2024-09-19T04:34:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 2366, 2024-09-19T04:34:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-09-19T04:34:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2024-09-19T04:34:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2024-09-19T04:34:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 61820, 2024-09-19T04:34:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 215328 2024-09-19T04:34:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-19T04:34:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2024-09-19T04:34:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-09-19T04:34:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2024-09-19T04:34:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2024-09-19T04:34:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-09-19T04:34:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-19T04:34:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-19T04:34:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-19T04:34:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-19T04:34:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-19T04:34:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-19T04:34:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-19T04:34:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-19T04:34:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-19T04:34:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-19T04:34:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-19T04:34:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-19T04:34:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-19T04:34:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-19T04:34:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-19T04:34:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-19T04:34:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-19T04:34:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-19T04:34:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-19T04:34:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-19T04:34:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-19T04:34:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-19T04:34:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-19T04:34:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-19T04:34:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-19T04:34:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-19T04:34:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-19T04:34:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-19T04:34:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-19T04:34:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:34:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:34:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-19T04:34:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-19T04:34:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-09-19T04:34:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 5, 2024-09-19T04:34:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 3, 2024-09-19T04:34:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 4, 2024-09-19T04:34:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-19T04:34:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2024-09-19T04:34:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2024-09-19T04:34:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 3616, 2024-09-19T04:34:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-19T04:34:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-19T04:34:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-19T04:34:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 251954, 2024-09-19T04:34:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 398, 2024-09-19T04:34:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:34:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-19T04:34:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-19T04:34:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-19T04:34:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 8032, 2024-09-19T04:34:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 177, 2024-09-19T04:34:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-19T04:34:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 4474276, 2024-09-19T04:34:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 37875, 2024-09-19T04:34:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-19T04:34:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-19T04:34:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-19T04:34:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-19T04:34:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-19T04:34:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 3655, 2024-09-19T04:34:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 26, 2024-09-19T04:34:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 4, 2024-09-19T04:34:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 3655, 2024-09-19T04:34:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-19T04:34:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3655, 2024-09-19T04:34:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.925942950, 2024-09-19T04:34:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000800531 2024-09-19T04:34:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-19T04:34:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-19T04:34:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3655, 2024-09-19T04:34:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3656 2024-09-19T04:34:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 3655, 2024-09-19T04:34:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 3655, 2024-09-19T04:34:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2024-09-19T04:34:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-19T04:34:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-19T04:34:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 2036, 2024-09-19T04:34:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2024-09-19T04:34:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1616, 2024-09-19T04:34:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2024-09-19T04:34:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2024-09-19T04:34:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-19T04:34:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-19T04:34:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-19T04:34:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-09-19T04:34:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2024-09-19T04:34:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-19T04:34:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-19T04:34:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-19T04:34:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 2, 2024-09-19T04:34:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2024-09-19T04:34:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:34:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-19T04:34:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:34:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-19T04:34:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-19T04:34:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-19T04:34:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:34:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-19T04:34:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:34:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-09-19T04:34:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-19T04:34:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-19T04:34:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-19T04:34:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-19T04:34:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-19T04:34:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-19T04:34:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-19T04:34:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-19T04:34:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-19T04:34:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-19T04:34:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-19T04:34:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-19T04:34:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-19T04:34:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-19T04:34:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-19T04:34:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-19T04:34:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 100, 2024-09-19T04:34:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-19T04:34:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 19, 2024-09-19T04:34:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 11, 2024-09-19T04:34:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 11, 2024-09-19T04:34:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-09-19T04:34:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-19T04:34:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2024-09-19T04:34:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2024-09-19T04:34:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2024-09-19T04:34:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-19T04:34:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2024-09-19T04:34:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:34:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-19T04:34:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-19T04:34:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-19T04:34:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-09-19T04:34:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-19T04:34:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-19T04:34:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:34:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-19T04:34:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-09-19T04:34:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-09-19T04:34:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-09-19T04:34:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-19T04:34:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-19T04:34:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 3996, 2024-09-19T04:34:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 303736719, 2024-09-19T04:34:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 3996, 2024-09-19T04:34:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3996, 2024-09-19T04:34:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 303736719, 2024-09-19T04:34:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-19T04:34:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-19T04:34:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3655, 2024-09-19T04:34:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 301134950, 2024-09-19T04:34:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3655, 2024-09-19T04:34:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 301134950, 2024-09-19T04:34:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-19T04:34:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-19T04:34:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3655, 2024-09-19T04:34:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 3655, 2024-09-19T04:34:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3655, 2024-09-19T04:34:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 3655, 2024-09-19T04:34:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:34:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-19T04:34:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-19T04:34:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-19T04:34:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x5603af33eb40": { 2024-09-19T04:34:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-19T04:34:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-19T04:34:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-19T04:34:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:39.629 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-19T04:34:40.384 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T04:34:40.384 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T04:34:40.384 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-19T04:34:40.384 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-19T04:34:40.384 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-19T04:34:40.384 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:34:40.385 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:34:40.385 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T04:34:40.385 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T04:34:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:34:41.113 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-19T04:34:41.113 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:41.114 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 5727, 2024-09-19T04:34:41.114 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 5724, 2024-09-19T04:34:41.114 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1579400, 2024-09-19T04:34:41.114 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 18813108, 2024-09-19T04:34:41.114 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 19, 2024-09-19T04:34:41.114 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-09-19T04:34:41.114 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.378419822, 2024-09-19T04:34:41.115 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.141452154, 2024-09-19T04:34:41.115 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.113655691, 2024-09-19T04:34:41.115 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.105821155, 2024-09-19T04:34:41.115 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:41.115 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5722, 2024-09-19T04:34:41.115 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.134905083, 2024-09-19T04:34:41.115 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000023576 2024-09-19T04:34:41.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.116 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:41.116 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1858, 2024-09-19T04:34:41.116 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000338204, 2024-09-19T04:34:41.116 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000182 2024-09-19T04:34:41.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.116 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1856, 2024-09-19T04:34:41.116 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 2864 2024-09-19T04:34:41.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.117 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:41.117 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 6666, 2024-09-19T04:34:41.117 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 6837, 2024-09-19T04:34:41.117 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1840081, 2024-09-19T04:34:41.117 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 18569593, 2024-09-19T04:34:41.117 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 13, 2024-09-19T04:34:41.117 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2024-09-19T04:34:41.118 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.434120198, 2024-09-19T04:34:41.118 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.154144062, 2024-09-19T04:34:41.118 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.131530402, 2024-09-19T04:34:41.118 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.127602714, 2024-09-19T04:34:41.118 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:41.118 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6835, 2024-09-19T04:34:41.118 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.182105708, 2024-09-19T04:34:41.118 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000026643 2024-09-19T04:34:41.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.119 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:41.119 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1851, 2024-09-19T04:34:41.119 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000296803, 2024-09-19T04:34:41.119 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000160 2024-09-19T04:34:41.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.119 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 5184, 2024-09-19T04:34:41.119 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 531456 2024-09-19T04:34:41.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.120 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:41.120 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 150007, 2024-09-19T04:34:41.120 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 135571, 2024-09-19T04:34:41.120 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 48736332, 2024-09-19T04:34:41.120 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 61571089, 2024-09-19T04:34:41.120 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 40, 2024-09-19T04:34:41.121 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2024-09-19T04:34:41.121 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 7.167456468, 2024-09-19T04:34:41.121 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 3.242584652, 2024-09-19T04:34:41.121 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 5.268162688, 2024-09-19T04:34:41.121 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.066484865, 2024-09-19T04:34:41.121 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:41.121 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 135570, 2024-09-19T04:34:41.121 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.904622256, 2024-09-19T04:34:41.122 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000014048 2024-09-19T04:34:41.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.122 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:41.122 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 232205, 2024-09-19T04:34:41.122 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.037001015, 2024-09-19T04:34:41.122 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000159 2024-09-19T04:34:41.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.123 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1993968, 2024-09-19T04:34:41.123 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 192272 2024-09-19T04:34:41.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.123 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-19T04:34:41.123 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-19T04:34:41.123 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-19T04:34:41.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.123 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-19T04:34:41.124 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-19T04:34:41.124 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-19T04:34:41.124 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 15895, 2024-09-19T04:34:41.124 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 6.365657347, 2024-09-19T04:34:41.124 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000400481 2024-09-19T04:34:41.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:41.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.124 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-19T04:34:41.125 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-19T04:34:41.125 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-19T04:34:41.125 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7145, 2024-09-19T04:34:41.125 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.324025418, 2024-09-19T04:34:41.125 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000045349 2024-09-19T04:34:41.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:41.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-19T04:34:41.126 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 25303, 2024-09-19T04:34:41.126 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 20571, 2024-09-19T04:34:41.126 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-19T04:34:41.126 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 20571, 2024-09-19T04:34:41.126 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 21.379923105, 2024-09-19T04:34:41.126 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001039323 2024-09-19T04:34:41.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.126 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-19T04:34:41.127 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 4732, 2024-09-19T04:34:41.127 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 1, 2024-09-19T04:34:41.127 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-19T04:34:41.127 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 11, 2024-09-19T04:34:41.127 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-09-19T04:34:41.127 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-09-19T04:34:41.127 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 549, 2024-09-19T04:34:41.127 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 58, 2024-09-19T04:34:41.127 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 463, 2024-09-19T04:34:41.128 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 28, 2024-09-19T04:34:41.128 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 467, 2024-09-19T04:34:41.128 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 320, 2024-09-19T04:34:41.128 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 151, 2024-09-19T04:34:41.128 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 151, 2024-09-19T04:34:41.128 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 12, 2024-09-19T04:34:41.128 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 54052, 2024-09-19T04:34:41.128 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 26620, 2024-09-19T04:34:41.129 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 4732, 2024-09-19T04:34:41.129 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 214, 2024-09-19T04:34:41.129 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:34:41.129 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-19T04:34:41.129 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 949, 2024-09-19T04:34:41.129 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 28592, 2024-09-19T04:34:41.129 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-09-19T04:34:41.129 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 2, 2024-09-19T04:34:41.129 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 161, 2024-09-19T04:34:41.130 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 5, 2024-09-19T04:34:41.130 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 433, 2024-09-19T04:34:41.130 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-19T04:34:41.130 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 161, 2024-09-19T04:34:41.130 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-19T04:34:41.131 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 43, 2024-09-19T04:34:41.131 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 7184928, 2024-09-19T04:34:41.131 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2024-09-19T04:34:41.131 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 161, 2024-09-19T04:34:41.131 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-19T04:34:41.131 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:34:41.131 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:34:41.131 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:34:41.132 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:34:41.132 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:34:41.132 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 161, 2024-09-19T04:34:41.132 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:34:41.132 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 2021, 2024-09-19T04:34:41.132 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 14505, 2024-09-19T04:34:41.133 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:34:41.133 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:34:41.133 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:34:41.133 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 12214, 2024-09-19T04:34:41.133 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 4831, 2024-09-19T04:34:41.133 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 910, 2024-09-19T04:34:41.133 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 20898 2024-09-19T04:34:41.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-19T04:34:41.134 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 828, 2024-09-19T04:34:41.134 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 1394, 2024-09-19T04:34:41.134 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 750, 2024-09-19T04:34:41.134 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 1430, 2024-09-19T04:34:41.134 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 1041, 2024-09-19T04:34:41.134 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-09-19T04:34:41.135 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-19T04:34:41.135 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:34:41.135 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 3210, 2024-09-19T04:34:41.135 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 3210, 2024-09-19T04:34:41.135 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-19T04:34:41.135 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-19T04:34:41.136 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-19T04:34:41.136 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:34:41.136 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:34:41.136 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-19T04:34:41.137 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-19T04:34:41.137 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:34:41.137 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:34:41.137 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:34:41.137 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 3, 2024-09-19T04:34:41.137 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-19T04:34:41.137 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:34:41.138 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-19T04:34:41.138 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-19T04:34:41.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-19T04:34:41.138 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-19T04:34:41.138 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 23574, 2024-09-19T04:34:41.138 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-09-19T04:34:41.138 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-09-19T04:34:41.138 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 23574, 2024-09-19T04:34:41.139 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-19T04:34:41.139 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-09-19T04:34:41.139 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 29, 2024-09-19T04:34:41.139 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-09-19T04:34:41.139 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-09-19T04:34:41.139 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 29, 2024-09-19T04:34:41.139 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 3, 2024-09-19T04:34:41.140 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-19T04:34:41.140 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-09-19T04:34:41.140 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-09-19T04:34:41.140 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 40523724, 2024-09-19T04:34:41.140 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-09-19T04:34:41.140 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-19T04:34:41.140 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7804, 2024-09-19T04:34:41.140 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 15.133022419, 2024-09-19T04:34:41.141 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001939136 2024-09-19T04:34:41.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.141 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-09-19T04:34:41.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-19T04:34:41.141 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 403, 2024-09-19T04:34:41.141 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 3849, 2024-09-19T04:34:41.141 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 3446, 2024-09-19T04:34:41.142 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 132, 2024-09-19T04:34:41.142 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 730, 2024-09-19T04:34:41.142 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 598, 2024-09-19T04:34:41.142 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 548, 2024-09-19T04:34:41.142 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 4330, 2024-09-19T04:34:41.142 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 3782, 2024-09-19T04:34:41.142 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 151, 2024-09-19T04:34:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 3668, 2024-09-19T04:34:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 3517, 2024-09-19T04:34:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 63684, 2024-09-19T04:34:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 223520 2024-09-19T04:34:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-19T04:34:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 47280, 2024-09-19T04:34:41.144 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 1207, 2024-09-19T04:34:41.144 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 25303, 2024-09-19T04:34:41.144 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 1303, 2024-09-19T04:34:41.144 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 8307, 2024-09-19T04:34:41.144 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-19T04:34:41.144 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3281, 2024-09-19T04:34:41.144 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 6.979378214, 2024-09-19T04:34:41.145 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002127210 2024-09-19T04:34:41.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.145 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-19T04:34:41.145 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 107, 2024-09-19T04:34:41.145 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.066299887, 2024-09-19T04:34:41.145 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000619625 2024-09-19T04:34:41.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.145 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-19T04:34:41.146 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.146 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.146 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.146 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-19T04:34:41.146 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.146 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.147 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.147 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-19T04:34:41.147 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4304, 2024-09-19T04:34:41.147 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.941465997, 2024-09-19T04:34:41.147 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000683426 2024-09-19T04:34:41.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.148 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-19T04:34:41.148 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.148 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.148 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.148 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-19T04:34:41.148 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.148 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.149 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.149 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-19T04:34:41.149 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.149 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.149 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.150 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-19T04:34:41.150 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.150 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.150 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.150 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-19T04:34:41.150 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.150 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.150 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.151 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-19T04:34:41.152 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.152 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.152 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.152 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-19T04:34:41.152 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.152 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.153 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.153 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-19T04:34:41.153 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.153 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.153 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.153 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-19T04:34:41.154 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.154 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.154 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.154 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-19T04:34:41.154 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.154 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.154 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.155 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-19T04:34:41.155 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 8240, 2024-09-19T04:34:41.155 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.361481525, 2024-09-19T04:34:41.155 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000407946 2024-09-19T04:34:41.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.155 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-19T04:34:41.156 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 986, 2024-09-19T04:34:41.156 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.125877683, 2024-09-19T04:34:41.156 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002156062 2024-09-19T04:34:41.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.156 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-19T04:34:41.156 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.156 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.157 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.157 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-19T04:34:41.157 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.157 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.157 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.157 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-19T04:34:41.158 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.158 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.160 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.160 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-19T04:34:41.160 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.160 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.160 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.161 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-19T04:34:41.161 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.161 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.161 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.161 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-19T04:34:41.161 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7, 2024-09-19T04:34:41.161 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.018880493, 2024-09-19T04:34:41.162 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002697213 2024-09-19T04:34:41.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.162 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-19T04:34:41.162 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.162 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.162 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.162 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-19T04:34:41.163 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 436, 2024-09-19T04:34:41.163 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.164410435, 2024-09-19T04:34:41.163 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000377088 2024-09-19T04:34:41.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.163 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-19T04:34:41.163 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.163 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.163 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.164 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-19T04:34:41.164 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.164 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.164 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.164 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-19T04:34:41.165 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.165 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.165 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.165 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-19T04:34:41.165 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3210, 2024-09-19T04:34:41.165 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 5.722128871, 2024-09-19T04:34:41.166 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001782594 2024-09-19T04:34:41.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.166 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:34:41.166 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:34:41.166 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-19T04:34:41.166 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.166 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.166 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:41.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.167 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-19T04:34:41.167 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 1, 2024-09-19T04:34:41.167 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 3, 2024-09-19T04:34:41.167 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 2, 2024-09-19T04:34:41.167 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 1, 2024-09-19T04:34:41.167 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-19T04:34:41.168 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 2582, 2024-09-19T04:34:41.168 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 2582, 2024-09-19T04:34:41.168 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 776, 2024-09-19T04:34:41.168 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-19T04:34:41.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.168 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-19T04:34:41.168 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:41.169 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:41.169 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:41.169 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:41.169 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:41.169 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:41.169 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:41.169 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:41.169 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:41.170 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:41.170 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:41.170 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:41.170 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:41.170 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:41.170 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:41.170 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:41.170 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:41.170 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:41.171 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:41.171 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:41.171 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:41.171 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:41.171 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:41.171 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:41.171 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:41.171 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:41.172 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:41.172 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:41.172 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:41.172 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:41.172 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:41.172 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-19T04:34:41.172 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:41.173 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:41.173 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:41.173 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:41.173 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 383292, 2024-09-19T04:34:41.173 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 533, 2024-09-19T04:34:41.173 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 88, 2024-09-19T04:34:41.173 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 1, 2024-09-19T04:34:41.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-19T04:34:41.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-19T04:34:41.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:41.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:41.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:41.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:41.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 8032, 2024-09-19T04:34:41.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 177, 2024-09-19T04:34:41.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:41.175 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:41.175 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:41.175 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-19T04:34:41.175 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 1672910, 2024-09-19T04:34:41.175 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 14258, 2024-09-19T04:34:41.175 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:41.175 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-19T04:34:41.175 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:41.175 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-19T04:34:41.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.176 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-19T04:34:41.176 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-19T04:34:41.176 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-19T04:34:41.176 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 13201, 2024-09-19T04:34:41.176 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 37078381, 2024-09-19T04:34:41.176 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 28, 2024-09-19T04:34:41.177 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 13201, 2024-09-19T04:34:41.177 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-19T04:34:41.177 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 13201, 2024-09-19T04:34:41.177 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 27.105470903, 2024-09-19T04:34:41.177 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002053289 2024-09-19T04:34:41.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.177 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-19T04:34:41.177 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-19T04:34:41.178 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 13201, 2024-09-19T04:34:41.178 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 13999 2024-09-19T04:34:41.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.178 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 13201, 2024-09-19T04:34:41.178 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 721, 2024-09-19T04:34:41.178 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 12480, 2024-09-19T04:34:41.178 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-19T04:34:41.179 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-19T04:34:41.179 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2024-09-19T04:34:41.179 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 10, 2024-09-19T04:34:41.179 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 559, 2024-09-19T04:34:41.179 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 8363, 2024-09-19T04:34:41.179 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 347, 2024-09-19T04:34:41.179 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-19T04:34:41.179 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-19T04:34:41.180 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-19T04:34:41.180 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-09-19T04:34:41.180 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 3230, 2024-09-19T04:34:41.180 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-19T04:34:41.180 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-19T04:34:41.180 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-19T04:34:41.180 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 161, 2024-09-19T04:34:41.181 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 20, 2024-09-19T04:34:41.181 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:34:41.181 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-19T04:34:41.181 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:34:41.181 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-19T04:34:41.181 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-19T04:34:41.181 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-19T04:34:41.181 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:34:41.182 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-19T04:34:41.182 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:34:41.182 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-09-19T04:34:41.182 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-19T04:34:41.182 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-19T04:34:41.182 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-19T04:34:41.182 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-19T04:34:41.182 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-19T04:34:41.183 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-19T04:34:41.183 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-19T04:34:41.183 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-19T04:34:41.183 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-19T04:34:41.183 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-19T04:34:41.183 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-19T04:34:41.183 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-19T04:34:41.183 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-19T04:34:41.184 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-19T04:34:41.184 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-19T04:34:41.184 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-19T04:34:41.184 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 100, 2024-09-19T04:34:41.184 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-19T04:34:41.184 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 19, 2024-09-19T04:34:41.184 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2024-09-19T04:34:41.185 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2024-09-19T04:34:41.185 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-09-19T04:34:41.185 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-19T04:34:41.185 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 944, 2024-09-19T04:34:41.185 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2024-09-19T04:34:41.185 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 363 2024-09-19T04:34:41.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.185 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-19T04:34:41.186 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-09-19T04:34:41.186 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:34:41.186 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-19T04:34:41.186 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-19T04:34:41.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.186 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-19T04:34:41.186 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-09-19T04:34:41.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.187 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-19T04:34:41.187 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-19T04:34:41.187 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T04:34:41.187 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-19T04:34:41.187 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 6, 2024-09-19T04:34:41.187 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 3210, 2024-09-19T04:34:41.187 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 3210, 2024-09-19T04:34:41.187 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-19T04:34:41.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.188 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:41.188 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:41.188 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-19T04:34:41.188 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:41.188 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 162396, 2024-09-19T04:34:41.188 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 38170553, 2024-09-19T04:34:41.189 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:41.189 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 162396, 2024-09-19T04:34:41.189 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:41.189 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:41.189 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 162396, 2024-09-19T04:34:41.189 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 38170553, 2024-09-19T04:34:41.189 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:41.189 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.190 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.190 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:41.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.190 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-19T04:34:41.190 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:41.190 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-19T04:34:41.190 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:41.191 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:41.191 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:41.191 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:41.191 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:41.191 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 13201, 2024-09-19T04:34:41.191 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 37403631, 2024-09-19T04:34:41.191 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 13201, 2024-09-19T04:34:41.192 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 37403631, 2024-09-19T04:34:41.192 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:41.192 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.192 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.192 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:41.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.193 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-19T04:34:41.193 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:41.193 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-19T04:34:41.193 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:41.193 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:41.193 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:41.193 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:41.193 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:41.194 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 13201, 2024-09-19T04:34:41.194 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 13201, 2024-09-19T04:34:41.194 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 13201, 2024-09-19T04:34:41.194 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 13201, 2024-09-19T04:34:41.194 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:41.194 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.194 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.194 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:41.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.195 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:34:41.195 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:41.195 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-19T04:34:41.195 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:41.195 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 3210, 2024-09-19T04:34:41.195 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 325266, 2024-09-19T04:34:41.195 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:41.196 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 3210, 2024-09-19T04:34:41.196 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:41.196 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:41.196 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 559, 2024-09-19T04:34:41.196 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 325266, 2024-09-19T04:34:41.196 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:41.196 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.197 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.197 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:41.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:41.197 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55808571c960": { 2024-09-19T04:34:41.197 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 2260, 2024-09-19T04:34:41.197 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-19T04:34:41.197 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:41.198 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 23574, 2024-09-19T04:34:41.198 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 36329420, 2024-09-19T04:34:41.198 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:41.198 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 23574, 2024-09-19T04:34:41.198 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:41.198 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:41.198 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 7804, 2024-09-19T04:34:41.198 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 36327160, 2024-09-19T04:34:41.198 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:41.199 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:41.199 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:41.199 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:41.199 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:41.199 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:41.199 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-19T04:34:41.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:41 smithi115 ceph-mon[35477]: pgmap v449: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 6.6 MiB/s wr, 133 op/s 2024-09-19T04:34:41.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:41 smithi145 ceph-mon[35096]: pgmap v449: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 6.6 MiB/s wr, 133 op/s 2024-09-19T04:34:41.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:41 smithi107 ceph-mon[30659]: pgmap v449: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 6.6 MiB/s wr, 133 op/s 2024-09-19T04:34:41.917 INFO:tasks.check_counter:Daemon mds.h mds.exported=2 2024-09-19T04:34:41.917 INFO:tasks.check_counter:Daemon mds.h mds.imported=5 2024-09-19T04:34:41.918 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=828 2024-09-19T04:34:41.918 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=1394 2024-09-19T04:34:41.918 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-19T04:34:41.918 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:34:41.918 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:34:41.918 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T04:34:41.918 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T04:34:42.166 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:34:42.672 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-19T04:34:42.672 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:42.673 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 525, 2024-09-19T04:34:42.673 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 481, 2024-09-19T04:34:42.673 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 6216910, 2024-09-19T04:34:42.673 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 231376, 2024-09-19T04:34:42.673 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 7, 2024-09-19T04:34:42.673 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 2, 2024-09-19T04:34:42.673 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.072025105, 2024-09-19T04:34:42.674 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.022330777, 2024-09-19T04:34:42.674 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.058777734, 2024-09-19T04:34:42.674 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.005036472, 2024-09-19T04:34:42.674 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:42.674 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 481, 2024-09-19T04:34:42.674 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.033673806, 2024-09-19T04:34:42.674 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000070007 2024-09-19T04:34:42.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.675 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:42.675 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.675 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.675 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.675 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1976976, 2024-09-19T04:34:42.675 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 174448 2024-09-19T04:34:42.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.676 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:42.676 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1006, 2024-09-19T04:34:42.676 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1215, 2024-09-19T04:34:42.676 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 15608662, 2024-09-19T04:34:42.676 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 847325, 2024-09-19T04:34:42.676 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 8, 2024-09-19T04:34:42.676 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2024-09-19T04:34:42.676 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.100323491, 2024-09-19T04:34:42.677 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.030108782, 2024-09-19T04:34:42.677 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.034241683, 2024-09-19T04:34:42.677 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.026760810, 2024-09-19T04:34:42.677 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:42.677 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1212, 2024-09-19T04:34:42.677 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.049483231, 2024-09-19T04:34:42.677 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000040827 2024-09-19T04:34:42.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.678 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:42.678 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.678 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.678 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.678 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T04:34:42.678 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 528048 2024-09-19T04:34:42.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.678 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:42.679 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2066, 2024-09-19T04:34:42.679 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 2068, 2024-09-19T04:34:42.679 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 17385721, 2024-09-19T04:34:42.679 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 657046, 2024-09-19T04:34:42.679 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 6, 2024-09-19T04:34:42.679 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-09-19T04:34:42.679 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.188442345, 2024-09-19T04:34:42.679 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.059154483, 2024-09-19T04:34:42.680 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.055051058, 2024-09-19T04:34:42.680 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.058248414, 2024-09-19T04:34:42.680 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:42.680 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2066, 2024-09-19T04:34:42.680 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.070270000, 2024-09-19T04:34:42.680 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000034012 2024-09-19T04:34:42.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.680 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:42.681 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.681 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.681 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.681 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:34:42.681 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:34:42.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.681 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-19T04:34:42.682 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-19T04:34:42.682 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-19T04:34:42.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.682 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-19T04:34:42.682 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-19T04:34:42.682 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-19T04:34:42.682 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5543, 2024-09-19T04:34:42.682 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.130581199, 2024-09-19T04:34:42.683 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000023557 2024-09-19T04:34:42.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:42.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.683 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-19T04:34:42.683 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-19T04:34:42.683 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-19T04:34:42.683 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.684 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.684 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:42.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.684 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-19T04:34:42.684 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2024-09-19T04:34:42.685 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2024-09-19T04:34:42.685 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-19T04:34:42.685 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.685 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.685 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.685 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-19T04:34:42.685 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-09-19T04:34:42.686 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-09-19T04:34:42.686 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-19T04:34:42.686 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2024-09-19T04:34:42.686 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-09-19T04:34:42.686 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-09-19T04:34:42.686 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 3671, 2024-09-19T04:34:42.686 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 9, 2024-09-19T04:34:42.686 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 3662, 2024-09-19T04:34:42.687 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-09-19T04:34:42.687 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 3659, 2024-09-19T04:34:42.687 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-09-19T04:34:42.687 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-09-19T04:34:42.687 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-09-19T04:34:42.687 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 5, 2024-09-19T04:34:42.687 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2024-09-19T04:34:42.687 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2024-09-19T04:34:42.688 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-09-19T04:34:42.688 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-09-19T04:34:42.688 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:34:42.688 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-19T04:34:42.688 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-09-19T04:34:42.688 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-09-19T04:34:42.688 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-09-19T04:34:42.688 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-09-19T04:34:42.689 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-09-19T04:34:42.689 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-09-19T04:34:42.689 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-09-19T04:34:42.689 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-19T04:34:42.689 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:34:42.689 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-19T04:34:42.689 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 0, 2024-09-19T04:34:42.689 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 0, 2024-09-19T04:34:42.690 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 0, 2024-09-19T04:34:42.690 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:34:42.690 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-19T04:34:42.690 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:34:42.690 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:34:42.690 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:34:42.690 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:34:42.690 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:34:42.691 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-09-19T04:34:42.691 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:34:42.691 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:34:42.691 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:34:42.691 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:34:42.691 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:34:42.691 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:34:42.691 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-09-19T04:34:42.692 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:34:42.692 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-09-19T04:34:42.692 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2024-09-19T04:34:42.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.692 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-19T04:34:42.692 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-09-19T04:34:42.692 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-09-19T04:34:42.693 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-09-19T04:34:42.693 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-09-19T04:34:42.693 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-09-19T04:34:42.693 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-09-19T04:34:42.693 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-19T04:34:42.693 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:34:42.693 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-09-19T04:34:42.693 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-09-19T04:34:42.694 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-19T04:34:42.694 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-19T04:34:42.694 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-19T04:34:42.694 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:34:42.694 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:34:42.694 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-19T04:34:42.694 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-19T04:34:42.694 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:34:42.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:34:42.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:34:42.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-09-19T04:34:42.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-19T04:34:42.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:34:42.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-19T04:34:42.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-19T04:34:42.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.695 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-19T04:34:42.696 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-19T04:34:42.696 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2024-09-19T04:34:42.696 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-09-19T04:34:42.696 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-09-19T04:34:42.696 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 23572, 2024-09-19T04:34:42.696 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-19T04:34:42.696 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-09-19T04:34:42.696 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2024-09-19T04:34:42.697 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-09-19T04:34:42.697 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-09-19T04:34:42.697 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 29, 2024-09-19T04:34:42.697 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 3, 2024-09-19T04:34:42.697 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-19T04:34:42.697 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-09-19T04:34:42.697 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-09-19T04:34:42.698 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 40521464, 2024-09-19T04:34:42.698 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 40519463, 2024-09-19T04:34:42.698 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-19T04:34:42.698 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.698 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.698 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.698 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 23572 2024-09-19T04:34:42.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.699 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-19T04:34:42.699 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 315, 2024-09-19T04:34:42.699 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 3749, 2024-09-19T04:34:42.699 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 3434, 2024-09-19T04:34:42.699 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 44, 2024-09-19T04:34:42.699 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 46, 2024-09-19T04:34:42.699 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 2, 2024-09-19T04:34:42.700 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 3671, 2024-09-19T04:34:42.700 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 3837, 2024-09-19T04:34:42.700 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 166, 2024-09-19T04:34:42.700 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-09-19T04:34:42.700 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2024-09-19T04:34:42.700 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2024-09-19T04:34:42.700 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 54076, 2024-09-19T04:34:42.701 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 215328 2024-09-19T04:34:42.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.701 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-19T04:34:42.701 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2024-09-19T04:34:42.701 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-09-19T04:34:42.701 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2024-09-19T04:34:42.701 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2024-09-19T04:34:42.701 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-09-19T04:34:42.701 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-19T04:34:42.702 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.702 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.702 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.702 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-19T04:34:42.702 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.702 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.702 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.703 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-19T04:34:42.703 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.703 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.703 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.703 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-19T04:34:42.703 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.704 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.704 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.704 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-19T04:34:42.704 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.704 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.704 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.705 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-19T04:34:42.705 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.705 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.705 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.705 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-19T04:34:42.705 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.705 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.706 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.706 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-19T04:34:42.706 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.706 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.706 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.706 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-19T04:34:42.707 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.707 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.707 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.707 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-19T04:34:42.707 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.707 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.707 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.708 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-19T04:34:42.708 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.708 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.708 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.708 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-19T04:34:42.709 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.709 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.709 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.710 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-19T04:34:42.710 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.710 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.710 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.710 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-19T04:34:42.710 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.711 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.711 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.711 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-19T04:34:42.711 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.711 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.711 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.711 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-19T04:34:42.712 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.712 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.712 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.712 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-19T04:34:42.712 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.712 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.713 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.713 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-19T04:34:42.713 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.713 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.713 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.714 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-19T04:34:42.714 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.714 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.714 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.714 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-19T04:34:42.714 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.714 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-19T04:34:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-19T04:34:42.716 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.716 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.716 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.716 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-19T04:34:42.716 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.716 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.717 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.717 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-19T04:34:42.717 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.717 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.717 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.717 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-19T04:34:42.718 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.718 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.718 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.718 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-19T04:34:42.718 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.718 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.718 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.719 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-19T04:34:42.719 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.719 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.719 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.719 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-19T04:34:42.719 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.719 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.720 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.720 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-19T04:34:42.720 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.720 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.720 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.720 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:34:42.721 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:34:42.721 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-19T04:34:42.721 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.721 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.721 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:42.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.722 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-19T04:34:42.722 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 1, 2024-09-19T04:34:42.722 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 3, 2024-09-19T04:34:42.722 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 2, 2024-09-19T04:34:42.722 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 3, 2024-09-19T04:34:42.722 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-19T04:34:42.722 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2024-09-19T04:34:42.722 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2024-09-19T04:34:42.723 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 5001, 2024-09-19T04:34:42.723 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-19T04:34:42.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.723 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-19T04:34:42.723 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:42.723 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:42.723 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:42.723 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:42.724 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:42.724 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:42.724 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:42.724 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:42.724 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:42.724 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:42.724 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:42.724 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:42.725 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:42.725 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:42.725 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:42.725 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:42.725 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:42.725 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:42.725 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:42.725 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:42.726 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:42.726 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:42.726 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:42.726 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:42.726 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:42.726 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:42.726 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:42.726 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:42.727 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:42.727 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:42.727 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:42.727 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-19T04:34:42.727 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:42.727 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:42.727 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:42.727 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:42.727 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 235289, 2024-09-19T04:34:42.728 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 396, 2024-09-19T04:34:42.728 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:34:42.728 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-19T04:34:42.728 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-19T04:34:42.728 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-19T04:34:42.728 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:42.728 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:42.729 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:42.729 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:42.729 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 8032, 2024-09-19T04:34:42.729 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 177, 2024-09-19T04:34:42.729 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:42.729 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:42.729 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:42.729 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-19T04:34:42.730 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 3447048, 2024-09-19T04:34:42.730 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 22291, 2024-09-19T04:34:42.730 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:42.730 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-19T04:34:42.730 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:42.730 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-19T04:34:42.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.730 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-19T04:34:42.731 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-19T04:34:42.731 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-19T04:34:42.731 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 3249, 2024-09-19T04:34:42.731 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 21, 2024-09-19T04:34:42.731 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 3, 2024-09-19T04:34:42.731 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 3249, 2024-09-19T04:34:42.731 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-19T04:34:42.731 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3249, 2024-09-19T04:34:42.732 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.027960442, 2024-09-19T04:34:42.732 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000624179 2024-09-19T04:34:42.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.732 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-19T04:34:42.732 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-19T04:34:42.732 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3249, 2024-09-19T04:34:42.732 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3250 2024-09-19T04:34:42.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.733 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 3249, 2024-09-19T04:34:42.733 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 3249, 2024-09-19T04:34:42.733 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2024-09-19T04:34:42.733 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-19T04:34:42.733 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-19T04:34:42.733 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 1834, 2024-09-19T04:34:42.733 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2024-09-19T04:34:42.734 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1413, 2024-09-19T04:34:42.734 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2024-09-19T04:34:42.734 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2024-09-19T04:34:42.734 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-19T04:34:42.734 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-19T04:34:42.734 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-19T04:34:42.734 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-09-19T04:34:42.734 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2024-09-19T04:34:42.735 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-19T04:34:42.735 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-19T04:34:42.735 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-19T04:34:42.735 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 1, 2024-09-19T04:34:42.735 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2024-09-19T04:34:42.735 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:34:42.735 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-19T04:34:42.735 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:34:42.736 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-19T04:34:42.736 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-19T04:34:42.736 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-19T04:34:42.736 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:34:42.736 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-19T04:34:42.736 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:34:42.736 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-09-19T04:34:42.736 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-19T04:34:42.737 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-19T04:34:42.737 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-19T04:34:42.737 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-19T04:34:42.737 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-19T04:34:42.737 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-19T04:34:42.737 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-19T04:34:42.737 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-19T04:34:42.737 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-19T04:34:42.737 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-19T04:34:42.738 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-19T04:34:42.738 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-19T04:34:42.738 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-19T04:34:42.738 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-19T04:34:42.738 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-19T04:34:42.738 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-19T04:34:42.738 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 100, 2024-09-19T04:34:42.738 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-19T04:34:42.739 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 19, 2024-09-19T04:34:42.739 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 9, 2024-09-19T04:34:42.739 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 10, 2024-09-19T04:34:42.739 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 1, 2024-09-19T04:34:42.739 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-19T04:34:42.739 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2024-09-19T04:34:42.739 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2024-09-19T04:34:42.739 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2024-09-19T04:34:42.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.740 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-19T04:34:42.740 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2024-09-19T04:34:42.740 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:34:42.740 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-19T04:34:42.740 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-19T04:34:42.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.740 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-19T04:34:42.741 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-09-19T04:34:42.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.741 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-19T04:34:42.741 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-19T04:34:42.741 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:34:42.741 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-19T04:34:42.741 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-09-19T04:34:42.741 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-09-19T04:34:42.742 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-09-19T04:34:42.742 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-19T04:34:42.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.742 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:42.742 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:42.742 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-19T04:34:42.742 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:42.742 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 3593, 2024-09-19T04:34:42.743 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 38867499, 2024-09-19T04:34:42.743 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:42.743 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 3593, 2024-09-19T04:34:42.743 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:42.743 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:42.743 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3593, 2024-09-19T04:34:42.743 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 38867499, 2024-09-19T04:34:42.743 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:42.744 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.744 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.744 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:42.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.744 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-19T04:34:42.745 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:42.745 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-19T04:34:42.745 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:42.745 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:42.745 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:42.745 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:42.745 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:42.745 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3249, 2024-09-19T04:34:42.746 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 36329963, 2024-09-19T04:34:42.746 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3249, 2024-09-19T04:34:42.746 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 36329963, 2024-09-19T04:34:42.746 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:42.746 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.746 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.746 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:42.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.747 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-19T04:34:42.747 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:42.747 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-19T04:34:42.747 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:42.747 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:42.747 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:42.747 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:42.748 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:42.748 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3249, 2024-09-19T04:34:42.748 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 3249, 2024-09-19T04:34:42.748 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3249, 2024-09-19T04:34:42.748 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 3249, 2024-09-19T04:34:42.748 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:42.748 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.748 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.749 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:42.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.749 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:34:42.749 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:42.749 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-19T04:34:42.749 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:42.749 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:42.750 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:42.750 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:42.750 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:42.750 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:42.750 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:42.750 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-19T04:34:42.750 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-19T04:34:42.750 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:42.750 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.751 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.751 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:42.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-19T04:34:42.751 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55c419f1eb40": { 2024-09-19T04:34:42.751 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-19T04:34:42.751 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-19T04:34:42.751 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-19T04:34:42.751 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-19T04:34:42.752 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-19T04:34:42.752 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:42.752 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:42.752 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-19T04:34:42.752 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-19T04:34:42.752 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-19T04:34:42.752 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-19T04:34:42.752 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-19T04:34:42.753 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-19T04:34:42.753 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-19T04:34:42.753 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-19T04:34:42.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:42.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-19T04:34:42.753 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-19T04:34:43.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:43 smithi115 ceph-mon[35477]: pgmap v450: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 6.6 MiB/s wr, 140 op/s 2024-09-19T04:34:43.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:43 smithi115 ceph-mon[35477]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:34:43.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:43 smithi145 ceph-mon[35096]: pgmap v450: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 6.6 MiB/s wr, 140 op/s 2024-09-19T04:34:43.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:43 smithi145 ceph-mon[35096]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:34:43.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:43 smithi107 ceph-mon[30659]: pgmap v450: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 6.6 MiB/s wr, 140 op/s 2024-09-19T04:34:43.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:43 smithi107 ceph-mon[30659]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:34:43.570 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T04:34:43.570 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T04:34:43.571 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-19T04:34:43.571 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-19T04:34:43.571 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-19T04:34:43.571 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:34:43.571 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:34:43.571 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T04:34:43.571 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T04:34:43.829 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:34:44.430 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-19T04:34:44.430 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:44.430 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 3935, 2024-09-19T04:34:44.430 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 4051, 2024-09-19T04:34:44.430 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1355986, 2024-09-19T04:34:44.431 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 10747169, 2024-09-19T04:34:44.431 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 46, 2024-09-19T04:34:44.431 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 5, 2024-09-19T04:34:44.431 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.293970047, 2024-09-19T04:34:44.431 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.118360557, 2024-09-19T04:34:44.431 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.095789571, 2024-09-19T04:34:44.431 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.067438432, 2024-09-19T04:34:44.431 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:44.432 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4049, 2024-09-19T04:34:44.432 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.139749303, 2024-09-19T04:34:44.432 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000034514 2024-09-19T04:34:44.432 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.432 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:44.432 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1583, 2024-09-19T04:34:44.432 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000268696, 2024-09-19T04:34:44.433 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000169 2024-09-19T04:34:44.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.433 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-19T04:34:44.433 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 399984 2024-09-19T04:34:44.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.433 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:44.433 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 38670, 2024-09-19T04:34:44.434 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 49454, 2024-09-19T04:34:44.434 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 14149625, 2024-09-19T04:34:44.434 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 47356792, 2024-09-19T04:34:44.434 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 12, 2024-09-19T04:34:44.434 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 9, 2024-09-19T04:34:44.434 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 1.870738487, 2024-09-19T04:34:44.434 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.911911057, 2024-09-19T04:34:44.434 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 1.305598834, 2024-09-19T04:34:44.435 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.043720832, 2024-09-19T04:34:44.435 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:44.435 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 49450, 2024-09-19T04:34:44.435 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.898254111, 2024-09-19T04:34:44.435 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000018164 2024-09-19T04:34:44.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.435 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:44.435 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 43769, 2024-09-19T04:34:44.436 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.011904974, 2024-09-19T04:34:44.436 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000271 2024-09-19T04:34:44.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.436 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 2003392, 2024-09-19T04:34:44.436 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 207808 2024-09-19T04:34:44.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.436 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:44.436 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 101452, 2024-09-19T04:34:44.437 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 63621, 2024-09-19T04:34:44.437 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 31506557, 2024-09-19T04:34:44.437 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 22536966, 2024-09-19T04:34:44.437 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 14, 2024-09-19T04:34:44.437 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 5, 2024-09-19T04:34:44.437 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 4.660521569, 2024-09-19T04:34:44.437 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 1.828204805, 2024-09-19T04:34:44.437 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 3.629400016, 2024-09-19T04:34:44.438 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.069318498, 2024-09-19T04:34:44.438 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:44.438 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 63622, 2024-09-19T04:34:44.438 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.102879685, 2024-09-19T04:34:44.438 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000017334 2024-09-19T04:34:44.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.438 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:44.438 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 159131, 2024-09-19T04:34:44.439 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.023083406, 2024-09-19T04:34:44.439 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000145 2024-09-19T04:34:44.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.439 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-19T04:34:44.439 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 130144 2024-09-19T04:34:44.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.439 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-19T04:34:44.439 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-09-19T04:34:44.440 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-19T04:34:44.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.440 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-09-19T04:34:44.440 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-19T04:34:44.440 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-19T04:34:44.440 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 7962, 2024-09-19T04:34:44.440 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3.641211028, 2024-09-19T04:34:44.441 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000457323 2024-09-19T04:34:44.441 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:44.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.441 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-09-19T04:34:44.441 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-19T04:34:44.441 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-19T04:34:44.444 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4819, 2024-09-19T04:34:44.444 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.210155818, 2024-09-19T04:34:44.444 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000043609 2024-09-19T04:34:44.444 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:44.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.444 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-09-19T04:34:44.444 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 23605, 2024-09-19T04:34:44.445 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 18701, 2024-09-19T04:34:44.445 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-09-19T04:34:44.445 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 18701, 2024-09-19T04:34:44.445 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 19.581166500, 2024-09-19T04:34:44.445 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001047065 2024-09-19T04:34:44.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.445 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-09-19T04:34:44.445 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 4904, 2024-09-19T04:34:44.446 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 1, 2024-09-19T04:34:44.446 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-09-19T04:34:44.446 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 11, 2024-09-19T04:34:44.446 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-09-19T04:34:44.446 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-09-19T04:34:44.446 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 587, 2024-09-19T04:34:44.446 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 62, 2024-09-19T04:34:44.446 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 476, 2024-09-19T04:34:44.447 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 49, 2024-09-19T04:34:44.447 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 499, 2024-09-19T04:34:44.447 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 185, 2024-09-19T04:34:44.447 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 155, 2024-09-19T04:34:44.447 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 156, 2024-09-19T04:34:44.447 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 14, 2024-09-19T04:34:44.447 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 45024, 2024-09-19T04:34:44.447 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 23243, 2024-09-19T04:34:44.448 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 4899, 2024-09-19T04:34:44.448 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 167, 2024-09-19T04:34:44.448 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:34:44.448 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-09-19T04:34:44.448 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 915, 2024-09-19T04:34:44.448 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 6927, 2024-09-19T04:34:44.448 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 1, 2024-09-19T04:34:44.448 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 3, 2024-09-19T04:34:44.449 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 142, 2024-09-19T04:34:44.449 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 6, 2024-09-19T04:34:44.449 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 471, 2024-09-19T04:34:44.449 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-09-19T04:34:44.449 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 185, 2024-09-19T04:34:44.449 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-09-19T04:34:44.449 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 43, 2024-09-19T04:34:44.450 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 7184928, 2024-09-19T04:34:44.450 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 6, 2024-09-19T04:34:44.450 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 185, 2024-09-19T04:34:44.450 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-09-19T04:34:44.450 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:34:44.450 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:34:44.450 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:34:44.450 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:34:44.450 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:34:44.451 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 185, 2024-09-19T04:34:44.451 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:34:44.451 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 3522, 2024-09-19T04:34:44.451 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 11890, 2024-09-19T04:34:44.451 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:34:44.451 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-09-19T04:34:44.451 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:34:44.451 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 10101, 2024-09-19T04:34:44.452 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 1782, 2024-09-19T04:34:44.452 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 1083, 2024-09-19T04:34:44.452 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 18000 2024-09-19T04:34:44.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.452 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-09-19T04:34:44.452 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 674, 2024-09-19T04:34:44.452 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 1400, 2024-09-19T04:34:44.452 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 754, 2024-09-19T04:34:44.453 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 1406, 2024-09-19T04:34:44.453 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 678, 2024-09-19T04:34:44.453 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-09-19T04:34:44.453 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-09-19T04:34:44.453 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:34:44.453 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 1912, 2024-09-19T04:34:44.453 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 1912, 2024-09-19T04:34:44.454 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-09-19T04:34:44.454 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-09-19T04:34:44.454 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-09-19T04:34:44.454 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:34:44.454 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:34:44.454 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-09-19T04:34:44.454 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-09-19T04:34:44.454 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:34:44.455 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:34:44.455 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:34:44.455 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 4, 2024-09-19T04:34:44.455 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-09-19T04:34:44.455 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:34:44.455 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-09-19T04:34:44.455 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-09-19T04:34:44.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.456 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-09-19T04:34:44.456 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-09-19T04:34:44.456 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 16365, 2024-09-19T04:34:44.456 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-09-19T04:34:44.456 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-09-19T04:34:44.456 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 16365, 2024-09-19T04:34:44.456 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-09-19T04:34:44.456 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-09-19T04:34:44.457 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 20, 2024-09-19T04:34:44.457 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-09-19T04:34:44.457 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-09-19T04:34:44.457 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 20, 2024-09-19T04:34:44.457 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 3, 2024-09-19T04:34:44.457 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-09-19T04:34:44.457 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-09-19T04:34:44.457 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-09-19T04:34:44.458 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 28319719, 2024-09-19T04:34:44.458 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 0, 2024-09-19T04:34:44.458 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-09-19T04:34:44.458 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3669, 2024-09-19T04:34:44.458 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 6.547390505, 2024-09-19T04:34:44.458 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001784516 2024-09-19T04:34:44.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.459 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 0 2024-09-19T04:34:44.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.459 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-09-19T04:34:44.459 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 434, 2024-09-19T04:34:44.459 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 2504, 2024-09-19T04:34:44.459 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 2070, 2024-09-19T04:34:44.459 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 134, 2024-09-19T04:34:44.460 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 742, 2024-09-19T04:34:44.460 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 608, 2024-09-19T04:34:44.460 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 587, 2024-09-19T04:34:44.460 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 4477, 2024-09-19T04:34:44.460 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 3890, 2024-09-19T04:34:44.460 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 148, 2024-09-19T04:34:44.460 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 2228, 2024-09-19T04:34:44.460 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 2080, 2024-09-19T04:34:44.461 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 61064, 2024-09-19T04:34:44.461 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 223520 2024-09-19T04:34:44.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.461 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-09-19T04:34:44.461 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 36168, 2024-09-19T04:34:44.461 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 12, 2024-09-19T04:34:44.461 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 23605, 2024-09-19T04:34:44.462 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 203, 2024-09-19T04:34:44.462 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 4735, 2024-09-19T04:34:44.462 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-09-19T04:34:44.462 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1896, 2024-09-19T04:34:44.462 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 4.229591872, 2024-09-19T04:34:44.462 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002230797 2024-09-19T04:34:44.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.462 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-09-19T04:34:44.463 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4, 2024-09-19T04:34:44.463 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.003689475, 2024-09-19T04:34:44.463 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000922368 2024-09-19T04:34:44.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.463 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-09-19T04:34:44.463 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.463 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.463 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.464 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-09-19T04:34:44.464 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.464 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.464 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.464 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-09-19T04:34:44.464 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4434, 2024-09-19T04:34:44.465 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 6.495837503, 2024-09-19T04:34:44.465 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001465006 2024-09-19T04:34:44.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.465 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-09-19T04:34:44.465 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.465 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.465 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.466 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-09-19T04:34:44.466 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1, 2024-09-19T04:34:44.466 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000110961, 2024-09-19T04:34:44.466 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000110961 2024-09-19T04:34:44.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.466 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-09-19T04:34:44.466 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.466 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.467 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.467 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-09-19T04:34:44.467 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.467 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.467 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.467 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-09-19T04:34:44.468 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.468 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.468 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.469 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-09-19T04:34:44.469 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.469 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.469 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.470 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-09-19T04:34:44.470 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.470 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.471 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.471 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-09-19T04:34:44.471 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.471 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.471 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.472 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-09-19T04:34:44.472 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.472 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.472 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.472 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-09-19T04:34:44.472 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.473 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.473 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.473 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-09-19T04:34:44.473 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 9541, 2024-09-19T04:34:44.473 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3.888990077, 2024-09-19T04:34:44.473 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000407608 2024-09-19T04:34:44.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.474 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-09-19T04:34:44.474 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 912, 2024-09-19T04:34:44.474 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.975102510, 2024-09-19T04:34:44.474 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002165682 2024-09-19T04:34:44.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.474 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-09-19T04:34:44.474 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.474 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.474 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.475 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-09-19T04:34:44.475 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.475 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.475 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.475 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-09-19T04:34:44.476 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.476 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.476 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.476 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-09-19T04:34:44.476 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.476 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.476 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.477 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-09-19T04:34:44.477 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.477 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.477 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.477 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-09-19T04:34:44.477 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1, 2024-09-19T04:34:44.478 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.002048955, 2024-09-19T04:34:44.478 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002048955 2024-09-19T04:34:44.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.478 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-09-19T04:34:44.478 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.478 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.478 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.479 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-09-19T04:34:44.479 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.479 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.479 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.479 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-09-19T04:34:44.479 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.480 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.480 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.480 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-09-19T04:34:44.480 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.480 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.480 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.481 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-09-19T04:34:44.481 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.481 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.481 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.481 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-09-19T04:34:44.481 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1912, 2024-09-19T04:34:44.481 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 2.985795147, 2024-09-19T04:34:44.482 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001561608 2024-09-19T04:34:44.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.482 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:34:44.482 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:34:44.482 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-09-19T04:34:44.482 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.482 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.482 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.482 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:44.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.483 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-09-19T04:34:44.483 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 2, 2024-09-19T04:34:44.483 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 4, 2024-09-19T04:34:44.483 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 2, 2024-09-19T04:34:44.483 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 2, 2024-09-19T04:34:44.483 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-09-19T04:34:44.483 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 2268, 2024-09-19T04:34:44.484 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 1134, 2024-09-19T04:34:44.484 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 835, 2024-09-19T04:34:44.484 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-09-19T04:34:44.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.484 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-19T04:34:44.484 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:44.484 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:44.485 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:44.485 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:44.485 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:44.485 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:44.485 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:44.485 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:44.485 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:44.485 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:44.486 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:44.486 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:44.486 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:44.486 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:44.486 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:44.486 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:44.486 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:44.486 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:44.487 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:44.487 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:44.487 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:44.487 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:44.487 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:44.487 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:44.487 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:44.487 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:44.488 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:44.488 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:44.488 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:44.488 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:44.488 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:44.488 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-19T04:34:44.488 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:44.488 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:44.489 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:44.489 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:44.489 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 309936, 2024-09-19T04:34:44.489 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 589, 2024-09-19T04:34:44.489 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 352, 2024-09-19T04:34:44.489 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 4, 2024-09-19T04:34:44.489 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-19T04:34:44.490 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-19T04:34:44.490 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:44.490 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:44.490 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:44.490 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:44.490 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 8032, 2024-09-19T04:34:44.490 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 177, 2024-09-19T04:34:44.490 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:44.491 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:44.491 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:44.491 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-19T04:34:44.491 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 1823234, 2024-09-19T04:34:44.491 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 15830, 2024-09-19T04:34:44.491 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:44.491 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-19T04:34:44.491 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:44.492 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-19T04:34:44.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.492 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-09-19T04:34:44.492 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-09-19T04:34:44.492 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-09-19T04:34:44.492 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 7473, 2024-09-19T04:34:44.492 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 24598931, 2024-09-19T04:34:44.492 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 33, 2024-09-19T04:34:44.493 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 7473, 2024-09-19T04:34:44.493 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-09-19T04:34:44.493 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 7473, 2024-09-19T04:34:44.493 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 14.953661441, 2024-09-19T04:34:44.493 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002001025 2024-09-19T04:34:44.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.493 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-09-19T04:34:44.493 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-09-19T04:34:44.494 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 7473, 2024-09-19T04:34:44.494 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 8130 2024-09-19T04:34:44.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.494 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 7473, 2024-09-19T04:34:44.494 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 641, 2024-09-19T04:34:44.495 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 6832, 2024-09-19T04:34:44.495 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-09-19T04:34:44.495 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-09-19T04:34:44.495 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 0, 2024-09-19T04:34:44.495 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 10, 2024-09-19T04:34:44.495 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 453, 2024-09-19T04:34:44.495 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 4122, 2024-09-19T04:34:44.495 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 296, 2024-09-19T04:34:44.496 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-09-19T04:34:44.496 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-09-19T04:34:44.496 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-09-19T04:34:44.496 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-09-19T04:34:44.496 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 1929, 2024-09-19T04:34:44.496 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-09-19T04:34:44.496 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-09-19T04:34:44.497 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-09-19T04:34:44.497 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 185, 2024-09-19T04:34:44.497 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 20, 2024-09-19T04:34:44.497 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:34:44.497 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-09-19T04:34:44.497 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:34:44.497 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-09-19T04:34:44.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-09-19T04:34:44.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-09-19T04:34:44.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:34:44.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-09-19T04:34:44.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:34:44.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-09-19T04:34:44.498 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-09-19T04:34:44.498 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-09-19T04:34:44.499 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-09-19T04:34:44.499 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-09-19T04:34:44.499 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-09-19T04:34:44.499 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-09-19T04:34:44.499 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-09-19T04:34:44.499 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-09-19T04:34:44.499 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-09-19T04:34:44.499 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-09-19T04:34:44.499 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-09-19T04:34:44.500 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-09-19T04:34:44.500 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-09-19T04:34:44.500 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-09-19T04:34:44.500 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-09-19T04:34:44.500 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-09-19T04:34:44.500 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 100, 2024-09-19T04:34:44.500 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-09-19T04:34:44.500 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 19, 2024-09-19T04:34:44.501 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 12, 2024-09-19T04:34:44.501 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 12, 2024-09-19T04:34:44.501 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 0, 2024-09-19T04:34:44.501 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-09-19T04:34:44.501 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 828, 2024-09-19T04:34:44.501 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 4, 2024-09-19T04:34:44.501 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 283 2024-09-19T04:34:44.501 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.502 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-09-19T04:34:44.502 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 1, 2024-09-19T04:34:44.502 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:34:44.502 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-09-19T04:34:44.502 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-09-19T04:34:44.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.502 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-09-19T04:34:44.502 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-09-19T04:34:44.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.503 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-09-19T04:34:44.503 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-09-19T04:34:44.503 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T04:34:44.503 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-09-19T04:34:44.503 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 6, 2024-09-19T04:34:44.503 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 1912, 2024-09-19T04:34:44.503 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 1912, 2024-09-19T04:34:44.504 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-09-19T04:34:44.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.504 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:44.504 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:44.504 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-19T04:34:44.504 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:44.504 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 144053, 2024-09-19T04:34:44.505 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 34605114, 2024-09-19T04:34:44.505 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:44.505 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 144053, 2024-09-19T04:34:44.505 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:44.505 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:44.505 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 144053, 2024-09-19T04:34:44.505 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 34605114, 2024-09-19T04:34:44.505 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:44.506 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.506 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.506 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.506 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:44.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.506 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-09-19T04:34:44.506 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:44.506 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-19T04:34:44.507 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:44.507 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-19T04:34:44.507 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-19T04:34:44.507 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:44.507 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:44.507 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 7473, 2024-09-19T04:34:44.507 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 24792940, 2024-09-19T04:34:44.507 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 7473, 2024-09-19T04:34:44.508 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 24792940, 2024-09-19T04:34:44.508 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:44.508 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.508 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.508 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.508 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:44.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.508 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-09-19T04:34:44.509 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:44.509 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-09-19T04:34:44.509 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:44.509 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-19T04:34:44.509 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-19T04:34:44.509 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:44.509 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:44.510 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 7473, 2024-09-19T04:34:44.510 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 7473, 2024-09-19T04:34:44.510 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 7473, 2024-09-19T04:34:44.510 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 7473, 2024-09-19T04:34:44.510 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:44.510 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.510 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.510 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.510 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:44.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.511 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:34:44.511 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:44.511 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-19T04:34:44.511 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:44.511 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 1912, 2024-09-19T04:34:44.511 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 194088, 2024-09-19T04:34:44.511 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:44.512 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 1912, 2024-09-19T04:34:44.512 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:44.512 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:44.512 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 453, 2024-09-19T04:34:44.512 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 194088, 2024-09-19T04:34:44.512 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:44.512 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.512 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.513 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.513 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:44.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:44.513 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x55fa74ebe960": { 2024-09-19T04:34:44.513 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 2260, 2024-09-19T04:34:44.513 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-19T04:34:44.513 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:44.513 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 16365, 2024-09-19T04:34:44.514 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 24125415, 2024-09-19T04:34:44.514 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:44.514 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 16365, 2024-09-19T04:34:44.514 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:44.514 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:44.514 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 3669, 2024-09-19T04:34:44.514 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 24123155, 2024-09-19T04:34:44.514 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:44.515 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:44.515 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:44.515 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:44.515 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:44.515 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:44.515 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-19T04:34:45.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:45 smithi145 ceph-mon[35096]: pgmap v451: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 622 KiB/s wr, 62 op/s 2024-09-19T04:34:45.422 INFO:tasks.check_counter:Daemon mds.c mds.exported=3 2024-09-19T04:34:45.422 INFO:tasks.check_counter:Daemon mds.c mds.imported=6 2024-09-19T04:34:45.422 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=674 2024-09-19T04:34:45.422 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=1400 2024-09-19T04:34:45.422 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-19T04:34:45.423 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:34:45.423 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:34:45.423 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T04:34:45.423 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T04:34:45.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:45 smithi115 ceph-mon[35477]: pgmap v451: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 622 KiB/s wr, 62 op/s 2024-09-19T04:34:45.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:45 smithi107 ceph-mon[30659]: pgmap v451: 97 pgs: 97 active+clean; 649 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 622 KiB/s wr, 62 op/s 2024-09-19T04:34:45.672 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:34:46.337 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-19T04:34:46.337 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:46.337 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 3851, 2024-09-19T04:34:46.337 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 3839, 2024-09-19T04:34:46.337 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 3068726, 2024-09-19T04:34:46.338 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 10612145, 2024-09-19T04:34:46.338 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 52, 2024-09-19T04:34:46.338 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 9, 2024-09-19T04:34:46.338 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.320317463, 2024-09-19T04:34:46.338 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.110955882, 2024-09-19T04:34:46.338 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.124061831, 2024-09-19T04:34:46.338 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.082261951, 2024-09-19T04:34:46.338 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:46.338 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3832, 2024-09-19T04:34:46.339 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.121031984, 2024-09-19T04:34:46.339 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000031584 2024-09-19T04:34:46.339 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.339 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:46.339 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 90, 2024-09-19T04:34:46.339 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000015237, 2024-09-19T04:34:46.339 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000169 2024-09-19T04:34:46.339 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.340 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1996768, 2024-09-19T04:34:46.340 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 204800 2024-09-19T04:34:46.340 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.340 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:46.340 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 123513, 2024-09-19T04:34:46.340 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 92914, 2024-09-19T04:34:46.340 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 39394699, 2024-09-19T04:34:46.340 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 44573660, 2024-09-19T04:34:46.341 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 11, 2024-09-19T04:34:46.341 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 6, 2024-09-19T04:34:46.341 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 5.527879257, 2024-09-19T04:34:46.341 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 2.270839237, 2024-09-19T04:34:46.341 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 4.379254251, 2024-09-19T04:34:46.341 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.028593453, 2024-09-19T04:34:46.341 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:46.341 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 92914, 2024-09-19T04:34:46.342 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.552567934, 2024-09-19T04:34:46.342 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000016709 2024-09-19T04:34:46.342 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.342 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:46.342 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 183677, 2024-09-19T04:34:46.342 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.030157678, 2024-09-19T04:34:46.342 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000164 2024-09-19T04:34:46.342 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.343 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T04:34:46.343 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 129168 2024-09-19T04:34:46.343 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.343 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:46.343 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 5001, 2024-09-19T04:34:46.343 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 5701, 2024-09-19T04:34:46.343 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1626450, 2024-09-19T04:34:46.343 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 17637937, 2024-09-19T04:34:46.344 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 9, 2024-09-19T04:34:46.344 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 5, 2024-09-19T04:34:46.344 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.378930346, 2024-09-19T04:34:46.344 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.170471925, 2024-09-19T04:34:46.344 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.146253377, 2024-09-19T04:34:46.344 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.065258845, 2024-09-19T04:34:46.344 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:46.344 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 5703, 2024-09-19T04:34:46.344 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.158685852, 2024-09-19T04:34:46.345 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000027824 2024-09-19T04:34:46.345 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.345 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:46.345 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4890, 2024-09-19T04:34:46.345 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000890866, 2024-09-19T04:34:46.345 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000182 2024-09-19T04:34:46.345 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.345 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T04:34:46.346 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 400944 2024-09-19T04:34:46.346 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.346 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-19T04:34:46.346 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-09-19T04:34:46.346 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-19T04:34:46.346 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.346 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-09-19T04:34:46.346 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-19T04:34:46.347 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-19T04:34:46.347 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 6888, 2024-09-19T04:34:46.347 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3.054172838, 2024-09-19T04:34:46.347 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000443404 2024-09-19T04:34:46.347 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:46.347 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.347 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-09-19T04:34:46.347 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-19T04:34:46.347 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-19T04:34:46.348 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4683, 2024-09-19T04:34:46.348 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.189262783, 2024-09-19T04:34:46.348 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000040414 2024-09-19T04:34:46.348 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:46.348 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.348 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-09-19T04:34:46.348 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 15247, 2024-09-19T04:34:46.348 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 14989, 2024-09-19T04:34:46.349 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-09-19T04:34:46.349 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 14989, 2024-09-19T04:34:46.349 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 34.212091618, 2024-09-19T04:34:46.349 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002282479 2024-09-19T04:34:46.349 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.349 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-09-19T04:34:46.349 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 258, 2024-09-19T04:34:46.350 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 3, 2024-09-19T04:34:46.350 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-09-19T04:34:46.350 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 11, 2024-09-19T04:34:46.350 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-09-19T04:34:46.350 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-09-19T04:34:46.350 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 997, 2024-09-19T04:34:46.350 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 60, 2024-09-19T04:34:46.350 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 923, 2024-09-19T04:34:46.351 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 14, 2024-09-19T04:34:46.351 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 911, 2024-09-19T04:34:46.351 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 237, 2024-09-19T04:34:46.351 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 95, 2024-09-19T04:34:46.351 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 95, 2024-09-19T04:34:46.351 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 15, 2024-09-19T04:34:46.351 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 29997, 2024-09-19T04:34:46.351 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 16000, 2024-09-19T04:34:46.352 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 256, 2024-09-19T04:34:46.352 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 123, 2024-09-19T04:34:46.352 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:34:46.352 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-09-19T04:34:46.352 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 302, 2024-09-19T04:34:46.352 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 4869, 2024-09-19T04:34:46.352 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-09-19T04:34:46.352 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 5, 2024-09-19T04:34:46.353 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 196, 2024-09-19T04:34:46.353 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 7, 2024-09-19T04:34:46.353 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 957, 2024-09-19T04:34:46.353 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-09-19T04:34:46.353 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 236, 2024-09-19T04:34:46.353 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-09-19T04:34:46.353 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 43, 2024-09-19T04:34:46.353 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 7184928, 2024-09-19T04:34:46.354 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 6, 2024-09-19T04:34:46.354 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 236, 2024-09-19T04:34:46.354 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-09-19T04:34:46.354 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:34:46.354 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:34:46.354 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:34:46.354 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:34:46.354 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:34:46.355 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 236, 2024-09-19T04:34:46.355 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:34:46.355 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 1619, 2024-09-19T04:34:46.355 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 9259, 2024-09-19T04:34:46.355 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:34:46.355 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 2, 2024-09-19T04:34:46.355 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:34:46.355 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 9255, 2024-09-19T04:34:46.355 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 1955, 2024-09-19T04:34:46.356 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 901, 2024-09-19T04:34:46.356 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 15171 2024-09-19T04:34:46.356 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.356 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-09-19T04:34:46.356 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 624, 2024-09-19T04:34:46.356 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 1598, 2024-09-19T04:34:46.356 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 859, 2024-09-19T04:34:46.357 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 1485, 2024-09-19T04:34:46.357 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 652, 2024-09-19T04:34:46.357 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-09-19T04:34:46.357 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-09-19T04:34:46.357 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:34:46.357 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 1939, 2024-09-19T04:34:46.357 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 1939, 2024-09-19T04:34:46.357 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-09-19T04:34:46.357 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-09-19T04:34:46.358 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-09-19T04:34:46.358 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:34:46.358 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:34:46.358 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-09-19T04:34:46.358 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-09-19T04:34:46.358 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:34:46.358 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:34:46.358 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:34:46.359 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 5, 2024-09-19T04:34:46.359 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-09-19T04:34:46.359 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:34:46.359 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-09-19T04:34:46.359 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-09-19T04:34:46.359 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.359 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-09-19T04:34:46.359 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-09-19T04:34:46.360 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 14919, 2024-09-19T04:34:46.360 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-09-19T04:34:46.360 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-09-19T04:34:46.360 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 14919, 2024-09-19T04:34:46.360 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-09-19T04:34:46.360 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-09-19T04:34:46.360 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 18, 2024-09-19T04:34:46.360 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-09-19T04:34:46.361 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-09-19T04:34:46.361 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 18, 2024-09-19T04:34:46.361 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 3, 2024-09-19T04:34:46.361 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-09-19T04:34:46.361 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-09-19T04:34:46.361 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-09-19T04:34:46.361 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 27705986, 2024-09-19T04:34:46.361 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 0, 2024-09-19T04:34:46.362 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-09-19T04:34:46.362 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3157, 2024-09-19T04:34:46.362 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 6.545602228, 2024-09-19T04:34:46.362 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002073361 2024-09-19T04:34:46.362 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.362 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 0 2024-09-19T04:34:46.362 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.362 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-09-19T04:34:46.363 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 857, 2024-09-19T04:34:46.363 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 3005, 2024-09-19T04:34:46.363 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 2148, 2024-09-19T04:34:46.363 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 139, 2024-09-19T04:34:46.363 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 789, 2024-09-19T04:34:46.363 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 650, 2024-09-19T04:34:46.363 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 997, 2024-09-19T04:34:46.363 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 3475, 2024-09-19T04:34:46.364 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 2478, 2024-09-19T04:34:46.364 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 95, 2024-09-19T04:34:46.364 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 2108, 2024-09-19T04:34:46.364 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 2013, 2024-09-19T04:34:46.364 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 63080, 2024-09-19T04:34:46.364 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 223520 2024-09-19T04:34:46.364 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.364 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-09-19T04:34:46.365 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 25943, 2024-09-19T04:34:46.365 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-09-19T04:34:46.365 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 15247, 2024-09-19T04:34:46.365 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 178, 2024-09-19T04:34:46.365 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 4182, 2024-09-19T04:34:46.365 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-09-19T04:34:46.365 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1920, 2024-09-19T04:34:46.366 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 4.456599810, 2024-09-19T04:34:46.366 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002321145 2024-09-19T04:34:46.366 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.366 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-09-19T04:34:46.366 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1, 2024-09-19T04:34:46.366 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.002999989, 2024-09-19T04:34:46.366 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002999989 2024-09-19T04:34:46.366 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.367 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-09-19T04:34:46.367 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.367 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.367 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.367 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.367 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-09-19T04:34:46.367 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.367 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.368 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.368 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.368 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-09-19T04:34:46.368 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2201, 2024-09-19T04:34:46.368 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 22.174310725, 2024-09-19T04:34:46.368 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.010074652 2024-09-19T04:34:46.368 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.368 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-09-19T04:34:46.368 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.369 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.369 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.369 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.369 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-09-19T04:34:46.369 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.369 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.369 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.369 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.370 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-09-19T04:34:46.370 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.370 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.370 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.370 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.370 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-09-19T04:34:46.370 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.370 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.371 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.371 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-09-19T04:34:46.371 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.371 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.371 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.372 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-09-19T04:34:46.372 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.372 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.372 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.372 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.372 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-09-19T04:34:46.372 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.372 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.372 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.373 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-09-19T04:34:46.373 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.373 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.373 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.373 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-09-19T04:34:46.373 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.374 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.374 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.374 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.374 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-09-19T04:34:46.374 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.374 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.374 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.375 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.375 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-09-19T04:34:46.375 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 8624, 2024-09-19T04:34:46.375 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3.611441377, 2024-09-19T04:34:46.375 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000418766 2024-09-19T04:34:46.375 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.375 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-09-19T04:34:46.375 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 304, 2024-09-19T04:34:46.375 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.685896986, 2024-09-19T04:34:46.376 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002256240 2024-09-19T04:34:46.376 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.376 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-09-19T04:34:46.376 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.376 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.377 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.377 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.377 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-09-19T04:34:46.377 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.377 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.377 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.377 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.377 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-09-19T04:34:46.378 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.378 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.378 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.378 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.378 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-09-19T04:34:46.378 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.378 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.378 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.379 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-09-19T04:34:46.379 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.379 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.379 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.379 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-09-19T04:34:46.379 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.380 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.380 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.380 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-09-19T04:34:46.380 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.380 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.380 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.381 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-09-19T04:34:46.381 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.381 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.381 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.381 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.381 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-09-19T04:34:46.381 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.381 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.382 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.382 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.382 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-09-19T04:34:46.382 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.382 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.382 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.382 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.382 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-09-19T04:34:46.383 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.383 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.383 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.383 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-09-19T04:34:46.383 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1939, 2024-09-19T04:34:46.383 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3.280842731, 2024-09-19T04:34:46.383 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001692028 2024-09-19T04:34:46.384 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.384 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:34:46.384 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:34:46.384 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-09-19T04:34:46.384 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.384 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.384 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.384 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:46.385 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.385 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-09-19T04:34:46.385 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 2, 2024-09-19T04:34:46.385 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 3, 2024-09-19T04:34:46.385 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 1, 2024-09-19T04:34:46.385 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 2, 2024-09-19T04:34:46.386 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-09-19T04:34:46.387 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 2558, 2024-09-19T04:34:46.387 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 1279, 2024-09-19T04:34:46.387 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 844, 2024-09-19T04:34:46.387 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-09-19T04:34:46.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.387 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-19T04:34:46.387 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:46.387 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:46.388 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:46.388 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:46.388 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:46.388 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:46.388 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:46.388 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:46.388 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:46.388 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:46.389 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:46.389 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:46.389 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:46.389 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:46.389 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:46.389 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:46.389 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:46.389 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:46.390 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:46.390 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:46.390 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:46.390 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:46.390 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:46.390 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:46.390 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:46.391 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:46.391 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:46.391 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:46.391 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:46.391 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:46.391 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:46.391 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-19T04:34:46.391 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:46.392 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:46.392 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:46.392 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:46.392 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 333237, 2024-09-19T04:34:46.392 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 1089, 2024-09-19T04:34:46.392 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 88, 2024-09-19T04:34:46.392 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 1, 2024-09-19T04:34:46.392 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-19T04:34:46.393 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-19T04:34:46.393 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:46.393 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:46.393 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:46.393 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:46.393 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 8032, 2024-09-19T04:34:46.393 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 177, 2024-09-19T04:34:46.393 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:46.394 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:46.394 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:46.394 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-19T04:34:46.394 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 3144528, 2024-09-19T04:34:46.394 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 29248, 2024-09-19T04:34:46.394 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:46.394 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-19T04:34:46.394 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:46.395 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-19T04:34:46.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.395 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-09-19T04:34:46.395 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-09-19T04:34:46.395 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-09-19T04:34:46.395 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 7003, 2024-09-19T04:34:46.395 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 23989876, 2024-09-19T04:34:46.395 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 33, 2024-09-19T04:34:46.396 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 7003, 2024-09-19T04:34:46.396 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-09-19T04:34:46.396 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 7003, 2024-09-19T04:34:46.396 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 14.512672253, 2024-09-19T04:34:46.396 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002072350 2024-09-19T04:34:46.396 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.396 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-09-19T04:34:46.396 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-09-19T04:34:46.397 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 7003, 2024-09-19T04:34:46.397 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 7655 2024-09-19T04:34:46.397 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.397 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 7003, 2024-09-19T04:34:46.397 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 677, 2024-09-19T04:34:46.397 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 6326, 2024-09-19T04:34:46.398 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-09-19T04:34:46.398 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-09-19T04:34:46.398 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 0, 2024-09-19T04:34:46.398 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 10, 2024-09-19T04:34:46.398 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 435, 2024-09-19T04:34:46.398 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 3592, 2024-09-19T04:34:46.398 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 285, 2024-09-19T04:34:46.398 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-09-19T04:34:46.399 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-09-19T04:34:46.399 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-09-19T04:34:46.399 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-09-19T04:34:46.399 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 1956, 2024-09-19T04:34:46.399 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-09-19T04:34:46.399 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-09-19T04:34:46.399 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-09-19T04:34:46.399 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 236, 2024-09-19T04:34:46.400 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 20, 2024-09-19T04:34:46.400 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:34:46.400 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-09-19T04:34:46.400 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:34:46.400 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-09-19T04:34:46.400 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-09-19T04:34:46.400 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-09-19T04:34:46.400 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:34:46.401 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-09-19T04:34:46.401 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:34:46.401 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-09-19T04:34:46.401 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-09-19T04:34:46.401 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-09-19T04:34:46.401 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-09-19T04:34:46.401 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-09-19T04:34:46.401 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-09-19T04:34:46.402 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-09-19T04:34:46.402 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-09-19T04:34:46.402 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-09-19T04:34:46.402 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-09-19T04:34:46.402 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-09-19T04:34:46.402 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-09-19T04:34:46.402 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-09-19T04:34:46.402 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-09-19T04:34:46.403 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-09-19T04:34:46.403 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-09-19T04:34:46.403 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-09-19T04:34:46.403 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 100, 2024-09-19T04:34:46.403 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-09-19T04:34:46.403 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 19, 2024-09-19T04:34:46.403 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 12, 2024-09-19T04:34:46.403 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 12, 2024-09-19T04:34:46.404 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 0, 2024-09-19T04:34:46.404 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-09-19T04:34:46.404 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 871, 2024-09-19T04:34:46.404 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 9, 2024-09-19T04:34:46.404 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 241 2024-09-19T04:34:46.404 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.404 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-09-19T04:34:46.404 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 1, 2024-09-19T04:34:46.405 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:34:46.405 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-09-19T04:34:46.405 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-09-19T04:34:46.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.405 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-09-19T04:34:46.405 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 1 2024-09-19T04:34:46.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.405 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-09-19T04:34:46.406 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-09-19T04:34:46.406 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T04:34:46.406 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-09-19T04:34:46.406 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 6, 2024-09-19T04:34:46.406 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 1939, 2024-09-19T04:34:46.406 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 1939, 2024-09-19T04:34:46.406 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-09-19T04:34:46.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.407 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:46.407 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:46.407 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-19T04:34:46.407 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:46.407 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 132361, 2024-09-19T04:34:46.407 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 32580537, 2024-09-19T04:34:46.408 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:46.408 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 132361, 2024-09-19T04:34:46.408 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:46.408 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:46.408 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 132361, 2024-09-19T04:34:46.408 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 32580537, 2024-09-19T04:34:46.408 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:46.408 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.409 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.409 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.409 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:46.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.409 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-09-19T04:34:46.409 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:46.409 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-19T04:34:46.409 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:46.410 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-19T04:34:46.410 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-19T04:34:46.410 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:46.410 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:46.410 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 7003, 2024-09-19T04:34:46.410 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 24186673, 2024-09-19T04:34:46.410 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 7003, 2024-09-19T04:34:46.411 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 24186673, 2024-09-19T04:34:46.411 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:46.411 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.411 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.411 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.411 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:46.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.411 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-09-19T04:34:46.412 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:46.412 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-09-19T04:34:46.412 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:46.412 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-19T04:34:46.412 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-19T04:34:46.412 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:46.412 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:46.412 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 7003, 2024-09-19T04:34:46.413 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 7003, 2024-09-19T04:34:46.413 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 7003, 2024-09-19T04:34:46.413 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 7003, 2024-09-19T04:34:46.413 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:46.413 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.413 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.413 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.413 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:46.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.414 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:34:46.414 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:46.414 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-19T04:34:46.414 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:46.414 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 1939, 2024-09-19T04:34:46.414 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 196935, 2024-09-19T04:34:46.414 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:46.415 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 1939, 2024-09-19T04:34:46.415 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:46.415 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:46.415 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 435, 2024-09-19T04:34:46.415 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 196935, 2024-09-19T04:34:46.415 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:46.415 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.415 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.416 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.416 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:46.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:46.416 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x55af330fe960": { 2024-09-19T04:34:46.416 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 574, 2024-09-19T04:34:46.416 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-19T04:34:46.416 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:46.416 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 14919, 2024-09-19T04:34:46.417 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 23511682, 2024-09-19T04:34:46.417 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:46.417 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 14919, 2024-09-19T04:34:46.417 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:46.417 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:46.417 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 3157, 2024-09-19T04:34:46.417 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 23511108, 2024-09-19T04:34:46.417 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:46.418 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:46.418 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:46.418 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:46.418 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:46.418 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:46.418 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-19T04:34:47.181 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:47 smithi145 ceph-mon[35096]: pgmap v452: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 646 KiB/s wr, 69 op/s 2024-09-19T04:34:47.181 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:47 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:34:47.208 INFO:tasks.check_counter:Daemon mds.f mds.exported=5 2024-09-19T04:34:47.208 INFO:tasks.check_counter:Daemon mds.f mds.imported=7 2024-09-19T04:34:47.208 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=624 2024-09-19T04:34:47.208 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=1598 2024-09-19T04:34:47.208 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-19T04:34:47.209 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:34:47.209 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:34:47.209 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T04:34:47.209 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T04:34:47.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:47 smithi115 ceph-mon[35477]: pgmap v452: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 646 KiB/s wr, 69 op/s 2024-09-19T04:34:47.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:47 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:34:47.456 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:34:47.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:47 smithi107 ceph-mon[30659]: pgmap v452: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 646 KiB/s wr, 69 op/s 2024-09-19T04:34:47.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:47 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:34:47.918 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-19T04:34:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 8996, 2024-09-19T04:34:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 9889, 2024-09-19T04:34:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 2435149, 2024-09-19T04:34:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 29048616, 2024-09-19T04:34:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 16, 2024-09-19T04:34:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 7, 2024-09-19T04:34:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.668936570, 2024-09-19T04:34:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.278629141, 2024-09-19T04:34:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.196606096, 2024-09-19T04:34:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.165230877, 2024-09-19T04:34:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 9883, 2024-09-19T04:34:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.247481907, 2024-09-19T04:34:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000025041 2024-09-19T04:34:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4894, 2024-09-19T04:34:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000812403, 2024-09-19T04:34:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000165 2024-09-19T04:34:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 2512, 2024-09-19T04:34:47.922 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 2416 2024-09-19T04:34:47.922 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.922 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:47.922 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 109231, 2024-09-19T04:34:47.923 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 82393, 2024-09-19T04:34:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 35682312, 2024-09-19T04:34:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 25588291, 2024-09-19T04:34:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 8, 2024-09-19T04:34:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 5, 2024-09-19T04:34:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 5.217783218, 2024-09-19T04:34:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 2.041494021, 2024-09-19T04:34:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 3.978994824, 2024-09-19T04:34:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.072976225, 2024-09-19T04:34:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 82397, 2024-09-19T04:34:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.401290344, 2024-09-19T04:34:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000017006 2024-09-19T04:34:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 185204, 2024-09-19T04:34:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.027461021, 2024-09-19T04:34:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000148 2024-09-19T04:34:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 2008832, 2024-09-19T04:34:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 194592 2024-09-19T04:34:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 50913, 2024-09-19T04:34:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 79020, 2024-09-19T04:34:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 16577111, 2024-09-19T04:34:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 67797933, 2024-09-19T04:34:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 49, 2024-09-19T04:34:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 7, 2024-09-19T04:34:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 2.534525306, 2024-09-19T04:34:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 1.220185978, 2024-09-19T04:34:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 1.505870135, 2024-09-19T04:34:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.122366689, 2024-09-19T04:34:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 79017, 2024-09-19T04:34:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.281961957, 2024-09-19T04:34:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000016223 2024-09-19T04:34:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 65434, 2024-09-19T04:34:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.015014432, 2024-09-19T04:34:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000229 2024-09-19T04:34:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-09-19T04:34:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 532624 2024-09-19T04:34:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-19T04:34:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-09-19T04:34:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-19T04:34:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-09-19T04:34:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-19T04:34:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-19T04:34:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 13782, 2024-09-19T04:34:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 6.991210692, 2024-09-19T04:34:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000507271 2024-09-19T04:34:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-09-19T04:34:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-19T04:34:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-19T04:34:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 10957, 2024-09-19T04:34:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.480792656, 2024-09-19T04:34:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000043879 2024-09-19T04:34:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-09-19T04:34:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 35810, 2024-09-19T04:34:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 32001, 2024-09-19T04:34:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-09-19T04:34:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 32001, 2024-09-19T04:34:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 42.570025036, 2024-09-19T04:34:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001330271 2024-09-19T04:34:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-09-19T04:34:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 3809, 2024-09-19T04:34:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-09-19T04:34:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-09-19T04:34:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 11, 2024-09-19T04:34:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-09-19T04:34:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-09-19T04:34:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 455, 2024-09-19T04:34:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 1, 2024-09-19T04:34:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 1, 2024-09-19T04:34:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 453, 2024-09-19T04:34:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 453, 2024-09-19T04:34:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 593, 2024-09-19T04:34:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 156, 2024-09-19T04:34:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 156, 2024-09-19T04:34:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 10, 2024-09-19T04:34:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 75097, 2024-09-19T04:34:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 30124, 2024-09-19T04:34:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 3808, 2024-09-19T04:34:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 195, 2024-09-19T04:34:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:34:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-09-19T04:34:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 541, 2024-09-19T04:34:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 8296, 2024-09-19T04:34:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-09-19T04:34:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 4, 2024-09-19T04:34:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 413, 2024-09-19T04:34:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 6, 2024-09-19T04:34:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 563, 2024-09-19T04:34:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-09-19T04:34:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 118, 2024-09-19T04:34:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-09-19T04:34:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 43, 2024-09-19T04:34:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 7184928, 2024-09-19T04:34:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 6, 2024-09-19T04:34:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 118, 2024-09-19T04:34:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-09-19T04:34:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:34:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:34:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:34:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:34:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:34:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 118, 2024-09-19T04:34:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:34:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 1449, 2024-09-19T04:34:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 19965, 2024-09-19T04:34:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:34:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:34:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:34:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 6132, 2024-09-19T04:34:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 1677, 2024-09-19T04:34:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 773, 2024-09-19T04:34:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 30522 2024-09-19T04:34:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-09-19T04:34:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 534, 2024-09-19T04:34:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 2399, 2024-09-19T04:34:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 1276, 2024-09-19T04:34:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 2304, 2024-09-19T04:34:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 589, 2024-09-19T04:34:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-09-19T04:34:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-09-19T04:34:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:34:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 5537, 2024-09-19T04:34:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 5537, 2024-09-19T04:34:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-09-19T04:34:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-09-19T04:34:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-09-19T04:34:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:34:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:34:47.942 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-09-19T04:34:47.942 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-09-19T04:34:47.942 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:34:47.942 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:34:47.942 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:34:47.942 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 5, 2024-09-19T04:34:47.942 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-09-19T04:34:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:34:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-09-19T04:34:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-09-19T04:34:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-09-19T04:34:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-09-19T04:34:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 26622, 2024-09-19T04:34:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-09-19T04:34:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-09-19T04:34:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 26622, 2024-09-19T04:34:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-09-19T04:34:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-09-19T04:34:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 33, 2024-09-19T04:34:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-09-19T04:34:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-09-19T04:34:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 33, 2024-09-19T04:34:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 4, 2024-09-19T04:34:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-09-19T04:34:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-09-19T04:34:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-09-19T04:34:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 48817984, 2024-09-19T04:34:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 0, 2024-09-19T04:34:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-09-19T04:34:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 6616, 2024-09-19T04:34:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 12.196670160, 2024-09-19T04:34:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001843511 2024-09-19T04:34:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 0 2024-09-19T04:34:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-09-19T04:34:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 319, 2024-09-19T04:34:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 6290, 2024-09-19T04:34:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 5971, 2024-09-19T04:34:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 60, 2024-09-19T04:34:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 1195, 2024-09-19T04:34:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 1135, 2024-09-19T04:34:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 455, 2024-09-19T04:34:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 9607, 2024-09-19T04:34:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 9152, 2024-09-19T04:34:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 156, 2024-09-19T04:34:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 5917, 2024-09-19T04:34:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 5761, 2024-09-19T04:34:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 67212, 2024-09-19T04:34:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 223520 2024-09-19T04:34:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-09-19T04:34:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 64920, 2024-09-19T04:34:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 16, 2024-09-19T04:34:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 35810, 2024-09-19T04:34:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 296, 2024-09-19T04:34:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 11647, 2024-09-19T04:34:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-09-19T04:34:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 5531, 2024-09-19T04:34:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 12.487495608, 2024-09-19T04:34:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002257728 2024-09-19T04:34:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-09-19T04:34:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 127, 2024-09-19T04:34:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.071339526, 2024-09-19T04:34:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000561728 2024-09-19T04:34:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-09-19T04:34:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-09-19T04:34:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-09-19T04:34:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 7891, 2024-09-19T04:34:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 10.275084746, 2024-09-19T04:34:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001302127 2024-09-19T04:34:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-09-19T04:34:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-09-19T04:34:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1, 2024-09-19T04:34:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000118075, 2024-09-19T04:34:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000118075 2024-09-19T04:34:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-09-19T04:34:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-09-19T04:34:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-09-19T04:34:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-09-19T04:34:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-09-19T04:34:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-09-19T04:34:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-09-19T04:34:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-09-19T04:34:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-09-19T04:34:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 12374, 2024-09-19T04:34:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 4.611747039, 2024-09-19T04:34:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000372696 2024-09-19T04:34:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-09-19T04:34:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 538, 2024-09-19T04:34:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.195228310, 2024-09-19T04:34:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002221613 2024-09-19T04:34:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-09-19T04:34:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-09-19T04:34:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-09-19T04:34:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-09-19T04:34:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-09-19T04:34:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-09-19T04:34:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2, 2024-09-19T04:34:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.005399298, 2024-09-19T04:34:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002699649 2024-09-19T04:34:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-09-19T04:34:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-09-19T04:34:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-09-19T04:34:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-09-19T04:34:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-09-19T04:34:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-09-19T04:34:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 5537, 2024-09-19T04:34:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 13.923612434, 2024-09-19T04:34:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002514649 2024-09-19T04:34:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:34:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:34:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-09-19T04:34:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-09-19T04:34:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 1, 2024-09-19T04:34:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 4, 2024-09-19T04:34:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 3, 2024-09-19T04:34:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 1, 2024-09-19T04:34:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-09-19T04:34:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 3903, 2024-09-19T04:34:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 3903, 2024-09-19T04:34:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 854, 2024-09-19T04:34:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-09-19T04:34:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-19T04:34:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-19T04:34:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 307370, 2024-09-19T04:34:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 319, 2024-09-19T04:34:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 176, 2024-09-19T04:34:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 2, 2024-09-19T04:34:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-19T04:34:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-19T04:34:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 8032, 2024-09-19T04:34:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 177, 2024-09-19T04:34:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-19T04:34:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 1303060, 2024-09-19T04:34:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 9909, 2024-09-19T04:34:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-19T04:34:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-19T04:34:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-09-19T04:34:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-09-19T04:34:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-09-19T04:34:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 13942, 2024-09-19T04:34:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 45765678, 2024-09-19T04:34:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 32, 2024-09-19T04:34:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 13942, 2024-09-19T04:34:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-09-19T04:34:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 13942, 2024-09-19T04:34:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 28.609353174, 2024-09-19T04:34:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002052026 2024-09-19T04:34:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-09-19T04:34:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-09-19T04:34:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 13942, 2024-09-19T04:34:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 14620 2024-09-19T04:34:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 13942, 2024-09-19T04:34:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 555, 2024-09-19T04:34:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 13387, 2024-09-19T04:34:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-09-19T04:34:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-09-19T04:34:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 0, 2024-09-19T04:34:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 10, 2024-09-19T04:34:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 437, 2024-09-19T04:34:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 7053, 2024-09-19T04:34:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 286, 2024-09-19T04:34:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-09-19T04:34:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-09-19T04:34:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-09-19T04:34:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-09-19T04:34:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 5559, 2024-09-19T04:34:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-09-19T04:34:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-09-19T04:34:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-09-19T04:34:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 118, 2024-09-19T04:34:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 20, 2024-09-19T04:34:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:34:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-09-19T04:34:47.982 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:34:47.982 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-09-19T04:34:47.982 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-09-19T04:34:47.982 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-09-19T04:34:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:34:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-09-19T04:34:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:34:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-09-19T04:34:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-09-19T04:34:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-09-19T04:34:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-09-19T04:34:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-09-19T04:34:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-09-19T04:34:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-09-19T04:34:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-09-19T04:34:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-09-19T04:34:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-09-19T04:34:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-09-19T04:34:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-09-19T04:34:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-09-19T04:34:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-09-19T04:34:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-09-19T04:34:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-09-19T04:34:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-09-19T04:34:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 100, 2024-09-19T04:34:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-09-19T04:34:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 19, 2024-09-19T04:34:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 12, 2024-09-19T04:34:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 12, 2024-09-19T04:34:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 0, 2024-09-19T04:34:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-09-19T04:34:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 839, 2024-09-19T04:34:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 0, 2024-09-19T04:34:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 298 2024-09-19T04:34:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-09-19T04:34:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 1, 2024-09-19T04:34:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:34:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-09-19T04:34:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-09-19T04:34:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-09-19T04:34:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-09-19T04:34:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-09-19T04:34:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-09-19T04:34:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T04:34:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-09-19T04:34:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 6, 2024-09-19T04:34:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 5537, 2024-09-19T04:34:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 5537, 2024-09-19T04:34:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-09-19T04:34:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-19T04:34:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 169136, 2024-09-19T04:34:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 40190427, 2024-09-19T04:34:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 169136, 2024-09-19T04:34:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 169136, 2024-09-19T04:34:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 40190427, 2024-09-19T04:34:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-09-19T04:34:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-19T04:34:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-19T04:34:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-19T04:34:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 13942, 2024-09-19T04:34:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 46325467, 2024-09-19T04:34:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 13942, 2024-09-19T04:34:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 46325467, 2024-09-19T04:34:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-09-19T04:34:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-09-19T04:34:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-19T04:34:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-19T04:34:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 13942, 2024-09-19T04:34:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 13942, 2024-09-19T04:34:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 13942, 2024-09-19T04:34:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 13942, 2024-09-19T04:34:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:34:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-19T04:34:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 5537, 2024-09-19T04:34:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 559789, 2024-09-19T04:34:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 5537, 2024-09-19T04:34:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 437, 2024-09-19T04:34:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 559789, 2024-09-19T04:34:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x564384f88960": { 2024-09-19T04:34:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-19T04:34:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 26622, 2024-09-19T04:34:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 44623680, 2024-09-19T04:34:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 26622, 2024-09-19T04:34:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 6616, 2024-09-19T04:34:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 44623680, 2024-09-19T04:34:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:47.999 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-19T04:34:48.224 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:48 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:34:48.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:48 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:34:48.225 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:48 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:34:48.438 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:48 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:34:48.438 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:48 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:34:48.438 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:48 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:34:48.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:48 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:34:48.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:48 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:34:48.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:48 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' 2024-09-19T04:34:48.694 INFO:tasks.check_counter:Daemon mds.i mds.exported=4 2024-09-19T04:34:48.694 INFO:tasks.check_counter:Daemon mds.i mds.imported=6 2024-09-19T04:34:48.694 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=534 2024-09-19T04:34:48.694 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=2399 2024-09-19T04:34:48.694 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-19T04:34:48.694 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:34:48.695 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:34:48.695 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T04:34:48.695 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T04:34:48.939 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:34:49.354 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:49 smithi145 ceph-mon[35096]: pgmap v453: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-09-19T04:34:49.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:49 smithi115 ceph-mon[35477]: pgmap v453: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-09-19T04:34:49.462 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-19T04:34:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:34:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 1568, 2024-09-19T04:34:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 1526, 2024-09-19T04:34:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 10754140, 2024-09-19T04:34:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 564834, 2024-09-19T04:34:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-09-19T04:34:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 2, 2024-09-19T04:34:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.179483767, 2024-09-19T04:34:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.054138514, 2024-09-19T04:34:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.090550564, 2024-09-19T04:34:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.040911341, 2024-09-19T04:34:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1525, 2024-09-19T04:34:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.072289087, 2024-09-19T04:34:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000047402 2024-09-19T04:34:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.467 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:49.467 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.467 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.467 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1974768, 2024-09-19T04:34:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 175920 2024-09-19T04:34:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:34:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 658, 2024-09-19T04:34:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 866, 2024-09-19T04:34:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 15432592, 2024-09-19T04:34:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 736070, 2024-09-19T04:34:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 8, 2024-09-19T04:34:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 6, 2024-09-19T04:34:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.074725892, 2024-09-19T04:34:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.021737869, 2024-09-19T04:34:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.029395787, 2024-09-19T04:34:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.016979757, 2024-09-19T04:34:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 864, 2024-09-19T04:34:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.138749680, 2024-09-19T04:34:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000160589 2024-09-19T04:34:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T04:34:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 528672 2024-09-19T04:34:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:34:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 1368, 2024-09-19T04:34:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 1370, 2024-09-19T04:34:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 21322886, 2024-09-19T04:34:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 435031, 2024-09-19T04:34:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-09-19T04:34:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 4, 2024-09-19T04:34:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.106958251, 2024-09-19T04:34:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.032494236, 2024-09-19T04:34:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.036102008, 2024-09-19T04:34:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.030699242, 2024-09-19T04:34:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:34:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1368, 2024-09-19T04:34:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.048084646, 2024-09-19T04:34:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000035149 2024-09-19T04:34:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:34:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:34:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:34:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-19T04:34:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-09-19T04:34:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-19T04:34:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-09-19T04:34:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-19T04:34:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-19T04:34:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 5542, 2024-09-19T04:34:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.135489110, 2024-09-19T04:34:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000024447 2024-09-19T04:34:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-09-19T04:34:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-19T04:34:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-19T04:34:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-09-19T04:34:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 0, 2024-09-19T04:34:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 0, 2024-09-19T04:34:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-09-19T04:34:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-09-19T04:34:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 0, 2024-09-19T04:34:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-09-19T04:34:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-09-19T04:34:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 0, 2024-09-19T04:34:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-09-19T04:34:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-09-19T04:34:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 5977, 2024-09-19T04:34:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 2, 2024-09-19T04:34:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 5975, 2024-09-19T04:34:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 0, 2024-09-19T04:34:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 5973, 2024-09-19T04:34:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 0, 2024-09-19T04:34:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 0, 2024-09-19T04:34:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 0, 2024-09-19T04:34:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 6, 2024-09-19T04:34:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 0, 2024-09-19T04:34:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 0, 2024-09-19T04:34:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 0, 2024-09-19T04:34:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 0, 2024-09-19T04:34:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:34:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-09-19T04:34:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-09-19T04:34:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-09-19T04:34:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-09-19T04:34:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 0, 2024-09-19T04:34:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 0, 2024-09-19T04:34:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 0, 2024-09-19T04:34:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 0, 2024-09-19T04:34:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-09-19T04:34:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:34:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-09-19T04:34:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 0, 2024-09-19T04:34:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 0, 2024-09-19T04:34:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 0, 2024-09-19T04:34:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:34:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-09-19T04:34:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:34:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:34:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:34:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:34:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:34:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-09-19T04:34:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:34:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:34:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:34:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:34:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:34:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:34:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 0, 2024-09-19T04:34:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:34:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 0, 2024-09-19T04:34:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 0 2024-09-19T04:34:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-09-19T04:34:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-09-19T04:34:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-09-19T04:34:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-09-19T04:34:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 0, 2024-09-19T04:34:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 0, 2024-09-19T04:34:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-09-19T04:34:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-09-19T04:34:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:34:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 0, 2024-09-19T04:34:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 0, 2024-09-19T04:34:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-09-19T04:34:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-09-19T04:34:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-09-19T04:34:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:34:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:34:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-09-19T04:34:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-09-19T04:34:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:34:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:34:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:34:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 0, 2024-09-19T04:34:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-09-19T04:34:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:34:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-09-19T04:34:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-09-19T04:34:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-09-19T04:34:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-09-19T04:34:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 0, 2024-09-19T04:34:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-09-19T04:34:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-09-19T04:34:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 26622, 2024-09-19T04:34:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-09-19T04:34:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-09-19T04:34:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 0, 2024-09-19T04:34:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-09-19T04:34:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-09-19T04:34:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 33, 2024-09-19T04:34:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 4, 2024-09-19T04:34:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-09-19T04:34:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-09-19T04:34:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-09-19T04:34:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 48817984, 2024-09-19T04:34:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 48807861, 2024-09-19T04:34:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-09-19T04:34:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 26622 2024-09-19T04:34:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-09-19T04:34:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 298, 2024-09-19T04:34:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 6122, 2024-09-19T04:34:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 5824, 2024-09-19T04:34:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 43, 2024-09-19T04:34:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 65, 2024-09-19T04:34:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 22, 2024-09-19T04:34:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 5977, 2024-09-19T04:34:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 6401, 2024-09-19T04:34:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 424, 2024-09-19T04:34:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 0, 2024-09-19T04:34:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 0, 2024-09-19T04:34:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 0, 2024-09-19T04:34:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 56420, 2024-09-19T04:34:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 215328 2024-09-19T04:34:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-09-19T04:34:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 0, 2024-09-19T04:34:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-09-19T04:34:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 0, 2024-09-19T04:34:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 0, 2024-09-19T04:34:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 0, 2024-09-19T04:34:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-09-19T04:34:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-09-19T04:34:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-09-19T04:34:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-09-19T04:34:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-09-19T04:34:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-09-19T04:34:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-09-19T04:34:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-09-19T04:34:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-09-19T04:34:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.507 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-09-19T04:34:49.507 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.507 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.507 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-09-19T04:34:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-09-19T04:34:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-09-19T04:34:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-09-19T04:34:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-09-19T04:34:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-09-19T04:34:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-09-19T04:34:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-09-19T04:34:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-09-19T04:34:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.514 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-09-19T04:34:49.514 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.514 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.514 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.515 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-09-19T04:34:49.515 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.515 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.515 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.516 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-09-19T04:34:49.516 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-09-19T04:34:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-09-19T04:34:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-09-19T04:34:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-09-19T04:34:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.520 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.520 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-09-19T04:34:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-09-19T04:34:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-09-19T04:34:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:34:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:34:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-09-19T04:34:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-09-19T04:34:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 1, 2024-09-19T04:34:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 4, 2024-09-19T04:34:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 3, 2024-09-19T04:34:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 3, 2024-09-19T04:34:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-09-19T04:34:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 0, 2024-09-19T04:34:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 0, 2024-09-19T04:34:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 5012, 2024-09-19T04:34:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-09-19T04:34:49.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.525 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-19T04:34:49.525 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:34:49.525 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-19T04:34:49.525 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:34:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:34:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:34:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:34:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:34:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:34:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:34:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:34:49.527 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:34:49.527 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:34:49.527 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:34:49.527 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:34:49.528 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:34:49.528 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-19T04:34:49.528 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:34:49.528 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-19T04:34:49.529 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:34:49.529 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:34:49.529 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:34:49.530 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:34:49.530 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:34:49.530 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:34:49.530 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:34:49.531 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-19T04:34:49.531 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:34:49.531 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:34:49.531 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:34:49.532 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-19T04:34:49.532 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-19T04:34:49.532 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-19T04:34:49.533 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:34:49.533 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:34:49.533 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:34:49.533 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:34:49.533 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 227361, 2024-09-19T04:34:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 377, 2024-09-19T04:34:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:34:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-09-19T04:34:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-19T04:34:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-19T04:34:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:34:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-19T04:34:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:34:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-19T04:34:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 8032, 2024-09-19T04:34:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 177, 2024-09-19T04:34:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:34:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:34:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-19T04:34:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-19T04:34:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 5035536, 2024-09-19T04:34:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 31181, 2024-09-19T04:34:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-19T04:34:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-19T04:34:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-19T04:34:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-19T04:34:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-09-19T04:34:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-09-19T04:34:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-09-19T04:34:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 3243, 2024-09-19T04:34:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 21, 2024-09-19T04:34:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 3, 2024-09-19T04:34:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 3243, 2024-09-19T04:34:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-09-19T04:34:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3243, 2024-09-19T04:34:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.822958761, 2024-09-19T04:34:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000562121 2024-09-19T04:34:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-09-19T04:34:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-09-19T04:34:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3243, 2024-09-19T04:34:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3244 2024-09-19T04:34:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 3243, 2024-09-19T04:34:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 3243, 2024-09-19T04:34:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 0, 2024-09-19T04:34:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-09-19T04:34:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-09-19T04:34:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 1867, 2024-09-19T04:34:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 0, 2024-09-19T04:34:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 1374, 2024-09-19T04:34:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 0, 2024-09-19T04:34:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 0, 2024-09-19T04:34:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-09-19T04:34:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-09-19T04:34:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-09-19T04:34:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-09-19T04:34:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 0, 2024-09-19T04:34:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-09-19T04:34:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-09-19T04:34:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-09-19T04:34:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 1, 2024-09-19T04:34:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 0, 2024-09-19T04:34:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:34:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-09-19T04:34:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:34:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-09-19T04:34:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-09-19T04:34:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-09-19T04:34:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:34:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-09-19T04:34:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:34:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-09-19T04:34:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-09-19T04:34:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-09-19T04:34:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-09-19T04:34:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-09-19T04:34:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-09-19T04:34:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-09-19T04:34:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-09-19T04:34:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-09-19T04:34:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-09-19T04:34:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-09-19T04:34:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-09-19T04:34:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-09-19T04:34:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-09-19T04:34:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-09-19T04:34:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-09-19T04:34:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-09-19T04:34:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 100, 2024-09-19T04:34:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-09-19T04:34:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 19, 2024-09-19T04:34:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 8, 2024-09-19T04:34:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 9, 2024-09-19T04:34:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 1, 2024-09-19T04:34:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-09-19T04:34:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 0, 2024-09-19T04:34:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 2, 2024-09-19T04:34:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 0 2024-09-19T04:34:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-09-19T04:34:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 0, 2024-09-19T04:34:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:34:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-09-19T04:34:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-09-19T04:34:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-09-19T04:34:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-09-19T04:34:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-09-19T04:34:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-09-19T04:34:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:34:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-09-19T04:34:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 0, 2024-09-19T04:34:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 0, 2024-09-19T04:34:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 0, 2024-09-19T04:34:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-09-19T04:34:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:34:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-19T04:34:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 3590, 2024-09-19T04:34:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 47165938, 2024-09-19T04:34:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 3590, 2024-09-19T04:34:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 3590, 2024-09-19T04:34:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 47165938, 2024-09-19T04:34:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-09-19T04:34:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-19T04:34:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-19T04:34:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-19T04:34:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 3243, 2024-09-19T04:34:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 44627582, 2024-09-19T04:34:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 3243, 2024-09-19T04:34:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 44627582, 2024-09-19T04:34:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-09-19T04:34:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-09-19T04:34:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-19T04:34:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-19T04:34:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 3243, 2024-09-19T04:34:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 3243, 2024-09-19T04:34:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 3243, 2024-09-19T04:34:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 3243, 2024-09-19T04:34:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:34:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-19T04:34:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-19T04:34:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-19T04:34:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-09-19T04:34:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-09-19T04:34:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-19T04:34:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x556c5372ab40": { 2024-09-19T04:34:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-19T04:34:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-19T04:34:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-19T04:34:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-19T04:34:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-19T04:34:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-19T04:34:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-19T04:34:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-19T04:34:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-19T04:34:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-09-19T04:34:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-09-19T04:34:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-19T04:34:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-19T04:34:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-19T04:34:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-19T04:34:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-19T04:34:49.560 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-19T04:34:49.561 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:49 smithi107 ceph-mon[30659]: pgmap v453: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-09-19T04:34:50.282 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T04:34:50.283 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T04:34:50.283 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-19T04:34:50.283 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-19T04:34:50.283 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-19T04:34:50.283 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-19T04:34:50.283 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T04:34:50.283 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T04:34:50.294 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T04:34:50.294 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:34:50.294 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:34:50.336 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-19T04:34:50.336 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:34:50.415 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T04:34:50.828 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:51.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:51 smithi107 ceph-mon[30659]: pgmap v454: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-09-19T04:34:51.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:51 smithi115 ceph-mon[35477]: pgmap v454: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-09-19T04:34:51.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:51 smithi145 ceph-mon[35096]: pgmap v454: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-09-19T04:34:51.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.485+0000 7ff809a00640 1 -- 172.21.15.107:0/1271485363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804112a50 msgr2=0x7ff804114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:51.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.485+0000 7ff809a00640 1 --2- 172.21.15.107:0/1271485363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804112a50 0x7ff804114e40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009ca0 tx=0x7ff7fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.485+0000 7ff809a00640 1 -- 172.21.15.107:0/1271485363 shutdown_connections 2024-09-19T04:34:51.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.485+0000 7ff809a00640 1 --2- 172.21.15.107:0/1271485363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804112a50 0x7ff804114e40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.485+0000 7ff809a00640 1 --2- 172.21.15.107:0/1271485363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff804077040 0x7ff804112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.485+0000 7ff809a00640 1 --2- 172.21.15.107:0/1271485363 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff804076720 0x7ff804076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.485+0000 7ff809a00640 1 -- 172.21.15.107:0/1271485363 >> 172.21.15.107:0/1271485363 conn(0x7ff8040ffa50 msgr2=0x7ff804101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:51.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.485+0000 7ff809a00640 1 -- 172.21.15.107:0/1271485363 shutdown_connections 2024-09-19T04:34:51.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.485+0000 7ff809a00640 1 -- 172.21.15.107:0/1271485363 wait complete. 2024-09-19T04:34:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.486+0000 7ff809a00640 1 Processor -- start 2024-09-19T04:34:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.486+0000 7ff809a00640 1 -- start start 2024-09-19T04:34:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.486+0000 7ff809a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff804076720 0x7ff8041a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.486+0000 7ff809a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804077040 0x7ff8041a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.486+0000 7ff809a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff804112a50 0x7ff80419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.486+0000 7ff809a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041185f0 con 0x7ff804077040 2024-09-19T04:34:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.486+0000 7ff809a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff804118470 con 0x7ff804076720 2024-09-19T04:34:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.486+0000 7ff809a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff804118770 con 0x7ff804112a50 2024-09-19T04:34:51.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff802000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804077040 0x7ff8041a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:51.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff802a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff804076720 0x7ff8041a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:51.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff802000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804077040 0x7ff8041a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57600/0 (socket says 172.21.15.107:57600) 2024-09-19T04:34:51.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff802a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff804076720 0x7ff8041a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47588/0 (socket says 172.21.15.107:47588) 2024-09-19T04:34:51.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff802000640 1 -- 172.21.15.107:0/3956205725 learned_addr learned my addr 172.21.15.107:0/3956205725 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:51.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff803400640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff804112a50 0x7ff80419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:51.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff802000640 1 -- 172.21.15.107:0/3956205725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff804112a50 msgr2=0x7ff80419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:51.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff802000640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff804112a50 0x7ff80419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff802000640 1 -- 172.21.15.107:0/3956205725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff804076720 msgr2=0x7ff8041a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:51.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff802000640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff804076720 0x7ff8041a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.487+0000 7ff802000640 1 -- 172.21.15.107:0/3956205725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80419efd0 con 0x7ff804077040 2024-09-19T04:34:51.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.488+0000 7ff802000640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804077040 0x7ff8041a4c20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000e9f0 tx=0x7ff7f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:51.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.488+0000 7ff7fb400640 1 -- 172.21.15.107:0/3956205725 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f000cd80 con 0x7ff804077040 2024-09-19T04:34:51.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.488+0000 7ff7fb400640 1 -- 172.21.15.107:0/3956205725 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f0010910 con 0x7ff804077040 2024-09-19T04:34:51.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.488+0000 7ff809a00640 1 -- 172.21.15.107:0/3956205725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80419f2c0 con 0x7ff804077040 2024-09-19T04:34:51.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.488+0000 7ff7fb400640 1 -- 172.21.15.107:0/3956205725 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0018990 con 0x7ff804077040 2024-09-19T04:34:51.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.488+0000 7ff809a00640 1 -- 172.21.15.107:0/3956205725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8041ab5b0 con 0x7ff804077040 2024-09-19T04:34:51.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.489+0000 7ff809a00640 1 -- 172.21.15.107:0/3956205725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff804076b00 con 0x7ff804077040 2024-09-19T04:34:51.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.491+0000 7ff7fb400640 1 -- 172.21.15.107:0/3956205725 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7ff7f0010ac0 con 0x7ff804077040 2024-09-19T04:34:51.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.493+0000 7ff7fb400640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff7e80a3430 0x7ff7e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:51.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.493+0000 7ff7fb400640 1 -- 172.21.15.107:0/3956205725 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7ff7f00cff30 con 0x7ff804077040 2024-09-19T04:34:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.493+0000 7ff802a00640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff7e80a3430 0x7ff7e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.494+0000 7ff802a00640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff7e80a3430 0x7ff7e80a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff80419f9e0 tx=0x7ff7f4005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:51.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.497+0000 7ff7fb400640 1 -- 172.21.15.107:0/3956205725 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff7f009c280 con 0x7ff804077040 2024-09-19T04:34:51.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.685+0000 7ff809a00640 1 -- 172.21.15.107:0/3956205725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff8041ab8d0 con 0x7ff804077040 2024-09-19T04:34:51.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.686+0000 7ff7fb400640 1 -- 172.21.15.107:0/3956205725 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+959 (secure 0 0 0) 0x7ff7f00a0580 con 0x7ff804077040 2024-09-19T04:34:51.688 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3991722475 2024-09-20T04:20:19.150743+0000 2024-09-19T04:34:51.688 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3866113757 2024-09-20T04:20:19.150743+0000 2024-09-19T04:34:51.688 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1307215535 2024-09-20T04:20:19.150743+0000 2024-09-19T04:34:51.688 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2144535101 2024-09-20T04:10:32.750566+0000 2024-09-19T04:34:51.688 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1606203977 2024-09-20T04:10:32.750566+0000 2024-09-19T04:34:51.688 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/2057620332 2024-09-20T04:10:32.750566+0000 2024-09-19T04:34:51.689 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3489252098 2024-09-20T04:20:19.150743+0000 2024-09-19T04:34:51.689 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/2057620332 2024-09-20T04:10:32.750566+0000 2024-09-19T04:34:51.689 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1762826901 2024-09-20T04:10:32.750566+0000 2024-09-19T04:34:51.689 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1481502377 2024-09-20T04:10:59.721218+0000 2024-09-19T04:34:51.689 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3237606700 2024-09-20T04:10:59.721218+0000 2024-09-19T04:34:51.689 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/972711045 2024-09-20T04:10:59.721218+0000 2024-09-19T04:34:51.689 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/972711045 2024-09-20T04:10:59.721218+0000 2024-09-19T04:34:51.689 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3991722475 2024-09-20T04:20:19.150743+0000 2024-09-19T04:34:51.690 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3889998535 2024-09-20T04:20:19.150743+0000 2024-09-19T04:34:51.690 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/498138874 2024-09-20T04:10:59.721218+0000 2024-09-19T04:34:51.690 INFO:teuthology.orchestra.run.smithi107.stderr:listed 16 entries 2024-09-19T04:34:51.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 -- 172.21.15.107:0/3956205725 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff7e80a3430 msgr2=0x7ff7e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:51.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff7e80a3430 0x7ff7e80a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff80419f9e0 tx=0x7ff7f4005f90 comp rx=0 tx=0).stop 2024-09-19T04:34:51.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 -- 172.21.15.107:0/3956205725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804077040 msgr2=0x7ff8041a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:51.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804077040 0x7ff8041a4c20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000e9f0 tx=0x7ff7f000eeb0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 -- 172.21.15.107:0/3956205725 shutdown_connections 2024-09-19T04:34:51.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff7e80a3430 0x7ff7e80a58f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff804112a50 0x7ff80419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804077040 0x7ff8041a4c20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 --2- 172.21.15.107:0/3956205725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff804076720 0x7ff8041a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:51.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 -- 172.21.15.107:0/3956205725 >> 172.21.15.107:0/3956205725 conn(0x7ff8040ffa50 msgr2=0x7ff804101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:51.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 -- 172.21.15.107:0/3956205725 shutdown_connections 2024-09-19T04:34:51.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:51.688+0000 7ff809a00640 1 -- 172.21.15.107:0/3956205725 wait complete. 2024-09-19T04:34:52.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:52 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/3956205725' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:34:52.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:52 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/3956205725' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:34:52.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:52 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/3956205725' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:34:52.584 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T04:34:52.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:34:52.585 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T04:34:52.807 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-09-19T04:34:52.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:34:52.807 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T04:34:52.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:34:52.887 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-09-19T04:34:52.964 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:34:52.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:34:52.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:34:53.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:34:53.032 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-09-19T04:34:53.361 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T04:34:53.371 INFO:tasks.cephadm:Teardown begin 2024-09-19T04:34:53.372 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:34:53.372 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:34:53.422 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:53 smithi107 ceph-mon[30659]: pgmap v455: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-09-19T04:34:53.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:53 smithi115 ceph-mon[35477]: pgmap v455: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-09-19T04:34:53.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:53 smithi145 ceph-mon[35096]: pgmap v455: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-09-19T04:34:53.744 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:54.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.308+0000 7f1f98a00640 1 -- 172.21.15.107:0/1488978650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f94103e30 msgr2=0x7f1f94104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:54.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.308+0000 7f1f98a00640 1 --2- 172.21.15.107:0/1488978650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f94103e30 0x7f1f94104210 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009ca0 tx=0x7f1f8402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.308+0000 7f1f98a00640 1 -- 172.21.15.107:0/1488978650 shutdown_connections 2024-09-19T04:34:54.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.308+0000 7f1f98a00640 1 --2- 172.21.15.107:0/1488978650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f94108f60 0x7f1f94110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.308+0000 7f1f98a00640 1 --2- 172.21.15.107:0/1488978650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f941047e0 0x7f1f94108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.308+0000 7f1f98a00640 1 --2- 172.21.15.107:0/1488978650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f94103e30 0x7f1f94104210 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.308+0000 7f1f98a00640 1 -- 172.21.15.107:0/1488978650 >> 172.21.15.107:0/1488978650 conn(0x7f1f940ff910 msgr2=0x7f1f94101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:54.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.308+0000 7f1f98a00640 1 -- 172.21.15.107:0/1488978650 shutdown_connections 2024-09-19T04:34:54.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.308+0000 7f1f98a00640 1 -- 172.21.15.107:0/1488978650 wait complete. 2024-09-19T04:34:54.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.309+0000 7f1f98a00640 1 Processor -- start 2024-09-19T04:34:54.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.309+0000 7f1f98a00640 1 -- start start 2024-09-19T04:34:54.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f98a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f94103e30 0x7f1f9419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:54.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f98a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941047e0 0x7f1f9419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:54.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f98a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f94108f60 0x7f1f9419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:54.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f98a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94113fc0 con 0x7f1f94103e30 2024-09-19T04:34:54.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f98a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94113e40 con 0x7f1f94108f60 2024-09-19T04:34:54.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f98a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94114140 con 0x7f1f941047e0 2024-09-19T04:34:54.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f92800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f94103e30 0x7f1f9419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:54.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f92800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f94103e30 0x7f1f9419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57610/0 (socket says 172.21.15.107:57610) 2024-09-19T04:34:54.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f92800640 1 -- 172.21.15.107:0/661338312 learned_addr learned my addr 172.21.15.107:0/661338312 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:54.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f93200640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f94108f60 0x7f1f9419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:54.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.310+0000 7f1f89e00640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941047e0 0x7f1f9419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:54.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f92800640 1 -- 172.21.15.107:0/661338312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941047e0 msgr2=0x7f1f9419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:54.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f92800640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941047e0 0x7f1f9419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f92800640 1 -- 172.21.15.107:0/661338312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f94108f60 msgr2=0x7f1f9419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:54.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f92800640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f94108f60 0x7f1f9419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f92800640 1 -- 172.21.15.107:0/661338312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9419f430 con 0x7f1f94103e30 2024-09-19T04:34:54.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f92800640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f94103e30 0x7f1f9419a480 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1f8402ade0 tx=0x7f1f840319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:54.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f8be00640 1 -- 172.21.15.107:0/661338312 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8402fa20 con 0x7f1f94103e30 2024-09-19T04:34:54.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f8be00640 1 -- 172.21.15.107:0/661338312 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8402fbc0 con 0x7f1f94103e30 2024-09-19T04:34:54.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f8be00640 1 -- 172.21.15.107:0/661338312 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f840466e0 con 0x7f1f94103e30 2024-09-19T04:34:54.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f98a00640 1 -- 172.21.15.107:0/661338312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9419f6c0 con 0x7f1f94103e30 2024-09-19T04:34:54.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.311+0000 7f1f98a00640 1 -- 172.21.15.107:0/661338312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f941a7000 con 0x7f1f94103e30 2024-09-19T04:34:54.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.313+0000 7f1f98a00640 1 -- 172.21.15.107:0/661338312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f94105310 con 0x7f1f94103e30 2024-09-19T04:34:54.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.314+0000 7f1f8be00640 1 -- 172.21.15.107:0/661338312 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f1f840305c0 con 0x7f1f94103e30 2024-09-19T04:34:54.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.315+0000 7f1f8be00640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f1f780a3430 0x7f1f780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:54.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.316+0000 7f1f8be00640 1 -- 172.21.15.107:0/661338312 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f1f840f10e0 con 0x7f1f94103e30 2024-09-19T04:34:54.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.316+0000 7f1f89e00640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f1f780a3430 0x7f1f780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:54.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.316+0000 7f1f89e00640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f1f780a3430 0x7f1f780a58f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1f940ffd30 tx=0x7f1f7c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:54.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.320+0000 7f1f8be00640 1 -- 172.21.15.107:0/661338312 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1f840bdb90 con 0x7f1f94103e30 2024-09-19T04:34:54.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.531+0000 7f1f98a00640 1 -- 172.21.15.107:0/661338312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f94104210 con 0x7f1f94103e30 2024-09-19T04:34:54.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.532+0000 7f1f8be00640 1 -- 172.21.15.107:0/661338312 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+12019 (secure 0 0 0) 0x7f1f840bd490 con 0x7f1f94103e30 2024-09-19T04:34:54.534 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:34:54.534 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":57,"btime":"2024-09-19T04:34:42:046352+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:34:41.865177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:34:54.534 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 57 2024-09-19T04:34:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.534+0000 7f1f98a00640 1 -- 172.21.15.107:0/661338312 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f1f780a3430 msgr2=0x7f1f780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.534+0000 7f1f98a00640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f1f780a3430 0x7f1f780a58f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1f940ffd30 tx=0x7f1f7c005f90 comp rx=0 tx=0).stop 2024-09-19T04:34:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.534+0000 7f1f98a00640 1 -- 172.21.15.107:0/661338312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f94103e30 msgr2=0x7f1f9419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.534+0000 7f1f98a00640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f94103e30 0x7f1f9419a480 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1f8402ade0 tx=0x7f1f840319f0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.534+0000 7f1f98a00640 1 -- 172.21.15.107:0/661338312 shutdown_connections 2024-09-19T04:34:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.534+0000 7f1f98a00640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f1f780a3430 0x7f1f780a58f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.534+0000 7f1f98a00640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f94108f60 0x7f1f9419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.534+0000 7f1f98a00640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941047e0 0x7f1f9419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.534+0000 7f1f98a00640 1 --2- 172.21.15.107:0/661338312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f94103e30 0x7f1f9419a480 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.534+0000 7f1f98a00640 1 -- 172.21.15.107:0/661338312 >> 172.21.15.107:0/661338312 conn(0x7f1f940ff910 msgr2=0x7f1f94100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.535+0000 7f1f98a00640 1 -- 172.21.15.107:0/661338312 shutdown_connections 2024-09-19T04:34:54.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:54.535+0000 7f1f98a00640 1 -- 172.21.15.107:0/661338312 wait complete. 2024-09-19T04:34:55.234 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:34:55.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:55 smithi115 ceph-mon[35477]: pgmap v456: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-19T04:34:55.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:55 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/661338312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:34:55.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:55 smithi145 ceph-mon[35096]: pgmap v456: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-19T04:34:55.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:55 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/661338312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:34:55.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:55 smithi107 ceph-mon[30659]: pgmap v456: 97 pgs: 97 active+clean; 650 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-19T04:34:55.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:55 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/661338312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:34:55.605 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.166+0000 7fcb2b800640 1 -- 172.21.15.107:0/210246825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb241049a0 msgr2=0x7fcb241089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.166+0000 7fcb2b800640 1 --2- 172.21.15.107:0/210246825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb241049a0 0x7fcb241089f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009900 tx=0x7fcb1002a410 comp rx=0 tx=0).stop 2024-09-19T04:34:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.166+0000 7fcb2b800640 1 -- 172.21.15.107:0/210246825 shutdown_connections 2024-09-19T04:34:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.166+0000 7fcb2b800640 1 --2- 172.21.15.107:0/210246825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb24109120 0x7fcb24110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.166+0000 7fcb2b800640 1 --2- 172.21.15.107:0/210246825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb241049a0 0x7fcb241089f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.166+0000 7fcb2b800640 1 --2- 172.21.15.107:0/210246825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24103ff0 0x7fcb241043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.166+0000 7fcb2b800640 1 -- 172.21.15.107:0/210246825 >> 172.21.15.107:0/210246825 conn(0x7fcb240ffa90 msgr2=0x7fcb24101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:56.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.166+0000 7fcb2b800640 1 -- 172.21.15.107:0/210246825 shutdown_connections 2024-09-19T04:34:56.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.166+0000 7fcb2b800640 1 -- 172.21.15.107:0/210246825 wait complete. 2024-09-19T04:34:56.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.167+0000 7fcb2b800640 1 Processor -- start 2024-09-19T04:34:56.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.167+0000 7fcb2b800640 1 -- start start 2024-09-19T04:34:56.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.167+0000 7fcb2b800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24103ff0 0x7fcb2419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:56.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.167+0000 7fcb2b800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb241049a0 0x7fcb2419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:56.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.167+0000 7fcb2b800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb24109120 0x7fcb2419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:56.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.167+0000 7fcb2b800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24114190 con 0x7fcb24109120 2024-09-19T04:34:56.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.167+0000 7fcb2b800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24114010 con 0x7fcb24103ff0 2024-09-19T04:34:56.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.167+0000 7fcb2b800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24114310 con 0x7fcb241049a0 2024-09-19T04:34:56.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb23e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb241049a0 0x7fcb2419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb28e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24103ff0 0x7fcb2419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb29800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb24109120 0x7fcb2419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb28e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24103ff0 0x7fcb2419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:47622/0 (socket says 172.21.15.107:47622) 2024-09-19T04:34:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb23e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb241049a0 0x7fcb2419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43834/0 (socket says 172.21.15.107:43834) 2024-09-19T04:34:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb28e00640 1 -- 172.21.15.107:0/2285771551 learned_addr learned my addr 172.21.15.107:0/2285771551 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb28e00640 1 -- 172.21.15.107:0/2285771551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb241049a0 msgr2=0x7fcb2419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb28e00640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb241049a0 0x7fcb2419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb28e00640 1 -- 172.21.15.107:0/2285771551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb24109120 msgr2=0x7fcb2419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb28e00640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb24109120 0x7fcb2419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:56.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb28e00640 1 -- 172.21.15.107:0/2285771551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2419f5a0 con 0x7fcb24103ff0 2024-09-19T04:34:56.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.168+0000 7fcb28e00640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24103ff0 0x7fcb2419a5f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9f0 tx=0x7fcb1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:56.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.169+0000 7fcb21600640 1 -- 172.21.15.107:0/2285771551 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800cd80 con 0x7fcb24103ff0 2024-09-19T04:34:56.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.169+0000 7fcb2b800640 1 -- 172.21.15.107:0/2285771551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2419f890 con 0x7fcb24103ff0 2024-09-19T04:34:56.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.169+0000 7fcb2b800640 1 -- 172.21.15.107:0/2285771551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb241a7170 con 0x7fcb24103ff0 2024-09-19T04:34:56.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.169+0000 7fcb21600640 1 -- 172.21.15.107:0/2285771551 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb18010910 con 0x7fcb24103ff0 2024-09-19T04:34:56.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.169+0000 7fcb21600640 1 -- 172.21.15.107:0/2285771551 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb180188f0 con 0x7fcb24103ff0 2024-09-19T04:34:56.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.170+0000 7fcb2b800640 1 -- 172.21.15.107:0/2285771551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb241043d0 con 0x7fcb24103ff0 2024-09-19T04:34:56.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.172+0000 7fcb21600640 1 -- 172.21.15.107:0/2285771551 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fcb18010ac0 con 0x7fcb24103ff0 2024-09-19T04:34:56.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.173+0000 7fcb21600640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcb0c0a3430 0x7fcb0c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:56.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.174+0000 7fcb21600640 1 -- 172.21.15.107:0/2285771551 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fcb180d0450 con 0x7fcb24103ff0 2024-09-19T04:34:56.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.174+0000 7fcb23e00640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcb0c0a3430 0x7fcb0c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:56.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.174+0000 7fcb23e00640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcb0c0a3430 0x7fcb0c0a58f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb10000c00 tx=0x7fcb10005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:56.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.175+0000 7fcb21600640 1 -- 172.21.15.107:0/2285771551 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcb1809c7a0 con 0x7fcb24103ff0 2024-09-19T04:34:56.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.386+0000 7fcb2b800640 1 -- 172.21.15.107:0/2285771551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb241a73a0 con 0x7fcb24103ff0 2024-09-19T04:34:56.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.387+0000 7fcb21600640 1 -- 172.21.15.107:0/2285771551 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+12019 (secure 0 0 0) 0x7fcb180a0aa0 con 0x7fcb24103ff0 2024-09-19T04:34:56.389 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:34:56.390 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":57,"btime":"2024-09-19T04:34:42:046352+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3986650541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3986650541},{"type":"v1","addr":"172.21.15.107:6835","nonce":3986650541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2692745433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2692745433},{"type":"v1","addr":"172.21.15.107:6837","nonce":2692745433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:18:13.349314+0000","modified":"2024-09-19T04:34:41.865177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24517,"mds_2":24500,"mds_3":24496,"mds_4":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14499":{"gid":14499,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/2243096646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2243096646},{"type":"v1","addr":"172.21.15.107:6839","nonce":2243096646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14511":{"gid":14511,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/305932896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":305932896},{"type":"v1","addr":"172.21.15.107:6841","nonce":305932896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/1661283120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1661283120},{"type":"v1","addr":"172.21.15.115:6833","nonce":1661283120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6833/2248561847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2248561847},{"type":"v1","addr":"172.21.15.145:6833","nonce":2248561847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/687464127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":687464127},{"type":"v1","addr":"172.21.15.115:6835","nonce":687464127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.145:6835/4237714429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4237714429},{"type":"v1","addr":"172.21.15.145:6835","nonce":4237714429}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.145:6837/1756017901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1756017901},{"type":"v1","addr":"172.21.15.145:6837","nonce":1756017901}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/191835918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":191835918},{"type":"v1","addr":"172.21.15.115:6837","nonce":191835918}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.145:6839/4220252383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4220252383},{"type":"v1","addr":"172.21.15.145:6839","nonce":4220252383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2907975805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2907975805},{"type":"v1","addr":"172.21.15.115:6839","nonce":2907975805}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24517,24539,24500,24496,24490]},"id":1}]} 2024-09-19T04:34:56.390 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 57 2024-09-19T04:34:56.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.390+0000 7fcb2b800640 1 -- 172.21.15.107:0/2285771551 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcb0c0a3430 msgr2=0x7fcb0c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:56.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.390+0000 7fcb2b800640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcb0c0a3430 0x7fcb0c0a58f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb10000c00 tx=0x7fcb10005b50 comp rx=0 tx=0).stop 2024-09-19T04:34:56.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.390+0000 7fcb2b800640 1 -- 172.21.15.107:0/2285771551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24103ff0 msgr2=0x7fcb2419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:56.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.390+0000 7fcb2b800640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24103ff0 0x7fcb2419a5f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9f0 tx=0x7fcb1800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:34:56.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.391+0000 7fcb2b800640 1 -- 172.21.15.107:0/2285771551 shutdown_connections 2024-09-19T04:34:56.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.391+0000 7fcb2b800640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fcb0c0a3430 0x7fcb0c0a58f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:56.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.391+0000 7fcb2b800640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb24109120 0x7fcb2419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:56.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.391+0000 7fcb2b800640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb241049a0 0x7fcb2419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:56.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.391+0000 7fcb2b800640 1 --2- 172.21.15.107:0/2285771551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24103ff0 0x7fcb2419a5f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:56.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.391+0000 7fcb2b800640 1 -- 172.21.15.107:0/2285771551 >> 172.21.15.107:0/2285771551 conn(0x7fcb240ffa90 msgr2=0x7fcb24101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:56.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.391+0000 7fcb2b800640 1 -- 172.21.15.107:0/2285771551 shutdown_connections 2024-09-19T04:34:56.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:56.391+0000 7fcb2b800640 1 -- 172.21.15.107:0/2285771551 wait complete. 2024-09-19T04:34:57.119 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:34:57.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:57 smithi115 ceph-mon[35477]: pgmap v457: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 154 KiB/s wr, 22 op/s 2024-09-19T04:34:57.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:57 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2285771551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:34:57.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:57 smithi145 ceph-mon[35096]: pgmap v457: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 154 KiB/s wr, 22 op/s 2024-09-19T04:34:57.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:57 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2285771551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:34:57.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:57 smithi107 ceph-mon[30659]: pgmap v457: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 154 KiB/s wr, 22 op/s 2024-09-19T04:34:57.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:57 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2285771551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:34:57.489 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:58.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.073+0000 7f8e81400640 1 -- 172.21.15.107:0/2048844985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e7c103ff0 msgr2=0x7f8e7c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:58.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.073+0000 7f8e81400640 1 --2- 172.21.15.107:0/2048844985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e7c103ff0 0x7f8e7c1043d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009860 tx=0x7f8e6c02a340 comp rx=0 tx=0).stop 2024-09-19T04:34:58.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.073+0000 7f8e81400640 1 -- 172.21.15.107:0/2048844985 shutdown_connections 2024-09-19T04:34:58.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.073+0000 7f8e81400640 1 --2- 172.21.15.107:0/2048844985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e7c109120 0x7f8e7c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:58.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.073+0000 7f8e81400640 1 --2- 172.21.15.107:0/2048844985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e7c1049a0 0x7f8e7c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:58.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.073+0000 7f8e81400640 1 --2- 172.21.15.107:0/2048844985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e7c103ff0 0x7f8e7c1043d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:58.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.073+0000 7f8e81400640 1 -- 172.21.15.107:0/2048844985 >> 172.21.15.107:0/2048844985 conn(0x7f8e7c0ffa90 msgr2=0x7f8e7c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:58.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.073+0000 7f8e81400640 1 -- 172.21.15.107:0/2048844985 shutdown_connections 2024-09-19T04:34:58.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.073+0000 7f8e81400640 1 -- 172.21.15.107:0/2048844985 wait complete. 2024-09-19T04:34:58.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.074+0000 7f8e81400640 1 Processor -- start 2024-09-19T04:34:58.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e81400640 1 -- start start 2024-09-19T04:34:58.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e81400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e7c103ff0 0x7f8e7c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:58.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e81400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e7c1049a0 0x7f8e7c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:58.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e81400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e7c109120 0x7f8e7c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:58.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e81400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c114120 con 0x7f8e7c109120 2024-09-19T04:34:58.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e81400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c113fa0 con 0x7f8e7c103ff0 2024-09-19T04:34:58.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e81400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1142a0 con 0x7f8e7c1049a0 2024-09-19T04:34:58.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e79e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e7c1049a0 0x7f8e7c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:58.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e79e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e7c1049a0 0x7f8e7c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43858/0 (socket says 172.21.15.107:43858) 2024-09-19T04:34:58.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e7b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e7c109120 0x7f8e7c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:58.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e79e00640 1 -- 172.21.15.107:0/1041467483 learned_addr learned my addr 172.21.15.107:0/1041467483 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:58.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.075+0000 7f8e7a800640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e7c103ff0 0x7f8e7c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:58.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e7a800640 1 -- 172.21.15.107:0/1041467483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e7c1049a0 msgr2=0x7f8e7c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:58.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e7a800640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e7c1049a0 0x7f8e7c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:58.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e7a800640 1 -- 172.21.15.107:0/1041467483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e7c109120 msgr2=0x7f8e7c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:58.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e7a800640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e7c109120 0x7f8e7c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:58.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e7a800640 1 -- 172.21.15.107:0/1041467483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c19f5b0 con 0x7f8e7c103ff0 2024-09-19T04:34:58.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e79e00640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e7c1049a0 0x7f8e7c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:34:58.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e7a800640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e7c103ff0 0x7f8e7c19a600 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c035040 tx=0x7f8e6c033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:58.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e73400640 1 -- 172.21.15.107:0/1041467483 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c03f020 con 0x7f8e7c103ff0 2024-09-19T04:34:58.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e81400640 1 -- 172.21.15.107:0/1041467483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7c19f840 con 0x7f8e7c103ff0 2024-09-19T04:34:58.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e73400640 1 -- 172.21.15.107:0/1041467483 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c030550 con 0x7f8e7c103ff0 2024-09-19T04:34:58.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.076+0000 7f8e73400640 1 -- 172.21.15.107:0/1041467483 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c037040 con 0x7f8e7c103ff0 2024-09-19T04:34:58.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.077+0000 7f8e81400640 1 -- 172.21.15.107:0/1041467483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7c1a7180 con 0x7f8e7c103ff0 2024-09-19T04:34:58.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.078+0000 7f8e81400640 1 -- 172.21.15.107:0/1041467483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e7c1043d0 con 0x7f8e7c103ff0 2024-09-19T04:34:58.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.080+0000 7f8e73400640 1 -- 172.21.15.107:0/1041467483 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f8e6c042020 con 0x7f8e7c103ff0 2024-09-19T04:34:58.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.081+0000 7f8e73400640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e600a33b0 0x7f8e600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:58.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.081+0000 7f8e73400640 1 -- 172.21.15.107:0/1041467483 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f8e6c0effe0 con 0x7f8e7c103ff0 2024-09-19T04:34:58.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.081+0000 7f8e79e00640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e600a33b0 0x7f8e600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:58.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.082+0000 7f8e79e00640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e600a33b0 0x7f8e600a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8e680096c0 tx=0x7f8e68009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:58.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.083+0000 7f8e73400640 1 -- 172.21.15.107:0/1041467483 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8e6c049020 con 0x7f8e7c103ff0 2024-09-19T04:34:58.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.262+0000 7f8e81400640 1 -- 172.21.15.107:0/1041467483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e7c10c300 con 0x7f8e7c103ff0 2024-09-19T04:34:58.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.263+0000 7f8e73400640 1 -- 172.21.15.107:0/1041467483 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23211 (secure 0 0 0) 0x7f8e6c0bcaf0 con 0x7f8e7c103ff0 2024-09-19T04:34:58.265 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-19T04:34:58.266 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":100,"fsid":"e2ec5b0e-763c-11ef-bceb-c7b262605968","created":"2024-09-19T04:10:05.820654+0000","modified":"2024-09-19T04:34:06.674424+0000","last_up_change":"2024-09-19T04:17:17.158156+0000","last_in_change":"2024-09-19T04:16:53.976879+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:18.308041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:09.031386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:11.455271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9218bb1e-b075-4d2f-9533-da54ea586a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6803","nonce":1501799384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6805","nonce":1501799384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6809","nonce":1501799384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1501799384},{"type":"v1","addr":"172.21.15.107:6807","nonce":1501799384}]},"public_addr":"172.21.15.107:6803/1501799384","cluster_addr":"172.21.15.107:6805/1501799384","heartbeat_back_addr":"172.21.15.107:6809/1501799384","heartbeat_front_addr":"172.21.15.107:6807/1501799384","state":["exists","up"]},{"osd":1,"uuid":"4bed3c76-ab42-4406-9064-a558f56ee785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6801","nonce":3371389240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6803","nonce":3371389240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6807","nonce":3371389240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3371389240},{"type":"v1","addr":"172.21.15.115:6805","nonce":3371389240}]},"public_addr":"172.21.15.115:6801/3371389240","cluster_addr":"172.21.15.115:6803/3371389240","heartbeat_back_addr":"172.21.15.115:6807/3371389240","heartbeat_front_addr":"172.21.15.115:6805/3371389240","state":["exists","up"]},{"osd":2,"uuid":"9bfdcafa-f7f4-4f78-af4b-af1a16e24be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6801","nonce":2439686508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6803","nonce":2439686508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6807","nonce":2439686508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2439686508},{"type":"v1","addr":"172.21.15.145:6805","nonce":2439686508}]},"public_addr":"172.21.15.145:6801/2439686508","cluster_addr":"172.21.15.145:6803/2439686508","heartbeat_back_addr":"172.21.15.145:6807/2439686508","heartbeat_front_addr":"172.21.15.145:6805/2439686508","state":["exists","up"]},{"osd":3,"uuid":"7b7ec0d9-e6f1-4f7b-8f07-472ec50faa12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212660448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212660448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6817","nonce":3212660448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212660448},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212660448}]},"public_addr":"172.21.15.107:6811/3212660448","cluster_addr":"172.21.15.107:6813/3212660448","heartbeat_back_addr":"172.21.15.107:6817/3212660448","heartbeat_front_addr":"172.21.15.107:6815/3212660448","state":["exists","up"]},{"osd":4,"uuid":"86388823-3a04-4923-8ece-d0e955be08e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6809","nonce":1123876846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6811","nonce":1123876846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6815","nonce":1123876846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1123876846},{"type":"v1","addr":"172.21.15.115:6813","nonce":1123876846}]},"public_addr":"172.21.15.115:6809/1123876846","cluster_addr":"172.21.15.115:6811/1123876846","heartbeat_back_addr":"172.21.15.115:6815/1123876846","heartbeat_front_addr":"172.21.15.115:6813/1123876846","state":["exists","up"]},{"osd":5,"uuid":"47753716-2254-426c-b279-28ca4388d41f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6809","nonce":1693312722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6811","nonce":1693312722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6815","nonce":1693312722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1693312722},{"type":"v1","addr":"172.21.15.145:6813","nonce":1693312722}]},"public_addr":"172.21.15.145:6809/1693312722","cluster_addr":"172.21.15.145:6811/1693312722","heartbeat_back_addr":"172.21.15.145:6815/1693312722","heartbeat_front_addr":"172.21.15.145:6813/1693312722","state":["exists","up"]},{"osd":6,"uuid":"23e84324-77d0-460a-ae79-96ce61c8d0c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6819","nonce":2695066565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6821","nonce":2695066565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6825","nonce":2695066565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2695066565},{"type":"v1","addr":"172.21.15.107:6823","nonce":2695066565}]},"public_addr":"172.21.15.107:6819/2695066565","cluster_addr":"172.21.15.107:6821/2695066565","heartbeat_back_addr":"172.21.15.107:6825/2695066565","heartbeat_front_addr":"172.21.15.107:6823/2695066565","state":["exists","up"]},{"osd":7,"uuid":"eb0c2813-44ea-4a41-9a51-28e0e327a400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6817","nonce":278000330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6819","nonce":278000330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6823","nonce":278000330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":278000330},{"type":"v1","addr":"172.21.15.115:6821","nonce":278000330}]},"public_addr":"172.21.15.115:6817/278000330","cluster_addr":"172.21.15.115:6819/278000330","heartbeat_back_addr":"172.21.15.115:6823/278000330","heartbeat_front_addr":"172.21.15.115:6821/278000330","state":["exists","up"]},{"osd":8,"uuid":"8262ec4a-c805-449a-9e9c-b55b5b36ffed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6817","nonce":2759275478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6819","nonce":2759275478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6823","nonce":2759275478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2759275478},{"type":"v1","addr":"172.21.15.145:6821","nonce":2759275478}]},"public_addr":"172.21.15.145:6817/2759275478","cluster_addr":"172.21.15.145:6819/2759275478","heartbeat_back_addr":"172.21.15.145:6823/2759275478","heartbeat_front_addr":"172.21.15.145:6821/2759275478","state":["exists","up"]},{"osd":9,"uuid":"8277cde7-50c0-4a5b-9283-24f3813de2c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6827","nonce":3784218099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6829","nonce":3784218099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6833","nonce":3784218099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3784218099},{"type":"v1","addr":"172.21.15.107:6831","nonce":3784218099}]},"public_addr":"172.21.15.107:6827/3784218099","cluster_addr":"172.21.15.107:6829/3784218099","heartbeat_back_addr":"172.21.15.107:6833/3784218099","heartbeat_front_addr":"172.21.15.107:6831/3784218099","state":["exists","up"]},{"osd":10,"uuid":"3df95b38-68a9-426e-83e4-7283c68a2ead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6825","nonce":442507474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6827","nonce":442507474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6831","nonce":442507474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":442507474},{"type":"v1","addr":"172.21.15.115:6829","nonce":442507474}]},"public_addr":"172.21.15.115:6825/442507474","cluster_addr":"172.21.15.115:6827/442507474","heartbeat_back_addr":"172.21.15.115:6831/442507474","heartbeat_front_addr":"172.21.15.115:6829/442507474","state":["exists","up"]},{"osd":11,"uuid":"5fa0931c-684b-448d-8d63-2bd10881d88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6825","nonce":399346726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6827","nonce":399346726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6831","nonce":399346726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":399346726},{"type":"v1","addr":"172.21.15.145:6829","nonce":399346726}]},"public_addr":"172.21.15.145:6825/399346726","cluster_addr":"172.21.15.145:6827/399346726","heartbeat_back_addr":"172.21.15.145:6831/399346726","heartbeat_front_addr":"172.21.15.145:6829/399346726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:25.510803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:12:51.487777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:15.753108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:43.841673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:10.094667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:14:36.191719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:04.400338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:30.926674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:15:56.240491+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:24.543515+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:16:50.299828+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:17:15.416274+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6800/3991722475":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/3866113757":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/1307215535":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/2144535101":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1606203977":"2024-09-20T04:10:32.750566+0000","172.21.15.107:6800/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/3489252098":"2024-09-20T04:20:19.150743+0000","172.21.15.107:6801/2057620332":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1762826901":"2024-09-20T04:10:32.750566+0000","172.21.15.107:0/1481502377":"2024-09-20T04:10:59.721218+0000","172.21.15.107:0/3237606700":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6800/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/972711045":"2024-09-20T04:10:59.721218+0000","172.21.15.107:6801/3991722475":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/3889998535":"2024-09-20T04:20:19.150743+0000","172.21.15.107:0/498138874":"2024-09-20T04:10:59.721218+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:34:58.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 -- 172.21.15.107:0/1041467483 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e600a33b0 msgr2=0x7f8e600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:58.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e600a33b0 0x7f8e600a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8e680096c0 tx=0x7f8e68009260 comp rx=0 tx=0).stop 2024-09-19T04:34:58.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 -- 172.21.15.107:0/1041467483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e7c103ff0 msgr2=0x7f8e7c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:58.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e7c103ff0 0x7f8e7c19a600 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c035040 tx=0x7f8e6c033b60 comp rx=0 tx=0).stop 2024-09-19T04:34:58.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 -- 172.21.15.107:0/1041467483 shutdown_connections 2024-09-19T04:34:58.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f8e600a33b0 0x7f8e600a5870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:58.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e7c109120 0x7f8e7c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:58.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e7c1049a0 0x7f8e7c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:58.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 --2- 172.21.15.107:0/1041467483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e7c103ff0 0x7f8e7c19a600 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:58.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 -- 172.21.15.107:0/1041467483 >> 172.21.15.107:0/1041467483 conn(0x7f8e7c0ffa90 msgr2=0x7f8e7c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:58.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 -- 172.21.15.107:0/1041467483 shutdown_connections 2024-09-19T04:34:58.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:58.266+0000 7f8e81400640 1 -- 172.21.15.107:0/1041467483 wait complete. 2024-09-19T04:34:58.999 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T04:34:59.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:59 smithi107 ceph-mon[30659]: pgmap v458: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 130 KiB/s wr, 15 op/s 2024-09-19T04:34:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:34:59 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1041467483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:34:59.370 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:34:59.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:59 smithi115 ceph-mon[35477]: pgmap v458: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 130 KiB/s wr, 15 op/s 2024-09-19T04:34:59.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:34:59 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1041467483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:34:59.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:59 smithi145 ceph-mon[35096]: pgmap v458: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 130 KiB/s wr, 15 op/s 2024-09-19T04:34:59.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:34:59 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1041467483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:34:59.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.917+0000 7f09c6a00640 1 -- 172.21.15.107:0/645014381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c01079d0 msgr2=0x7f09c0107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:59.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.917+0000 7f09c6a00640 1 --2- 172.21.15.107:0/645014381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c01079d0 0x7f09c0107db0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009930 tx=0x7f09b402a420 comp rx=0 tx=0).stop 2024-09-19T04:34:59.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.917+0000 7f09c6a00640 1 -- 172.21.15.107:0/645014381 shutdown_connections 2024-09-19T04:34:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.917+0000 7f09c6a00640 1 --2- 172.21.15.107:0/645014381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09c0102370 0x7f09c010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.917+0000 7f09c6a00640 1 --2- 172.21.15.107:0/645014381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c01019d0 0x7f09c0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.917+0000 7f09c6a00640 1 --2- 172.21.15.107:0/645014381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c01079d0 0x7f09c0107db0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.917+0000 7f09c6a00640 1 -- 172.21.15.107:0/645014381 >> 172.21.15.107:0/645014381 conn(0x7f09c00fd710 msgr2=0x7f09c00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:34:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.917+0000 7f09c6a00640 1 -- 172.21.15.107:0/645014381 shutdown_connections 2024-09-19T04:34:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.917+0000 7f09c6a00640 1 -- 172.21.15.107:0/645014381 wait complete. 2024-09-19T04:34:59.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.918+0000 7f09c6a00640 1 Processor -- start 2024-09-19T04:34:59.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.918+0000 7f09c6a00640 1 -- start start 2024-09-19T04:34:59.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c01019d0 0x7f09c01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:59.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09c0102370 0x7f09c0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c01079d0 0x7f09c019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c6a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0111e20 con 0x7f09c01019d0 2024-09-19T04:34:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c6a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0111ca0 con 0x7f09c01079d0 2024-09-19T04:34:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c6a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0111fa0 con 0x7f09c0102370 2024-09-19T04:34:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c5600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c01019d0 0x7f09c01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c5600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c01019d0 0x7f09c01982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39982/0 (socket says 172.21.15.107:39982) 2024-09-19T04:34:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c5600640 1 -- 172.21.15.107:0/2905686534 learned_addr learned my addr 172.21.15.107:0/2905686534 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:34:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c4c00640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09c0102370 0x7f09c0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c6000640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c01079d0 0x7f09c019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c5600640 1 -- 172.21.15.107:0/2905686534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09c0102370 msgr2=0x7f09c0198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c5600640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09c0102370 0x7f09c0198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c5600640 1 -- 172.21.15.107:0/2905686534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c01079d0 msgr2=0x7f09c019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:34:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c5600640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c01079d0 0x7f09c019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:34:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.919+0000 7f09c5600640 1 -- 172.21.15.107:0/2905686534 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c019d270 con 0x7f09c01019d0 2024-09-19T04:34:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.920+0000 7f09c5600640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c01019d0 0x7f09c01982c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f09b402a940 tx=0x7f09b4003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:59.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.920+0000 7f09be000640 1 -- 172.21.15.107:0/2905686534 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b403f070 con 0x7f09c01019d0 2024-09-19T04:34:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.920+0000 7f09be000640 1 -- 172.21.15.107:0/2905686534 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4031c90 con 0x7f09c01019d0 2024-09-19T04:34:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.920+0000 7f09c6a00640 1 -- 172.21.15.107:0/2905686534 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c019d500 con 0x7f09c01019d0 2024-09-19T04:34:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.920+0000 7f09be000640 1 -- 172.21.15.107:0/2905686534 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4035030 con 0x7f09c01019d0 2024-09-19T04:34:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.920+0000 7f09c6a00640 1 -- 172.21.15.107:0/2905686534 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c01a4e40 con 0x7f09c01019d0 2024-09-19T04:34:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.921+0000 7f09c6a00640 1 -- 172.21.15.107:0/2905686534 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c0103110 con 0x7f09c01019d0 2024-09-19T04:34:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.923+0000 7f09be000640 1 -- 172.21.15.107:0/2905686534 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7f09b4042070 con 0x7f09c01019d0 2024-09-19T04:34:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.924+0000 7f09be000640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09a80a3430 0x7f09a80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:34:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.924+0000 7f09be000640 1 -- 172.21.15.107:0/2905686534 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f09b40f02e0 con 0x7f09c01019d0 2024-09-19T04:34:59.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.924+0000 7f09c4c00640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09a80a3430 0x7f09a80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:34:59.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.925+0000 7f09c4c00640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09a80a3430 0x7f09a80a58f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0095f0 tx=0x7f09ac005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:34:59.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:34:59.928+0000 7f09be000640 1 -- 172.21.15.107:0/2905686534 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f09b40bc630 con 0x7f09c01019d0 2024-09-19T04:35:00.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.142+0000 7f09c6a00640 1 -- 172.21.15.107:0/2905686534 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f09c0109f40 con 0x7f09c01019d0 2024-09-19T04:35:00.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.145+0000 7f09be000640 1 -- 172.21.15.107:0/2905686534 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7f09b40c0930 con 0x7f09c01019d0 2024-09-19T04:35:00.148 INFO:teuthology.orchestra.run.smithi107.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:35:00.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.149+0000 7f09c6a00640 1 -- 172.21.15.107:0/2905686534 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09a80a3430 msgr2=0x7f09a80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:00.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.149+0000 7f09c6a00640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09a80a3430 0x7f09a80a58f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0095f0 tx=0x7f09ac005f90 comp rx=0 tx=0).stop 2024-09-19T04:35:00.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.149+0000 7f09c6a00640 1 -- 172.21.15.107:0/2905686534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c01019d0 msgr2=0x7f09c01982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:00.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.149+0000 7f09c6a00640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c01019d0 0x7f09c01982c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f09b402a940 tx=0x7f09b4003b20 comp rx=0 tx=0).stop 2024-09-19T04:35:00.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.149+0000 7f09c6a00640 1 -- 172.21.15.107:0/2905686534 shutdown_connections 2024-09-19T04:35:00.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.149+0000 7f09c6a00640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7f09a80a3430 0x7f09a80a58f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:00.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.149+0000 7f09c6a00640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c01079d0 0x7f09c019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:00.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.149+0000 7f09c6a00640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09c0102370 0x7f09c0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:00.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.149+0000 7f09c6a00640 1 --2- 172.21.15.107:0/2905686534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c01019d0 0x7f09c01982c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:00.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.149+0000 7f09c6a00640 1 -- 172.21.15.107:0/2905686534 >> 172.21.15.107:0/2905686534 conn(0x7f09c00fd710 msgr2=0x7f09c00fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:35:00.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.150+0000 7f09c6a00640 1 -- 172.21.15.107:0/2905686534 shutdown_connections 2024-09-19T04:35:00.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:00.150+0000 7f09c6a00640 1 -- 172.21.15.107:0/2905686534 wait complete. 2024-09-19T04:35:00.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2905686534' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:35:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-mon[35477]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:35:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-mon[35477]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:35:00.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2905686534' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:35:00.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-mon[35477]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T04:35:00.448 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-mon[35477]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T04:35:00.448 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b[65574]: ignoring --setuser ceph since I am not root 2024-09-19T04:35:00.449 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b[65574]: ignoring --setgroup ceph since I am not root 2024-09-19T04:35:00.449 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-mds[65578]: -- 172.21.15.115:0/3226047017 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649e2759c20 con 0x5649e2737400 2024-09-19T04:35:00.449 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b[65574]: starting mds.b at 2024-09-19T04:35:00.450 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e[64955]: ignoring --setuser ceph since I am not root 2024-09-19T04:35:00.450 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e[64955]: ignoring --setgroup ceph since I am not root 2024-09-19T04:35:00.451 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-mds[64959]: -- 172.21.15.115:0/2117026160 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596962b9860 con 0x559696296800 2024-09-19T04:35:00.451 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e[64955]: starting mds.e at 2024-09-19T04:35:00.451 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k[64644]: ignoring --setuser ceph since I am not root 2024-09-19T04:35:00.452 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k[64644]: ignoring --setgroup ceph since I am not root 2024-09-19T04:35:00.452 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-mds[64648]: -- 172.21.15.115:0/880639398 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9c75b1860 con 0x55e9c758f400 2024-09-19T04:35:00.452 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k[64644]: starting mds.k at 2024-09-19T04:35:00.453 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h[65274]: ignoring --setuser ceph since I am not root 2024-09-19T04:35:00.453 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h[65274]: ignoring --setgroup ceph since I am not root 2024-09-19T04:35:00.453 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-mds[65278]: -- 172.21.15.115:0/2323274831 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b52d0ff860 con 0x55b52d0dd400 2024-09-19T04:35:00.453 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:35:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h[65274]: starting mds.h at 2024-09-19T04:35:00.474 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f[64878]: ignoring --setuser ceph since I am not root 2024-09-19T04:35:00.475 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f[64878]: ignoring --setgroup ceph since I am not root 2024-09-19T04:35:00.475 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mds[64882]: -- 172.21.15.145:0/3284452727 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed9e245860 con 0x55ed9e223000 2024-09-19T04:35:00.475 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f[64878]: starting mds.f at 2024-09-19T04:35:00.475 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c[65180]: ignoring --setuser ceph since I am not root 2024-09-19T04:35:00.475 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c[65180]: ignoring --setgroup ceph since I am not root 2024-09-19T04:35:00.476 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mds[65185]: -- 172.21.15.145:0/2319193179 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5630bcf47860 con 0x5630bcf25000 2024-09-19T04:35:00.476 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c[65180]: starting mds.c at 2024-09-19T04:35:00.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2905686534' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:35:00.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mon[35096]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:35:00.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mon[35096]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:35:00.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2905686534' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:35:00.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mon[35096]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T04:35:00.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mon[35096]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T04:35:00.477 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i[65493]: ignoring --setuser ceph since I am not root 2024-09-19T04:35:00.477 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i[65493]: ignoring --setgroup ceph since I am not root 2024-09-19T04:35:00.477 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mds[65497]: -- 172.21.15.145:0/2932460143 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cd7733e680 con 0x55cd77312800 2024-09-19T04:35:00.477 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mds[65497]: -- 172.21.15.145:0/2932460143 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cd77335860 con 0x55cd77312800 2024-09-19T04:35:00.478 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i[65493]: starting mds.i at 2024-09-19T04:35:00.478 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l[64577]: ignoring --setuser ceph since I am not root 2024-09-19T04:35:00.478 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l[64577]: ignoring --setgroup ceph since I am not root 2024-09-19T04:35:00.478 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-mds[64581]: -- 172.21.15.145:0/1443318276 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2bd11d860 con 0x55a2bd0fa800 2024-09-19T04:35:00.478 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l[64577]: starting mds.l at 2024-09-19T04:35:00.478 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:35:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l[64577]: 2024-09-19T04:35:00.202+0000 7f196108d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-19T04:35:00.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a[30655]: 2024-09-19T04:35:00.143+0000 7f143da00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:35:00.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2905686534' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:35:00.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-mon[30659]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:35:00.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-mon[30659]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:35:00.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2905686534' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:35:00.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-mon[30659]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T04:35:00.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-mon[30659]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T04:35:00.482 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a[71817]: ignoring --setuser ceph since I am not root 2024-09-19T04:35:00.482 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a[71817]: ignoring --setgroup ceph since I am not root 2024-09-19T04:35:00.482 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-mds[71839]: -- 172.21.15.107:0/2275868602 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f6d0659860 con 0x55f6d0637000 2024-09-19T04:35:00.482 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a[71817]: starting mds.a at 2024-09-19T04:35:00.482 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g[71463]: ignoring --setuser ceph since I am not root 2024-09-19T04:35:00.483 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g[71463]: ignoring --setgroup ceph since I am not root 2024-09-19T04:35:00.483 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-mds[71467]: -- 172.21.15.107:0/2036469959 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c64cf3860 con 0x563c64cd1400 2024-09-19T04:35:00.483 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:35:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g[71463]: starting mds.g at 2024-09-19T04:35:00.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:35:01.292 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:35:01.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:01 smithi115 ceph-mon[35477]: pgmap v459: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 130 KiB/s wr, 15 op/s 2024-09-19T04:35:01.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:01 smithi145 ceph-mon[35096]: pgmap v459: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 130 KiB/s wr, 15 op/s 2024-09-19T04:35:01.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:01 smithi107 ceph-mon[30659]: pgmap v459: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 130 KiB/s wr, 15 op/s 2024-09-19T04:35:01.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.902+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1806953433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e01047e0 msgr2=0x7fb9e0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:01.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.902+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/1806953433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e01047e0 0x7fb9e0108830 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009900 tx=0x7fb9cc02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:35:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.902+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1806953433 shutdown_connections 2024-09-19T04:35:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.902+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/1806953433 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9e0108f60 0x7fb9e0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.902+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/1806953433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e01047e0 0x7fb9e0108830 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.902+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/1806953433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e0103e30 0x7fb9e0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.902+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1806953433 >> 172.21.15.107:0/1806953433 conn(0x7fb9e00ff910 msgr2=0x7fb9e0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:35:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.902+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1806953433 shutdown_connections 2024-09-19T04:35:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.903+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1806953433 wait complete. 2024-09-19T04:35:01.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.904+0000 7fb9e4e00640 1 Processor -- start 2024-09-19T04:35:01.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.904+0000 7fb9e4e00640 1 -- start start 2024-09-19T04:35:01.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.904+0000 7fb9e4e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e0103e30 0x7fb9e019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:01.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.904+0000 7fb9e4e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9e01047e0 0x7fb9e019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:01.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.904+0000 7fb9e4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0108f60 0x7fb9e019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:01.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.904+0000 7fb9e4e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0114010 con 0x7fb9e0108f60 2024-09-19T04:35:01.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.904+0000 7fb9e4e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0113e90 con 0x7fb9e0103e30 2024-09-19T04:35:01.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.904+0000 7fb9e4e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0114190 con 0x7fb9e01047e0 2024-09-19T04:35:01.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0108f60 0x7fb9e019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9df400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e0103e30 0x7fb9e019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0108f60 0x7fb9e019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40028/0 (socket says 172.21.15.107:40028) 2024-09-19T04:35:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9dfe00640 1 -- 172.21.15.107:0/1431449084 learned_addr learned my addr 172.21.15.107:0/1431449084 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:35:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9dea00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9e01047e0 0x7fb9e019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9dfe00640 1 -- 172.21.15.107:0/1431449084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9e01047e0 msgr2=0x7fb9e019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9dfe00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9e01047e0 0x7fb9e019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9dfe00640 1 -- 172.21.15.107:0/1431449084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e0103e30 msgr2=0x7fb9e019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9dfe00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e0103e30 0x7fb9e019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9dfe00640 1 -- 172.21.15.107:0/1431449084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e019f420 con 0x7fb9e0108f60 2024-09-19T04:35:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9df400640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e0103e30 0x7fb9e019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:35:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.905+0000 7fb9dfe00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0108f60 0x7fb9e019ed40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800eb10 tx=0x7fb9d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:35:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.906+0000 7fb9d7e00640 1 -- 172.21.15.107:0/1431449084 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8004b20 con 0x7fb9e0108f60 2024-09-19T04:35:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.906+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1431449084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e019f710 con 0x7fb9e0108f60 2024-09-19T04:35:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.906+0000 7fb9d7e00640 1 -- 172.21.15.107:0/1431449084 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d8004cc0 con 0x7fb9e0108f60 2024-09-19T04:35:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.906+0000 7fb9d7e00640 1 -- 172.21.15.107:0/1431449084 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8010760 con 0x7fb9e0108f60 2024-09-19T04:35:01.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.906+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1431449084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e01a6ff0 con 0x7fb9e0108f60 2024-09-19T04:35:01.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.907+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1431449084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e0105310 con 0x7fb9e0108f60 2024-09-19T04:35:01.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.909+0000 7fb9d7e00640 1 -- 172.21.15.107:0/1431449084 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fb9d8005020 con 0x7fb9e0108f60 2024-09-19T04:35:01.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.910+0000 7fb9d7e00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9c40a33b0 0x7fb9c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:01.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.910+0000 7fb9d7e00640 1 -- 172.21.15.107:0/1431449084 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8870+0+0 (secure 0 0 0) 0x7fb9d80d05c0 con 0x7fb9e0108f60 2024-09-19T04:35:01.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.910+0000 7fb9df400640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9c40a33b0 0x7fb9c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:01.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.911+0000 7fb9df400640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9c40a33b0 0x7fb9c40a5870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009fc0 tx=0x7fb9d0005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:35:01.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:01.914+0000 7fb9d7e00640 1 -- 172.21.15.107:0/1431449084 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb9d809cde0 con 0x7fb9e0108f60 2024-09-19T04:35:02.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.125+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1431449084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb9e002d7c0 con 0x7fb9e0108f60 2024-09-19T04:35:02.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.199+0000 7fb9d7e00640 1 -- 172.21.15.107:0/1431449084 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7fb9d809c6e0 con 0x7fb9e0108f60 2024-09-19T04:35:02.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.202+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1431449084 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9c40a33b0 msgr2=0x7fb9c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:02.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.202+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9c40a33b0 0x7fb9c40a5870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009fc0 tx=0x7fb9d0005f90 comp rx=0 tx=0).stop 2024-09-19T04:35:02.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.202+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1431449084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0108f60 msgr2=0x7fb9e019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:02.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.202+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0108f60 0x7fb9e019ed40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800eb10 tx=0x7fb9d800c2d0 comp rx=0 tx=0).stop 2024-09-19T04:35:02.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.202+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1431449084 shutdown_connections 2024-09-19T04:35:02.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.202+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fb9c40a33b0 0x7fb9c40a5870 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:02.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.202+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0108f60 0x7fb9e019ed40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:02.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.202+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9e01047e0 0x7fb9e019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:02.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.202+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/1431449084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e0103e30 0x7fb9e019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:02.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.202+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1431449084 >> 172.21.15.107:0/1431449084 conn(0x7fb9e00ff910 msgr2=0x7fb9e0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:35:02.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.203+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1431449084 shutdown_connections 2024-09-19T04:35:02.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:02.203+0000 7fb9e4e00640 1 -- 172.21.15.107:0/1431449084 wait complete. 2024-09-19T04:35:02.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.115:6832/1533637756,v1:172.21.15.115:6833/1533637756] up:boot 2024-09-19T04:35:02.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.115:6834/3595294678,v1:172.21.15.115:6835/3595294678] up:boot 2024-09-19T04:35:02.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6833/2601364147,v1:172.21.15.145:6835/2601364147] up:boot 2024-09-19T04:35:02.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6832/3452658980,v1:172.21.15.145:6834/3452658980] up:boot 2024-09-19T04:35:02.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6836/1282334866,v1:172.21.15.145:6838/1282334866] up:boot 2024-09-19T04:35:02.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.145:6837/1567562681,v1:172.21.15.145:6839/1567562681] up:boot 2024-09-19T04:35:02.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.115:6836/3188917855,v1:172.21.15.115:6838/3188917855] up:boot 2024-09-19T04:35:02.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.115:6837/833296954,v1:172.21.15.115:6839/833296954] up:boot 2024-09-19T04:35:02.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.107:6838/2050090726,v1:172.21.15.107:6839/2050090726] up:boot 2024-09-19T04:35:02.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: mds.? [v2:172.21.15.107:6840/2144332403,v1:172.21.15.107:6841/2144332403] up:boot 2024-09-19T04:35:02.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T04:35:02.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:35:02.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:35:02.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:35:02.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:35:02.476 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:35:02.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:35:02.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:35:02.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:35:02.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:35:02.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:35:02.477 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:02 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1431449084' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:35:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.115:6832/1533637756,v1:172.21.15.115:6833/1533637756] up:boot 2024-09-19T04:35:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.115:6834/3595294678,v1:172.21.15.115:6835/3595294678] up:boot 2024-09-19T04:35:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6833/2601364147,v1:172.21.15.145:6835/2601364147] up:boot 2024-09-19T04:35:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6832/3452658980,v1:172.21.15.145:6834/3452658980] up:boot 2024-09-19T04:35:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6836/1282334866,v1:172.21.15.145:6838/1282334866] up:boot 2024-09-19T04:35:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.145:6837/1567562681,v1:172.21.15.145:6839/1567562681] up:boot 2024-09-19T04:35:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.115:6836/3188917855,v1:172.21.15.115:6838/3188917855] up:boot 2024-09-19T04:35:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.115:6837/833296954,v1:172.21.15.115:6839/833296954] up:boot 2024-09-19T04:35:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.107:6838/2050090726,v1:172.21.15.107:6839/2050090726] up:boot 2024-09-19T04:35:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: mds.? [v2:172.21.15.107:6840/2144332403,v1:172.21.15.107:6841/2144332403] up:boot 2024-09-19T04:35:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T04:35:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:35:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:35:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:35:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:35:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:35:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:35:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:35:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:35:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:35:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:35:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:02 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1431449084' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:35:02.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.115:6832/1533637756,v1:172.21.15.115:6833/1533637756] up:boot 2024-09-19T04:35:02.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.115:6834/3595294678,v1:172.21.15.115:6835/3595294678] up:boot 2024-09-19T04:35:02.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6833/2601364147,v1:172.21.15.145:6835/2601364147] up:boot 2024-09-19T04:35:02.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6832/3452658980,v1:172.21.15.145:6834/3452658980] up:boot 2024-09-19T04:35:02.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6836/1282334866,v1:172.21.15.145:6838/1282334866] up:boot 2024-09-19T04:35:02.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.145:6837/1567562681,v1:172.21.15.145:6839/1567562681] up:boot 2024-09-19T04:35:02.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.115:6836/3188917855,v1:172.21.15.115:6838/3188917855] up:boot 2024-09-19T04:35:02.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.115:6837/833296954,v1:172.21.15.115:6839/833296954] up:boot 2024-09-19T04:35:02.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.107:6838/2050090726,v1:172.21.15.107:6839/2050090726] up:boot 2024-09-19T04:35:02.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: mds.? [v2:172.21.15.107:6840/2144332403,v1:172.21.15.107:6841/2144332403] up:boot 2024-09-19T04:35:02.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T04:35:02.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:35:02.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:35:02.698 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:35:02.699 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:35:02.699 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:35:02.699 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:35:02.699 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:35:02.699 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:35:02.699 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:35:02.699 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='mgr.15114 172.21.15.107:0/2923682089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:35:02.699 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:02 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1431449084' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:35:02.968 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:35:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:35:03.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:03 smithi145 ceph-mon[35096]: pgmap v461: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 164 KiB/s wr, 19 op/s 2024-09-19T04:35:03.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:03 smithi145 ceph-mon[35096]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:35:03.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:03 smithi145 ceph-mon[35096]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:35:03.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:03 smithi145 ceph-mon[35096]: Cluster is now healthy 2024-09-19T04:35:03.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:03 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/1431449084' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:35:03.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:03 smithi145 ceph-mon[35096]: fsmap 12 up:standby 2024-09-19T04:35:03.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:03 smithi107 ceph-mon[30659]: pgmap v461: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 164 KiB/s wr, 19 op/s 2024-09-19T04:35:03.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:03 smithi107 ceph-mon[30659]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:35:03.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:03 smithi107 ceph-mon[30659]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:35:03.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:03 smithi107 ceph-mon[30659]: Cluster is now healthy 2024-09-19T04:35:03.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:03 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/1431449084' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:35:03.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:03 smithi107 ceph-mon[30659]: fsmap 12 up:standby 2024-09-19T04:35:03.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:03 smithi115 ceph-mon[35477]: pgmap v461: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 164 KiB/s wr, 19 op/s 2024-09-19T04:35:03.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:03 smithi115 ceph-mon[35477]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:35:03.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:03 smithi115 ceph-mon[35477]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:35:03.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:03 smithi115 ceph-mon[35477]: Cluster is now healthy 2024-09-19T04:35:03.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:03 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/1431449084' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:35:03.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:03 smithi115 ceph-mon[35477]: fsmap 12 up:standby 2024-09-19T04:35:03.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.916+0000 7fc81d600640 1 -- 172.21.15.107:0/4211169434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8181047e0 msgr2=0x7fc818108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:03.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.916+0000 7fc81d600640 1 --2- 172.21.15.107:0/4211169434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8181047e0 0x7fc818108830 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc804009900 tx=0x7fc80402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:35:03.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.916+0000 7fc81d600640 1 -- 172.21.15.107:0/4211169434 shutdown_connections 2024-09-19T04:35:03.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.916+0000 7fc81d600640 1 --2- 172.21.15.107:0/4211169434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc818108f60 0x7fc818110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:03.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.916+0000 7fc81d600640 1 --2- 172.21.15.107:0/4211169434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8181047e0 0x7fc818108830 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:03.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.916+0000 7fc81d600640 1 --2- 172.21.15.107:0/4211169434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc818103e30 0x7fc818104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:03.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.916+0000 7fc81d600640 1 -- 172.21.15.107:0/4211169434 >> 172.21.15.107:0/4211169434 conn(0x7fc8180ff8d0 msgr2=0x7fc818101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:35:03.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.916+0000 7fc81d600640 1 -- 172.21.15.107:0/4211169434 shutdown_connections 2024-09-19T04:35:03.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.916+0000 7fc81d600640 1 -- 172.21.15.107:0/4211169434 wait complete. 2024-09-19T04:35:03.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.917+0000 7fc81d600640 1 Processor -- start 2024-09-19T04:35:03.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.917+0000 7fc81d600640 1 -- start start 2024-09-19T04:35:03.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc81d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc818103e30 0x7fc81819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:03.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc81d600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8181047e0 0x7fc81819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:03.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc81d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc818108f60 0x7fc81819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:03.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc81d600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc818113fc0 con 0x7fc818103e30 2024-09-19T04:35:03.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc81d600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc818113e40 con 0x7fc8181047e0 2024-09-19T04:35:03.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc81d600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc818114140 con 0x7fc818108f60 2024-09-19T04:35:03.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc817e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc818103e30 0x7fc81819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:03.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc817400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8181047e0 0x7fc81819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:03.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc817400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8181047e0 0x7fc81819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.107:59996/0 (socket says 172.21.15.107:59996) 2024-09-19T04:35:03.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc817e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc818103e30 0x7fc81819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40048/0 (socket says 172.21.15.107:40048) 2024-09-19T04:35:03.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc817400640 1 -- 172.21.15.107:0/2426926150 learned_addr learned my addr 172.21.15.107:0/2426926150 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:35:03.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc81cc00640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc818108f60 0x7fc81819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:03.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc817400640 1 -- 172.21.15.107:0/2426926150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc818108f60 msgr2=0x7fc81819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:03.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc817400640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc818108f60 0x7fc81819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:03.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc817400640 1 -- 172.21.15.107:0/2426926150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc818103e30 msgr2=0x7fc81819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:03.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc817400640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc818103e30 0x7fc81819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:03.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.918+0000 7fc817400640 1 -- 172.21.15.107:0/2426926150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81819f470 con 0x7fc8181047e0 2024-09-19T04:35:03.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.919+0000 7fc817e00640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc818103e30 0x7fc81819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:35:03.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.919+0000 7fc817400640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8181047e0 0x7fc81819aa00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc804006010 tx=0x7fc8040042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:35:03.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.919+0000 7fc814c00640 1 -- 172.21.15.107:0/2426926150 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80403f030 con 0x7fc8181047e0 2024-09-19T04:35:03.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.919+0000 7fc814c00640 1 -- 172.21.15.107:0/2426926150 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc804003ce0 con 0x7fc8181047e0 2024-09-19T04:35:03.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.919+0000 7fc81d600640 1 -- 172.21.15.107:0/2426926150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81819f700 con 0x7fc8181047e0 2024-09-19T04:35:03.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.919+0000 7fc814c00640 1 -- 172.21.15.107:0/2426926150 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804035070 con 0x7fc8181047e0 2024-09-19T04:35:03.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.919+0000 7fc81d600640 1 -- 172.21.15.107:0/2426926150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8181a7040 con 0x7fc8181047e0 2024-09-19T04:35:03.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.920+0000 7fc81d600640 1 -- 172.21.15.107:0/2426926150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc818104210 con 0x7fc8181047e0 2024-09-19T04:35:03.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.923+0000 7fc814c00640 1 -- 172.21.15.107:0/2426926150 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7fc804042030 con 0x7fc8181047e0 2024-09-19T04:35:03.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.924+0000 7fc814c00640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc7fc0a3430 0x7fc7fc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:03.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.925+0000 7fc814c00640 1 -- 172.21.15.107:0/2426926150 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8870+0+0 (secure 0 0 0) 0x7fc8040f0700 con 0x7fc8181047e0 2024-09-19T04:35:03.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.925+0000 7fc817e00640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc7fc0a3430 0x7fc7fc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:03.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.925+0000 7fc817e00640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc7fc0a3430 0x7fc7fc0a58f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc80800bfd0 tx=0x7fc80800b390 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:35:03.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:03.926+0000 7fc814c00640 1 -- 172.21.15.107:0/2426926150 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc8040bc990 con 0x7fc8181047e0 2024-09-19T04:35:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.102+0000 7fc81d600640 1 -- 172.21.15.107:0/2426926150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc81810c1d0 con 0x7fc8181047e0 2024-09-19T04:35:04.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.211+0000 7fc814c00640 1 -- 172.21.15.107:0/2426926150 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7fc8040c0c90 con 0x7fc8181047e0 2024-09-19T04:35:04.214 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:35:04.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.214+0000 7fc81d600640 1 -- 172.21.15.107:0/2426926150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc7fc0a3430 msgr2=0x7fc7fc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:04.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc7fc0a3430 0x7fc7fc0a58f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc80800bfd0 tx=0x7fc80800b390 comp rx=0 tx=0).stop 2024-09-19T04:35:04.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 -- 172.21.15.107:0/2426926150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8181047e0 msgr2=0x7fc81819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:04.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8181047e0 0x7fc81819aa00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc804006010 tx=0x7fc8040042f0 comp rx=0 tx=0).stop 2024-09-19T04:35:04.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 -- 172.21.15.107:0/2426926150 shutdown_connections 2024-09-19T04:35:04.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7fc7fc0a3430 0x7fc7fc0a58f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:04.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc818108f60 0x7fc81819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:04.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8181047e0 0x7fc81819aa00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:04.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 --2- 172.21.15.107:0/2426926150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc818103e30 0x7fc81819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:04.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 -- 172.21.15.107:0/2426926150 >> 172.21.15.107:0/2426926150 conn(0x7fc8180ff8d0 msgr2=0x7fc818100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:35:04.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 -- 172.21.15.107:0/2426926150 shutdown_connections 2024-09-19T04:35:04.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:04.215+0000 7fc81d600640 1 -- 172.21.15.107:0/2426926150 wait complete. 2024-09-19T04:35:04.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:04 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/2426926150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:04.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:04 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:04.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:04 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/2426926150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:04.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:04 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:04.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:04 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/2426926150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:04.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:04 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:05.087 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:35:05.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:05 smithi145 ceph-mon[35096]: pgmap v462: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 164 KiB/s wr, 19 op/s 2024-09-19T04:35:05.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:05 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:35:05.475 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:05 smithi145 ceph-mon[35096]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T04:35:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:05 smithi107 ceph-mon[30659]: pgmap v462: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 164 KiB/s wr, 19 op/s 2024-09-19T04:35:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:05 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:35:05.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:05 smithi107 ceph-mon[30659]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T04:35:05.493 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:35:05.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:05 smithi115 ceph-mon[35477]: pgmap v462: 97 pgs: 97 active+clean; 651 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 164 KiB/s wr, 19 op/s 2024-09-19T04:35:05.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:05 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:35:05.697 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:05 smithi115 ceph-mon[35477]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T04:35:06.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.065+0000 7ff681200640 1 -- 172.21.15.107:0/1854081293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff67c10e420 msgr2=0x7ff67c110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.065+0000 7ff681200640 1 --2- 172.21.15.107:0/1854081293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff67c10e420 0x7ff67c110810 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff674009ca0 tx=0x7ff67402a950 comp rx=0 tx=0).stop 2024-09-19T04:35:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.065+0000 7ff681200640 1 -- 172.21.15.107:0/1854081293 shutdown_connections 2024-09-19T04:35:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.065+0000 7ff681200640 1 --2- 172.21.15.107:0/1854081293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff67c10e420 0x7ff67c110810 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.065+0000 7ff681200640 1 --2- 172.21.15.107:0/1854081293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c074ee0 0x7ff67c10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.065+0000 7ff681200640 1 --2- 172.21.15.107:0/1854081293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff67c0745c0 0x7ff67c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.065+0000 7ff681200640 1 -- 172.21.15.107:0/1854081293 >> 172.21.15.107:0/1854081293 conn(0x7ff67c0fd8c0 msgr2=0x7ff67c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:35:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.065+0000 7ff681200640 1 -- 172.21.15.107:0/1854081293 shutdown_connections 2024-09-19T04:35:06.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.065+0000 7ff681200640 1 -- 172.21.15.107:0/1854081293 wait complete. 2024-09-19T04:35:06.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.066+0000 7ff681200640 1 Processor -- start 2024-09-19T04:35:06.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.066+0000 7ff681200640 1 -- start start 2024-09-19T04:35:06.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff681200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c0745c0 0x7ff67c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:06.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff681200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff67c074ee0 0x7ff67c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:06.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff681200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff67c10e420 0x7ff67c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:06.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff681200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c1141c0 con 0x7ff67c074ee0 2024-09-19T04:35:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff681200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c114040 con 0x7ff67c10e420 2024-09-19T04:35:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff681200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c114340 con 0x7ff67c0745c0 2024-09-19T04:35:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff671400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff67c074ee0 0x7ff67c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff67a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff67c10e420 0x7ff67c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff671400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff67c074ee0 0x7ff67c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40072/0 (socket says 172.21.15.107:40072) 2024-09-19T04:35:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff671400640 1 -- 172.21.15.107:0/942678702 learned_addr learned my addr 172.21.15.107:0/942678702 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-19T04:35:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.067+0000 7ff679e00640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c0745c0 0x7ff67c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff67a800640 1 -- 172.21.15.107:0/942678702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c0745c0 msgr2=0x7ff67c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff67a800640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c0745c0 0x7ff67c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff67a800640 1 -- 172.21.15.107:0/942678702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff67c074ee0 msgr2=0x7ff67c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff67a800640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff67c074ee0 0x7ff67c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff67a800640 1 -- 172.21.15.107:0/942678702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c19f6b0 con 0x7ff67c10e420 2024-09-19T04:35:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff67a800640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff67c10e420 0x7ff67c19efd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff67402af40 tx=0x7ff674003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:35:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff673400640 1 -- 172.21.15.107:0/942678702 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674034040 con 0x7ff67c10e420 2024-09-19T04:35:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff673400640 1 -- 172.21.15.107:0/942678702 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff67402f940 con 0x7ff67c10e420 2024-09-19T04:35:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff681200640 1 -- 172.21.15.107:0/942678702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67c19f940 con 0x7ff67c10e420 2024-09-19T04:35:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff673400640 1 -- 172.21.15.107:0/942678702 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6740045a0 con 0x7ff67c10e420 2024-09-19T04:35:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.068+0000 7ff681200640 1 -- 172.21.15.107:0/942678702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67c19fdf0 con 0x7ff67c10e420 2024-09-19T04:35:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.069+0000 7ff681200640 1 -- 172.21.15.107:0/942678702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff67c0749a0 con 0x7ff67c10e420 2024-09-19T04:35:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.072+0000 7ff673400640 1 -- 172.21.15.107:0/942678702 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) ==== 138487+0+0 (secure 0 0 0) 0x7ff674037020 con 0x7ff67c10e420 2024-09-19T04:35:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.073+0000 7ff673400640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff6600a33b0 0x7ff6600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:35:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.073+0000 7ff673400640 1 -- 172.21.15.107:0/942678702 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8345+0+0 (secure 0 0 0) 0x7ff6740f0a00 con 0x7ff67c10e420 2024-09-19T04:35:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.074+0000 7ff679e00640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff6600a33b0 0x7ff6600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:35:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.074+0000 7ff679e00640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff6600a33b0 0x7ff6600a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0095f0 tx=0x7ff66c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:35:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.077+0000 7ff673400640 1 -- 172.21.15.107:0/942678702 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff674081210 con 0x7ff67c10e420 2024-09-19T04:35:06.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:06.255+0000 7ff681200640 1 -- 172.21.15.107:0/942678702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff67c02d7c0 con 0x7ff67c10e420 2024-09-19T04:35:07.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.226+0000 7ff673400640 1 -- 172.21.15.107:0/942678702 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7ff6740bd540 con 0x7ff67c10e420 2024-09-19T04:35:07.228 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' removed 2024-09-19T04:35:07.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.230+0000 7ff681200640 1 -- 172.21.15.107:0/942678702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff6600a33b0 msgr2=0x7ff6600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:07.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.230+0000 7ff681200640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff6600a33b0 0x7ff6600a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0095f0 tx=0x7ff66c005f90 comp rx=0 tx=0).stop 2024-09-19T04:35:07.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.230+0000 7ff681200640 1 -- 172.21.15.107:0/942678702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff67c10e420 msgr2=0x7ff67c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:35:07.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.230+0000 7ff681200640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff67c10e420 0x7ff67c19efd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff67402af40 tx=0x7ff674003b80 comp rx=0 tx=0).stop 2024-09-19T04:35:07.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.230+0000 7ff681200640 1 -- 172.21.15.107:0/942678702 shutdown_connections 2024-09-19T04:35:07.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.230+0000 7ff681200640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.107:6800/1724794486,v1:172.21.15.107:6801/1724794486] conn(0x7ff6600a33b0 0x7ff6600a5870 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:07.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.230+0000 7ff681200640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff67c10e420 0x7ff67c19efd0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:07.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.230+0000 7ff681200640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff67c074ee0 0x7ff67c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:07.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.230+0000 7ff681200640 1 --2- 172.21.15.107:0/942678702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c0745c0 0x7ff67c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:35:07.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.230+0000 7ff681200640 1 -- 172.21.15.107:0/942678702 >> 172.21.15.107:0/942678702 conn(0x7ff67c0fd8c0 msgr2=0x7ff67c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:35:07.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.231+0000 7ff681200640 1 -- 172.21.15.107:0/942678702 shutdown_connections 2024-09-19T04:35:07.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-19T04:35:07.231+0000 7ff681200640 1 -- 172.21.15.107:0/942678702 wait complete. 2024-09-19T04:35:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:07 smithi115 ceph-mon[35477]: pgmap v464: 65 pgs: 65 active+clean; 238 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:35:07.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:07 smithi115 ceph-mon[35477]: from='client.? 172.21.15.107:0/942678702' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:07.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:07 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:07 smithi145 ceph-mon[35096]: pgmap v464: 65 pgs: 65 active+clean; 238 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:35:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:07 smithi145 ceph-mon[35096]: from='client.? 172.21.15.107:0/942678702' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:07.474 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:07 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:07.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:07 smithi107 ceph-mon[30659]: pgmap v464: 65 pgs: 65 active+clean; 238 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:35:07.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:07 smithi107 ceph-mon[30659]: from='client.? 172.21.15.107:0/942678702' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:07.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:07 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:35:08.087 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:35:08.146 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:35:08.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:35:08.229 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T04:35:08.229 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:35:08.267 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:35:08.283 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:35:08.300 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T04:35:08.300 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T04:35:08.300 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.a 2024-09-19T04:35:08.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:08 smithi107 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:35:08.355 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:08 smithi107 ceph-mon[30659]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T04:35:08.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:08 smithi115 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:35:08.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:08 smithi115 ceph-mon[35477]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T04:35:08.692 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:08 smithi107 systemd[1]: Stopping Ceph mon.a for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:08.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:08 smithi145 ceph-mon[35096]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:35:08.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:08 smithi145 ceph-mon[35096]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T04:35:08.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:08 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a[30655]: 2024-09-19T04:35:08.689+0000 7f1444800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:08.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:08 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a[30655]: 2024-09-19T04:35:08.689+0000 7f1444800640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T04:35:08.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:08 smithi107 podman[111151]: 2024-09-19 04:35:08.721121583 +0000 UTC m=+0.134608124 container died 7e651d195b90d496def2b6acae0421e01f804fabfee60a21c1461bcd30457811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T04:35:09.269 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:09 smithi107 podman[111151]: 2024-09-19 04:35:09.245277259 +0000 UTC m=+0.658763799 container cleanup 7e651d195b90d496def2b6acae0421e01f804fabfee60a21c1461bcd30457811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T04:35:09.269 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:09 smithi107 bash[111151]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a 2024-09-19T04:35:09.773 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 19 04:35:09 smithi107 podman[111163]: 2024-09-19 04:35:09.513701292 +0000 UTC m=+0.783652215 container remove 7e651d195b90d496def2b6acae0421e01f804fabfee60a21c1461bcd30457811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:09.802 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T04:35:09.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:09.889 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T04:35:09.889 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T04:35:09.889 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.b 2024-09-19T04:35:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:09 smithi115 systemd[1]: Stopping Ceph mon.b for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:10.630 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:10 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-b[35473]: 2024-09-19T04:35:10.236+0000 7f4cce569640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:10.630 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:10 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-b[35473]: 2024-09-19T04:35:10.236+0000 7f4cce569640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T04:35:10.630 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:10 smithi115 podman[72394]: 2024-09-19 04:35:10.329006079 +0000 UTC m=+0.246283253 container died 96e16a26aaa9b97091cd71e657dc916ae9ba18f10c99991a503dc3337523f7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T04:35:10.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:10 smithi115 podman[72394]: 2024-09-19 04:35:10.728224623 +0000 UTC m=+0.645501804 container cleanup 96e16a26aaa9b97091cd71e657dc916ae9ba18f10c99991a503dc3337523f7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-09-19T04:35:10.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:10 smithi115 bash[72394]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-b 2024-09-19T04:35:11.194 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T04:35:11.216 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:10 smithi115 podman[72408]: 2024-09-19 04:35:10.954802285 +0000 UTC m=+0.626132315 container remove 96e16a26aaa9b97091cd71e657dc916ae9ba18f10c99991a503dc3337523f7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T04:35:11.216 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:11 smithi115 systemd[1]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-19T04:35:11.216 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:11 smithi115 systemd[1]: Stopped Ceph mon.b for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:35:11.216 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 19 04:35:11 smithi115 systemd[1]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.b.service: Consumed 14.385s CPU time. 2024-09-19T04:35:11.286 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:11.287 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T04:35:11.287 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T04:35:11.287 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.c 2024-09-19T04:35:11.607 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:11 smithi145 systemd[1]: Stopping Ceph mon.c for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:11.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:11 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-c[35092]: 2024-09-19T04:35:11.604+0000 7f9221d64640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:11.974 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:11 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-c[35092]: 2024-09-19T04:35:11.604+0000 7f9221d64640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T04:35:11.975 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:11 smithi145 podman[71772]: 2024-09-19 04:35:11.687230344 +0000 UTC m=+0.199613398 container died 63c8ed9438d42ad5c054dd3a42c26cfbf567e53f0712c9e39685fdc2da0be182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-c, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T04:35:12.292 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:12 smithi145 podman[71772]: 2024-09-19 04:35:12.081152084 +0000 UTC m=+0.593535135 container cleanup 63c8ed9438d42ad5c054dd3a42c26cfbf567e53f0712c9e39685fdc2da0be182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T04:35:12.292 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 19 04:35:12 smithi145 bash[71772]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mon-c 2024-09-19T04:35:12.502 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T04:35:12.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:12.548 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T04:35:12.548 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T04:35:12.548 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.x 2024-09-19T04:35:12.981 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:35:12 smithi107 systemd[1]: Stopping Ceph mgr.x for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:13.370 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:35:12 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:35:12.994+0000 7fd9d6800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:13.371 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:35:12 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x[30884]: 2024-09-19T04:35:12.994+0000 7fd9d6800640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-19T04:35:13.371 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:35:13 smithi107 podman[111273]: 2024-09-19 04:35:13.077460365 +0000 UTC m=+0.235986703 container died c7ad715785fee54e102083cae12f2cb19eb851520e7a423ef7f9f2bcbddef977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:13.963 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:35:13 smithi107 podman[111273]: 2024-09-19 04:35:13.643197881 +0000 UTC m=+0.801724217 container cleanup c7ad715785fee54e102083cae12f2cb19eb851520e7a423ef7f9f2bcbddef977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:35:13.963 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Sep 19 04:35:13 smithi107 bash[111273]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-x 2024-09-19T04:35:14.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T04:35:14.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:14.203 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T04:35:14.203 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T04:35:14.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.y 2024-09-19T04:35:14.555 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:35:14 smithi115 systemd[1]: Stopping Ceph mgr.y for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:14.860 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:35:14 smithi115 podman[72516]: 2024-09-19 04:35:14.554078668 +0000 UTC m=+0.150263666 container died e28287cf2a5ba9254500b884708586e98d2d6d457e3312f907d34c8eb697a6a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T04:35:15.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:35:14 smithi115 podman[72516]: 2024-09-19 04:35:14.958380545 +0000 UTC m=+0.554565535 container cleanup e28287cf2a5ba9254500b884708586e98d2d6d457e3312f907d34c8eb697a6a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T04:35:15.196 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 19 04:35:14 smithi115 bash[72516]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-y 2024-09-19T04:35:15.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T04:35:15.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:15.386 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T04:35:15.387 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T04:35:15.387 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.z 2024-09-19T04:35:15.699 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:35:15 smithi145 systemd[1]: Stopping Ceph mgr.z for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:15.974 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:35:15 smithi145 podman[71894]: 2024-09-19 04:35:15.698246864 +0000 UTC m=+0.123206496 container died 42f6183e2e30dcd4d9dd99ebcd790a88edd1701eba63fef57b9bdd24cf940500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T04:35:16.474 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:35:16 smithi145 podman[71894]: 2024-09-19 04:35:16.203034983 +0000 UTC m=+0.627994616 container cleanup 42f6183e2e30dcd4d9dd99ebcd790a88edd1701eba63fef57b9bdd24cf940500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T04:35:16.475 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 19 04:35:16 smithi145 bash[71894]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mgr-z 2024-09-19T04:35:16.769 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T04:35:16.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:16.803 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T04:35:16.803 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T04:35:16.804 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.0 2024-09-19T04:35:17.210 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:16 smithi107 systemd[1]: Stopping Ceph osd.0 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:17.480 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0[43968]: 2024-09-19T04:35:17.209+0000 7f136f600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:17.481 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0[43968]: 2024-09-19T04:35:17.209+0000 7f136f600640 -1 osd.0 103 *** Got signal Terminated *** 2024-09-19T04:35:17.481 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0[43968]: 2024-09-19T04:35:17.209+0000 7f136f600640 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:35:23.117 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:22 smithi107 podman[111390]: 2024-09-19 04:35:22.810133478 +0000 UTC m=+5.722672912 container died 38607649c50daa4ca6af1ed36b1272bf8ae206c94ea7ced16d8b78097a0d8f66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T04:35:23.692 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:23 smithi107 podman[111390]: 2024-09-19 04:35:23.373173937 +0000 UTC m=+6.285713371 container cleanup 38607649c50daa4ca6af1ed36b1272bf8ae206c94ea7ced16d8b78097a0d8f66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T04:35:23.693 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:23 smithi107 bash[111390]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0 2024-09-19T04:35:23.980 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:23 smithi107 podman[111408]: 2024-09-19 04:35:23.691640707 +0000 UTC m=+0.869066925 container remove 38607649c50daa4ca6af1ed36b1272bf8ae206c94ea7ced16d8b78097a0d8f66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-19T04:35:24.463 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:24 smithi107 podman[111493]: 2024-09-19 04:35:24.05096647 +0000 UTC m=+0.027641020 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:35:24.463 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:24 smithi107 podman[111493]: 2024-09-19 04:35:24.167520817 +0000 UTC m=+0.144195351 container create d27a4510d67a47287781ba943ee05ea822e42addf31589c20c5aa66212bc2cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:24.726 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:24 smithi107 podman[111493]: 2024-09-19 04:35:24.462006243 +0000 UTC m=+0.438680793 container init d27a4510d67a47287781ba943ee05ea822e42addf31589c20c5aa66212bc2cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:35:24.726 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:24 smithi107 podman[111493]: 2024-09-19 04:35:24.469355004 +0000 UTC m=+0.446029540 container start d27a4510d67a47287781ba943ee05ea822e42addf31589c20c5aa66212bc2cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T04:35:24.726 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:24 smithi107 podman[111493]: 2024-09-19 04:35:24.570149854 +0000 UTC m=+0.546824401 container attach d27a4510d67a47287781ba943ee05ea822e42addf31589c20c5aa66212bc2cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T04:35:24.980 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 19 04:35:24 smithi107 podman[111493]: 2024-09-19 04:35:24.725534139 +0000 UTC m=+0.702208689 container died d27a4510d67a47287781ba943ee05ea822e42addf31589c20c5aa66212bc2cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T04:35:25.511 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T04:35:25.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:25.588 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T04:35:25.589 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T04:35:25.589 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.1 2024-09-19T04:35:25.930 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:25 smithi115 systemd[1]: Stopping Ceph osd.1 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:26.192 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:25 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1[39953]: 2024-09-19T04:35:25.928+0000 7fd62b030640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:26.192 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:25 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1[39953]: 2024-09-19T04:35:25.928+0000 7fd62b030640 -1 osd.1 103 *** Got signal Terminated *** 2024-09-19T04:35:26.193 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:25 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1[39953]: 2024-09-19T04:35:25.928+0000 7fd62b030640 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:35:31.696 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:31 smithi115 podman[72638]: 2024-09-19 04:35:31.43073086 +0000 UTC m=+5.648076037 container died 311ebaa9575871610f5ac90cb371adf4dcb2aa782f265fc06454680946432bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:32.097 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:31 smithi115 podman[72638]: 2024-09-19 04:35:31.845272128 +0000 UTC m=+6.062617293 container cleanup 311ebaa9575871610f5ac90cb371adf4dcb2aa782f265fc06454680946432bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:32.111 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:31 smithi115 bash[72638]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1 2024-09-19T04:35:32.446 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:32 smithi115 podman[72666]: 2024-09-19 04:35:32.096818436 +0000 UTC m=+0.664780365 container remove 311ebaa9575871610f5ac90cb371adf4dcb2aa782f265fc06454680946432bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T04:35:32.843 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:32 smithi115 podman[72755]: 2024-09-19 04:35:32.41511118 +0000 UTC m=+0.016209634 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:35:32.843 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:32 smithi115 podman[72755]: 2024-09-19 04:35:32.582137552 +0000 UTC m=+0.183236042 container create f9090f9b64def441d3037b001eeb3c180bafdafb41b1dcf707cc739cbce9b894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:32.844 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:32 smithi115 podman[72755]: 2024-09-19 04:35:32.842512482 +0000 UTC m=+0.443610933 container init f9090f9b64def441d3037b001eeb3c180bafdafb41b1dcf707cc739cbce9b894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T04:35:33.196 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:32 smithi115 podman[72755]: 2024-09-19 04:35:32.846084982 +0000 UTC m=+0.447183429 container start f9090f9b64def441d3037b001eeb3c180bafdafb41b1dcf707cc739cbce9b894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T04:35:33.197 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:32 smithi115 podman[72755]: 2024-09-19 04:35:32.942448039 +0000 UTC m=+0.543546495 container attach f9090f9b64def441d3037b001eeb3c180bafdafb41b1dcf707cc739cbce9b894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-19T04:35:33.197 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 19 04:35:33 smithi115 podman[72755]: 2024-09-19 04:35:33.028020479 +0000 UTC m=+0.629118924 container died f9090f9b64def441d3037b001eeb3c180bafdafb41b1dcf707cc739cbce9b894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T04:35:33.649 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T04:35:33.689 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:33.689 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T04:35:33.690 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T04:35:33.690 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.2 2024-09-19T04:35:33.974 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:33 smithi145 systemd[1]: Stopping Ceph osd.2 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:34.475 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:34 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2[40074]: 2024-09-19T04:35:34.009+0000 7ff69890f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:34.475 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:34 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2[40074]: 2024-09-19T04:35:34.009+0000 7ff69890f640 -1 osd.2 103 *** Got signal Terminated *** 2024-09-19T04:35:34.475 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:34 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2[40074]: 2024-09-19T04:35:34.009+0000 7ff69890f640 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:35:39.724 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:39 smithi145 podman[72031]: 2024-09-19 04:35:39.469993188 +0000 UTC m=+5.587347799 container died 31538d34f8ed9b301caabeb20cbbd58efdd8833e78e4b14c913ba7dd6350fa53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T04:35:40.118 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:39 smithi145 podman[72031]: 2024-09-19 04:35:39.823860735 +0000 UTC m=+5.941215345 container cleanup 31538d34f8ed9b301caabeb20cbbd58efdd8833e78e4b14c913ba7dd6350fa53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T04:35:40.118 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:39 smithi145 bash[72031]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2 2024-09-19T04:35:40.438 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:40 smithi145 podman[72047]: 2024-09-19 04:35:40.117434893 +0000 UTC m=+0.646778550 container remove 31538d34f8ed9b301caabeb20cbbd58efdd8833e78e4b14c913ba7dd6350fa53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T04:35:40.713 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:40 smithi145 podman[72134]: 2024-09-19 04:35:40.34062116 +0000 UTC m=+0.015404996 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:35:40.714 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:40 smithi145 podman[72134]: 2024-09-19 04:35:40.485086331 +0000 UTC m=+0.159870155 container create 2948c564f83d05ec1acb7168e789481fdc65a245cf10e9ba5ac4e7ddafe3b317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:35:40.974 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:40 smithi145 podman[72134]: 2024-09-19 04:35:40.712899513 +0000 UTC m=+0.387683338 container init 2948c564f83d05ec1acb7168e789481fdc65a245cf10e9ba5ac4e7ddafe3b317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T04:35:40.975 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:40 smithi145 podman[72134]: 2024-09-19 04:35:40.71690889 +0000 UTC m=+0.391692718 container start 2948c564f83d05ec1acb7168e789481fdc65a245cf10e9ba5ac4e7ddafe3b317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:40.975 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:40 smithi145 podman[72134]: 2024-09-19 04:35:40.796119132 +0000 UTC m=+0.470902973 container attach 2948c564f83d05ec1acb7168e789481fdc65a245cf10e9ba5ac4e7ddafe3b317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-09-19T04:35:40.975 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 19 04:35:40 smithi145 podman[72134]: 2024-09-19 04:35:40.912704346 +0000 UTC m=+0.587488180 container died 2948c564f83d05ec1acb7168e789481fdc65a245cf10e9ba5ac4e7ddafe3b317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:41.563 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T04:35:41.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:41.604 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T04:35:41.604 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T04:35:41.604 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.3 2024-09-19T04:35:41.948 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:41 smithi107 systemd[1]: Stopping Ceph osd.3 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:42.230 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:42 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3[50730]: 2024-09-19T04:35:42.028+0000 7f1b6ce00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:42.231 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:42 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3[50730]: 2024-09-19T04:35:42.028+0000 7f1b6ce00640 -1 osd.3 103 *** Got signal Terminated *** 2024-09-19T04:35:42.231 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:42 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3[50730]: 2024-09-19T04:35:42.028+0000 7f1b6ce00640 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:35:43.583 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:43 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3[50730]: 2024-09-19T04:35:43.350+0000 7f1b63800640 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:18.097737+0000 front 2024-09-19T04:35:18.097834+0000 (oldest deadline 2024-09-19T04:35:43.298818+0000) 2024-09-19T04:35:44.224 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:43 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:43.768+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:44.730 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:44 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3[50730]: 2024-09-19T04:35:44.389+0000 7f1b63800640 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:18.097737+0000 front 2024-09-19T04:35:18.097834+0000 (oldest deadline 2024-09-19T04:35:43.298818+0000) 2024-09-19T04:35:44.731 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:44 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:44.623+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:45.224 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:44 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:44.763+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:45.655 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:45 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3[50730]: 2024-09-19T04:35:45.341+0000 7f1b63800640 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:18.097737+0000 front 2024-09-19T04:35:18.097834+0000 (oldest deadline 2024-09-19T04:35:43.298818+0000) 2024-09-19T04:35:45.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:45 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:45.654+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:46.224 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:45 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:45.725+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:46.624 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:46 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3[50730]: 2024-09-19T04:35:46.377+0000 7f1b63800640 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:18.097737+0000 front 2024-09-19T04:35:18.097834+0000 (oldest deadline 2024-09-19T04:35:43.298818+0000) 2024-09-19T04:35:46.624 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:46 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:46.267+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:46.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:46 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:46.702+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:46.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:46 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:46.623+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:47.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:47.217+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:47.581 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:47 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:47.252+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:47.894 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:47 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:47.639+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:47.894 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:47 smithi107 podman[111595]: 2024-09-19 04:35:47.580470861 +0000 UTC m=+5.689872428 container died 281a3cebe5355c809abda728fad1cbfce15a976386e3077f5a732b2ada9afefb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T04:35:48.199 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:47.750+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:48.199 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:48 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:48.198+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:48.230 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:48 smithi107 podman[111595]: 2024-09-19 04:35:48.03362325 +0000 UTC m=+6.143024824 container cleanup 281a3cebe5355c809abda728fad1cbfce15a976386e3077f5a732b2ada9afefb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T04:35:48.231 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:48 smithi107 bash[111595]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3 2024-09-19T04:35:48.622 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:48 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:48.268+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:48.623 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:48 smithi107 podman[111624]: 2024-09-19 04:35:48.268579776 +0000 UTC m=+0.677987214 container remove 281a3cebe5355c809abda728fad1cbfce15a976386e3077f5a732b2ada9afefb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-19T04:35:48.934 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:48 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:48.621+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:48.935 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:48 smithi107 podman[111708]: 2024-09-19 04:35:48.794487636 +0000 UTC m=+0.115897229 container create e06c576650d4d50c210029b1a01ab7aa1b5363bf72af0a0d5c22ba6d0c19a760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:48.935 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:48 smithi107 podman[111708]: 2024-09-19 04:35:48.706259449 +0000 UTC m=+0.027669055 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:35:48.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:48 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:48.478+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:48.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:48 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:48.713+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:49.231 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:49 smithi107 podman[111708]: 2024-09-19 04:35:49.047084632 +0000 UTC m=+0.368494224 container init e06c576650d4d50c210029b1a01ab7aa1b5363bf72af0a0d5c22ba6d0c19a760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T04:35:49.231 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:49 smithi107 podman[111708]: 2024-09-19 04:35:49.053857914 +0000 UTC m=+0.375267508 container start e06c576650d4d50c210029b1a01ab7aa1b5363bf72af0a0d5c22ba6d0c19a760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T04:35:49.231 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:49 smithi107 podman[111708]: 2024-09-19 04:35:49.155299563 +0000 UTC m=+0.476709167 container attach e06c576650d4d50c210029b1a01ab7aa1b5363bf72af0a0d5c22ba6d0c19a760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T04:35:49.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:49 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:49.214+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:49.651 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:49 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:49.266+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:49.651 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 19 04:35:49 smithi107 podman[111731]: 2024-09-19 04:35:49.360431399 +0000 UTC m=+0.026708504 container died e06c576650d4d50c210029b1a01ab7aa1b5363bf72af0a0d5c22ba6d0c19a760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-19T04:35:49.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:49 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:49.458+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:49.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:49 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:49.678+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:49.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:49 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:49.649+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:50.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T04:35:50.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:50.189 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T04:35:50.189 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T04:35:50.189 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.4 2024-09-19T04:35:50.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:50 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:50.166+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:50.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:50 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:50.235+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:50.506 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:50 smithi115 systemd[1]: Stopping Ceph osd.4 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:50.506 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:50 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:50.447+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:50.946 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:50 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4[46844]: 2024-09-19T04:35:50.504+0000 7f477ec2b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:50.946 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:50 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4[46844]: 2024-09-19T04:35:50.504+0000 7f477ec2b640 -1 osd.4 103 *** Got signal Terminated *** 2024-09-19T04:35:50.946 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:50 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4[46844]: 2024-09-19T04:35:50.504+0000 7f477ec2b640 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:35:50.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:50 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:50.709+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:50.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:50 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:50.684+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:51.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:51.131+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:51.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:51 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:51.206+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:51.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:51 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:51.447+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:51.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:51.687+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:51.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:51 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:51.662+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:52.446 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:52 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4[46844]: 2024-09-19T04:35:52.108+0000 7f4777239640 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.520406+0000 front 2024-09-19T04:35:30.520436+0000 (oldest deadline 2024-09-19T04:35:51.620647+0000) 2024-09-19T04:35:52.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:52 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:52.180+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:52.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:52 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:52.168+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:52.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:52 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:52.168+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:35:52.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:52 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:52.474+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:52.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:52 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:52.722+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:52.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:52 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:52.669+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:52.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:52 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:52.669+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:35:53.446 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:53 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4[46844]: 2024-09-19T04:35:53.100+0000 7f4777239640 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.520406+0000 front 2024-09-19T04:35:30.520436+0000 (oldest deadline 2024-09-19T04:35:51.620647+0000) 2024-09-19T04:35:53.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:53 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:53.180+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:53.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:53 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:53.197+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:53.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:53 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:53.197+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:35:53.749 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:53 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:53.702+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:53.749 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:53 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:53.702+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:25.309263+0000 front 2024-09-19T04:35:25.309240+0000 (oldest deadline 2024-09-19T04:35:52.909747+0000) 2024-09-19T04:35:53.941 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:53 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:53.499+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:53.941 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:53 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:53.499+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:35:53.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:53 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:53.710+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:53.981 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:53 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:53.710+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:35:54.196 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:54 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4[46844]: 2024-09-19T04:35:54.086+0000 7f4777239640 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.520406+0000 front 2024-09-19T04:35:30.520436+0000 (oldest deadline 2024-09-19T04:35:51.620647+0000) 2024-09-19T04:35:54.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:35:53 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:35:53.939+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:35:54.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:54 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:54.183+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:54.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:54 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:54.183+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:35:54.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:54.152+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:54.481 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:54.152+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:35:54.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:54 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:54.462+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:54.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:54 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:54.462+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:35:54.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:54 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:54.683+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:54.975 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:54 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:54.683+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:25.309263+0000 front 2024-09-19T04:35:25.309240+0000 (oldest deadline 2024-09-19T04:35:52.909747+0000) 2024-09-19T04:35:54.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:54.677+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:54.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:54.677+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:35:55.446 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:55 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4[46844]: 2024-09-19T04:35:55.113+0000 7f4777239640 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.520406+0000 front 2024-09-19T04:35:30.520436+0000 (oldest deadline 2024-09-19T04:35:51.620647+0000) 2024-09-19T04:35:55.447 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:35:54 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:35:54.972+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:35:55.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:35:55 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:35:55.212+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:35:55.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:55 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:55.186+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:55.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:55 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:55.186+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:35:55.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:55 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:55.173+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:55.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:55 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:55.173+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:35:55.726 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:55 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:55.511+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:55.727 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:55 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:55.511+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:35:55.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:55 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:55.675+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:55.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:55 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:55.675+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:25.309263+0000 front 2024-09-19T04:35:25.309240+0000 (oldest deadline 2024-09-19T04:35:52.909747+0000) 2024-09-19T04:35:55.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:55 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:55.631+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:55.981 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:55 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:55.631+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:35:55.981 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:35:55 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:35:55.950+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:35:56.281 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:55 smithi115 podman[72858]: 2024-09-19 04:35:55.980702058 +0000 UTC m=+5.597318397 container died b7995b9ee83af09fc1ce4f1961937ea778348d802ba26f10b21d198dc04c14e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T04:35:56.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:35:56 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:35:56.231+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:35:56.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:56 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:56.137+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:56.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:56 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:56.137+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:35:56.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:56 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:56.219+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:56.481 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:56 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:56.219+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:35:56.552 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:56 smithi115 podman[72858]: 2024-09-19 04:35:56.396907281 +0000 UTC m=+6.013523609 container cleanup b7995b9ee83af09fc1ce4f1961937ea778348d802ba26f10b21d198dc04c14e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:35:56.552 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:56 smithi115 bash[72858]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4 2024-09-19T04:35:56.946 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:56 smithi115 podman[72882]: 2024-09-19 04:35:56.631698661 +0000 UTC m=+0.646799028 container remove b7995b9ee83af09fc1ce4f1961937ea778348d802ba26f10b21d198dc04c14e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:56.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:56 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:56.551+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:56.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:56 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:56.551+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:35:56.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:56 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:56.626+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:56.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:56 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:56.626+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:25.309263+0000 front 2024-09-19T04:35:25.309240+0000 (oldest deadline 2024-09-19T04:35:52.909747+0000) 2024-09-19T04:35:56.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:56 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:56.613+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:56.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:56 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:56.613+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:35:57.202 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:35:56 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:35:56.950+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:35:57.203 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:57 smithi115 podman[72970]: 2024-09-19 04:35:56.95804009 +0000 UTC m=+0.015988580 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:35:57.203 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:57 smithi115 podman[72970]: 2024-09-19 04:35:57.08284641 +0000 UTC m=+0.140794934 container create 3df3a1b7705d9c0af28032515428b0d4277c44c76f9e3f5adf2c7211b17ae5a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T04:35:57.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:35:57 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:35:57.205+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:35:57.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:57 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:57.092+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:57.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:57 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:57.092+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:35:57.529 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:57 smithi115 podman[72970]: 2024-09-19 04:35:57.327028728 +0000 UTC m=+0.384977220 container init 3df3a1b7705d9c0af28032515428b0d4277c44c76f9e3f5adf2c7211b17ae5a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:35:57.529 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:57 smithi115 podman[72970]: 2024-09-19 04:35:57.330498037 +0000 UTC m=+0.388446522 container start 3df3a1b7705d9c0af28032515428b0d4277c44c76f9e3f5adf2c7211b17ae5a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T04:35:57.529 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:57 smithi115 podman[72970]: 2024-09-19 04:35:57.426910248 +0000 UTC m=+0.484858730 container attach 3df3a1b7705d9c0af28032515428b0d4277c44c76f9e3f5adf2c7211b17ae5a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T04:35:57.577 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:57 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:57.234+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:57.577 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:57 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:57.234+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:35:57.897 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 19 04:35:57 smithi115 podman[72970]: 2024-09-19 04:35:57.528547648 +0000 UTC m=+0.586496130 container died 3df3a1b7705d9c0af28032515428b0d4277c44c76f9e3f5adf2c7211b17ae5a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T04:35:57.897 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:57 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:57.537+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:57.897 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:57 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:57.537+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:35:57.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:57 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:57.578+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:57.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:57 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:57.578+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:25.309263+0000 front 2024-09-19T04:35:25.309240+0000 (oldest deadline 2024-09-19T04:35:52.909747+0000) 2024-09-19T04:35:57.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:57 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:57.575+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:57.981 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:57 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:57.575+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:35:58.173 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:35:57 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:35:57.975+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:35:58.204 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T04:35:58.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:35:58.243 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T04:35:58.244 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T04:35:58.244 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.5 2024-09-19T04:35:58.474 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:58 smithi145 systemd[1]: Stopping Ceph osd.5 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:35:58.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:35:58 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:35:58.172+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:35:58.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:58 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:58.094+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:58.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:58 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:58.094+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:35:58.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:58.208+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:58.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:58.208+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:35:58.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:58 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:58.559+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:58.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:58 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:58.559+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:35:58.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:58 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:58.551+0000 7efe971b2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:35:58.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:58 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:58.551+0000 7efe971b2640 -1 osd.5 103 *** Got signal Terminated *** 2024-09-19T04:35:58.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:58 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:58.551+0000 7efe971b2640 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:35:58.975 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:58 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:58.600+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:58.975 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:58 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:58.600+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:25.309263+0000 front 2024-09-19T04:35:25.309240+0000 (oldest deadline 2024-09-19T04:35:52.909747+0000) 2024-09-19T04:35:58.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:58.621+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:58.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:58 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:58.621+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:35:59.446 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:35:58 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:35:58.955+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:35:59.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:35:59 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:35:59.201+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:35:59.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:59 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:59.065+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:35:59.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:35:59 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:35:59.065+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:35:59.637 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:59 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:59.255+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:35:59.637 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:35:59 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:35:59.255+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:35:59.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:59 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:59.600+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:35:59.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:35:59 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:35:59.600+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:35:59.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:59 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:59.635+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:35:59.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:35:59 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:35:59.635+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:25.309263+0000 front 2024-09-19T04:35:25.309240+0000 (oldest deadline 2024-09-19T04:35:52.909747+0000) 2024-09-19T04:35:59.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:59 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:59.635+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:35:59.981 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:35:59 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:35:59.635+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:00.446 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:00.004+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:00.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:00.201+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:00.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:00.025+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:00.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:00.025+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:00.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:00.025+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:00.616 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:00.276+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:00.616 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:00.276+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:00.948 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:00.560+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:00.948 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:00.560+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:00.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:36:00.665+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:36:00.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:36:00.665+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:25.309263+0000 front 2024-09-19T04:35:25.309240+0000 (oldest deadline 2024-09-19T04:35:52.909747+0000) 2024-09-19T04:36:00.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:00 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:36:00.665+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:38.111483+0000 front 2024-09-19T04:35:38.111516+0000 (oldest deadline 2024-09-19T04:35:59.712161+0000) 2024-09-19T04:36:00.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:00.614+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:00.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:00.614+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:01.446 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:00.995+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:01.446 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:00 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:00.995+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:01.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:01 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:01.180+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:01.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:01 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:01.180+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:01.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:01 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:01.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:01.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:01 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:01.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:01.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:01 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:01.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:01.655 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:01.284+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:01.655 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:01.284+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:01.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:01 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:01.521+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:01.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:01 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:01.521+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:01.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:01 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:36:01.638+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:36:01.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:01 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:36:01.638+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:25.309263+0000 front 2024-09-19T04:35:25.309240+0000 (oldest deadline 2024-09-19T04:35:52.909747+0000) 2024-09-19T04:36:01.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:01 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:36:01.638+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:38.111483+0000 front 2024-09-19T04:35:38.111516+0000 (oldest deadline 2024-09-19T04:35:59.712161+0000) 2024-09-19T04:36:01.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:01.653+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:01.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:01.653+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:01.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:01 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:01.653+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.714474+0000 front 2024-09-19T04:35:37.714444+0000 (oldest deadline 2024-09-19T04:36:01.214423+0000) 2024-09-19T04:36:02.446 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:01 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:01.957+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:02.446 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:01 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:01.957+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:02.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:02.204+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:02.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:02.204+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:02.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:02.059+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:02.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:02.059+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:02.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:02.059+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:02.697 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:02 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:02.321+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:02.697 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:02 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:02.321+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:02.697 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:02 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:02.321+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:02.930 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:02.532+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:02.930 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:02.532+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:02.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:36:02.595+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.707999+0000 front 2024-09-19T04:35:20.707833+0000 (oldest deadline 2024-09-19T04:35:43.607959+0000) 2024-09-19T04:36:02.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:36:02.595+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:25.309263+0000 front 2024-09-19T04:35:25.309240+0000 (oldest deadline 2024-09-19T04:35:52.909747+0000) 2024-09-19T04:36:02.974 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:02 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5[46538]: 2024-09-19T04:36:02.595+0000 7efe937c8640 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:38.111483+0000 front 2024-09-19T04:35:38.111516+0000 (oldest deadline 2024-09-19T04:35:59.712161+0000) 2024-09-19T04:36:02.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:02 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:02.695+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:02.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:02 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:02.695+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:02.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:02 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:02.695+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.714474+0000 front 2024-09-19T04:35:37.714444+0000 (oldest deadline 2024-09-19T04:36:01.214423+0000) 2024-09-19T04:36:03.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:02.929+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:03.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:02 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:02.929+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:03.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:03.205+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:03.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:03.205+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:03.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:03.012+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:03.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:03.012+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:03.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:03.012+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:03.706 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:03 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:03.343+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:03.706 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:03 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:03.343+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:03.706 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:03 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:03.343+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:03.931 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:03 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:03.549+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:03.931 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:03 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:03.549+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:03.931 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:03 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:03.549+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:03.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:03 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:03.704+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:03.981 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:03 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:03.704+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:03.981 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:03 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:03.704+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.714474+0000 front 2024-09-19T04:35:37.714444+0000 (oldest deadline 2024-09-19T04:36:01.214423+0000) 2024-09-19T04:36:04.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:03 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:03.930+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:04.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:03 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:03.930+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:04.224 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:03 smithi145 podman[72264]: 2024-09-19 04:36:03.994264093 +0000 UTC m=+5.550368745 container died 1e202a27246c7a1b45111fce13c646dd7b1180a622455dd837c38f263f64108e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T04:36:04.224 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:04 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:04.157+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:04.225 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:04 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:04.157+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:04.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:03.970+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:04.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:03.970+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:04.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:03 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:03.970+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:04.613 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:04 smithi145 podman[72264]: 2024-09-19 04:36:04.402994607 +0000 UTC m=+5.959099258 container cleanup 1e202a27246c7a1b45111fce13c646dd7b1180a622455dd837c38f263f64108e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:36:04.614 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:04 smithi145 bash[72264]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5 2024-09-19T04:36:04.719 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:04 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:04.386+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:04.720 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:04 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:04.386+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:04.720 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:04 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:04.386+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:04.892 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:04 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:04.590+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:04.893 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:04 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:04.590+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:04.893 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:04 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:04.590+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:04.946 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:04 smithi145 podman[72281]: 2024-09-19 04:36:04.612733095 +0000 UTC m=+0.618984169 container remove 1e202a27246c7a1b45111fce13c646dd7b1180a622455dd837c38f263f64108e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:36:04.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:04 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:04.718+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:04.981 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:04 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:04.718+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:04.981 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:04 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:04.718+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.714474+0000 front 2024-09-19T04:35:37.714444+0000 (oldest deadline 2024-09-19T04:36:01.214423+0000) 2024-09-19T04:36:05.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:04 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:04.891+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:05.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:04 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:04.891+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:05.200 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:05 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:05.162+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:05.200 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:05 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:05.162+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:05.200 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:04 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:04.994+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:05.201 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:04 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:04.994+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:05.201 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:04 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:04.994+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:05.201 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:04 smithi145 podman[72369]: 2024-09-19 04:36:04.848579651 +0000 UTC m=+0.015845636 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:36:05.201 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:04 smithi145 podman[72369]: 2024-09-19 04:36:04.972224852 +0000 UTC m=+0.139490826 container create 6de3c83252d65ff3ef4e952ed517fbf3b4b00bcb7001f0b9e6e0923b4b7ea143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:36:05.201 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:05 smithi145 podman[72369]: 2024-09-19 04:36:05.199902769 +0000 UTC m=+0.367168748 container init 6de3c83252d65ff3ef4e952ed517fbf3b4b00bcb7001f0b9e6e0923b4b7ea143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:36:05.474 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:05 smithi145 podman[72369]: 2024-09-19 04:36:05.203207304 +0000 UTC m=+0.370473282 container start 6de3c83252d65ff3ef4e952ed517fbf3b4b00bcb7001f0b9e6e0923b4b7ea143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T04:36:05.474 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:05 smithi145 podman[72369]: 2024-09-19 04:36:05.291491603 +0000 UTC m=+0.458757586 container attach 6de3c83252d65ff3ef4e952ed517fbf3b4b00bcb7001f0b9e6e0923b4b7ea143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:36:05.475 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 19 04:36:05 smithi145 podman[72369]: 2024-09-19 04:36:05.393455285 +0000 UTC m=+0.560721270 container died 6de3c83252d65ff3ef4e952ed517fbf3b4b00bcb7001f0b9e6e0923b4b7ea143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T04:36:05.721 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:05 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:05.426+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:05.722 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:05 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:05.426+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:05.722 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:05 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:05.426+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:05.907 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:05.586+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:05.907 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:05.586+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:05.908 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:05.586+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:05.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:05 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:05.720+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:05.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:05 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:05.720+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:05.981 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:05 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:05.720+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.714474+0000 front 2024-09-19T04:35:37.714444+0000 (oldest deadline 2024-09-19T04:36:01.214423+0000) 2024-09-19T04:36:06.037 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:05 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:05.958+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:06.038 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:05 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:05.958+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:06.038 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:05 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:05.958+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:06.061 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T04:36:06.098 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:06.099 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T04:36:06.099 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T04:36:06.099 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.6 2024-09-19T04:36:06.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:05.906+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:06.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:05 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:05.906+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:06.463 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:06 smithi107 systemd[1]: Stopping Ceph osd.6 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:06.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:06 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:06.170+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:06.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:06 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:06.170+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:06.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:06 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:06.461+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:06.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:06 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:06.461+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:06.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:06 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:06.461+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:06.731 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:06 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:06.521+0000 7f6407200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:06.731 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:06 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:06.521+0000 7f6407200640 -1 osd.6 103 *** Got signal Terminated *** 2024-09-19T04:36:06.731 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:06 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:06.521+0000 7f6407200640 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:36:06.731 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:06 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:06.673+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:06.732 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:06 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:06.673+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:06.732 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:06 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:06.673+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.714474+0000 front 2024-09-19T04:35:37.714444+0000 (oldest deadline 2024-09-19T04:36:01.214423+0000) 2024-09-19T04:36:06.928 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:06 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:06.636+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:06.928 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:06 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:06.636+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:06.929 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:06 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:06.636+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:07.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:06 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:06.927+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:07.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:06 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:06.927+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:07.216 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:06 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:06.940+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:07.216 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:06 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:06.940+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:07.216 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:06 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:06.940+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:07.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:07 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:07.214+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:07.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:07 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:07.214+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:07.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:07 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:07.416+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:07.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:07 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:07.416+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:07.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:07 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:07.416+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:07.731 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:07 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:07.664+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:07.731 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:07 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:07.664+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:07.731 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:07 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:07.664+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.714474+0000 front 2024-09-19T04:35:37.714444+0000 (oldest deadline 2024-09-19T04:36:01.214423+0000) 2024-09-19T04:36:07.906 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:07 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:07.607+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:07.906 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:07 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:07.607+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:07.906 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:07 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:07.607+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:08.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:07 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:07.905+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:08.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:07 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:07.905+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:08.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:07 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:07.945+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:08.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:07 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:07.945+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:08.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:07 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:07.945+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:08.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:08 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:08.235+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:08.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:08 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:08.235+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:08.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:08 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:08.235+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:08.732 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:08 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:08.631+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:08.732 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:08 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:08.631+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:08.732 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:08 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:08.631+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.714474+0000 front 2024-09-19T04:35:37.714444+0000 (oldest deadline 2024-09-19T04:36:01.214423+0000) 2024-09-19T04:36:08.732 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:08 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:08.414+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:08.733 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:08 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:08.414+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:08.733 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:08 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:08.414+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:08.897 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:08 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:08.558+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:08.897 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:08 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:08.558+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:08.897 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:08 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:08.558+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:09.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:08 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:08.896+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:09.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:08 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:08.896+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:09.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:08 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:08.896+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:09.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:08 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:08.937+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:09.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:08 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:08.937+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:09.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:08 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:08.937+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:09.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:09 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:09.244+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:09.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:09 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:09.244+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:09.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:09 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:09.244+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:09.730 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:09 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:09.584+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:09.730 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:09 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:09.584+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:09.731 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:09 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:09.584+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.714474+0000 front 2024-09-19T04:35:37.714444+0000 (oldest deadline 2024-09-19T04:36:01.214423+0000) 2024-09-19T04:36:09.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:09 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:09.461+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:09.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:09 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:09.461+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:09.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:09 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:09.461+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:09.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:09 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:09.461+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:09.851 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:09 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:09.557+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:09.851 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:09 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:09.557+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:09.851 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:09 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:09.557+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:10.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:09 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:09.850+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:10.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:09 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:09.850+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:10.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:09 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:09.850+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:10.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:09 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:09.931+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:10.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:09 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:09.931+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:10.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:09 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:09.931+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:10.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:09 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:09.931+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:10.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:10 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:10.287+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:10.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:10 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:10.287+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:10.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:10 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:10.287+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:10.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:10 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:10.833+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:10.947 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:10 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:10.833+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:10.947 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:10 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:10.833+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:10.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:10 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:10.603+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:10.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:10 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:10.603+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:10.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:10 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:10.603+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:10.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:10 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:10.618+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:19.011391+0000 front 2024-09-19T04:35:19.011361+0000 (oldest deadline 2024-09-19T04:35:43.711374+0000) 2024-09-19T04:36:10.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:10 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:10.618+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.913215+0000 front 2024-09-19T04:35:28.913254+0000 (oldest deadline 2024-09-19T04:35:51.813239+0000) 2024-09-19T04:36:10.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:10 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6[57685]: 2024-09-19T04:36:10.618+0000 7f6402c00640 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.714474+0000 front 2024-09-19T04:35:37.714444+0000 (oldest deadline 2024-09-19T04:36:01.214423+0000) 2024-09-19T04:36:10.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:10 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:10.487+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:10.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:10 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:10.487+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:10.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:10 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:10.487+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:10.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:10 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:10.487+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:11.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:10 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:10.887+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:11.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:10 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:10.887+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:11.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:10 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:10.887+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:11.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:10 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:10.887+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:11.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:11 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:11.294+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:11.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:11 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:11.294+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:11.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:11 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:11.294+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:11.882 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:11 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:11.523+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:11.882 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:11 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:11.523+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:11.883 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:11 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:11.523+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:11.883 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:11 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:11.523+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:11.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:11 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:11.794+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:11.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:11 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:11.794+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:11.947 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:11 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:11.794+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:11.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:11 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:11.584+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:11.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:11 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:11.584+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:11.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:11 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:11.584+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:11.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:11 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:11.584+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:12.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:11 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:11.897+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:12.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:11 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:11.897+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:12.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:11 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:11.897+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:12.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:11 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:11.897+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:12.230 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:12 smithi107 podman[111809]: 2024-09-19 04:36:12.036128303 +0000 UTC m=+5.652666838 container died 985a75a4ff67ca2b0813c610a965f40b44b78e031a6d638cfa7c38dddb3c4c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T04:36:12.578 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:12 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:12.485+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:12.578 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:12 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:12.485+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:12.578 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:12 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:12.485+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:12.578 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:12 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:12.485+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:12.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:12 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:12.287+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:12.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:12 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:12.287+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:12.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:12 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:12.287+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:12.892 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:12 smithi107 podman[111809]: 2024-09-19 04:36:12.598315212 +0000 UTC m=+6.214853745 container cleanup 985a75a4ff67ca2b0813c610a965f40b44b78e031a6d638cfa7c38dddb3c4c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T04:36:12.893 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:12 smithi107 bash[111809]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6 2024-09-19T04:36:12.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:12 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:12.811+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:12.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:12 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:12.811+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:12.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:12 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:12.811+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:12.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:12 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:12.627+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:12.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:12 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:12.627+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:12.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:12 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:12.627+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:12.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:12 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:12.627+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:13.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:12 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:12.874+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:13.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:12 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:12.874+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:13.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:12 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:12.874+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:13.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:12 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:12.874+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:13.230 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:12 smithi107 podman[111827]: 2024-09-19 04:36:12.945497049 +0000 UTC m=+0.898114051 container remove 985a75a4ff67ca2b0813c610a965f40b44b78e031a6d638cfa7c38dddb3c4c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T04:36:13.716 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:13.502+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:13.716 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:13.502+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:13.716 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:13.502+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:13.716 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:13 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:13.502+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:13.717 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:13 smithi107 podman[111922]: 2024-09-19 04:36:13.334691674 +0000 UTC m=+0.027675057 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:36:13.717 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:13 smithi107 podman[111922]: 2024-09-19 04:36:13.437809897 +0000 UTC m=+0.130793280 container create 9169b467287b30d8491dbf8e76cd3affaf98bf2f67039477eda1d42d04a9e917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T04:36:13.717 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:13 smithi107 podman[111922]: 2024-09-19 04:36:13.715428849 +0000 UTC m=+0.408412233 container init 9169b467287b30d8491dbf8e76cd3affaf98bf2f67039477eda1d42d04a9e917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T04:36:13.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:13.278+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:13.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:13.278+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:13.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:13.278+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:13.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:13.851+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:13.947 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:13.851+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:13.947 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:13.851+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:13.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:13.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:13.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:13.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:13.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:13.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:13.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:13.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:13.980 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:13 smithi107 podman[111922]: 2024-09-19 04:36:13.722419022 +0000 UTC m=+0.415402407 container start 9169b467287b30d8491dbf8e76cd3affaf98bf2f67039477eda1d42d04a9e917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T04:36:13.981 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:13 smithi107 podman[111922]: 2024-09-19 04:36:13.83200895 +0000 UTC m=+0.524992334 container attach 9169b467287b30d8491dbf8e76cd3affaf98bf2f67039477eda1d42d04a9e917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T04:36:14.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:13.858+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:14.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:13.858+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:14.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:13.858+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:14.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:13 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:13.858+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:14.351 INFO:journalctl@ceph.osd.6.smithi107.stdout:Sep 19 04:36:13 smithi107 podman[111922]: 2024-09-19 04:36:13.98354592 +0000 UTC m=+0.676529305 container died 9169b467287b30d8491dbf8e76cd3affaf98bf2f67039477eda1d42d04a9e917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T04:36:14.628 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:14.502+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:14.628 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:14.502+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:14.628 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:14.502+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:14.628 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:14 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:14.502+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:14.658 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T04:36:14.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:14.323+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:14.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:14.323+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:14.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:14.323+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:14.733 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:14.734 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T04:36:14.734 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T04:36:14.734 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.7 2024-09-19T04:36:14.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:14 smithi115 systemd[1]: Stopping Ceph osd.7 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:14.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:14 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:14.828+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:14.947 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:14 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:14.828+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:14.947 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:14 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:14.828+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:14.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:14 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:14.688+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:14.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:14 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:14.688+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:14.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:14 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:14.688+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:14.948 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:14 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:14.688+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:15.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:14.861+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:15.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:14.861+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:15.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:14.861+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:15.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:14 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:14.861+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:15.446 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:15.064+0000 7f82746a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:15.447 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:15.064+0000 7f82746a7640 -1 osd.7 103 *** Got signal Terminated *** 2024-09-19T04:36:15.447 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:15.064+0000 7f82746a7640 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:36:15.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:15 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:15.284+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:15.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:15 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:15.284+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:15.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:15 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:15.284+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:15.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:15 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:15.505+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:15.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:15 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:15.505+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:15.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:15 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:15.505+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:15.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:15 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:15.505+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:16.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:15.869+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:16.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:15.869+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:16.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:15.869+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:16.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:15.711+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:16.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:15.711+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:16.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:15.711+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:16.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:15 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:15.711+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:16.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:15 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:15.850+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:16.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:15 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:15.850+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:16.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:15 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:15.850+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:16.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:15 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:15.850+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:16.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:16.311+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:16.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:16.311+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:16.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:16.311+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:16.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:16.852+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:16.947 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:16.852+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:16.947 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:16.852+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:16.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:16.667+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:16.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:16.667+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:16.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:16.667+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:16.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:16 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:16.667+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:16.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:16 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:16.522+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:16.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:16 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:16.522+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:16.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:16 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:16.522+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:16.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:16 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:16.522+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:17.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:16.850+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:17.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:16.850+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:17.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:16.850+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:17.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:16.850+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:17.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:16 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:16.850+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:17.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:17.286+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:17.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:17.286+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:17.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:17.286+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:17.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:17.286+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:17.948 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:17.871+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:17.949 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:17.871+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:17.949 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:17.871+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:17.949 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:17.651+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:17.949 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:17.651+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:17.949 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:17.651+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:17.950 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:17.651+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:17.950 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:17.651+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:17.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:17.541+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:17.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:17.541+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:17.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:17.541+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:17.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:17 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:17.541+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:18.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:17.896+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:18.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:17.896+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:18.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:17.896+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:18.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:17.896+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:18.226 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:17 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:17.896+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:18.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:18 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:18.264+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:18.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:18 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:18.264+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:18.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:18 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:18.264+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:18.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:18 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:18.264+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:18.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:18 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:18.678+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:18.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:18 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:18.678+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:18.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:18 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:18.678+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:18.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:18 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:18.678+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:18.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:18 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:18.678+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:18.947 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:18 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:18.891+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:18.948 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:18 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:18.891+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:18.948 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:18 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:18.891+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:18.948 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:18 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:18.891+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:49.860356+0000 front 2024-09-19T04:35:49.860331+0000 (oldest deadline 2024-09-19T04:36:18.061043+0000) 2024-09-19T04:36:18.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:18.558+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:18.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:18.558+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:18.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:18.558+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:18.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:18 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:18.558+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:19.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:18 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:18.920+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:19.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:18 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:18.920+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:19.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:18 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:18.920+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:19.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:18 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:18.920+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:19.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:18 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:18.920+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:19.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:19 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:19.299+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:19.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:19 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:19.299+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:19.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:19 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:19.299+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:19.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:19 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:19.299+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:19.941 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:19 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:19.675+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:19.941 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:19 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:19.675+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:19.941 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:19 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:19.675+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:19.941 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:19 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:19.675+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:19.941 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:19 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:19.675+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:19.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:19.545+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:19.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:19.545+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:19.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:19.545+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:19.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:19 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:19.545+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:20.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:19 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:19.939+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:28.356639+0000 front 2024-09-19T04:35:28.356683+0000 (oldest deadline 2024-09-19T04:35:53.656803+0000) 2024-09-19T04:36:20.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:19 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:19.939+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:34.757754+0000 front 2024-09-19T04:35:34.757800+0000 (oldest deadline 2024-09-19T04:36:00.558382+0000) 2024-09-19T04:36:20.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:19 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:19.939+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.059467+0000 front 2024-09-19T04:35:44.059451+0000 (oldest deadline 2024-09-19T04:36:08.759523+0000) 2024-09-19T04:36:20.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:19 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7[53367]: 2024-09-19T04:36:19.939+0000 7f8270cbd640 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:49.860356+0000 front 2024-09-19T04:35:49.860331+0000 (oldest deadline 2024-09-19T04:36:18.061043+0000) 2024-09-19T04:36:20.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:19 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:19.964+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:20.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:19 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:19.964+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:20.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:19 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:19.964+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:20.224 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:19 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:19.964+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:20.225 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:19 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:19.964+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:20.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:20 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:20.333+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:20.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:20 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:20.333+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:20.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:20 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:20.333+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:20.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:20 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:20.333+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:20.946 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:20 smithi115 podman[73071]: 2024-09-19 04:36:20.652384614 +0000 UTC m=+5.715477348 container died c714ebdca20ab29f9e7043998ce8537c781db5d191ed32467cd6cf46213d4152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T04:36:20.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:20 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:20.642+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:20.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:20 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:20.642+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:20.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:20 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:20.642+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:20.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:20 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:20.642+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:20.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:20 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:20.642+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:20.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:20.498+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:20.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:20.498+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:20.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:20.498+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:20.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:20 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:20.498+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:21.266 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:21 smithi115 podman[73071]: 2024-09-19 04:36:21.06629036 +0000 UTC m=+6.129383091 container cleanup c714ebdca20ab29f9e7043998ce8537c781db5d191ed32467cd6cf46213d4152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T04:36:21.267 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:21 smithi115 bash[73071]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7 2024-09-19T04:36:21.338 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:20 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:20.981+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:21.338 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:20 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:20.981+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:21.338 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:20 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:20.981+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:21.338 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:20 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:20.981+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:21.339 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:20 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:20.981+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:21.619 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:21 smithi115 podman[73086]: 2024-09-19 04:36:21.265786457 +0000 UTC m=+0.608260497 container remove c714ebdca20ab29f9e7043998ce8537c781db5d191ed32467cd6cf46213d4152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T04:36:21.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:21 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:21.337+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:21.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:21 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:21.337+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:21.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:21 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:21.337+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:21.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:21 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:21.337+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:21.895 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:21 smithi115 podman[73175]: 2024-09-19 04:36:21.568864472 +0000 UTC m=+0.015835973 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:36:21.895 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:21 smithi115 podman[73175]: 2024-09-19 04:36:21.68371104 +0000 UTC m=+0.130682577 container create b760acaac1be2d46a4ae8dba24e29df0afd2d0b41480f59bb39e4f18bce52ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T04:36:21.895 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:21 smithi115 podman[73175]: 2024-09-19 04:36:21.894162155 +0000 UTC m=+0.341133658 container init b760acaac1be2d46a4ae8dba24e29df0afd2d0b41480f59bb39e4f18bce52ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T04:36:21.896 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:21 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:21.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:21.896 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:21 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:21.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:21.896 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:21 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:21.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:21.897 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:21 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:21.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:21.897 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:21 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:21.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:21.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:21 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:21.490+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:21.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:21 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:21.490+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:21.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:21 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:21.490+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:21.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:21 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:21.490+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:21.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:21 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:21.490+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:22.196 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:21 smithi115 podman[73175]: 2024-09-19 04:36:21.897736913 +0000 UTC m=+0.344708410 container start b760acaac1be2d46a4ae8dba24e29df0afd2d0b41480f59bb39e4f18bce52ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:36:22.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:21 smithi115 podman[73175]: 2024-09-19 04:36:21.99419255 +0000 UTC m=+0.441164056 container attach b760acaac1be2d46a4ae8dba24e29df0afd2d0b41480f59bb39e4f18bce52ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:36:22.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 19 04:36:22 smithi115 podman[73175]: 2024-09-19 04:36:22.081048088 +0000 UTC m=+0.528019582 container died b760acaac1be2d46a4ae8dba24e29df0afd2d0b41480f59bb39e4f18bce52ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T04:36:22.326 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:22 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:22.016+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:22.327 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:22 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:22.016+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:22.327 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:22 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:22.016+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:22.327 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:22 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:22.016+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:22.328 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:22 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:22.016+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:22.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:22 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:22.325+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:22.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:22 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:22.325+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:22.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:22 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:22.325+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:22.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:22 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:22.325+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:22.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:22 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:22.471+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:22.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:22 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:22.471+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:22.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:22 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:22.471+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:22.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:22 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:22.471+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:22.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:22 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:22.471+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:22.782 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T04:36:22.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:22.820 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T04:36:22.820 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T04:36:22.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.8 2024-09-19T04:36:22.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:22 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:22.663+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:22.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:22 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:22.663+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:22.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:22 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:22.663+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:22.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:22 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:22.663+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:22.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:22 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:22.663+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:23.115 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:22 smithi145 systemd[1]: Stopping Ceph osd.8 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:23.115 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:23.050+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:23.116 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:23.050+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:23.116 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:23.050+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:23.116 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:23.050+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:23.116 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:23.050+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:23.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:23.113+0000 7f9c98caf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:23.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:23.113+0000 7f9c98caf640 -1 osd.8 103 *** Got signal Terminated *** 2024-09-19T04:36:23.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:23.113+0000 7f9c98caf640 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:36:23.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:23.333+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:23.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:23.333+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:23.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:23.333+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:23.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:23 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:23.333+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:23.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:23 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:23.444+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:23.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:23 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:23.444+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:23.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:23 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:23.444+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:23.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:23 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:23.444+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:23.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:23 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:23.444+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:23.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:23 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:23.631+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:23.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:23 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:23.631+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:23.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:23 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:23.631+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:23.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:23 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:23.631+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:23.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:23 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:23.631+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:24.320 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:24 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:24.067+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:24.321 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:24 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:24.067+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:24.321 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:24 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:24.067+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:24.321 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:24 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:24.067+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:24.321 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:24 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:24.067+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:24.724 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:24 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:24.319+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:24.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:24 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:24.319+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:24.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:24 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:24.319+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:24.725 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:24 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:24.319+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:24.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:24 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:24.445+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:24.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:24 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:24.445+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:24.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:24 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:24.445+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:24.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:24 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:24.445+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:24.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:24 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:24.445+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:24.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:24 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:24.664+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:24.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:24 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:24.664+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:24.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:24 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:24.664+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:24.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:24 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:24.664+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:24.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:24 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:24.664+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:24.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:24 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:24.664+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:25.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:25.288+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:25.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:25.288+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:25.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:25.288+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:25.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:25.288+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:25.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:25.288+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.029060+0000 front 2024-09-19T04:36:00.029093+0000 (oldest deadline 2024-09-19T04:36:24.729301+0000) 2024-09-19T04:36:25.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:25.051+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:25.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:25.051+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:25.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:25.051+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:25.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:25.051+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:25.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:25.051+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:25.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:25 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:25.413+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:25.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:25 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:25.413+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:25.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:25 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:25.413+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:25.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:25 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:25.413+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:25.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:25 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:25.413+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:25.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:25 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:25.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:25.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:25 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:25.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:25.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:25 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:25.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:25.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:25 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:25.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:25.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:25 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:25.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:25.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:25 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:25.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:26.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:26 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:26.262+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:26.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:26 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:26.262+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:26.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:26 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:26.262+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:26.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:26 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:26.262+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:26.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:26 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:26.262+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.029060+0000 front 2024-09-19T04:36:00.029093+0000 (oldest deadline 2024-09-19T04:36:24.729301+0000) 2024-09-19T04:36:26.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:26 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:26.069+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:26.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:26 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:26.069+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:26.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:26 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:26.069+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:26.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:26 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:26.069+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:26.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:26 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:26.069+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:26.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:26 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:26.434+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:26.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:26 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:26.434+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:26.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:26 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:26.434+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:26.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:26 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:26.434+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:26.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:26 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:26.434+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:26.871 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:26 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:26.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:26.871 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:26 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:26.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:26.871 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:26 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:26.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:26.872 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:26 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:26.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:26.872 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:26 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:26.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:26.872 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:26 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:26.617+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:27.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:27 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:27.263+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:29.724305+0000 front 2024-09-19T04:35:29.724285+0000 (oldest deadline 2024-09-19T04:35:54.424209+0000) 2024-09-19T04:36:27.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:27 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:27.263+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:35.525109+0000 front 2024-09-19T04:35:35.525078+0000 (oldest deadline 2024-09-19T04:36:00.225252+0000) 2024-09-19T04:36:27.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:27 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:27.263+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:44.926301+0000 front 2024-09-19T04:35:44.926287+0000 (oldest deadline 2024-09-19T04:36:07.826085+0000) 2024-09-19T04:36:27.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:27 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:27.263+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:53.028197+0000 front 2024-09-19T04:35:53.028208+0000 (oldest deadline 2024-09-19T04:36:17.128040+0000) 2024-09-19T04:36:27.475 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:27 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-19T04:36:27.263+0000 7f9c912bd640 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.029060+0000 front 2024-09-19T04:36:00.029093+0000 (oldest deadline 2024-09-19T04:36:24.729301+0000) 2024-09-19T04:36:27.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:27 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:27.091+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:27.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:27 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:27.091+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:27.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:27 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:27.091+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:27.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:27 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:27.091+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:27.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:27 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:27.091+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:27.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:27.392+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:27.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:27.392+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:27.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:27.392+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:27.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:27.392+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:27.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:27.392+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:27.732 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:27 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:27.392+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.800183+0000 front 2024-09-19T04:36:00.800168+0000 (oldest deadline 2024-09-19T04:36:26.700040+0000) 2024-09-19T04:36:27.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:27 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:27.666+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:27.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:27 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:27.666+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:27.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:27 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:27.666+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:27.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:27 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:27.666+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:27.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:27 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:27.666+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:27.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:27 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:27.666+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:28.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:28 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:28.126+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:28.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:28 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:28.126+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:28.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:28 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:28.126+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:28.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:28 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:28.126+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:28.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:28 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:28.126+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:28.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:28 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:28.382+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:28.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:28 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:28.382+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:28.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:28 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:28.382+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:28.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:28 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:28.382+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:28.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:28 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:28.382+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:28.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:28 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:28.382+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.800183+0000 front 2024-09-19T04:36:00.800168+0000 (oldest deadline 2024-09-19T04:36:26.700040+0000) 2024-09-19T04:36:28.864 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:28 smithi145 podman[72472]: 2024-09-19 04:36:28.588260041 +0000 UTC m=+5.574663300 container died bda2c4e8f7856cb1ad83ee4e8413b45ff2a7c3c4864223bd500ecde6683ba629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T04:36:28.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:28 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:28.632+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:28.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:28 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:28.632+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:28.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:28 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:28.632+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:28.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:28 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:28.632+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:28.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:28 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:28.632+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:28.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:28 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:28.632+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:29.138 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:28 smithi145 podman[72472]: 2024-09-19 04:36:28.946679615 +0000 UTC m=+5.933082867 container cleanup bda2c4e8f7856cb1ad83ee4e8413b45ff2a7c3c4864223bd500ecde6683ba629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:36:29.138 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:28 smithi145 bash[72472]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8 2024-09-19T04:36:29.474 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:29 smithi145 podman[72491]: 2024-09-19 04:36:29.214783093 +0000 UTC m=+0.626460726 container remove bda2c4e8f7856cb1ad83ee4e8413b45ff2a7c3c4864223bd500ecde6683ba629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T04:36:29.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:29 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:29.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:29.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:29 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:29.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:29.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:29 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:29.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:29.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:29 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:29.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:29.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:29 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:29.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:29.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:29 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:29.410+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:29.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:29 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:29.410+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:29.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:29 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:29.410+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:29.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:29 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:29.410+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:29.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:29 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:29.410+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:29.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:29 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:29.410+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.800183+0000 front 2024-09-19T04:36:00.800168+0000 (oldest deadline 2024-09-19T04:36:26.700040+0000) 2024-09-19T04:36:29.860 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:29 smithi145 podman[72581]: 2024-09-19 04:36:29.470740675 +0000 UTC m=+0.015810813 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:36:29.861 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:29 smithi145 podman[72581]: 2024-09-19 04:36:29.573986864 +0000 UTC m=+0.119057042 container create 2dae5affcd29e9818b228cd71740d236214f5391cb69717c3e851b428474b8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T04:36:29.861 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:29 smithi145 podman[72581]: 2024-09-19 04:36:29.860065809 +0000 UTC m=+0.405135942 container init 2dae5affcd29e9818b228cd71740d236214f5391cb69717c3e851b428474b8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T04:36:29.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:29 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:29.660+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:29.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:29 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:29.660+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:29.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:29 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:29.660+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:29.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:29 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:29.660+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:29.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:29 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:29.660+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:29.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:29 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:29.660+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:30.141 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:29 smithi145 podman[72581]: 2024-09-19 04:36:29.863791443 +0000 UTC m=+0.408861576 container start 2dae5affcd29e9818b228cd71740d236214f5391cb69717c3e851b428474b8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T04:36:30.141 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:29 smithi145 podman[72581]: 2024-09-19 04:36:29.943344762 +0000 UTC m=+0.488414895 container attach 2dae5affcd29e9818b228cd71740d236214f5391cb69717c3e851b428474b8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T04:36:30.141 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 19 04:36:30 smithi145 podman[72581]: 2024-09-19 04:36:30.0501786 +0000 UTC m=+0.595248733 container died 2dae5affcd29e9818b228cd71740d236214f5391cb69717c3e851b428474b8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T04:36:30.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:30 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:30.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:30.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:30 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:30.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:30.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:30 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:30.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:30.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:30 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:30.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:30.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:30 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:30.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:30.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:30 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:30.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:30.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T04:36:30.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:30.627 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T04:36:30.627 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T04:36:30.627 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.9 2024-09-19T04:36:30.681 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:30 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:30.370+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:30.681 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:30 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:30.370+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:30.681 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:30 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:30.370+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:30.681 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:30 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:30.370+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:30.681 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:30 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:30.370+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:30.682 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:30 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:30.370+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.800183+0000 front 2024-09-19T04:36:00.800168+0000 (oldest deadline 2024-09-19T04:36:26.700040+0000) 2024-09-19T04:36:30.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:30 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:30.637+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:30.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:30 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:30.637+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:30.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:30 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:30.637+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:30.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:30 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:30.637+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:30.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:30 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:30.637+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:30.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:30 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:30.637+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:30.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:30 smithi107 systemd[1]: Stopping Ceph osd.9 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:31.347 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:31.055+0000 7f8114800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:31.347 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:31.055+0000 7f8114800640 -1 osd.9 103 *** Got signal Terminated *** 2024-09-19T04:36:31.347 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:31.055+0000 7f8114800640 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:36:31.348 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:31.346+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:31.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:31 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:31.100+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:31.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:31 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:31.100+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:31.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:31 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:31.100+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:31.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:31 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:31.100+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:31.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:31 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:31.100+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:31.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:31 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:31.100+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:31.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:31.346+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:31.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:31.346+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:31.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:31.346+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:31.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:31.346+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:31.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:31 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:31.346+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.800183+0000 front 2024-09-19T04:36:00.800168+0000 (oldest deadline 2024-09-19T04:36:26.700040+0000) 2024-09-19T04:36:31.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:31 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:31.633+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:31.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:31 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:31.633+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:31.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:31 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:31.633+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:31.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:31 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:31.633+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:31.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:31 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:31.633+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:31.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:31 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:31.633+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:32.476 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:32 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:32.116+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:32.477 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:32 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:32.116+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:32.477 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:32 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:32.116+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:32.477 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:32 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:32.116+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:32.477 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:32 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:32.116+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:32.477 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:32 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:32.116+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:32.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:32 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:32.318+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:32.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:32 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:32.318+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:32.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:32 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:32.318+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:32.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:32 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:32.318+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:32.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:32 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:32.318+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:32.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:32 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:32.318+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.800183+0000 front 2024-09-19T04:36:00.800168+0000 (oldest deadline 2024-09-19T04:36:26.700040+0000) 2024-09-19T04:36:32.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:32 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:32.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:32.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:32 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:32.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:32.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:32 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:32.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:32.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:32 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:32.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:32.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:32 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:32.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:32.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:32 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:32.656+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:33.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:33 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:33.148+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:33.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:33 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:33.148+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:33.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:33 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:33.148+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:33.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:33 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:33.148+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:33.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:33 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:33.148+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:33.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:33 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:33.148+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:33.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:33 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:33.354+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:33.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:33 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:33.354+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:33.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:33 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:33.354+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:33.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:33 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:33.354+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:33.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:33 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:33.354+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:33.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:33 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:33.354+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.800183+0000 front 2024-09-19T04:36:00.800168+0000 (oldest deadline 2024-09-19T04:36:26.700040+0000) 2024-09-19T04:36:33.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:33 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:33.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:33.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:33 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:33.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:33.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:33 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:33.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:33.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:33 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:33.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:33.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:33 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:33.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:33.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:33 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:33.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:33.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:33 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:33.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:34.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:34 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:34.132+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:34.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:34 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:34.132+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:34.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:34 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:34.132+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:34.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:34 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:34.132+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:34.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:34 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:34.132+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:34.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:34 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:34.132+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:34.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:34.384+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:34.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:34.384+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:34.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:34.384+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:34.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:34.384+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:34.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:34.384+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:34.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:34 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:34.384+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.800183+0000 front 2024-09-19T04:36:00.800168+0000 (oldest deadline 2024-09-19T04:36:26.700040+0000) 2024-09-19T04:36:34.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:34 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:34.686+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:34.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:34 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:34.686+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:34.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:34 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:34.686+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:34.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:34 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:34.686+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:34.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:34 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:34.686+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:34.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:34 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:34.686+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:34.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:34 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:34.686+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:35.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:35 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:35.173+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:35.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:35 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:35.173+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:35.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:35 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:35.173+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:35.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:35 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:35.173+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:35.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:35 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:35.173+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:35.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:35 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:35.173+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:35.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:35 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:35.426+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.893002+0000 front 2024-09-19T04:35:21.893210+0000 (oldest deadline 2024-09-19T04:35:45.993091+0000) 2024-09-19T04:36:35.731 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:35 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:35.426+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:26.494499+0000 front 2024-09-19T04:35:26.494470+0000 (oldest deadline 2024-09-19T04:35:51.794548+0000) 2024-09-19T04:36:35.732 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:35 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:35.426+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:36.396286+0000 front 2024-09-19T04:35:36.396402+0000 (oldest deadline 2024-09-19T04:36:01.696316+0000) 2024-09-19T04:36:35.732 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:35 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:35.426+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:45.197159+0000 front 2024-09-19T04:35:45.197128+0000 (oldest deadline 2024-09-19T04:36:08.697136+0000) 2024-09-19T04:36:35.732 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:35 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:35.426+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:54.899668+0000 front 2024-09-19T04:35:54.899649+0000 (oldest deadline 2024-09-19T04:36:20.799591+0000) 2024-09-19T04:36:35.733 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:35 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9[64677]: 2024-09-19T04:36:35.426+0000 7f810b200640 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:00.800183+0000 front 2024-09-19T04:36:00.800168+0000 (oldest deadline 2024-09-19T04:36:26.700040+0000) 2024-09-19T04:36:36.196 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:35 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:35.706+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:36.196 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:35 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:35.706+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:36.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:35 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:35.706+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:36.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:35 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:35.706+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:36.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:35 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:35.706+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:36.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:35 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:35.706+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:36.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:35 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:35.706+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:36.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:36 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:36.144+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:36.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:36 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:36.144+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:36.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:36 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:36.144+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:36.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:36 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:36.144+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:36.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:36 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:36.144+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:36.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:36 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:36.144+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:36.730 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:36 smithi107 podman[112024]: 2024-09-19 04:36:36.473048071 +0000 UTC m=+5.558090057 container died c3f86b136dc7be267336ee736a85654725e32980428412c38879cba2c6ca5401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T04:36:36.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:36 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:36.687+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:36.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:36 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:36.687+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:36.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:36 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:36.687+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:36.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:36 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:36.687+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:36.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:36 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:36.687+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:36.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:36 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:36.687+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:36.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:36 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:36.687+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:37.179 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:36 smithi107 podman[112024]: 2024-09-19 04:36:36.902425574 +0000 UTC m=+5.987467560 container cleanup c3f86b136dc7be267336ee736a85654725e32980428412c38879cba2c6ca5401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T04:36:37.179 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:36 smithi107 bash[112024]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9 2024-09-19T04:36:37.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:37 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:37.187+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:37.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:37 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:37.187+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:37.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:37 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:37.187+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:37.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:37 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:37.187+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:37.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:37 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:37.187+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:37.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:37 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:37.187+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:37.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:37 smithi107 podman[112041]: 2024-09-19 04:36:37.178563877 +0000 UTC m=+0.690704945 container remove c3f86b136dc7be267336ee736a85654725e32980428412c38879cba2c6ca5401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T04:36:37.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:37 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:37.653+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:37.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:37 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:37.653+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:37.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:37 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:37.653+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:37.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:37 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:37.653+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:37.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:37 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:37.653+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:37.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:37 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:37.653+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:37.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:37 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:37.653+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:37.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:37 smithi107 podman[112128]: 2024-09-19 04:36:37.62248913 +0000 UTC m=+0.027223352 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:36:37.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:37 smithi107 podman[112128]: 2024-09-19 04:36:37.73781029 +0000 UTC m=+0.142544501 container create 92109706da05d67bc6efeee47a272168d5abca181efe4c26fc873adcef586bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T04:36:37.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:37 smithi107 podman[112128]: 2024-09-19 04:36:37.956983968 +0000 UTC m=+0.361718180 container init 92109706da05d67bc6efeee47a272168d5abca181efe4c26fc873adcef586bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T04:36:37.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:37 smithi107 podman[112128]: 2024-09-19 04:36:37.963689553 +0000 UTC m=+0.368423765 container start 92109706da05d67bc6efeee47a272168d5abca181efe4c26fc873adcef586bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T04:36:38.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:38 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:38.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:38.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:38 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:38.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:38.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:38 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:38.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:38.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:38 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:38.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:38.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:38 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:38.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:38.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:38 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:38.140+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:38.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:38 smithi107 podman[112128]: 2024-09-19 04:36:38.073325634 +0000 UTC m=+0.478059862 container attach 92109706da05d67bc6efeee47a272168d5abca181efe4c26fc873adcef586bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T04:36:38.480 INFO:journalctl@ceph.osd.9.smithi107.stdout:Sep 19 04:36:38 smithi107 podman[112128]: 2024-09-19 04:36:38.215472737 +0000 UTC m=+0.620206952 container died 92109706da05d67bc6efeee47a272168d5abca181efe4c26fc873adcef586bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:36:38.838 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T04:36:38.916 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:38.917 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T04:36:38.917 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T04:36:38.917 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.10 2024-09-19T04:36:38.944 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:38 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:38.667+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:38.945 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:38 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:38.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:38.945 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:38 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:38.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:38.945 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:38 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:38.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:38.945 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:38 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:38.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:38.945 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:38 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:38.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:38.945 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:38 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:38.668+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:39.196 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:38 smithi115 systemd[1]: Stopping Ceph osd.10 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:39.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:39 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:39.134+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:39.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:39 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:39.134+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:39.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:39 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:39.134+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:39.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:39 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:39.134+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:39.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:39 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:39.134+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:39.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:39 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:39.134+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:39.646 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:39 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:39.223+0000 7f168b1df640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:39.646 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:39 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:39.223+0000 7f168b1df640 -1 osd.10 103 *** Got signal Terminated *** 2024-09-19T04:36:39.646 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:39 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:39.223+0000 7f168b1df640 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:36:39.646 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:39 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:39.644+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:39.647 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:39 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:39.644+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:39.647 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:39 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:39.644+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:39.647 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:39 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:39.644+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:39.647 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:39 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:39.644+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:39.647 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:39 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:39.644+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:39.647 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:39 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:39.644+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:40.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:40 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:40.165+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:40.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:40 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:40.165+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:40.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:40 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:40.165+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:40.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:40 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:40.165+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:40.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:40 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:40.165+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:40.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:40 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:40.165+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:40.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:40 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:40.641+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:40.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:40 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:40.641+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:40.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:40 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:40.641+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:40.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:40 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:40.641+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:40.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:40 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:40.641+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:40.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:40 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:40.641+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:40.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:40 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:40.641+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:41.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:41 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:41.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:41.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:41 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:41.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:41.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:41 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:41.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:41.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:41 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:41.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:41.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:41 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:41.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:41.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:41 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:41.136+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:41.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:41 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:41.597+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:41.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:41 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:41.597+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:41.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:41 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:41.597+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:41.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:41 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:41.597+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:41.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:41 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:41.597+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:41.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:41 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:41.597+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:41.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:41 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:41.597+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:42.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:42 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:42.092+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:42.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:42 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:42.092+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:42.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:42 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:42.092+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:42.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:42 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:42.092+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:42.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:42 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:42.092+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:42.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:42 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:42.092+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:42.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:42 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:42.576+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:42.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:42 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:42.576+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:42.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:42 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:42.576+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:42.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:42 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:42.576+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:42.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:42 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:42.576+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:42.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:42 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:42.576+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:42.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:42 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:42.576+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:43.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:43 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:43.074+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:43.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:43 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:43.074+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:43.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:43 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:43.074+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:43.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:43 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:43.074+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:43.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:43 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:43.074+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:43.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:43 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:43.074+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:43.946 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:43 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:43.554+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:21.633672+0000 front 2024-09-19T04:35:21.633600+0000 (oldest deadline 2024-09-19T04:35:47.533605+0000) 2024-09-19T04:36:43.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:43 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:43.554+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:27.534083+0000 front 2024-09-19T04:35:27.534060+0000 (oldest deadline 2024-09-19T04:35:52.834209+0000) 2024-09-19T04:36:43.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:43 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:43.554+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:37.435929+0000 front 2024-09-19T04:35:37.436027+0000 (oldest deadline 2024-09-19T04:36:02.735989+0000) 2024-09-19T04:36:43.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:43 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:43.554+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:46.837376+0000 front 2024-09-19T04:35:46.837395+0000 (oldest deadline 2024-09-19T04:36:10.937302+0000) 2024-09-19T04:36:43.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:43 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:43.554+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:50.937733+0000 front 2024-09-19T04:35:50.937755+0000 (oldest deadline 2024-09-19T04:36:16.837885+0000) 2024-09-19T04:36:43.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:43 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:43.554+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:02.739094+0000 front 2024-09-19T04:36:02.738939+0000 (oldest deadline 2024-09-19T04:36:24.438780+0000) 2024-09-19T04:36:43.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:43 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10[59908]: 2024-09-19T04:36:43.554+0000 7f16837ed640 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-09-19T04:36:09.040388+0000 front 2024-09-19T04:36:09.040329+0000 (oldest deadline 2024-09-19T04:36:33.140228+0000) 2024-09-19T04:36:44.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:44 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:44.062+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:44.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:44 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:44.062+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:44.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:44 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:44.062+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:44.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:44 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:44.062+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:44.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:44 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:44.062+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:44.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:44 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:44.062+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:45.041 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:44 smithi115 podman[73290]: 2024-09-19 04:36:44.748549774 +0000 UTC m=+5.639730257 container died b031c91309225cd77779c1db10efac06b8d0bdcc1b25a4056e0a591a3372f744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:36:45.374 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:45 smithi115 podman[73290]: 2024-09-19 04:36:45.123040585 +0000 UTC m=+6.014221060 container cleanup b031c91309225cd77779c1db10efac06b8d0bdcc1b25a4056e0a591a3372f744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T04:36:45.375 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:45 smithi115 bash[73290]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10 2024-09-19T04:36:45.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:45 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:45.036+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:45.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:45 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:45.036+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:45.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:45 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:45.036+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:45.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:45 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:45.036+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:45.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:45 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:45.036+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:45.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:45 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:45.036+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:45.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:45 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:45.036+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-19T04:36:18.535463+0000 front 2024-09-19T04:36:18.535435+0000 (oldest deadline 2024-09-19T04:36:44.435319+0000) 2024-09-19T04:36:45.696 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:45 smithi115 podman[73306]: 2024-09-19 04:36:45.374013775 +0000 UTC m=+0.623927754 container remove b031c91309225cd77779c1db10efac06b8d0bdcc1b25a4056e0a591a3372f744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T04:36:46.053 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:45 smithi115 podman[73398]: 2024-09-19 04:36:45.631889038 +0000 UTC m=+0.015559026 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:36:46.053 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:45 smithi115 podman[73398]: 2024-09-19 04:36:45.741470251 +0000 UTC m=+0.125140269 container create 89ac97a8ecae4e9d5506d27f2f5a8e63506272e05210840484ac93eefea0b58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T04:36:46.053 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:45 smithi115 podman[73398]: 2024-09-19 04:36:45.968585288 +0000 UTC m=+0.352255264 container init 89ac97a8ecae4e9d5506d27f2f5a8e63506272e05210840484ac93eefea0b58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:36:46.053 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:45 smithi115 podman[73398]: 2024-09-19 04:36:45.971886496 +0000 UTC m=+0.355556491 container start 89ac97a8ecae4e9d5506d27f2f5a8e63506272e05210840484ac93eefea0b58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10-deactivate, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T04:36:46.446 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:46 smithi115 podman[73398]: 2024-09-19 04:36:46.051834783 +0000 UTC m=+0.435504781 container attach 89ac97a8ecae4e9d5506d27f2f5a8e63506272e05210840484ac93eefea0b58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T04:36:46.446 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:46 smithi115 podman[73398]: 2024-09-19 04:36:46.155118636 +0000 UTC m=+0.538788621 container died 89ac97a8ecae4e9d5506d27f2f5a8e63506272e05210840484ac93eefea0b58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:36:46.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:46 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:46.059+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:46.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:46 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:46.059+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:46.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:46 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:46.059+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:46.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:46 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:46.059+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:46.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:46 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:46.059+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:46.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:46 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:46.059+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:46.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:46 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:46.059+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-19T04:36:18.535463+0000 front 2024-09-19T04:36:18.535435+0000 (oldest deadline 2024-09-19T04:36:44.435319+0000) 2024-09-19T04:36:46.746 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 19 04:36:46 smithi115 podman[73421]: 2024-09-19 04:36:46.730110417 +0000 UTC m=+0.569288017 container remove 89ac97a8ecae4e9d5506d27f2f5a8e63506272e05210840484ac93eefea0b58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:36:46.751 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T04:36:46.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:46.828 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T04:36:46.829 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T04:36:46.829 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.11 2024-09-19T04:36:47.174 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:46 smithi145 systemd[1]: Stopping Ceph osd.11 for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:47.174 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.033+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:47.174 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.033+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:47.175 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.033+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:47.175 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.033+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:47.175 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.033+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:47.175 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.033+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:47.175 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.033+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-19T04:36:18.535463+0000 front 2024-09-19T04:36:18.535435+0000 (oldest deadline 2024-09-19T04:36:44.435319+0000) 2024-09-19T04:36:47.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.172+0000 7f546a1fa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:47.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.172+0000 7f546a1fa640 -1 osd.11 103 *** Got signal Terminated *** 2024-09-19T04:36:47.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.172+0000 7f546a1fa640 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:36:48.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.992+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:48.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.992+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:48.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.992+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:48.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.992+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:48.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.992+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:48.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.992+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:48.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:47 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:47.992+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-19T04:36:18.535463+0000 front 2024-09-19T04:36:18.535435+0000 (oldest deadline 2024-09-19T04:36:44.435319+0000) 2024-09-19T04:36:49.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:48 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:48.978+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:49.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:48 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:48.978+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:49.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:48 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:48.978+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:49.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:48 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:48.978+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:49.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:48 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:48.978+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:49.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:48 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:48.978+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:49.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:48 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:48.978+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-19T04:36:18.535463+0000 front 2024-09-19T04:36:18.535435+0000 (oldest deadline 2024-09-19T04:36:44.435319+0000) 2024-09-19T04:36:50.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:50 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:50.013+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:50.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:50 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:50.013+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:50.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:50 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:50.013+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:50.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:50 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:50.013+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:50.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:50 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:50.013+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:50.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:50 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:50.013+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:50.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:50 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:50.013+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-19T04:36:18.535463+0000 front 2024-09-19T04:36:18.535435+0000 (oldest deadline 2024-09-19T04:36:44.435319+0000) 2024-09-19T04:36:50.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:50 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:50.013+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6820 osd.8 since back 2024-09-19T04:36:24.935957+0000 front 2024-09-19T04:36:24.935988+0000 (oldest deadline 2024-09-19T04:36:49.635958+0000) 2024-09-19T04:36:51.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:51.048+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:51.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:51.048+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:51.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:51.048+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:51.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:51.048+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:51.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:51.048+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:51.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:51.048+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:51.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:51.048+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-19T04:36:18.535463+0000 front 2024-09-19T04:36:18.535435+0000 (oldest deadline 2024-09-19T04:36:44.435319+0000) 2024-09-19T04:36:51.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:51 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:51.048+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6820 osd.8 since back 2024-09-19T04:36:24.935957+0000 front 2024-09-19T04:36:24.935988+0000 (oldest deadline 2024-09-19T04:36:49.635958+0000) 2024-09-19T04:36:52.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:52.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-19T04:35:20.926664+0000 front 2024-09-19T04:35:20.926764+0000 (oldest deadline 2024-09-19T04:35:46.226621+0000) 2024-09-19T04:36:52.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:52.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-19T04:35:30.828383+0000 front 2024-09-19T04:35:30.828414+0000 (oldest deadline 2024-09-19T04:35:53.728370+0000) 2024-09-19T04:36:52.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:52.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-19T04:35:33.729019+0000 front 2024-09-19T04:35:33.729003+0000 (oldest deadline 2024-09-19T04:35:59.628974+0000) 2024-09-19T04:36:52.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:52.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-09-19T04:35:43.630825+0000 front 2024-09-19T04:35:43.630816+0000 (oldest deadline 2024-09-19T04:36:09.530799+0000) 2024-09-19T04:36:52.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:52.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-19T04:35:55.431701+0000 front 2024-09-19T04:35:55.431756+0000 (oldest deadline 2024-09-19T04:36:16.531699+0000) 2024-09-19T04:36:52.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:52.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-09-19T04:36:01.633243+0000 front 2024-09-19T04:36:01.633265+0000 (oldest deadline 2024-09-19T04:36:29.233684+0000) 2024-09-19T04:36:52.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:52.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-19T04:36:18.535463+0000 front 2024-09-19T04:36:18.535435+0000 (oldest deadline 2024-09-19T04:36:44.435319+0000) 2024-09-19T04:36:52.475 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11[59902]: 2024-09-19T04:36:52.057+0000 7f5462808640 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.145:6820 osd.8 since back 2024-09-19T04:36:24.935957+0000 front 2024-09-19T04:36:24.935988+0000 (oldest deadline 2024-09-19T04:36:49.635958+0000) 2024-09-19T04:36:52.799 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 podman[72683]: 2024-09-19 04:36:52.534944939 +0000 UTC m=+5.511928208 container died c30ed192c252f4580a117840bd238a91e5ce30e7901a088709431b4b047ba6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True) 2024-09-19T04:36:53.150 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 podman[72683]: 2024-09-19 04:36:52.906252576 +0000 UTC m=+5.883235834 container cleanup c30ed192c252f4580a117840bd238a91e5ce30e7901a088709431b4b047ba6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T04:36:53.150 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:52 smithi145 bash[72683]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11 2024-09-19T04:36:53.474 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:53 smithi145 podman[72699]: 2024-09-19 04:36:53.148852302 +0000 UTC m=+0.608717718 container remove c30ed192c252f4580a117840bd238a91e5ce30e7901a088709431b4b047ba6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T04:36:53.736 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:53 smithi145 podman[72789]: 2024-09-19 04:36:53.37590662 +0000 UTC m=+0.015763180 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:36:53.736 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:53 smithi145 podman[72789]: 2024-09-19 04:36:53.474573041 +0000 UTC m=+0.114429586 container create 256efa5727099df499e85b4dae525497d37a8129b491be9c6f310e79d00a75fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T04:36:54.206 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:53 smithi145 podman[72789]: 2024-09-19 04:36:53.735635421 +0000 UTC m=+0.375491965 container init 256efa5727099df499e85b4dae525497d37a8129b491be9c6f310e79d00a75fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:36:54.206 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:53 smithi145 podman[72789]: 2024-09-19 04:36:53.738956975 +0000 UTC m=+0.378813520 container start 256efa5727099df499e85b4dae525497d37a8129b491be9c6f310e79d00a75fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T04:36:54.206 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:53 smithi145 podman[72789]: 2024-09-19 04:36:53.810565117 +0000 UTC m=+0.450421675 container attach 256efa5727099df499e85b4dae525497d37a8129b491be9c6f310e79d00a75fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:36:54.206 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:53 smithi145 podman[72789]: 2024-09-19 04:36:53.92359754 +0000 UTC m=+0.563454095 container died 256efa5727099df499e85b4dae525497d37a8129b491be9c6f310e79d00a75fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-19T04:36:54.438 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T04:36:54.462 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:54 smithi145 podman[72812]: 2024-09-19 04:36:54.413900409 +0000 UTC m=+0.484471843 container remove 256efa5727099df499e85b4dae525497d37a8129b491be9c6f310e79d00a75fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T04:36:54.462 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:54 smithi145 systemd[1]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-19T04:36:54.462 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:54 smithi145 systemd[1]: Stopped Ceph osd.11 for e2ec5b0e-763c-11ef-bceb-c7b262605968. 2024-09-19T04:36:54.463 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 19 04:36:54 smithi145 systemd[1]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@osd.11.service: Consumed 38.941s CPU time. 2024-09-19T04:36:54.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:54.478 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T04:36:54.478 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T04:36:54.478 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.a 2024-09-19T04:36:54.882 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:36:54 smithi107 systemd[1]: Stopping Ceph mds.a for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:55.230 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:36:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a[71817]: 2024-09-19T04:36:54.880+0000 7f2f8f200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:55.231 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:36:54 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a[71817]: 2024-09-19T04:36:54.880+0000 7f2f8f200640 -1 mds.a *** got signal Terminated *** 2024-09-19T04:36:56.521 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:36:56 smithi107 podman[112242]: 2024-09-19 04:36:56.255502889 +0000 UTC m=+1.498293293 container died d992b5b44b7c72fbd26ab3023cd88a2752dbecc2e78c9de52656bf47d6ff7d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T04:36:56.980 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:36:56 smithi107 podman[112242]: 2024-09-19 04:36:56.727264059 +0000 UTC m=+1.970054465 container cleanup d992b5b44b7c72fbd26ab3023cd88a2752dbecc2e78c9de52656bf47d6ff7d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T04:36:56.981 INFO:journalctl@ceph.mds.a.smithi107.stdout:Sep 19 04:36:56 smithi107 bash[112242]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-a 2024-09-19T04:36:57.176 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T04:36:57.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:36:57.248 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T04:36:57.248 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T04:36:57.248 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.d 2024-09-19T04:36:57.635 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:36:57 smithi107 systemd[1]: Stopping Ceph mds.d for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:36:57.980 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:36:57 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-d[70649]: 2024-09-19T04:36:57.633+0000 7fb81d400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:36:57.981 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:36:57 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-d[70649]: 2024-09-19T04:36:57.633+0000 7fb81d400640 -1 mds.d *** got signal Terminated *** 2024-09-19T04:36:59.233 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:36:58 smithi107 podman[112363]: 2024-09-19 04:36:58.823040759 +0000 UTC m=+1.309614235 container died 937c61de8fef775e232ebc5b794a9c8cbd27c4f25e167fde361900839d77fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-d, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T04:36:59.730 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:36:59 smithi107 podman[112363]: 2024-09-19 04:36:59.589419687 +0000 UTC m=+2.075993203 container cleanup 937c61de8fef775e232ebc5b794a9c8cbd27c4f25e167fde361900839d77fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-d, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T04:36:59.730 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:36:59 smithi107 bash[112363]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-d 2024-09-19T04:37:00.205 INFO:journalctl@ceph.mds.d.smithi107.stdout:Sep 19 04:36:59 smithi107 podman[112374]: 2024-09-19 04:36:59.890934552 +0000 UTC m=+1.057243090 container remove 937c61de8fef775e232ebc5b794a9c8cbd27c4f25e167fde361900839d77fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T04:37:00.238 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T04:37:00.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:00.308 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T04:37:00.308 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T04:37:00.308 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.g 2024-09-19T04:37:00.681 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:37:00 smithi107 systemd[1]: Stopping Ceph mds.g for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:37:00.980 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:37:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g[71463]: 2024-09-19T04:37:00.679+0000 7f3b97e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:37:00.981 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:37:00 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g[71463]: 2024-09-19T04:37:00.679+0000 7f3b97e00640 -1 mds.g *** got signal Terminated *** 2024-09-19T04:37:04.536 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:37:04 smithi107 podman[112487]: 2024-09-19 04:37:04.249082804 +0000 UTC m=+3.676728410 container died 64d4ddb63aeaec62f4342985a40426da308e43e9edaae814242f11a3a0dc4813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T04:37:05.111 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:37:04 smithi107 podman[112487]: 2024-09-19 04:37:04.823976541 +0000 UTC m=+4.251622144 container cleanup 64d4ddb63aeaec62f4342985a40426da308e43e9edaae814242f11a3a0dc4813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T04:37:05.112 INFO:journalctl@ceph.mds.g.smithi107.stdout:Sep 19 04:37:04 smithi107 bash[112487]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-g 2024-09-19T04:37:05.309 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T04:37:05.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:05.379 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T04:37:05.379 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T04:37:05.379 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.j 2024-09-19T04:37:05.730 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:37:05 smithi107 systemd[1]: Stopping Ceph mds.j for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:37:06.230 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:37:05 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-j[70262]: 2024-09-19T04:37:05.766+0000 7fdc93400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:37:06.231 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:37:05 smithi107 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-j[70262]: 2024-09-19T04:37:05.766+0000 7fdc93400640 -1 mds.j *** got signal Terminated *** 2024-09-19T04:37:07.144 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:37:06 smithi107 podman[112605]: 2024-09-19 04:37:06.835674503 +0000 UTC m=+1.185776812 container died 2b706c582c8015970dda86c97f6f6ec40e36fbc3cfa6999f03f71765ee1f555b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T04:37:07.474 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:37:07 smithi107 podman[112605]: 2024-09-19 04:37:07.254971932 +0000 UTC m=+1.605074241 container cleanup 2b706c582c8015970dda86c97f6f6ec40e36fbc3cfa6999f03f71765ee1f555b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:37:07.474 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:37:07 smithi107 bash[112605]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-j 2024-09-19T04:37:07.730 INFO:journalctl@ceph.mds.j.smithi107.stdout:Sep 19 04:37:07 smithi107 podman[112617]: 2024-09-19 04:37:07.472719739 +0000 UTC m=+0.620080212 container remove 2b706c582c8015970dda86c97f6f6ec40e36fbc3cfa6999f03f71765ee1f555b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T04:37:07.743 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T04:37:07.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:07.811 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T04:37:07.811 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T04:37:07.811 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.b 2024-09-19T04:37:08.131 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:37:07 smithi115 systemd[1]: Stopping Ceph mds.b for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:37:08.446 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:37:08 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b[65574]: 2024-09-19T04:37:08.129+0000 7f7e4c9a9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:37:08.446 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:37:08 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b[65574]: 2024-09-19T04:37:08.129+0000 7f7e4c9a9640 -1 mds.b *** got signal Terminated *** 2024-09-19T04:37:12.430 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:37:12 smithi115 podman[73514]: 2024-09-19 04:37:12.134858699 +0000 UTC m=+4.131226370 container died 9d1512338f7a6d07e0370e84d348593cba05d85872bc4eb9f75885047a54c8d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T04:37:12.696 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:37:12 smithi115 podman[73514]: 2024-09-19 04:37:12.528876019 +0000 UTC m=+4.525243688 container cleanup 9d1512338f7a6d07e0370e84d348593cba05d85872bc4eb9f75885047a54c8d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:37:12.696 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 19 04:37:12 smithi115 bash[73514]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-b 2024-09-19T04:37:13.030 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T04:37:13.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:13.066 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T04:37:13.066 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T04:37:13.066 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.e 2024-09-19T04:37:13.196 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:37:13 smithi115 systemd[1]: Stopping Ceph mds.e for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:37:13.696 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:37:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e[64955]: 2024-09-19T04:37:13.381+0000 7f561bcf2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:37:13.697 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:37:13 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e[64955]: 2024-09-19T04:37:13.381+0000 7f561bcf2640 -1 mds.e *** got signal Terminated *** 2024-09-19T04:37:16.696 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:37:16 smithi115 podman[73635]: 2024-09-19 04:37:16.229742171 +0000 UTC m=+2.952579782 container died 2e28e55b62ee6afc50789f74f93b2e31066df47283c84c1fc0ca8fbac19d36ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T04:37:17.075 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:37:16 smithi115 podman[73635]: 2024-09-19 04:37:16.832273547 +0000 UTC m=+3.555111152 container cleanup 2e28e55b62ee6afc50789f74f93b2e31066df47283c84c1fc0ca8fbac19d36ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T04:37:17.075 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 19 04:37:16 smithi115 bash[73635]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-e 2024-09-19T04:37:17.218 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T04:37:17.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:17.250 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T04:37:17.250 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T04:37:17.251 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.h 2024-09-19T04:37:17.696 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:37:17 smithi115 systemd[1]: Stopping Ceph mds.h for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:37:17.696 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:37:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h[65274]: 2024-09-19T04:37:17.576+0000 7f7e7d869640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:37:17.697 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:37:17 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h[65274]: 2024-09-19T04:37:17.576+0000 7f7e7d869640 -1 mds.h *** got signal Terminated *** 2024-09-19T04:37:20.586 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:37:20 smithi115 podman[73756]: 2024-09-19 04:37:20.219737118 +0000 UTC m=+2.745058858 container died 00340df01aeec6a96c6d7454c70c04f497c8cff70ddf0b47afd9ca3c35b94d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T04:37:20.946 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:37:20 smithi115 podman[73756]: 2024-09-19 04:37:20.668457866 +0000 UTC m=+3.193779606 container cleanup 00340df01aeec6a96c6d7454c70c04f497c8cff70ddf0b47afd9ca3c35b94d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-19T04:37:20.946 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 19 04:37:20 smithi115 bash[73756]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-h 2024-09-19T04:37:21.094 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T04:37:21.126 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:21.126 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T04:37:21.126 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T04:37:21.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.k 2024-09-19T04:37:21.465 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:37:21 smithi115 systemd[1]: Stopping Ceph mds.k for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:37:21.946 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:37:21 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k[64644]: 2024-09-19T04:37:21.463+0000 7f6728eff640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:37:21.946 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:37:21 smithi115 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k[64644]: 2024-09-19T04:37:21.463+0000 7f6728eff640 -1 mds.k *** got signal Terminated *** 2024-09-19T04:37:24.582 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:37:24 smithi115 podman[73874]: 2024-09-19 04:37:24.229670234 +0000 UTC m=+2.891322442 container died f6e1855e54d27240a15cca636d2406415a8f64c60cff6384ffdd353497f48b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T04:37:24.946 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:37:24 smithi115 podman[73874]: 2024-09-19 04:37:24.688446324 +0000 UTC m=+3.350098532 container cleanup f6e1855e54d27240a15cca636d2406415a8f64c60cff6384ffdd353497f48b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T04:37:24.946 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 19 04:37:24 smithi115 bash[73874]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-k 2024-09-19T04:37:25.108 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T04:37:25.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:25.140 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T04:37:25.140 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T04:37:25.140 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.c 2024-09-19T04:37:25.431 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:37:25 smithi145 systemd[1]: Stopping Ceph mds.c for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:37:25.684 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:37:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c[65180]: 2024-09-19T04:37:25.428+0000 7f0a2e32e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:37:25.684 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:37:25 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c[65180]: 2024-09-19T04:37:25.428+0000 7f0a2e32e640 -1 mds.c *** got signal Terminated *** 2024-09-19T04:37:28.474 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:37:28 smithi145 podman[72907]: 2024-09-19 04:37:28.219995283 +0000 UTC m=+2.889755933 container died 1e3bf76d9ee6e563a0e3bb3a798405900868d2ec46aa42384ed30768453725b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T04:37:28.839 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:37:28 smithi145 podman[72907]: 2024-09-19 04:37:28.662795981 +0000 UTC m=+3.332556675 container cleanup 1e3bf76d9ee6e563a0e3bb3a798405900868d2ec46aa42384ed30768453725b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T04:37:28.839 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:37:28 smithi145 bash[72907]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c 2024-09-19T04:37:29.089 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 19 04:37:28 smithi145 podman[72931]: 2024-09-19 04:37:28.838845667 +0000 UTC m=+0.608036102 container remove 1e3bf76d9ee6e563a0e3bb3a798405900868d2ec46aa42384ed30768453725b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T04:37:29.093 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T04:37:29.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:29.129 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T04:37:29.130 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T04:37:29.130 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.f 2024-09-19T04:37:29.451 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:37:29 smithi145 systemd[1]: Stopping Ceph mds.f for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:37:29.724 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:37:29 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f[64878]: 2024-09-19T04:37:29.449+0000 7f2e0fb11640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:37:29.724 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:37:29 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f[64878]: 2024-09-19T04:37:29.449+0000 7f2e0fb11640 -1 mds.f *** got signal Terminated *** 2024-09-19T04:37:32.518 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:37:32 smithi145 podman[73041]: 2024-09-19 04:37:32.228099104 +0000 UTC m=+2.879395905 container died 8fe700ea027eaa3af1cb476e5ec541d1c606b1f63fccf024629387aee9ba99b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T04:37:32.826 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:37:32 smithi145 podman[73041]: 2024-09-19 04:37:32.591132056 +0000 UTC m=+3.242428855 container cleanup 8fe700ea027eaa3af1cb476e5ec541d1c606b1f63fccf024629387aee9ba99b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T04:37:32.826 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 19 04:37:32 smithi145 bash[73041]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-f 2024-09-19T04:37:32.967 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T04:37:33.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:33.001 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T04:37:33.001 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T04:37:33.001 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.i 2024-09-19T04:37:33.474 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:37:33 smithi145 systemd[1]: Stopping Ceph mds.i for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:37:33.474 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:37:33 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i[65493]: 2024-09-19T04:37:33.319+0000 7f6c5f656640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:37:33.475 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:37:33 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i[65493]: 2024-09-19T04:37:33.319+0000 7f6c5f656640 -1 mds.i *** got signal Terminated *** 2024-09-19T04:37:36.455 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:37:36 smithi145 podman[73163]: 2024-09-19 04:37:36.229277257 +0000 UTC m=+3.013458104 container died ce9d8167bf21a30d43f441b1fc7acad21ec28c731f75dc1697212f9832da6af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T04:37:36.707 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:37:36 smithi145 podman[73163]: 2024-09-19 04:37:36.553046471 +0000 UTC m=+3.337227310 container cleanup ce9d8167bf21a30d43f441b1fc7acad21ec28c731f75dc1697212f9832da6af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T04:37:36.708 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:37:36 smithi145 bash[73163]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i 2024-09-19T04:37:36.974 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 19 04:37:36 smithi145 podman[73175]: 2024-09-19 04:37:36.746002009 +0000 UTC m=+0.519855050 container remove ce9d8167bf21a30d43f441b1fc7acad21ec28c731f75dc1697212f9832da6af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T04:37:36.980 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T04:37:37.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:37.011 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T04:37:37.012 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T04:37:37.012 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.l 2024-09-19T04:37:37.341 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:37:37 smithi145 systemd[1]: Stopping Ceph mds.l for e2ec5b0e-763c-11ef-bceb-c7b262605968... 2024-09-19T04:37:37.724 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:37:37 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l[64577]: 2024-09-19T04:37:37.339+0000 7f195f089640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:37:37.724 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:37:37 smithi145 ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l[64577]: 2024-09-19T04:37:37.339+0000 7f195f089640 -1 mds.l *** got signal Terminated *** 2024-09-19T04:37:40.474 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:37:40 smithi145 podman[73283]: 2024-09-19 04:37:40.217978547 +0000 UTC m=+2.990407368 container died 59f8105a5daa102163b90fb0d2b3653ad4e6aa053faf174b43f2029921cd73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T04:37:40.915 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:37:40 smithi145 podman[73283]: 2024-09-19 04:37:40.565183792 +0000 UTC m=+3.337612571 container cleanup 59f8105a5daa102163b90fb0d2b3653ad4e6aa053faf174b43f2029921cd73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:37:40.915 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:37:40 smithi145 bash[73283]: ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l 2024-09-19T04:37:40.915 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 19 04:37:40 smithi145 podman[73296]: 2024-09-19 04:37:40.707978927 +0000 UTC m=+0.480285436 container remove 59f8105a5daa102163b90fb0d2b3653ad4e6aa053faf174b43f2029921cd73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T04:37:40.932 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e2ec5b0e-763c-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T04:37:40.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:37:40.964 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T04:37:40.964 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T04:37:41.240 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:37:45.143 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T04:37:45.349 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:37:47.972 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T04:37:48.171 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:37:50.527 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:50.591 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:50.624 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:37:50.651 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T04:37:50.653 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627/remote/smithi107/crash 2024-09-19T04:37:50.654 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/crash -- . 2024-09-19T04:37:50.719 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T04:37:50.719 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-09-19T04:37:50.722 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627/remote/smithi115/crash 2024-09-19T04:37:50.723 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/crash -- . 2024-09-19T04:37:50.765 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T04:37:50.765 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2024-09-19T04:37:50.767 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627/remote/smithi145/crash 2024-09-19T04:37:50.768 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/crash -- . 2024-09-19T04:37:50.801 INFO:teuthology.orchestra.run.smithi145.stderr:tar: /var/lib/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T04:37:50.802 INFO:teuthology.orchestra.run.smithi145.stderr:tar: Error is not recoverable: exiting now 2024-09-19T04:37:50.803 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T04:37:50.803 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T04:37:50.887 INFO:tasks.cephadm:Compressing logs... 2024-09-19T04:37:50.887 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:37:50.891 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:37:50.894 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:37:50.919 INFO:teuthology.orchestra.run.smithi145.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T04:37:50.919 INFO:teuthology.orchestra.run.smithi145.stderr:: No such file or directory 2024-09-19T04:37:50.919 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T04:37:50.919 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T04:37:50.919 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T04:37:50.920 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T04:37:50.922 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T04:37:50.922 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T04:37:50.922 INFO:teuthology.orchestra.run.smithi145.stderr: 82.1% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T04:37:50.922 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T04:37:50.922 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.log 2024-09-19T04:37:50.923 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T04:37:50.923 INFO:teuthology.orchestra.run.smithi115.stderr:: No such file or directory 2024-09-19T04:37:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T04:37:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T04:37:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T04:37:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T04:37:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T04:37:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T04:37:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.b.log: /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T04:37:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.4.log: /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T04:37:50.925 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T04:37:50.925 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.log: 90.6% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T04:37:50.925 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T04:37:50.925 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.l.log: 87.3% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T04:37:50.926 INFO:teuthology.orchestra.run.smithi115.stderr: 90.6% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T04:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T04:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.cephadm.log: 82.2% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T04:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.log 2024-09-19T04:37:50.934 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T04:37:50.934 INFO:teuthology.orchestra.run.smithi145.stderr: 91.0% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T04:37:50.935 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mgr.y.log: 91.0% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T04:37:50.937 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T04:37:50.938 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T04:37:50.939 INFO:teuthology.orchestra.run.smithi115.stderr: 92.1% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T04:37:50.941 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T04:37:50.941 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T04:37:50.942 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-09-19T04:37:50.942 INFO:teuthology.orchestra.run.smithi107.stderr: -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T04:37:50.942 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T04:37:50.943 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T04:37:50.943 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T04:37:50.944 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T04:37:50.945 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.h.log: 87.3% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T04:37:50.945 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T04:37:50.946 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T04:37:50.946 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T04:37:50.948 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T04:37:50.949 INFO:teuthology.orchestra.run.smithi107.stderr: 90.4% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T04:37:50.949 INFO:teuthology.orchestra.run.smithi107.stderr: 84.5% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T04:37:50.949 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T04:37:50.950 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.f.log: 92.2% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T04:37:50.950 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.log 2024-09-19T04:37:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.g.log: /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.6.log: /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T04:37:50.954 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T04:37:50.957 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T04:37:50.958 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T04:37:50.959 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mgr.x.log: 91.0% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T04:37:50.962 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T04:37:50.963 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T04:37:50.965 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.j.log: 87.3% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T04:37:50.966 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T04:37:50.973 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T04:37:50.974 INFO:teuthology.orchestra.run.smithi107.stderr: 94.6% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T04:37:50.975 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T04:37:50.990 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T04:37:50.996 INFO:teuthology.orchestra.run.smithi107.stderr: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T04:37:51.418 INFO:teuthology.orchestra.run.smithi107.stderr: 90.3% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T04:37:51.663 INFO:teuthology.orchestra.run.smithi145.stderr: 91.6% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T04:37:51.699 INFO:teuthology.orchestra.run.smithi115.stderr: 91.6% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T04:37:51.811 INFO:teuthology.orchestra.run.smithi107.stderr: 94.9% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T04:37:51.914 INFO:teuthology.orchestra.run.smithi107.stderr: 94.9% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T04:37:52.254 INFO:teuthology.orchestra.run.smithi115.stderr: 95.1% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T04:37:52.566 INFO:teuthology.orchestra.run.smithi107.stderr: 90.8% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T04:37:52.585 INFO:teuthology.orchestra.run.smithi145.stderr: 95.6% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T04:37:54.300 INFO:teuthology.orchestra.run.smithi115.stderr: 95.6% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T04:37:57.687 INFO:teuthology.orchestra.run.smithi107.stderr: 91.2% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T04:37:58.175 INFO:teuthology.orchestra.run.smithi115.stderr: 92.0% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T04:37:58.987 INFO:teuthology.orchestra.run.smithi107.stderr: 92.0% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T04:37:59.057 INFO:teuthology.orchestra.run.smithi107.stderr: 91.2% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T04:37:59.773 INFO:teuthology.orchestra.run.smithi115.stderr: 91.4% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T04:38:00.073 INFO:teuthology.orchestra.run.smithi107.stderr: 91.5% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T04:38:00.076 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-09-19T04:38:00.076 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m9.157s 2024-09-19T04:38:00.077 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m32.775s 2024-09-19T04:38:00.077 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m2.730s 2024-09-19T04:38:00.188 INFO:teuthology.orchestra.run.smithi145.stderr: 91.0% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T04:38:01.789 INFO:teuthology.orchestra.run.smithi115.stderr: 91.5% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T04:38:03.281 INFO:teuthology.orchestra.run.smithi145.stderr: 91.7% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T04:38:04.478 INFO:teuthology.orchestra.run.smithi145.stderr: 90.9% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T04:38:05.505 INFO:teuthology.orchestra.run.smithi145.stderr: 91.3% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T04:38:05.506 INFO:teuthology.orchestra.run.smithi115.stderr: 91.1% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T04:38:06.699 INFO:teuthology.orchestra.run.smithi145.stderr: 94.0% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T04:38:08.451 INFO:teuthology.orchestra.run.smithi145.stderr: 94.0% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T04:38:10.572 INFO:teuthology.orchestra.run.smithi115.stderr: 94.0% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T04:38:18.573 INFO:teuthology.orchestra.run.smithi145.stderr: 94.0% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T04:38:18.574 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-09-19T04:38:18.575 INFO:teuthology.orchestra.run.smithi145.stderr:real 0m27.666s 2024-09-19T04:38:18.575 INFO:teuthology.orchestra.run.smithi145.stderr:user 1m48.868s 2024-09-19T04:38:18.575 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m3.248s 2024-09-19T04:38:44.358 INFO:teuthology.orchestra.run.smithi115.stderr: 93.7% -- replaced with /var/log/ceph/e2ec5b0e-763c-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T04:38:44.359 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-19T04:38:44.360 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m53.451s 2024-09-19T04:38:44.360 INFO:teuthology.orchestra.run.smithi115.stderr:user 1m55.485s 2024-09-19T04:38:44.360 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m3.606s 2024-09-19T04:38:44.360 INFO:tasks.cephadm:Archiving logs... 2024-09-19T04:38:44.361 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627/remote/smithi107/log 2024-09-19T04:38:44.362 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:38:49.379 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627/remote/smithi115/log 2024-09-19T04:38:49.380 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:39:05.351 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627/remote/smithi145/log 2024-09-19T04:39:05.352 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:39:18.807 INFO:tasks.cephadm:Removing cluster... 2024-09-19T04:39:18.808 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 --force 2024-09-19T04:39:19.080 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:39:19.621 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 --force 2024-09-19T04:39:19.823 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:39:20.227 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e2ec5b0e-763c-11ef-bceb-c7b262605968 --force 2024-09-19T04:39:20.428 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: e2ec5b0e-763c-11ef-bceb-c7b262605968 2024-09-19T04:39:20.828 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T04:39:20.829 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T04:39:20.869 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T04:39:20.884 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T04:39:20.901 INFO:tasks.cephadm:Teardown complete 2024-09-19T04:39:20.901 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T04:39:20.916 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T04:39:20.917 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:39:20.921 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:39:20.927 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:39:21.038 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T04:39:21.048 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T04:39:21.048 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:39:21.053 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:39:21.056 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:39:21.067 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:39:21.070 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:39:21.071 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:39:21.072 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-19T04:39:21.072 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +37us[ +41us] +/- 42ms 2024-09-19T04:39:21.072 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 +82us[ +82us] +/- 72ms 2024-09-19T04:39:21.072 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -123us[ -123us] +/- 81ms 2024-09-19T04:39:21.072 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:39:21.074 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:39:21.074 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-09-19T04:39:21.074 INFO:teuthology.orchestra.run.smithi145.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 +40us[ +48us] +/- 42ms 2024-09-19T04:39:21.074 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 +73us[ +82us] +/- 72ms 2024-09-19T04:39:21.074 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -119us[ -119us] +/- 81ms 2024-09-19T04:39:21.074 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:39:21.083 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:39:21.090 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:39:21.090 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-19T04:39:21.090 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +42us[ +46us] +/- 42ms 2024-09-19T04:39:21.090 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 +69us[ +73us] +/- 72ms 2024-09-19T04:39:21.090 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -118us[ -113us] +/- 81ms 2024-09-19T04:39:21.091 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:39:21.092 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T04:39:21.117 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T04:39:21.118 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T04:39:21.136 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T04:39:21.269 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T04:39:21.337 DEBUG:teuthology.orchestra.run.smithi145:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T04:39:21.405 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:39:21.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:39:21.466 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:39:21.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:39:21.495 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:39:21.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:39:21.523 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T04:39:21.534 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T04:39:21.593 INFO:teuthology.task.internal:Duration was 2211.291040 seconds 2024-09-19T04:39:21.593 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T04:39:21.680 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T04:39:21.680 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:39:21.684 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:39:21.687 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:39:21.724 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:39:21.726 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:39:21.767 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:39:22.241 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T04:39:22.244 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-09-19T04:39:22.245 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:39:22.295 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-09-19T04:39:22.295 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:39:22.347 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2024-09-19T04:39:22.348 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:39:22.382 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T04:39:22.382 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:39:22.393 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:39:22.403 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:39:22.459 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T04:39:22.460 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:39:22.475 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:39:22.478 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:39:23.841 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T04:39:24.194 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T04:39:24.195 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:39:24.252 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:39:24.284 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:39:24.316 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T04:39:24.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:39:24.650 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:39:24.653 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:39:24.678 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-09-19T04:39:24.681 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2024-09-19T04:39:24.718 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-09-19T04:39:24.748 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:39:24.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:39:24.782 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:39:24.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:39:24.798 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:39:24.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:39:24.815 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T04:39:24.835 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T04:39:24.836 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627/remote/smithi107 2024-09-19T04:39:24.836 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:39:24.932 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627/remote/smithi115 2024-09-19T04:39:24.932 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:39:24.971 DEBUG:teuthology.misc:Transferring archived files from smithi145:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911627/remote/smithi145 2024-09-19T04:39:24.971 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:39:25.012 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T04:39:25.013 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:39:25.017 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:39:25.019 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:39:25.064 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T04:39:25.075 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T04:39:25.075 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T04:39:25.164 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T04:39:25.164 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:39:25.168 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:39:25.171 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:39:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:39 /home/ubuntu/cephtest 2024-09-19T04:39:25.188 INFO:teuthology.orchestra.run.smithi145.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:39 /home/ubuntu/cephtest 2024-09-19T04:39:25.199 INFO:teuthology.orchestra.run.smithi107.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:39 /home/ubuntu/cephtest 2024-09-19T04:39:25.201 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T04:39:25.212 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T04:39:25.276 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} duration: 2211.2910401821136 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi107 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=469ea1fddc8e4ef2be57d4c0b0b28fe7 status: fail success: false 2024-09-19T04:39:25.277 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T04:39:25.351 INFO:teuthology.run:FAIL